7
votes
Accepted
Is there a connection between the bias term in a linear regression model and the bias that can lead to under-fitting?
In machine learning, the term bias can refer to at least 2 related concepts
A (learnable) parameter of a model, such as a linear regression model, which allows you to learn a shifted function. For ...
- 41.4k
5
votes
Accepted
Does the correlation between inputs affect the model performance?
Non-correlation does not imply independence, that is, if two features are not correlated (i.e. zero correlation), it does not mean that they are independent. But (non-zero) correlation implies ...
- 41.4k
5
votes
Linear regression: why is distance *squared* used as an error metric?
The squared form is sometimes called the Euclidean norm or L2 norm. One of its very helpful properties is that it has an easily defined derivative, which can be used in mathematical analysis and ...
- 367
4
votes
Accepted
Linear regression: why is distance *squared* used as an error metric?
Brief Background
The error metric (an appropriate term used in the question title) quantifies the fitness of a linear or nonlinear model.
It aggregates individual errors across a set of observations ...
- 7,513
3
votes
Accepted
What makes a machine learning algorithm a low variance one or a high variance one?
What this is talking about is how much a machine learning algorithm is good at "memorizing" the data. Decision trees, for their nature, tend to overfit very easily, this is because they can separate ...
- 146
3
votes
Is Deep Learning the repeated application of Linear Regression?
There's no point to fit a linear regression model (such as OLS) with neural network because it's really designed for non-linear models. But if you want to do that, you'll just need to set linear ...
- 1,421
3
votes
Linear regression: why is distance *squared* used as an error metric?
One justification comes from the central limit theorem. If the noise in your data is the result of the sum of many independent effects, then it will tend to be normally distributed. And normally ...
- 139
3
votes
Is Deep Learning the repeated application of Linear Regression?
A neural network can be reduced to a linear regression model only if we use linear activation functions (i.e. $\sigma(x) = x$), and only if we do not use any neural network specific techniques such ...
- 1,690
3
votes
Linear regression: why is distance *squared* used as an error metric?
It simply derives itself from the maximum likelihood estimation. where in we maximise the log likelihood function., for detailed insight see this lecture: The Method of Maximum Likelihood for Simple ...
- 181
3
votes
Accepted
Is there a machine learning algorithm to find similar sales patterns?
If I understand correctly you want to find companies with similar patterns to yours.
I would start with measuring cosine similarity between your company and ...
- 216
3
votes
How is regression machine learning?
So in a sense you are correct. Using your jargon: linear regression will only "work" if the true function is approximately $y=h(x)=\beta^{T}x+\beta_0$. Advantages to using this is that its light, its ...
- 2,387
3
votes
If features are always positives, why do we use RELU activation functions?
The fact that features are always positive values don't guarantee that outputs of hidden layers are positive too.
Due to multiplication, output of an hidden layer could contain negative values, i.e., ...
- 1,040
3
votes
How parameter adjustment works in Gradient Descent?
Imagine we have the curve $f(x) = x^2$, and we want to find the minimum of this function. The derivate of $f$ with respect to $x$ is $2x$. Now, gradient descent works by updating our current estimate ...
- 5,030
3
votes
Accepted
Not able to understand Pytorch Tensor (Weight & Biases) Size for Linear Regression
The size of the parameters tensor is depended on what type of layer that you want to build. Convolutional, fully connected, attention or even custom layer, each layer has a difference in the way it ...
- 912
3
votes
Training a regression model on a set of values in 0-1 range to give 0-1 continual values
Many machine learning models used for regression will interpolate their predictions as you seem to want, and can return target values not seen in the training set.
For example, basic linear regression ...
- 33.3k
2
votes
Linear regression: why is distance *squared* used as an error metric?
One justification is that under homoscedasticity the L2 norm produces the minimum variance unbiased estimator (MVUE), see Gauss-Markov Theorem. It means that the fitted values are the conditional ...
- 29
2
votes
What to do if CNN cannot overfit a training set on adding dropout?
Sorry if this is a bad use of answer to add comment but since my reputation is not high enough this is only way to leave a comment to OP's question.
I think some of the answers misunderstood the OP's ...
2
votes
Regression on extreme values
I don't know of a pre-canned algorithm but I would just sweep on the angle from zero to ninety degrees with a triangular region and count the points. For each step in the sweep, record the angle and ...
- 1,987
2
votes
Accepted
Understanding the math behind using maximum likelihood for linear regression
Note first that the first $=$ (equals) in $\frac{dl(\theta)}{d\theta}
= 0 = −\frac{1}{2\sigma^2}(0−2X^TY + X^TX \theta)$ should be interpreted as a "is set to", that is, we set $\frac{dl(\theta)}{d\...
- 41.4k
2
votes
In the multi-linear regression, how is the value of weight $b_2$ calculated?
It is calculated the same way $b_1$ is calculated.
Nearly following your notation, say your multiple linear regression function is
$H(X_i) = b_0 + b_1x_{1,i} + ...+ b_nx_{n, i}$
for data ...
- 823
2
votes
Accepted
Will LMS always be convex function? If yes, then why do we change it for neural networks?
Square loss is fine for regression, since minimizing it is the same as maximizing the likelihood of the model parameters (under assumption that the error is Gaussian). However, if the model directly ...
- 499
2
votes
How to make machine learning model that reports ambiguity of the input?
Another specific way to do this if one uses a neural network for this. Use a dropout a layer in your network and instead of scaling the activations at test time, one can sample the activations (just ...
- 414
2
votes
Is there a machine learning algorithm to find similar sales patterns?
I would recommend a hierarchical cluster algorithm, after normalising your numbers into proportions. Then the clustering should be able to identify similar patterns. Depending at which level you make ...
- 5,417
2
votes
Accepted
TensorFlow estimator DNNClassifier fails to fit simple data
Normalise your inputs.
Neural networks work poorly outside of relatively small numerical ranges on input. An ideal range is for each feature to be drawn from $\mathcal{N}(0,1)$ i.e. a Normal ...
- 33.3k
2
votes
Accepted
What is the difference between linear and non-linear regression?
The difference is simply that non-linear regression learns parameters that in some way control the non-linearity - e.g. any weight or bias that is applied before a non-linear function.
For instance:
...
- 33.3k
2
votes
Is there any domain in machine learning that solves a problem by using only analytical algorithms?
In some cases, you can solve a linear regression problem with an analytical (or closed-form) solution/expression (although this may not always be the best approach). See this answer for more details.
...
- 41.4k
2
votes
Accepted
Why is the cross-entropy a cost function?
Optimizing the cross-entropy is equivalent to optimizing the log-likelihood of the parameters given the data, $\ell(\theta)$, which is what we want, i.e. find the parameters that most likely generated ...
- 41.4k
2
votes
Accepted
Training a regression model on a set of values in 0-1 range to give 0-1 continual values
As long as you train the model with a proper loss function for regression the model will learn to output any continuous values, not restricted to and most likely not exactly equal to the labels your ...
- 5,428
2
votes
Can Adaline do multiple linear regression being equivalent to the least squares method?
Though conceptually they're very similar, your own reference points out their essential difference:
This update rule minimizes $E$, the square of the error, and is in fact the stochastic gradient ...
- 5,422
Only top scored, non community-wiki answers of a minimum length are eligible