Tag Info

Hot answers tagged linear-regression

6 votes
Accepted

Is there a connection between the bias term in a linear regression model and the bias that can lead to under-fitting?

In machine learning, the term bias can refer to at least 2 related concepts A (learnable) parameter of a model, such as a linear regression model, which allows you to learn a shifted function. For ...
• 33.8k
5 votes
Accepted

Does the correlation between inputs affect the model performance?

Non-correlation does not imply independence, that is, if two features are not correlated (i.e. zero correlation), it does not mean that they are independent. But (non-zero) correlation implies ...
• 33.8k
5 votes

Linear regression: why is distance *squared* used as an error metric?

The squared form is sometimes called the Euclidean norm or L2 norm. One of its very helpful properties is that it has an easily defined derivative, which can be used in mathematical analysis and ...
3 votes

Is Deep Learning the repeated application of Linear Regression?

There's no point to fit a linear regression model (such as OLS) with neural network because it's really designed for non-linear models. But if you want to do that, you'll just need to set linear ...
• 1,390
3 votes

Linear regression: why is distance *squared* used as an error metric?

One justification comes from the central limit theorem. If the noise in your data is the result of the sum of many independent effects, then it will tend to be normally distributed. And normally ...
• 139
3 votes

Is Deep Learning the repeated application of Linear Regression?

A neural network can be reduced to a linear regression model only if we use linear activation functions (i.e. $\sigma(x) = x$), and only if we do not use any neural network specific techniques such ...
• 1,230
3 votes

Linear regression: why is distance *squared* used as an error metric?

It simply derives itself from the maximum likelihood estimation. where in we maximise the log likelihood function., for detailed insight see this lecture: The Method of Maximum Likelihood for Simple ...
• 181
3 votes

How is regression machine learning?

So in a sense you are correct. Using your jargon: linear regression will only "work" if the true function is approximately $y=h(x)=\beta^{T}x+\beta_0$. Advantages to using this is that its light, its ...
• 2,249
3 votes

If features are always positives, why do we use RELU activation functions?

The fact that features are always positive values don't guarantee that outputs of hidden layers are positive too. Due to multiplication, output of an hidden layer could contain negative values, i.e., ...
• 810
3 votes

How parameter adjustment works in Gradient Descent?

Imagine we have the curve $f(x) = x^2$, and we want to find the minimum of this function. The derivate of $f$ with respect to $x$ is $2x$. Now, gradient descent works by updating our current estimate ...
• 4,055
2 votes

Linear regression: why is distance *squared* used as an error metric?

One justification is that under homoscedasticity the L2 norm produces the minimum variance unbiased estimator (MVUE), see Gauss-Markov Theorem. It means that the fitted values are the conditional ...
2 votes

What to do if CNN cannot overfit a training set on adding dropout?

Sorry if this is a bad use of answer to add comment but since my reputation is not high enough this is only way to leave a comment to OP's question. I think some of the answers misunderstood the OP's ...
2 votes

In the multi-linear regression, how is the value of weight $b_2$ calculated?

It is calculated the same way $b_1$ is calculated. Nearly following your notation, say your multiple linear regression function is $H(X_i) = b_0 + b_1x_{1,i} + ...+ b_nx_{n, i}$ for data ...
• 769
2 votes

Regression on extreme values

I don't know of a pre-canned algorithm but I would just sweep on the angle from zero to ninety degrees with a triangular region and count the points. For each step in the sweep, record the angle and ...
• 1,423
2 votes
Accepted

• 33.8k
2 votes
Accepted

Training a regression model on a set of values in 0-1 range to give 0-1 continual values

As long as you train the model with a proper loss function for regression the model will learn to output any continuous values, not restricted to and most likely not exactly equal to the labels your ...
• 3,513
2 votes

Training a regression model on a set of values in 0-1 range to give 0-1 continual values

Many machine learning models used for regression will interpolate their predictions as you seem to want, and can return target values not seen in the training set. For example, basic linear regression ...
• 23.8k
1 vote
Accepted

Calculating Parameter value Using Gradient Descent for Linear Regression Model

...
• 2,249

Only top scored, non community-wiki answers of a minimum length are eligible