6 votes
Accepted

Is there a connection between the bias term in a linear regression model and the bias that can lead to under-fitting?

In machine learning, the term bias can refer to at least 2 related concepts A (learnable) parameter of a model, such as a linear regression model, which allows you to learn a shifted function. For ...
user avatar
  • 33.8k
5 votes
Accepted

Does the correlation between inputs affect the model performance?

Non-correlation does not imply independence, that is, if two features are not correlated (i.e. zero correlation), it does not mean that they are independent. But (non-zero) correlation implies ...
user avatar
  • 33.8k
5 votes

Linear regression: why is distance *squared* used as an error metric?

The squared form is sometimes called the Euclidean norm or L2 norm. One of its very helpful properties is that it has an easily defined derivative, which can be used in mathematical analysis and ...
user avatar
3 votes

Is Deep Learning the repeated application of Linear Regression?

There's no point to fit a linear regression model (such as OLS) with neural network because it's really designed for non-linear models. But if you want to do that, you'll just need to set linear ...
user avatar
  • 1,390
3 votes

Linear regression: why is distance *squared* used as an error metric?

One justification comes from the central limit theorem. If the noise in your data is the result of the sum of many independent effects, then it will tend to be normally distributed. And normally ...
user avatar
3 votes

Is Deep Learning the repeated application of Linear Regression?

A neural network can be reduced to a linear regression model only if we use linear activation functions (i.e. $\sigma(x) = x$), and only if we do not use any neural network specific techniques such ...
user avatar
  • 1,230
3 votes

Linear regression: why is distance *squared* used as an error metric?

It simply derives itself from the maximum likelihood estimation. where in we maximise the log likelihood function., for detailed insight see this lecture: The Method of Maximum Likelihood for Simple ...
user avatar
  • 181
3 votes

How is regression machine learning?

So in a sense you are correct. Using your jargon: linear regression will only "work" if the true function is approximately $y=h(x)=\beta^{T}x+\beta_0$. Advantages to using this is that its light, its ...
user avatar
  • 2,249
3 votes

If features are always positives, why do we use RELU activation functions?

The fact that features are always positive values don't guarantee that outputs of hidden layers are positive too. Due to multiplication, output of an hidden layer could contain negative values, i.e., ...
user avatar
3 votes

How parameter adjustment works in Gradient Descent?

Imagine we have the curve $f(x) = x^2$, and we want to find the minimum of this function. The derivate of $f$ with respect to $x$ is $2x$. Now, gradient descent works by updating our current estimate ...
user avatar
2 votes

Linear regression: why is distance *squared* used as an error metric?

One justification is that under homoscedasticity the L2 norm produces the minimum variance unbiased estimator (MVUE), see Gauss-Markov Theorem. It means that the fitted values are the conditional ...
user avatar
2 votes

What to do if CNN cannot overfit a training set on adding dropout?

Sorry if this is a bad use of answer to add comment but since my reputation is not high enough this is only way to leave a comment to OP's question. I think some of the answers misunderstood the OP's ...
user avatar
2 votes

In the multi-linear regression, how is the value of weight $b_2$ calculated?

It is calculated the same way $b_1$ is calculated. Nearly following your notation, say your multiple linear regression function is $H(X_i) = b_0 + b_1x_{1,i} + ...+ b_nx_{n, i}$ for data ...
user avatar
  • 769
2 votes

Regression on extreme values

I don't know of a pre-canned algorithm but I would just sweep on the angle from zero to ninety degrees with a triangular region and count the points. For each step in the sweep, record the angle and ...
user avatar
2 votes
Accepted

Understanding the math behind using maximum likelihood for linear regression

Note first that the first $=$ (equals) in $\frac{dl(\theta)}{d\theta} = 0 = −\frac{1}{2\sigma^2}(0−2X^TY + X^TX \theta)$ should be interpreted as a "is set to", that is, we set $\frac{dl(\theta)}{d\...
user avatar
  • 33.8k
2 votes
Accepted

Will LMS always be convex function? If yes, then why do we change it for neural networks?

Square loss is fine for regression, since minimizing it is the same as maximizing the likelihood of the model parameters (under assumption that the error is Gaussian). However, if the model directly ...
user avatar
  • 469
2 votes
Accepted

What makes a machine learning algorithm a low variance one or a high variance one?

What this is talking about is how much a machine learning algorithm is good at "memorizing" the data. Decision trees, for their nature, tend to overfit very easily, this is because they can separate ...
user avatar
  • 136
2 votes

How to make machine learning model that reports ambiguity of the input?

Another specific way to do this if one uses a neural network for this. Use a dropout a layer in your network and instead of scaling the activations at test time, one can sample the activations (just ...
user avatar
  • 384
2 votes
Accepted

Is there a machine learning algorithm to find similar sales patterns?

If I understand correctly you want to find companies with similar patterns to yours. I would start with measuring cosine similarity between your company and ...
user avatar
  • 206
2 votes

Is there a machine learning algorithm to find similar sales patterns?

I would recommend a hierarchical cluster algorithm, after normalising your numbers into proportions. Then the clustering should be able to identify similar patterns. Depending at which level you make ...
user avatar
  • 5,062
2 votes
Accepted

TensorFlow estimator DNNClassifier fails to fit simple data

Normalise your inputs. Neural networks work poorly outside of relatively small numerical ranges on input. An ideal range is for each feature to be drawn from $\mathcal{N}(0,1)$ i.e. a Normal ...
user avatar
  • 23.8k
2 votes
Accepted

What is the difference between linear and non-linear regression?

The difference is simply that non-linear regression learns parameters that in some way control the non-linearity - e.g. any weight or bias that is applied before a non-linear function. For instance: ...
user avatar
  • 23.8k
2 votes

Is there any domain in machine learning that solves a problem by using only analytical algorithms?

In some cases, you can solve a linear regression problem with an analytical (or closed-form) solution/expression (although this may not always be the best approach). See this answer for more details. ...
user avatar
  • 33.8k
2 votes
Accepted

Not able to understand Pytorch Tensor (Weight & Biases) Size for Linear Regression

The size of the parameters tensor is depended on what type of layer that you want to build. Convolutional, fully connected, attention or even custom layer, each layer has a difference in the way it ...
user avatar
  • 817
2 votes
Accepted

Why is the cross-entropy a cost function?

Optimizing the cross-entropy is equivalent to optimizing the log-likelihood of the parameters given the data, $\ell(\theta)$, which is what we want, i.e. find the parameters that most likely generated ...
user avatar
  • 33.8k
2 votes
Accepted

Training a regression model on a set of values in 0-1 range to give 0-1 continual values

As long as you train the model with a proper loss function for regression the model will learn to output any continuous values, not restricted to and most likely not exactly equal to the labels your ...
user avatar
2 votes

Training a regression model on a set of values in 0-1 range to give 0-1 continual values

Many machine learning models used for regression will interpolate their predictions as you seem to want, and can return target values not seen in the training set. For example, basic linear regression ...
user avatar
  • 23.8k
1 vote
Accepted

Calculating Parameter value Using Gradient Descent for Linear Regression Model

...
user avatar
  • 2,249

Only top scored, non community-wiki answers of a minimum length are eligible