4

The squared form is sometimes called the Euclidean norm or L2 norm. One of its very helpful properties is that it has an easily defined derivative, which can be used in mathematical analysis and translated fairly easily into code. Intuitively it is thought that it is advantageous to exaggerate the differences according to the value of the error, which ...


4

Brief Background The error metric (an appropriate term used in the question title) quantifies the fitness of a linear or nonlinear model. It aggregates individual errors across a set of observations (instances of training data). In typical use, an error function is applied to the difference between the dependent variable vector predicted by the model and ...


4

Non-correlation does not imply independence, that is, if two features are not correlated (i.e. zero correlation), it does not mean that they are independent. But (non-zero) correlation implies dependence (see https://stats.stackexchange.com/q/113417/82135 for more details). So, if you have non-zero correlation between two features, it means they are ...


4

So in a sense you are correct. Using your jargon: linear regression will only "work" if the true function is approximately $y=h(x)=\beta^{T}x+\beta_0$. Advantages to using this is that its light, its convex, and all-around easy. but for alot of larger problems, this wont work. As you said you want the machine to do the work, so this is (kinda) where deeper ...


4

In machine learning, the term bias can refer to at least 2 related concepts A (learnable) parameter of a model, such as a linear regression model, which allows you to learn a shifted function. For example, in the case of a linear regression model $y = f(x) = mx + b$, the bias $b$ allows you to shift the straight-line up an down: without the bias, you would ...


3

What this is talking about is how much a machine learning algorithm is good at "memorizing" the data. Decision trees, for their nature, tend to overfit very easily, this is because they can separate the space along very non-linear curves, especially if you get a very deep tree. Simpler algorithms, on the other hand, tend to separate the space along linear ...


3

A neural network can be reduced to a linear regression model only if we use linear activation functions (i.e. $\sigma(x) = x$), and only if we do not use any neural network specific techniques such as convolution, residuals, etc., as shown below: $\text{neural network}(x) = \sigma_n(W_{n} \sigma_{n-1}(W_{n-1}\dots\sigma_1(W_1 x + b_1) + \dots + b_{n-1}) + ...


3

The fact that features are always positive values don't guarantee that outputs of hidden layers are positive too. Due to multiplication, output of an hidden layer could contain negative values, i.e., a hidden layer can contain weights that have opposites signs as its input. Remember that only layer outputs, not their weights, are passed through ReLu, so, ...


3

Imagine we have the curve $f(x) = x^2$, and we want to find the minimum of this function. The derivate of $f$ with respect to $x$ is $2x$. Now, gradient descent works by updating our current estimate of the minimum, say $c_t$, by the following iterative process $$c_{t+1} = c_t - \alpha \times \nabla_xf(x=c_t),$$ where $\alpha$ is some constant to control how ...


2

It simply derives itself from the maximum likelihood estimation. where in we maximise the log likelihood function., for detailed insight see this lecture: The Method of Maximum Likelihood for Simple Linear Regression.


2

There's no point to fit a linear regression model (such as OLS) with neural network because it's really designed for non-linear models. But if you want to do that, you'll just need to set linear activation units.


2

One justification comes from the central limit theorem. If the noise in your data is the result of the sum of many independent effects, then it will tend to be normally distributed. And normally distributed means that the likelihood of the data is inversely proportional the exponential of the square of the distance to the mean. In other words, minimizing ...


2

Sorry if this is a bad use of answer to add comment but since my reputation is not high enough this is only way to leave a comment to OP's question. I think some of the answers misunderstood the OP's intention. Over fitting is used as a means to test the complexity of the model - if a model cannot overfit a small dataset then it's likely not able to ...


2

It is calculated the same way $b_1$ is calculated. Nearly following your notation, say your multiple linear regression function is $H(X_i) = b_0 + b_1x_{1,i} + ...+ b_nx_{n, i}$ for data instance $X_i=x_{1,i},...,x_{n, i}$ and weights $b_0,...,b_n$. And say your error function is $E(X,Y) = \sum_i(H(X_i)-Y_i)^2$ where $X$ is the collection of all data ...


2

I don't know of a pre-canned algorithm but I would just sweep on the angle from zero to ninety degrees with a triangular region and count the points. For each step in the sweep, record the angle and the count. When the sweep is done you will have an array of angles with bin counts and then you can convert to percentage of total count. You will have to ...


2

Square loss is fine for regression, since minimizing it is the same as maximizing the likelihood of the model parameters (under assumption that the error is Gaussian). However, if the model directly produces probabilities, then it is natural to use these probabilities directly within the loss. Hence, in all classification models we prefer to minimize ...


2

Another specific way to do this if one uses a neural network for this. Use a dropout a layer in your network and instead of scaling the activations at test time, one can sample the activations (just like in training-time) and predict multiple times for a given input, then look at distribution of your outputs. Intuitively this would add "probabilistic, ...


2

If I understand correctly you want to find companies with similar patterns to yours. I would start with measuring cosine similarity between your company and others. It is really easy with Python, for example: In [21]: from sklearn.metrics.pairwise import cosine_similarity In [22]: cosine_similarity([[1,4,2,6], [1,9,5,4]]) Out[22]: array([[1. , 0....


2

I would recommend a hierarchical cluster algorithm, after normalising your numbers into proportions. Then the clustering should be able to identify similar patterns. Depending at which level you make the cut, you can decide how many clusters you want. A great resource on this topic is Kaufman, L., & Roussew, P. J. (1990). "Finding Groups in Data - An ...


2

Normalise your inputs. Neural networks work poorly outside of relatively small numerical ranges on input. An ideal range is for each feature to be drawn from $\mathcal{N}(0,1)$ i.e. a Normal distribution with mean $0$ and standard deviation $1$. In your case, divide both parts of $\mathbf{x}$ by $25$ and subtract $1$ would probably suffice. Your neural ...


2

The difference is simply that non-linear regression learns parameters that in some way control the non-linearity - e.g. any weight or bias that is applied before a non-linear function. For instance: $$y = (w_1 x_1 + w_2 x_2)^2 + w_3$$ With such a function to learn, you cannot separate out transformed values of $w_1$ and $w_2$ and turn this into a linear ...


1

The predictions tend to move towards the mean of the series as one predicts for longer horizons. Also, in general, optimal long range forecast is the process mean. In other words, the past of the process contains no information on the development of the process in the distant future. And, this might be the reason that you are getting poor forecasts. ...


1

Predicting with confidence: the best machine learning idea you never heard of by Scott Locklin might provide you an idea. The name of this basket of ideas is “conformal prediction.”


1

Note first that the first $=$ (equals) in $\frac{dl(\theta)}{d\theta} = 0 = −\frac{1}{2\sigma^2}(0−2X^TY + X^TX \theta)$ should be interpreted as a "is set to", that is, we set $\frac{dl(\theta)}{d\theta} = 0$. Given that (apparently) $\frac{dl(\theta)}{d\theta} = −\frac{1}{2\sigma^2}(0−2X^TY + X^TX \theta)$, $\frac{dl(\theta)}{d\theta} = 0$ is ...


1

This is a good example of what happens when you take text out of context. The passage that was added in the edited question makes a difference, but it's not quite sufficient, and it doesn't help that the notation is all over the place. I found the textbook and the relevant passages (Section 2.3, p. 10-11). Here is a quick attempt at an explanation. The ...


1

RLS is a second order optimizer, so, unlike LMS which takes into account an approximation of the derivative of the gradient, RLS also considers the second order derivative. You can study more about second order methods in sub-section "8.6 Approximate Second-Order Methods" of the following book available online: https://www.deeplearningbook.org/...


1

The prediction made by linear regression can simply be thought of as a vector dot product. $$\overrightarrow{x}^T \cdot \overrightarrow{y}$$ One of those two vectors is the "data" for one case (like a row in your data matrix), the other is a vector of the model's parameters, which is usually called $\overrightarrow{\theta}$ or $\overrightarrow{\beta}$. So ...


Only top voted, non community-wiki answers of a minimum length are eligible