# Tag Info

4

The squared form is sometimes called the Euclidean norm or L2 norm. One of its very helpful properties is that it has an easily defined derivative, which can be used in mathematical analysis and translated fairly easily into code. Intuitively it is thought that it is advantageous to exaggerate the differences according to the value of the error, which ...

4

Brief Background The error metric (an appropriate term used in the question title) quantifies the fitness of a linear or nonlinear model. It aggregates individual errors across a set of observations (instances of training data). In typical use, an error function is applied to the difference between the dependent variable vector predicted by the model and ...

4

Non-correlation does not imply independence, that is, if two features are not correlated (i.e. zero correlation), it does not mean that they are independent. But (non-zero) correlation implies dependence (see https://stats.stackexchange.com/q/113417/82135 for more details). So, if you have non-zero correlation between two features, it means they are ...

4

So in a sense you are correct. Using your jargon: linear regression will only "work" if the true function is approximately $y=h(x)=\beta^{T}x+\beta_0$. Advantages to using this is that its light, its convex, and all-around easy. but for alot of larger problems, this wont work. As you said you want the machine to do the work, so this is (kinda) where deeper ...

3

What this is talking about is how much a machine learning algorithm is good at "memorizing" the data. Decision trees, for their nature, tend to overfit very easily, this is because they can separate the space along very non-linear curves, especially if you get a very deep tree. Simpler algorithms, on the other hand, tend to separate the space along linear ...

3

I think it depends on you application and what data you have available. If the prediction of body temperature itself doesn't have to be accurate and classes like COLD, NORMAL, and HOT will suffice, you should stay with a classification. There isn't a cut off but as you increase the number of classes that represent numbers on the same scale, it may become ...

2

Since you're termed the problem you're trying to solve is to "measure the body temperature of a person", the output should be a continuous valued number. If the problem statement had been to rank or classify a person's body temperature as fever/healthy, then this could be a classification problem. Deep learning employ neuron units at the output layer ...

2

One justification comes from the central limit theorem. If the noise in your data is the result of the sum of many independent effects, then it will tend to be normally distributed. And normally distributed means that the likelihood of the data is inversely proportional the exponential of the square of the distance to the mean. In other words, minimizing ...

2

There's no point to fit a linear regression model (such as OLS) with neural network because it's really designed for non-linear models. But if you want to do that, you'll just need to set linear activation units.

2

It simply derives itself from the maximum likelihood estimation. where in we maximise the log likelihood function., for detailed insight see this lecture: The Method of Maximum Likelihood for Simple Linear Regression.

2

Linear regression is always associated with an activation function, the weights between layers and the structure of the network. The weights between layers are $\theta_0$ and $\theta_1$. These weights and the input features undergo the dot product operation, which is then the input to the activation function of the next layer's nodes. An apparently ...

2

It is calculated the same way $b_1$ is calculated. Nearly following your notation, say your multiple linear regression function is $H(X_i) = b_0 + b_1x_{1,i} + ...+ b_nx_{n, i}$ for data instance $X_i=x_{1,i},...,x_{n, i}$ and weights $b_0,...,b_n$. And say your error function is $E(X,Y) = \sum_i(H(X_i)-Y_i)^2$ where $X$ is the collection of all data ...

2

I don't know of a pre-canned algorithm but I would just sweep on the angle from zero to ninety degrees with a triangular region and count the points. For each step in the sweep, record the angle and the count. When the sweep is done you will have an array of angles with bin counts and then you can convert to percentage of total count. You will have to ...

2

Square loss is fine for regression, since minimizing it is the same as maximizing the likelihood of the model parameters (under assumption that the error is Gaussian). However, if the model directly produces probabilities, then it is natural to use these probabilities directly within the loss. Hence, in all classification models we prefer to minimize ...

2

Another specific way to do this if one uses a neural network for this. Use a dropout a layer in your network and instead of scaling the activations at test time, one can sample the activations (just like in training-time) and predict multiple times for a given input, then look at distribution of your outputs. Intuitively this would add "probabilistic, ...

2

If I understand correctly you want to find companies with similar patterns to yours. I would start with measuring cosine similarity between your company and others. It is really easy with Python, for example: In : from sklearn.metrics.pairwise import cosine_similarity In : cosine_similarity([[1,4,2,6], [1,9,5,4]]) Out: array([[1. , 0....

2

I would recommend a hierarchical cluster algorithm, after normalising your numbers into proportions. Then the clustering should be able to identify similar patterns. Depending at which level you make the cut, you can decide how many clusters you want. A great resource on this topic is Kaufman, L., & Roussew, P. J. (1990). "Finding Groups in Data - An ...

2

Normalise your inputs. Neural networks work poorly outside of relatively small numerical ranges on input. An ideal range is for each feature to be drawn from $\mathcal{N}(0,1)$ i.e. a Normal distribution with mean $0$ and standard deviation $1$. In your case, divide both parts of $\mathbf{x}$ by $25$ and subtract $1$ would probably suffice. Your neural ...

2

The difference is simply that non-linear regression learns parameters that in some way control the non-linearity - e.g. any weight or bias that is applied before a non-linear function. For instance: $$y = (w_1 x_1 + w_2 x_2)^2 + w_3$$ With such a function to learn, you cannot separate out transformed values of $w_1$ and $w_2$ and turn this into a linear ...

1

X = np.array([1,2,3]) Y = np.array([2,1,2]) params = np.array([1, 0]) def loss(y, yhat): return ((y - yhat)**2).mean() def model(x): return params + params*x def loss_grad(y, yhat, x): return np.array([(2*(yhat-y)).mean(), (2*(yhat-y)*x).mean()]) lr = .1 for _ in range(3): yhat = model(X) l = loss(Y, yhat) g = loss_grad(Y, ...

1

The predictions tend to move towards the mean of the series as one predicts for longer horizons. Also, in general, optimal long range forecast is the process mean. In other words, the past of the process contains no information on the development of the process in the distant future. And, this might be the reason that you are getting poor forecasts. ...

1

Predicting with confidence: the best machine learning idea you never heard of by Scott Locklin might provide you an idea. The name of this basket of ideas is “conformal prediction.”

1

Note first that the first $=$ (equals) in $\frac{dl(\theta)}{d\theta} = 0 = −\frac{1}{2\sigma^2}(0−2X^TY + X^TX \theta)$ should be interpreted as a "is set to", that is, we set $\frac{dl(\theta)}{d\theta} = 0$. Given that (apparently) $\frac{dl(\theta)}{d\theta} = −\frac{1}{2\sigma^2}(0−2X^TY + X^TX \theta)$, $\frac{dl(\theta)}{d\theta} = 0$ is ...

1

Consider a target function $f: x \mapsto f(x)$. A hypothesis refers to an approximation of $f$. A hypothesis space refers to the set of possible approximations that algorithm can create for $f$. The hypothesis space consists of the set of functions the model is limited to learn. For instance, linear regression can be limited to linear functions as its ...

1

This is a good example of what happens when you take text out of context. The passage that was added in the edited question makes a difference, but it's not quite sufficient, and it doesn't help that the notation is all over the place. I found the textbook and the relevant passages (Section 2.3, p. 10-11). Here is a quick attempt at an explanation. The ...

1

I have used W as beta and Y_pred as Y_hat. Apparently, as far as conventions go both feature vector X and weight vector W are assumed as column vectors. Although this is not important in your case but when we use Neural Nets, this is particularly important, and a weight vector for a layer is given by p*N_n where p is the number of features being input from ...

1

First thing is that what does gradient descent do? Gradient Descent is a tool of calculus which we use to determine the parameters (here weights) used in a Machine Learning algorithm or a Neural Network, by running the Gradient Descent algorithm iteratively. What does the vector obtained from one iteration of gradient descent tell us? It tells us the ...

1

The prediction made by linear regression can simply be thought of as a vector dot product. $$\overrightarrow{x}^T \cdot \overrightarrow{y}$$ One of those two vectors is the "data" for one case (like a row in your data matrix), the other is a vector of the model's parameters, which is usually called $\overrightarrow{\theta}$ or $\overrightarrow{\beta}$. So ...

1

As said above, they are weights to your hypothesis function that are changed during training to minimize your Error function. You can think of them like slope and y intercept in basic algebra. However, a linear regression hypothesis function can be parameterized by many more weight terms than just theta_0, and theta_1. I detail this process more in this ...

1

At first we should define, what overfitting means. The OP has used 3000 training examples for reducing the error-rate of his neural network. Overfitting is happening, if it is not possible to use the learned model on new data. That means, the error rate for data outside of training-data is high. Overcome the problem is not possible. Because today's hardware ...

Only top voted, non community-wiki answers of a minimum length are eligible