# Tag Info

4

Non-correlation does not imply independence, that is, if two features are not correlated (i.e. zero correlation), it does not mean that they are independent. But (non-zero) correlation implies dependence (see https://stats.stackexchange.com/q/113417/82135 for more details). So, if you have non-zero correlation between two features, it means they are ...

4

The squared form is sometimes called the Euclidean norm or L2 norm. One of its very helpful properties is that it has an easily defined derivative, which can be used in mathematical analysis and translated fairly easily into code. Intuitively it is thought that it is advantageous to exaggerate the differences according to the value of the error, which ...

4

Brief Background The error metric (an appropriate term used in the question title) quantifies the fitness of a linear or nonlinear model. It aggregates individual errors across a set of observations (instances of training data). In typical use, an error function is applied to the difference between the dependent variable vector predicted by the model and ...

4

So in a sense you are correct. Using your jargon: linear regression will only "work" if the true function is approximately $y=h(x)=\beta^{T}x+\beta_0$. Advantages to using this is that its light, its convex, and all-around easy. but for alot of larger problems, this wont work. As you said you want the machine to do the work, so this is (kinda) where deeper ...

3

What this is talking about is how much a machine learning algorithm is good at "memorizing" the data. Decision trees, for their nature, tend to overfit very easily, this is because they can separate the space along very non-linear curves, especially if you get a very deep tree. Simpler algorithms, on the other hand, tend to separate the space along linear ...

3

The fact that features are always positive values don't guarantee that outputs of hidden layers are positive too. Due to multiplication, output of an hidden layer could contain negative values, i.e., a hidden layer can contain weights that have opposites signs as its input. Remember that only layer outputs, not their weights, are passed through ReLu, so, ...

3

Imagine we have the curve $f(x) = x^2$, and we want to find the minimum of this function. The derivate of $f$ with respect to $x$ is $2x$. Now, gradient descent works by updating our current estimate of the minimum, say $c_t$, by the following iterative process $$c_{t+1} = c_t - \alpha \times \nabla_xf(x=c_t),$$ where $\alpha$ is some constant to control how ...

2

It simply derives itself from the maximum likelihood estimation. where in we maximise the log likelihood function., for detailed insight see this lecture: The Method of Maximum Likelihood for Simple Linear Regression.

2

There's no point to fit a linear regression model (such as OLS) with neural network because it's really designed for non-linear models. But if you want to do that, you'll just need to set linear activation units.

2

One justification comes from the central limit theorem. If the noise in your data is the result of the sum of many independent effects, then it will tend to be normally distributed. And normally distributed means that the likelihood of the data is inversely proportional the exponential of the square of the distance to the mean. In other words, minimizing ...

2

Linear regression is always associated with an activation function, the weights between layers and the structure of the network. The weights between layers are $\theta_0$ and $\theta_1$. These weights and the input features undergo the dot product operation, which is then the input to the activation function of the next layer's nodes. An apparently ...

2

It is calculated the same way $b_1$ is calculated. Nearly following your notation, say your multiple linear regression function is $H(X_i) = b_0 + b_1x_{1,i} + ...+ b_nx_{n, i}$ for data instance $X_i=x_{1,i},...,x_{n, i}$ and weights $b_0,...,b_n$. And say your error function is $E(X,Y) = \sum_i(H(X_i)-Y_i)^2$ where $X$ is the collection of all data ...

2

I don't know of a pre-canned algorithm but I would just sweep on the angle from zero to ninety degrees with a triangular region and count the points. For each step in the sweep, record the angle and the count. When the sweep is done you will have an array of angles with bin counts and then you can convert to percentage of total count. You will have to ...

2

Square loss is fine for regression, since minimizing it is the same as maximizing the likelihood of the model parameters (under assumption that the error is Gaussian). However, if the model directly produces probabilities, then it is natural to use these probabilities directly within the loss. Hence, in all classification models we prefer to minimize ...

2

Another specific way to do this if one uses a neural network for this. Use a dropout a layer in your network and instead of scaling the activations at test time, one can sample the activations (just like in training-time) and predict multiple times for a given input, then look at distribution of your outputs. Intuitively this would add "probabilistic, ...

2

If I understand correctly you want to find companies with similar patterns to yours. I would start with measuring cosine similarity between your company and others. It is really easy with Python, for example: In : from sklearn.metrics.pairwise import cosine_similarity In : cosine_similarity([[1,4,2,6], [1,9,5,4]]) Out: array([[1. , 0....

2

I would recommend a hierarchical cluster algorithm, after normalising your numbers into proportions. Then the clustering should be able to identify similar patterns. Depending at which level you make the cut, you can decide how many clusters you want. A great resource on this topic is Kaufman, L., & Roussew, P. J. (1990). "Finding Groups in Data - An ...

2

Normalise your inputs. Neural networks work poorly outside of relatively small numerical ranges on input. An ideal range is for each feature to be drawn from $\mathcal{N}(0,1)$ i.e. a Normal distribution with mean $0$ and standard deviation $1$. In your case, divide both parts of $\mathbf{x}$ by $25$ and subtract $1$ would probably suffice. Your neural ...

2

The difference is simply that non-linear regression learns parameters that in some way control the non-linearity - e.g. any weight or bias that is applied before a non-linear function. For instance: $$y = (w_1 x_1 + w_2 x_2)^2 + w_3$$ With such a function to learn, you cannot separate out transformed values of $w_1$ and $w_2$ and turn this into a linear ...

1

Actually regression comes under the statistical analysis. As you know many business activity(decision making) relies in the previous trends that can be grabbed from the organizations transaction data. When regression is performed on those organizational data. One can understand what decision can be made. One could even simulate the different conditions when ...

1

X = np.array([1,2,3]) Y = np.array([2,1,2]) params = np.array([1, 0]) def loss(y, yhat): return ((y - yhat)**2).mean() def model(x): return params + params*x def loss_grad(y, yhat, x): return np.array([(2*(yhat-y)).mean(), (2*(yhat-y)*x).mean()]) lr = .1 for _ in range(3): yhat = model(X) l = loss(Y, yhat) g = loss_grad(Y, ...

1

The predictions tend to move towards the mean of the series as one predicts for longer horizons. Also, in general, optimal long range forecast is the process mean. In other words, the past of the process contains no information on the development of the process in the distant future. And, this might be the reason that you are getting poor forecasts. ...

1

Predicting with confidence: the best machine learning idea you never heard of by Scott Locklin might provide you an idea. The name of this basket of ideas is “conformal prediction.”

1

Note first that the first $=$ (equals) in $\frac{dl(\theta)}{d\theta} = 0 = −\frac{1}{2\sigma^2}(0−2X^TY + X^TX \theta)$ should be interpreted as a "is set to", that is, we set $\frac{dl(\theta)}{d\theta} = 0$. Given that (apparently) $\frac{dl(\theta)}{d\theta} = −\frac{1}{2\sigma^2}(0−2X^TY + X^TX \theta)$, $\frac{dl(\theta)}{d\theta} = 0$ is ...

1

A hypothesis space is defined as the set of functions $\mathcal H$ that can be chosen by a learning algorithm to minimize loss (in general). $$\mathcal H = \{h_1, h_2,....h_n\}$$ The hypothesis class can be finite or infinite, for example a discrete set of shapes to encircle certain portion of the input space is a finite hypothesis space, whereas hpyothesis ...

1

Consider a target function $f: x \mapsto f(x)$. A hypothesis refers to an approximation of $f$. A hypothesis space refers to the set of possible approximations that an algorithm can create for $f$. The hypothesis space consists of the set of functions the model is limited to learn. For instance, linear regression can be limited to linear functions as its ...

1

This is a good example of what happens when you take text out of context. The passage that was added in the edited question makes a difference, but it's not quite sufficient, and it doesn't help that the notation is all over the place. I found the textbook and the relevant passages (Section 2.3, p. 10-11). Here is a quick attempt at an explanation. The ...

1

I have used W as beta and Y_pred as Y_hat. Apparently, as far as conventions go both feature vector X and weight vector W are assumed as column vectors. Although this is not important in your case but when we use Neural Nets, this is particularly important, and a weight vector for a layer is given by p*N_n where p is the number of features being input from ...

1

First thing is that what does gradient descent do? Gradient Descent is a tool of calculus which we use to determine the parameters (here weights) used in a Machine Learning algorithm or a Neural Network, by running the Gradient Descent algorithm iteratively. What does the vector obtained from one iteration of gradient descent tell us? It tells us the ...

1

The prediction made by linear regression can simply be thought of as a vector dot product. $$\overrightarrow{x}^T \cdot \overrightarrow{y}$$ One of those two vectors is the "data" for one case (like a row in your data matrix), the other is a vector of the model's parameters, which is usually called $\overrightarrow{\theta}$ or $\overrightarrow{\beta}$. So ...

Only top voted, non community-wiki answers of a minimum length are eligible