# Questions tagged [linear-regression]

For questions related to the theory or application of linear regression.

66 questions
Filter by
Sorted by
Tagged with
1 vote
49 views

### Unclear steps in derivation of normal equations in linear regression using linear algebra approach

How are eqs.(3.55) and (3.56) obtained? Especially, it is unclear how triangle inequality implies eq.(3.56) because we have squared norms.
32 views

### Can multiple linear regression using the least squares(OLS) method, also be used to solve simple linear regression problems? Would both be equivalent?

Simple Linear Regression reference: https://online.stat.psu.edu/stat462/node/93/ Multiple Linear Regression reference: https://online.stat.psu.edu/stat462/node/131/ I see that the way to calculate the ...
93 views

### Can Adaline do multiple linear regression being equivalent to the least squares method?

https://en.wikipedia.org/wiki/ADALINE Can Adaline(Adaptive Linear Neuron) be used to do a multiple linear regression being equivalent to the least squares method?
15 views

### Using nearest neighbor in RANSAC

I found many resources online talking about nearest neighbor concept in RANSAC. For example, figure 2 of this paper, this article and this repo talk about nearest neighbor in the context of RANSAC. ...
27 views

### What are w0 and w1 respectively after training with the following two examples of (x, y) in the given order?

The answer is supposed to be w0 = 2.8229, w1 = 2.4686. I'm not sure how that is the case. Can you please also show how you arrived at the solution?
1 vote
57 views

### How to compute an estimate of the expected value of a stochastic random variable in Reinforcement Learning?

In the section on LSTD in SuttonBarto's book on RL, there is a proof on convergence of semi-gradient TD(0) using a linear function approximator. Later on they estimated A and b as I was under the ...
1 vote
130 views

### Why my best fit line is not having a single straight line | Multiple Linear Regression

I am working on Multiple Linear Regression (Multiple variables). I am been able to predict and get a good r2 score. But I am not sure that I understood the part of plotting the best fit line, I can't ...
74 views

### Loss function of logistic Regression Geometric

In linear Regression, I train the Model so the Graph runs best through the Data Points, so the geometric distance between f(x) and y^i is minimized. Now is it correct that in logistic Regression I do ...
78 views

### cross_val_score of sklearn and LinearRegression scoring method

cross_val_score (https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.cross_val_score.html) uses the estimator’s default scorer (if available) and LinearRgression (the estimator I ...
20 views

### Confused about interaction terms in polynomial regression

I am trying to code multivariate polynomial regression from scratch and I got confused about how interaction terms work. I saw that a polynomial regression with 2 inputs and with interaction terms ...
36 views

### SQL Machine Learning using matrix multiplication

What is the easiest classification algorithm in SQL when my data looks like this? ...
48 views

### Multi-layer network only predicts linear trends

I have made a neural network from scratch (in java), which is refusing to switch out of linear regression. I have pushed up the layer sizes (it now has 2 hidden layers, both with 5 neurons), and yet ...
1 vote
49 views

### Linear Actor Critic for continuing task and 1 continuous action => Any comment?

I wish to implement an Actor Critic agent using linear functions for a continuing task with one continuous action. Below the resulting pseudo-code I have reached by my own (the initialization part is ...
1k views

### Training a regression model on a set of values in 0-1 range to give 0-1 continual values

I have a textual dataset that has a set of real numbers as labels: L={0.0, 0.33, 0.5, 0.75, 1.0}, and I have a model that takes the text as input and has a Sigmoid output. If I train the model on this ...
44 views

### Simple Polynomial Gradient Descent algorithm not working

I am trying to implement a simple 2nd order polynomial gradient descent algorithm in Java. It is not converging and becomes unstable. How do I fix it? ...
205 views

### Why is the cross-entropy a cost function?

The question looks foolish, but I think cross-entropy is somewhat weird as a cost function. As a cost function for linear regression, the mean square error $\sum_{i=1}^{n} (y_i - (ax_i+b)) ^2$ seems ...
1 vote
699 views

### Not able to understand Pytorch Tensor (Weight & Biases) Size for Linear Regression

Below are the two tensors ...
44 views

### Why isn't my perceptron having the expected costs?

I want to implement a single perceptron for linear regression using the following formulas: The input data for the first case is one column (x(392, 1); y(392, 1)) ...
312 views

### Which of the following two implementations of a Least Squares classifier in Python is correct?

I am trying to solve a classification problem by implementing the Least Squares algorithm in Python. To solve this problem, I am implementing the linear algebra formula to train the classifier, which ...
241 views

### Is there any domain in machine learning that solves a problem by using only analytical algorithms?

Most of the algorithms in machine learning I am aware of use datasets and learning happens in an iterative manner given some examples. The examples can also be understood as experience in the case of ...
621 views

### Would either $L_1$ or $L_2$ regularisation lower the MSE on the training and test data?

Consider linear regression. The mean squared error (MSE) is 120.5 for the training dataset. We've reached the minimum for the training data. Is it possible that by applying Lasso (L1 regularization) ...
45 views

### Which machine learning technique can I use to match one set of data points to another?

I have two measuring devices. Both measure the same thing. One is accurate, the other is not, but does correlate with a non-fixed offset, some outliers, and some noise. I won't always be using the ...
1 vote
96 views

### Linear output layer back propagation

So I'm stack to something that it's probably very easy but I can't get my head around it. I'm building a Neural Network that will consist of many layers with non-linear activation functions (probably ...
2k views

### Is there a connection between the bias term in a linear regression model and the bias that can lead to under-fitting?

Here is a linear regression model $$y = mx + b,$$ where $b$ is known as $y$-intercept, but also known as the bias , $m$ is the slope, and $x$ is the feature vector. As I understood, in machine ...
1 vote
177 views

I am trying to comprehend how the Gradient Descent works. I understand we have a cost function which is defined in terms of the following parameters, $J(𝑤_{1},𝑤_{2},.... , w_{n}, b)$ the derivative ...
81 views

### Is there a UCB type algorithm for linear stochastic bandit with lasso regression?

Why is there no upper confidence bound algorithm for linear stochastic bandits that uses lasso regression in the case that the regression parameters are sparse in the features? In particular, I don't ...
283 views

### Hyper-plane in logistic regression vs linear regression for same number of features

Geometric interpretation of Logistic Regression and Linear regression is considered here. I was going through Logistic regression and Linear regression. In the optimization equation of both following ...
51 views

### Effect of adding an Independent Variable in Multiple Linear Regression

I am new in machine learning and learning linear regression concept. Please help with answers to below queries. I want to understand effect on existing independent variable(X1) if I add a new ...
147 views

### What ML algorithm should I use that suits this data?

What if I have some data, let's say I'm trying to answer if education level and IQ affect earnings, and I want to analyze this data and put in a regression model to predict earnings based on the IQ ...
580 views

### Do correlations matter when building neural networks?

I am new to working with neural networks. However, I have built some linear regression models in the past. My question is, is it worth looking for features with a correlation to my target variable as ...
213 views

### Why is the hypothesis function $h_{\theta}(x)$ equivalent to $E[y | x; \theta]$ in generalised linear models?

Reading through the CS229 lecture notes on generalised linear models, I came across the idea that a linear regression problem can be modelled as a Gaussian distribution, which is a form of the ...
1 vote
115 views

### If features are always positives, why do we use RELU activation functions?

When does it happen that a layer (either first or hidden) outputs negative values in order to justify the use of RELU? As far as I know, features are never negative or converted to negative in any ...
1 vote
49 views

### 3d representation of a regression with two independent variables one of them is categorical and another is continuous

I have hopefully a fundamental question of Do I understand things right. (Thank you in advance and sorry for my English which might be not so good) 1-Preambula 1: I know that if we have 2 independent ...
1 vote
69 views

### Is there any way to apply linear transformations on a vector other than matrix multiplication?

I am trying to optimize the cost function calculation in regression analysis using a non-matrix multiplication based approach. More specifically, I have a point $x = (1, 1, 2, 3)$, to which I want to ...
171 views

### Do I need to denormalise results in linear regression?

I have learned so far how to linear regression with one or multiple features. So far, so good, everything seems to work fine, at least for my first simple examples. However, I now need to normalise my ...
1 vote
49 views

### What is the difference between an generalised estimating equation and a recurrent neural network?

What is the difference between a generalised estimating equation (GEE) model and a recurrent neural network (RNN) model, in terms of what these two models are doing? Apart from the differences in the ...
696 views

1k views

### Is there a machine learning algorithm to find similar sales patterns?

I have a dataset as follows (and the table extends to include an extra 146 columns for companies 4-149) Is there an algorithm I could use effectively to find similar patterns in sales from the other ...
1 vote
79 views

### Auto-regression - Reduce error in prediction

I am trying to develop a time series model using autoregression. The data set is like as follows ...
1 vote
293 views

### Actor-critic algorithm using gaussian Radial Basis Function, Local Linear Regression and shallow Neural Network

I'm attempting to implement the actor-critic algorithm on Matlab using Radial Basis Function, Local Linear Regression, and shallow Neural Network for inverted pendulum system. the state space and the ...
1 vote
37 views

### What Model Used for Forecasting Sales with Dynamic Holiday

I'm working on a project where I need to forecast sales data where I have history of 1 year (2017) daily data. I am new on Artificial Intelligence topic and after searching for a while, I think ARIMA ...
1 vote
360 views

### How to make machine learning model that reports ambiguity of the input?

Suppose I want to build a neural network regression model that takes one input and return one output. Here's the training data: ...