Questions tagged [linear-regression]

For questions related to the theory or application of linear regression.

Filter by
Sorted by
Tagged with
0
votes
0answers
26 views

Why isn't my perceptron having the expected costs?

I want to implement a single perceptron for linear regression using the following formulas: The input data for the first case is one column (x(392, 1); y(392, 1)) ...
0
votes
1answer
69 views

Which of the following two implementations of a Least Squares classifier in Python is correct?

I am trying to solve a classification problem by implementing the Least Squares algorithm in Python. To solve this problem, I am implementing the linear algebra formula to train the classifier, which ...
2
votes
3answers
101 views

Is there any domain in machine learning that solves a problem by using only analytical algorithms?

Most of the algorithms in machine learning I am aware of use datasets and learning happens in an iterative manner given some examples. The examples can also be understood as experience in the case of ...
0
votes
0answers
29 views

Fine tuning a BERT model for text classification

An article written by Jay Alammar (http://jalammar.github.io/a-visual-guide-to-using-bert-for-the-first-time/) on using a BERT transformer for text classification. The article mentions the following ...
-2
votes
1answer
54 views

Assumptions of a Linear Regression [closed]

I was going through the concept of Linear Regression and ran into the concept of deciding whether a Linear Regression Model is the best fit for your data by 5 assumptions: Linearity Homoscedasticity ...
0
votes
1answer
48 views

Difficult to build learning curve for a simple linear regression

In some part of my online ML course, I should build a code for plotting a learning curve, to be more specific, it's just a linear regression implementation. So the first thing to consider is the ...
0
votes
0answers
7 views

Hand-crafted regression overshoots despite learning rate provided from hessian of the loss

I tried to do my own linear regression but I have seen that there are differences with LinearRegression from sklearn. Indeed, ...
0
votes
0answers
53 views

How can you grade an input?

I'm wondering how one can apply linear regression or some other method to calculate a grade/score for an input. More specifically, I'm having multiple objects in 2D-space, with some (but not all) of ...
2
votes
1answer
82 views

Would either $L_1$ or $L_2$ regularisation lower the MSE on the training and test data?

Consider linear regression. The mean squared error (MSE) is 120.5 for the training dataset. We've reached the minimum for the training data. Is it possible that by applying Lasso (L1 regularization) ...
0
votes
1answer
29 views

Which machine learning technique can I use to match one set of data points to another?

I have two measuring devices. Both measure the same thing. One is accurate, the other is not, but does correlate with a non-fixed offset, some outliers, and some noise. I won't always be using the ...
1
vote
0answers
31 views

Linear output layer back propagation

So I'm stack to something that it's probably very easy but I can't get my head around it. I'm building a Neural Network that will consist of many layers with non-linear activation functions (probably ...
2
votes
1answer
169 views

Is there a connection between the bias term in a linear regression model and the bias that can lead to under-fitting?

Here is a linear regression model $$y = mx + b,$$ where $b$ is known as $y$-intercept, but also known as the bias [1], $m$ is the slope, and $x$ is the feature vector. As I understood, in machine ...
1
vote
1answer
59 views

How parameter adjustment works in Gradient Descent?

I am trying to comprehend how the Gradient Descent works. I understand we have a cost function which is defined in terms of the following parameters, $J(𝑤_{1},𝑤_{2},.... , w_{n}, b)$ the derivative ...
2
votes
0answers
26 views

Is there a UCB type algorithm for linear stochastic bandit with lasso regression?

Why is there no upper confidence bound algorithm for linear stochastic bandits that uses lasso regression in the case that the regression parameters are sparse in the features? In particular, I don't ...
0
votes
1answer
90 views

Hyper-plane in logistic regression vs linear regression for same number of features

Geometric interpretation of Logistic Regression and Linear regression is considered here. I was going through Logistic regression and Linear regression. In the optimization equation of both following ...
0
votes
1answer
28 views

Effect of adding an Independent Variable in Multiple Linear Regression

I am new in machine learning and learning linear regression concept. Please help with answers to below queries. I want to understand effect on existing independent variable(X1) if I add a new ...
0
votes
2answers
80 views

What ML algorithm should I use that suits this data?

What if I have some data, let's say I'm trying to answer if education level and IQ affect earnings, and I want to analyze this data and put in a regression model to predict earnings based on the IQ ...
1
vote
1answer
70 views

Do correlations matter when building neural networks?

I am new to working with neural networks. However, I have built some linear regression models in the past. My question is, is it worth looking for features with a correlation to my target variable as ...
2
votes
1answer
80 views

Why is the hypothesis function $h_{\theta}(x)$ equivalent to $E[y | x; \theta]$ in generalised linear models?

Reading through the CS229 lecture notes on generalised linear models, I came across the idea that a linear regression problem can be modelled as a Gaussian distribution, which is a form of the ...
1
vote
1answer
72 views

If features are always positives, why do we use RELU activation functions?

When does it happen that a layer (either first or hidden) outputs negative values in order to justify the use of RELU? As far as I know, features are never negative or converted to negative in any ...
1
vote
0answers
31 views

3d representation of a regression with two independent variables one of them is categorical and another is continuous

I have hopefully a fundamental question of Do I understand things right. (Thank you in advance and sorry for my English which might be not so good) 1-Preambula 1: I know that if we have 2 independent ...
1
vote
1answer
63 views

Is there any way to apply linear transformations on a vector other than matrix multiplication?

I am trying to optimize the cost function calculation in regression analysis using a non-matrix multiplication based approach. More specifically, I have a point $x = (1, 1, 2, 3)$, to which I want to ...
3
votes
0answers
234 views

How to implement Mean square error loss function in mini batch GD

I have a vectorized implementation of the neural network in c++. I successfully solve the classification problems of Fashion MNIST and CIFAR. Now I am modifying my code to do the Linear regression. I ...
2
votes
1answer
74 views

Do I need to denormalise results in linear regression?

I have learned so far how to linear regression with one or multiple features. So far, so good, everything seems to work fine, at least for my first simple examples. However, I now need to normalise ...
1
vote
0answers
34 views

What is the difference between an generalised estimating equation and a recurrent neural network?

What is the difference between a generalised estimating equation (GEE) model and a recurrent neural network (RNN) model, in terms of what these two models are doing? Apart from the differences in the ...
2
votes
1answer
108 views

What is the difference between linear and non-linear regression?

In machine learning, I understand that linear regression assumes that parameters or weights in equation should be linear. For Example: $$y = w_1x_1 + w_2x_2$$ is a linear equation where $x_1$ and $...
1
vote
2answers
331 views

How do we choose the activation function for each hidden node? [duplicate]

I am new to neural networks. I would like to use them as a fitting or forecasting method. A simple NN model that does not contain hidden layers, that is, the input nodes are directly connected to the ...
0
votes
2answers
56 views

Is it still called linear separation with a layer of more than 1 neuron

A single neuron will be able to do linear separation. For example, XOR simulator network: ...
0
votes
1answer
34 views

Solution to classify product names

I have a bunch of training data for classifying product names, around 30,000 samples. The task is to classify these product names into types of product, around 100 classes (single words). For example:...
1
vote
1answer
67 views

TensorFlow estimator DNNClassifier fails to fit simple data

The ready-to-use DNNClassifier in tf.estimator seems not able to fit these data: ...
2
votes
4answers
141 views

How is regression machine learning?

In regression, in order to minimize an error function, a functional form of hypothesis $h$ must be decided upon, and it must be assumed (as far as I'm concerned) that $f$, the true mapping of instance ...
2
votes
1answer
112 views

Calculating Parameter value Using Gradient Descent for Linear Regression Model

Consider the following data with one input (x) and one output (y): (x=1, y=2) (x=2, y=1) (x=3, y=2) Apply linear regression on this data, using the hypothesis $h_Θ(x) = Θ_0 + Θ_1 x$, where $...
5
votes
2answers
673 views

Is there a machine learning algorithm to find similar sales patterns?

I have a dataset as follows (and the table extends to include an extra 146 columns for companies 4-149) Is there an algorithm I could use effectively to find similar patterns in sales from the other ...
1
vote
1answer
54 views

Auto-regression - Reduce error in prediction

I am trying to develop a time series model using autoregression. The data set is like as follows ...
1
vote
0answers
191 views

Actor-critic algorithm using gaussian Radial Basis Function, Local Linear Regression and shallow Neural Network

I'm attempting to implement the actor-critic algorithm on Matlab using Radial Basis Function, Local Linear Regression, and shallow Neural Network for inverted pendulum system. the state space and the ...
0
votes
0answers
44 views

Can we compare MAE MSE results with categorical_crossentropy?

can i compare MAE and MSE loss results of a regression CNN with categorical_crossentropy loss of a classification CNN if they both have similar tasks? is yes how to?
1
vote
0answers
28 views

What Model Used for Forecasting Sales with Dynamic Holiday

I'm working on a project where I need to forecast sales data where I have history of 1 year (2017) daily data. I am new on Artificial Intelligence topic and after searching for a while, I think ARIMA ...
1
vote
2answers
237 views

How to make machine learning model that reports ambiguity of the input?

Suppose I want to build a neural network regression model that takes one input and return one output. Here's the training data: ...
2
votes
1answer
75 views

Will LMS always be convex function? If yes, then why do we change it for neural networks?

In LMS(least mean square) since, we use a quadratic error function, and quadratic functions are generally parabola in (some convex like shape). I wonder whether that is the reason why we use least ...
3
votes
2answers
243 views

What makes a machine learning algorithm a low variance one or a high variance one?

Some examples of low-variance machine learning algorithms include linear regression, linear discriminant analysis, and logistic regression. Examples of high-variance machine learning algorithms ...
2
votes
1answer
243 views

Does the correlation between inputs affect the model performance?

I'm currently working on a regression problem and I have 10 inputs/attributes. What should I do if there are correlations between different features of the input data? Does the correlation between ...
2
votes
1answer
93 views

Understanding the math behind using maximum likelihood for linear regression

I understand both terms, linear regression and maximum likelihood, but, when it comes to the math, I am totally lost. So I am reading this article The Principle of Maximum Likelihood (by Suriyadeepan ...
3
votes
2answers
2k views

Matrix Dimension for Linear regression coefficients

While reading about least squares implementation for machine learning I came across this passage in the following two photos: Perhaps I’m misinterpreting the meaning of beta but if X^T has dimension ...
5
votes
1answer
274 views

Regression on extreme values

I have a data set that looks like this: I would like to estimate a relationship between x-values and the corresponding 5% extreme y-values, something that might look like that : Do you have an idea ...
4
votes
1answer
1k views

How do you handle multiple categorical values in a single column for wide_deep model in tensorflow? [closed]

To start, let me just say that I am very new to tensorflow and Machine Learning in general. But, as part of my learning project I am trying to adapt the tensorflow wide and deep model to generate ...
6
votes
1answer
425 views

Can we use the recursive least squares as a learning algorithm to an ADALINE?

I'm new to neural network, I study electrical engineering, and I just started working with ADALINEs. I use Matlab, and in their Documentation they cite : However, here the LMS (least mean squares) ...
2
votes
2answers
791 views

How is direction of weight change determined by Gradient Descent algorithm

The result of gradient descent algorithm is a vector. So how does this algorithm decide the direction for weight change? We Give hyperparameters for step size. But how is the vector direction for ...
0
votes
1answer
99 views

multi vs one prediction using Regression

I was trying to build a prediction system where I have the input data arranged in multiple columns. The input data would be of the type where I have weather, service type (bronze, silver, gold), size ...
1
vote
2answers
70 views

In the multi-linear regression, how is the value of weight $b_2$ calculated?

In multivariate linear regression (linear regression with more than one variable) the model is $yi = b_0 + b_1x_{1i} + b_2x_{2i} + ...$ , and so on. But how is the $b_n$ value calculated iteratively? ...
2
votes
3answers
237 views

Understanding a few terms in Andrew Ng's definition of the cost function for linear regression

I have completed week 1 of Andrew Ng's course. I understand that the cost function for linear regression is defined as $J (\theta_0, \theta_1) = 1/2m*\sum (h(x)-y)^2$ and the $h$ is defined as $h(x) = ...