# Calculating Parameter value Using Gradient Descent for Linear Regression Model

Consider the following data with one input (x) and one output (y):
(x=1, y=2)
(x=2, y=1)
(x=3, y=2)
Apply linear regression on this data, using the hypothesis $$h_Θ(x) = Θ_0 + Θ_1 x$$, where $$Θ_0$$ and $$Θ_1$$ represent the parameters to be learned. Considering the initial values $$Θ_0$$= 1.0, and $$Θ_1$$ = 0.0, and learning rate 0.1, what will be the values of $$Θ_0$$ and $$Θ_1$$ after the first three iterations of Gradient Descent

From least squares method I took the derivative with respect to $$Θ_0$$ and $$Θ_1$$ and plugged in the initial values to get the slope/intercept and multiplied it by the learning rate 0.1 to get the step size.The step size was used to calculate the new $$Θ_0$$ and $$Θ_1$$ values.

I am getting $$Θ_0$$ as 1.7821 when following the above. Please let me know if the approach followed and the solution correct or there is a better way to solve

## 1 Answer

X = np.array([1,2,3])
Y = np.array([2,1,2])

params = np.array([1, 0])

def loss(y, yhat):
return ((y - yhat)**2).mean()

def model(x):
return params + params*x

def loss_grad(y, yhat, x):
return np.array([(2*(yhat-y)).mean(), (2*(yhat-y)*x).mean()])

lr = .1
for _ in range(3):
yhat = model(X)
l = loss(Y, yhat)
g = loss_grad(Y, yhat, X)
params = params - lr*g
print(f'thetas are now {params} with new loss of {loss(Y, yhat)}')


outputs

thetas are now [1.13333333 0.26666667] with new loss of 0.6666666666666666
thetas are now [1.13333333 0.23111111] with new loss of 0.2696296296296296
thetas are now [1.14755556 0.22874074] with new loss of 0.262887242798354


I double checked this with keras too, but in numpy i explicitly wrote the gradients, i reccomend double checking your gradient or arithmetic