Calculating Parameter value Using Gradient Descent for Linear Regression Model

Question

Consider the following data with one input (x) and one output (y):
(x=1, y=2)
(x=2, y=1)
(x=3, y=2)
Apply linear regression on this data, using the hypothesis $h_Θ(x) = Θ_0 + Θ_1 x$, where $Θ_0$ and $Θ_1$ represent the parameters to be learned. Considering the initial values $Θ_0$= 1.0, and $Θ_1$ = 0.0, and learning rate 0.1, what will be the values of $Θ_0$ and $Θ_1$ after the first three iterations of Gradient Descent

From least squares method I took the derivative with respect to $Θ_0$ and $Θ_1$ and plugged in the initial values to get the slope/intercept and multiplied it by the learning rate 0.1 to get the step size.The step size was used to calculate the new $Θ_0$ and $Θ_1$ values.

I am getting $Θ_0$ as 1.7821 when following the above. Please let me know if the approach followed and the solution correct or there is a better way to solve

Answer 1

X = np.array([1,2,3])
Y = np.array([2,1,2])

params = np.array([1, 0])

def loss(y, yhat):
    return ((y - yhat)**2).mean()

def model(x):
    return params[0] + params[1]*x

def loss_grad(y, yhat, x):
    return np.array([(2*(yhat-y)).mean(), (2*(yhat-y)*x).mean()])

lr = .1
for _ in range(3):
    yhat = model(X)
    l = loss(Y, yhat)
    g = loss_grad(Y, yhat, X)
    params = params - lr*g
    print(f'thetas are now {params} with new loss of {loss(Y, yhat)}')

outputs

thetas are now [1.13333333 0.26666667] with new loss of 0.6666666666666666
thetas are now [1.13333333 0.23111111] with new loss of 0.2696296296296296
thetas are now [1.14755556 0.22874074] with new loss of 0.262887242798354

I double checked this with keras too, but in numpy i explicitly wrote the gradients, i reccomend double checking your gradient or arithmetic