I am novice in machine learning and try to implement Linear regression algorithm using Gradient Descent. I am using Google Co-lab to implement the algorithm. First step: Implementing Y=mX + C where I have to find out m(slope). I am using Kaggle dataset for predicting No. of rooms available depending upon price of house.

My code block is

import numpy as np
import pandas as pd
import io
import matplotlib.pyplot as plt
from google.colab import files 
uploaded = files.upload()
df = pd.read_csv(io.BytesIO(uploaded['datasets_46927_85203_data.csv'])) 

# Mean of house_price and No_of_bedrooms
mean_x= np.mean(house_price)

#Total number of datapoints
numerator = 0;
denominator = 0;
for i in range (n):
   numerator += (house_price[i] - mean_x) * (No_of_bedrooms[i] - mean_y)
   denominator += (house_price[i] - mean_x) ** 2
slope= numerator / denominator
constant = mean_y - (slope*mean_x)
print(slope, constant) 

I got the Gradient decent implementation from this link

Gradient Descent Iteration #1 Let’s start with values of 0.0 for both coefficients.

Note: As per my code slope and constant are B1 and B0. However, I did not understand as per the tutorial from where it got B1=0, B0=0 The tutorial already provides a dataset

x   y
1   1
2   3
4   3
3   2
5   5

If we follow this dataset then slope and constant should be something else.

B0 = 0.0

B1 = 0.0

y = 0.0 + 0.0 * x

We can calculate the error for a prediction as follows:

error = p(i) – y(i)

Where p(i) is the prediction for the i’th instance in our dataset and y(i) is the i’th output variable for the instance in the dataset.

We can now calculate he predicted value for y using our starting point coefficients for the first training instance:

x=1, y=1

p(i) = 0.0 + 0.0 * 1

This portion is fine for me. They simply use y(i)= m+ x(i)+ c to predict the value of y(i)

p(i) = 0

Using the predicted output, we can calculate our error:

error = 0 – 1

error = -1

Here is my next doubt: to find out the error we have to subtract the predicted value from the original value. As per the table original value of y=1 but from where they get predicted value 0?

We can now use this error in our equation for gradient descent to update the weights. We will start with updating the intercept first, because it is easier.

We can say that B0 is accountable for all of the error. This is to say that updating the weight will use just the error as the gradient. We can calculate the update for the B0 coefficient as follows:

B0(t+1) = B0(t) – alpha * error

Where B0(t+1) is the updated version of the coefficient we will use on the next training instance, B0(t) is the current value for B0, alpha is our learning rate and error is the error we calculate for the training instance. Let’s use a small learning rate of 0.01 and plug the values into the equation to work out what the new and slightly optimized value of B0 will be:

B0(t+1) = 0.0 – 0.01 * -1.0

B0(t+1) = 0.01

Now, let’s look at updating the value for B1. We use the same equation with one small change. The error is filtered by the input that caused it. We can update B1 using the equation:

B1(t+1) = B1(t) – alpha * error * x

Where B1(t+1) is the update coefficient, B1(t) is the current version of the coefficient, alpha is the same learning rate described above, error is the same error calculated above and x is the input value.

We can plug in our numbers into the equation and calculate the updated value for B1:

B1(t+1) = 0.0 – 0.01 * -1 * 1

B1(t+1) = 0.01

We have just finished the first iteration of gradient descent and we have updated our weights to be B0=0.01 and B1=0.01. This process must be repeated for the remaining 4 instances from our dataset.

One pass through the training dataset is called an epoch.

I am confused about the learning rate(alpha) as well. From where I get the value of alpha? They are using 0.01 as learning rate. Can I use it as my example?

I am also went through the link Calculating Parameter value Using Gradient Descent for Linear Regression Model.

I also watched this video

Now I am exhausted thinking of what do to do and what not to do?

Any stepwise suggestion to solve the linear regression using Gradient decent will be helpful for me.

Thank you in advance.

  • 1
    $\begingroup$ This post is a bit too long. If you have multiple questions, ask each of them in their separate post, even though they are related. Also, please, put your question in the title. $\endgroup$ – nbro Sep 12 '20 at 10:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.