I want to implement a single perceptron for linear regression using the following formulas:


The input data for the first case is one column (x(392, 1); y(392, 1)) and for the second case is (x(392, 7); y(392, 1)). The NaN values have been removed and x values have been standardized x-x.mean()/x.std()

This is my Python implementation:

class LinearRegression(object):

    def __init__(self, x, y, n_iter):
        self.x = x
        self.y = y
        self.n_iter = n_iter
        self.cost_iteration = []
        # Initializing model parameters (w, b) to zeros
        self.weights = np.zeros((1, self.x.shape[1]))   # w: weights
        self.biases  = np.zeros((1, 1))                 # b: bias 

    def feedforward(self):
        # return the feedforward value for x
        #self.weights, self.biases = self.update_params()
        z = self.x @ self.weights.T  + self.biases
        return z
    def loss(self):
      # return the loss value for given x and y
      z = self.feedforward() 
      loss = self.y-z
      cost = np.sum(loss**2)/self.y.shape[0]
      return loss, cost 

    def backpropagation(self):
      # return the derivatives with respect to weight matrix and biases
      loss, cost  = self.loss()
      db = -2*np.sum(loss)/self.y.shape[0]                    # dJ/db
      dw = -2*np.dot(self.x.T, loss)/self.y.shape[0]          # dJ/dw
      return dw, db   
    def update_params(self):
      # update weights and biases based on the output
      dw, db = self.backpropagation()
      self.weights -= dw.T
      self.biases  -= db
      return self.weights, self.biases
    def fit(self):
      # fit method for the training data
      for iter in range(self.n_iter):
        l, c = self.loss()
        self.cost_iteration.append (c)
      return self.cost_iteration

The final cost should be approximately 23.9 and 11.6 for the two models, respectively. But I can't figure out why it's not the case when I use my code.

  • $\begingroup$ Hello. Could you please put your specific question in the title? $\endgroup$
    – nbro
    Sep 28, 2021 at 12:09
  • $\begingroup$ I am not sure how to formulate the question correctly. But I think the problem is my implementation of the loss and update methods. I think they are wrong based on the resulting cost. $\endgroup$
    – Rim Sleimi
    Sep 28, 2021 at 12:32
  • $\begingroup$ Ok, I tried to give a more descriptive title to your post. Please, make sure that it's consistent with what you were asking. How do you know that the cost should approximately be 23.9 and 11.6? $\endgroup$
    – nbro
    Sep 28, 2021 at 13:59
  • $\begingroup$ It is said at the end of the document (from which I took the screenshot above). $\endgroup$
    – Rim Sleimi
    Sep 28, 2021 at 17:50


You must log in to answer this question.

Browse other questions tagged .