I have been trying to write code to implement plain neural net without convolution from scratch. I took some help online here and added my code to my github account.

I don't understand why the prediction made by my code is only 88%-90% accurate after the 1st epoch, whereas his code is 95% accurate after 1st epoch with the same parameters (Same Xavier initialization for weights, biases are not initialized, same hidden layer neurons). While his architecture uses 2 hidden layers, my code performed worse with 2 hidden layers. For 1 hidden layer, his code performs similar (~96%).

  • $\begingroup$ Different starting weights or ordering of the training data might make a difference in the first epoch. Do the two programs perform similarly after they've been given a chance to converge? And do you see similar differences in performance when you rerun your code with a different random seed? $\endgroup$
    – Ray
    May 3 '20 at 19:45
  • $\begingroup$ No my program only has around 90-92% accuracy even after 30 epochs (with 200 neurons in hidden layer), and performs similarly on different seeds. It only reaches 93% on first epoch when i increase hidden layer neurons to 1000. $\endgroup$
    – shekhar
    May 3 '20 at 22:05
  • 1
    $\begingroup$ Ok I found the mistake. The weights and biases of the input to hidden layer were not getting updated. $\endgroup$
    – shekhar
    May 4 '20 at 8:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.