Skip to main content
Artificial Intelligence

Return to Question

added 12 characters in body; edited tags; edited title
Source Link
nbro
nbro
  • 41.4k
  • 12
  • 114
  • 205

Neural Why would my neural network have either at 90%+an accuracy of 90% or stuck at 10% based on the validation data, given a random initialization?

I'm making a custom neural network framework, in (in C++, if that is of any help). When I train the model on MNIST, depending on how happy the network is feeling, it'll give me either 90%+ accuracy, or get stuck at 10-9% (on validation set). 

I shuffle all my data before feeding it to the neural net. 

Is there a better randomizer I should be using, or maybe I am not initializing my weights properly (Using srandsrand to generate values between +/-0.1). Did I somehow hit a saddle point?

My network consists of 784 size input layer, 256, 64, 32, 16 neuron hidden layers, all with RELU, and 10 output with SMAX

Where should I start investigating based on this kind of behavior, when I can't even replicate what is going on?

Neural network either at 90%+ accuracy or stuck at 10% based on random initialization?

I'm making a custom neural network framework, in C++ if that is of any help. When I train the model on MNIST, depending on how happy the network is feeling, it'll give me either 90%+ accuracy, or get stuck at 10-9% (on validation set). I shuffle all my data before feeding it to the neural net. Is there a better randomizer I should be using, or maybe I am not initializing my weights properly (Using srand to generate values between +/-0.1). Did I somehow hit a saddle point?

My network consists of 784 size input layer, 256, 64, 32, 16 neuron hidden layers, all with RELU, and 10 output with SMAX

Where should I start investigating based on this kind of behavior, when I can't even replicate what is going on?

Why would my neural network have either an accuracy of 90% or 10% on the validation data, given a random initialization?

I'm making a custom neural network framework (in C++, if that is of any help). When I train the model on MNIST, depending on how happy the network is feeling, it'll give me either 90%+ accuracy, or get stuck at 10-9% (on validation set). 

I shuffle all my data before feeding it to the neural net. 

Is there a better randomizer I should be using, or maybe I am not initializing my weights properly (Using srand to generate values between +/-0.1). Did I somehow hit a saddle point?

My network consists of 784 size input layer, 256, 64, 32, 16 neuron hidden layers, all with RELU, and 10 output with SMAX

Where should I start investigating based on this kind of behavior, when I can't even replicate what is going on?

Source Link
Ilknur Mustafa
Ilknur Mustafa
  • 115
  • 4

Neural network either at 90%+ accuracy or stuck at 10% based on random initialization?

I'm making a custom neural network framework, in C++ if that is of any help. When I train the model on MNIST, depending on how happy the network is feeling, it'll give me either 90%+ accuracy, or get stuck at 10-9% (on validation set). I shuffle all my data before feeding it to the neural net. Is there a better randomizer I should be using, or maybe I am not initializing my weights properly (Using srand to generate values between +/-0.1). Did I somehow hit a saddle point?

My network consists of 784 size input layer, 256, 64, 32, 16 neuron hidden layers, all with RELU, and 10 output with SMAX

Where should I start investigating based on this kind of behavior, when I can't even replicate what is going on?