I am training a simple convolutional neural network to recognize two types of 1024-point frequency spectra (FFT). This is the model I'm using:

cnn = Sequential()
cnn.add(Conv1D(filters=64, kernel_size=3, activation=LeakyReLU(), input_shape=(nInput,1)))
cnn.add(Conv1D(filters=64, kernel_size=3, activation=LeakyReLU()))
cnn.add(Dense(nFinalDense, activation=LeakyReLU()))
cnn.add(Dense(nOutput, activation='sigmoid'))

However I get the following accuracy and loss during training: enter image description here

Why do I get the large peak in both plots? How can it be explained? Is there a problem with the data I'm using (I mention that I obtain a similar peak when training an autoencoder for denoising using the same data)?

  • $\begingroup$ Are you getting it multiple times assuming you have randomly initialized weights Everytime or is it a one time thing? $\endgroup$
    – user9947
    Dec 24 '19 at 13:12
  • $\begingroup$ @DuttaA I get it multiple times. I'm using Tensorflow to train the model with default weight initializers for each layer, so it automatically randomizes the weights. I mention that I set the seed values for numpy and tensorflow at the beginning of the script to get a more deterministic output. I've changed the seed value and I get a smaller peak at a different epoch. What do you think it is? $\endgroup$
    – Cristian M
    Dec 24 '19 at 13:36

I found that the peak was caused by the data I am using. Specifically, the MinMaxScaler changed the data shape and I resolved the issue by simply dividing to the max value.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.