2
$\begingroup$

I have run into a strange behavior of my multi label classification ANN

model = Sequential()
model.add(Dense(6, input_shape=(input_size,), activation='elu'))
#model.add(BatchNormalization(axis=-1))
model.add(Dropout(0.2))
#model.add(BatchNormalization(axis=-1))
model.add(Dense(6, activation='elu'))
model.add(Dropout(0.2))
#model.add(BatchNormalization(axis=-1))
model.add(Dense(6, activation='elu'))
model.add(Dropout(0.2))

# model.add(keras.layers.BatchNormalization(axis=-1))
model.add(Dense(6, activation='sigmoid'))
model.compile(loss='binary_crossentropy',
              optimizer='nadam',
              metrics=['accuracy'])
history = model.fit(X_train, Y_train,batch_size=64 ,epochs=300,
                    validation_data = (X_test, Y_test), verbose=2)

The result is quite strange, I have a feeling that my model could not improve any more. Why does the loss and the accuracy does not change overtime ?

P/S For clarification, I have 6 output and the value of each output is 0 or 1 that is output 1: can be 0 or 1

output 2: can be 0 or 1

output 3: can be 0 or 1

output 4: can be 0 or 1

output 5: can be 0 or 1

output 6: can be 0 or 1

enter image description here

$\endgroup$
4
  • $\begingroup$ Hi ! How many samples do you have in X_train ? Can you try to visualize the outputs on your test set (for instance, are your predictions constants ?) ? Maybe one problem is that you put a lot of Dropout, and your network is already narrow so instead of preventing overfitting this could hinder the learning... $\endgroup$
    – 16Aghnar
    Aug 18 '20 at 8:28
  • $\begingroup$ about 4000 training samples, and validate 1000 samples.The output for each test set data point is a binary output vector that is [out1 out2 out3 out4 out5 out6] Particularly output 1: can be 0 or 1 output 2: can be 0 or 1 output 3: can be 0 or 1 output 4: can be 0 or 1 output 5: can be 0 or 1 output 6: can be 0 or 1 $\endgroup$
    – User2741
    Aug 18 '20 at 10:40
  • $\begingroup$ Okay. Why did you use Dropout and Batch norm ? It may be useful to visualize the actual predictions of your trained net. You know, using for instance plt.scatter(Y_test, model.predict(X_test)) $\endgroup$
    – 16Aghnar
    Aug 18 '20 at 11:30
  • $\begingroup$ (As you have multi label classification, you may plot plt.scatter(Y_test, model.predict(X_test)) for each particular label) $\endgroup$
    – 16Aghnar
    Aug 18 '20 at 11:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.