CNN multi output scores and evaluation

I am building a CNN with two outputs. I still have to put time in the network itself, but I was trying to get a good evaluation/classification report of the results. My code is the following:

scores = model.evaluate(data_test, [Y1_test, Y2_test], verbose=0)

for i, j in zip(model.metrics_names, scores):
print(i,'=', j)


Output:

loss = 5.124477842579717
Y1_output_loss = 1.3782909
Y2_output_loss = 4.10769
Y1_output_accuracy = 0.6304348
Y2_output_accuracy = 0.54347825


Not great, but that is not the point. My code for the classification repot is as follows:

Y1_pred, Y2_pred = model.predict(data_test)
Y1_true, Y2_true = Y1_test.argmax(axis=-1), Y2_test.argmax(axis=-1)
Y1_pred, Y2_pred = Y1_pred.argmax(axis=-1), Y2_pred.argmax(axis=-1)

print(classification_report(Y1_true, Y1_pred))
print(classification_report(Y2_true, Y2_pred))


Output:

Classification report Y1
precision    recall  f1-score   support

0       0.20      0.33      0.25         6
3       0.00      0.00      0.00         3
6       0.00      0.00      0.00         6
8       0.00      0.00      0.00         2
9       0.00      0.00      0.00         7
10       0.03      0.50      0.06         2
11       0.00      0.00      0.00         3
12       0.00      0.00      0.00         7
13       0.00      0.00      0.00         2
14       0.00      0.00      0.00         7
15       0.00      0.00      0.00         1

accuracy                           0.07        46
macro avg       0.02      0.08      0.03        46
weighted avg       0.03      0.07      0.04        46

Classification report Y2
precision    recall  f1-score   support

0       0.00      0.00      0.00         9
2       0.00      0.00      0.00        10
3       0.15      1.00      0.26         7
4       0.00      0.00      0.00         9
5       0.00      0.00      0.00         6
6       0.00      0.00      0.00         2
7       0.00      0.00      0.00         3

accuracy                           0.15        46
macro avg       0.02      0.14      0.04        46
weighted avg       0.02      0.15      0.04        46


Now the average accuracy is extremely low suddenly, so I have the feeling it isn't lining up correctly. But I don't see where?

Thank you all

• It looks like your model was trained to reproduce output vector of input and we have accuracy only a bit higher than random guesses. The chance that an index of a maximum element of prediction and from a sample will be the same will depend mainly on a size of your vector in this case. Other than that, I think that it's worth trying MLP with SoftMax as an output on top of CNN you have. – Stepan Novikov Nov 1 '19 at 13:41