0
$\begingroup$

I am training a multilabel text neural network and the model metric I chose, to measure the performance of the training and the validation sets, is the f1 score (Micro average, None average). However, I have a hard time trying to increase the value achieved by most of the model estimators. Specifically, the majority of my neural networks achieve an f1 score (micro averaged) between 0.69 - 0.71 points while a simple f1 score around 0.48 - 0.50. However, the loss of the neural network continued to decrease despite the f1 score being stuck around 0.70 units.

I would like to know if my models fall into a local minimum and this causes the performance metric of the algorithm can no longer increase. Or it's just my imagination and I simply can't get more of 0.70 units of f1 score due to other reasons, like the data, the multi labeled target, etc.

My neural network has the following specs:

  • 1 dense (hidden layer)
  • 32 - 128 batch size (hidden units/neurons)
  • Drop out: 0.1
  • Activation function of the last layer: 'sigmoid'
  • loss metric: 'binary cross-entropy'
  • model metric: [fi_score(average=micro), fi_score(average=None)]

Structure of the neural network

enter image description here

Training per epoch - As you can see the f1 score (micro) cannot increase more than 0.71 units. enter image description here

F1 score (None average) per genre tag - You can see that some of my genres cannot achieve a very high f1 score enter image description here

What can I do to tackle this? I would really like to increase the f1 score above 0.71 units but it would be helpful to know if this is feasible and the ways to achieve this.

Thank you in advance for any proposal and please let me know in the comments if you would like any additional information or you want me to share with you my colab notebook.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.