1
$\begingroup$

I am developing an ANN from scratch which classifies MNIST digits.

These are the curves I get using only one hidden layer composed of 100 neurons activated by ReLU function. The output's neurons are activated by the softmax function:

curves

Is it correct that training and validation loss are almost identical? Does it mean that my model perfectly fit the data?

Improve this question
$\endgroup$
6
  • $\begingroup$ Usually it means you did something wrong. Double check everything. $\endgroup$
    – lpounng
    Commented Nov 3, 2022 at 10:38
  • $\begingroup$ @lpounng What If I do not spot errors? $\endgroup$
    – tail
    Commented Nov 3, 2022 at 10:42
  • $\begingroup$ If you have implemented from scratch, I would verify that your validation data is being held out during training. It can happen that the two track quite tightly (as these do) but in my experience it is somewhat unusual for them to reflect the same hills and valleys throughout. $\endgroup$
    – David Hoelzer
    Commented Nov 3, 2022 at 11:07
  • $\begingroup$ @DavidHoelzer No bugs spotted... $\endgroup$
    – tail
    Commented Nov 3, 2022 at 15:33
  • $\begingroup$ @tail try get someone else to look at it, or post more details here so we can help. $\endgroup$
    – lpounng
    Commented Nov 4, 2022 at 1:15

1 Answer 1

Reset to default
0
$\begingroup$

If your accuracy is 50% in train and val, would you say that the model fit the data perfectly?

One guess is that your val and train data are pretty similar. Have you split the data before training the model?

The loss seems quite high. What does your accuracy look like? Have you tried to train for more epochs?

Improve this answer
$\endgroup$
5
  • $\begingroup$ Accuracy on both data are around 95% at the end of the learning. Yes I have split the data. Yes I have tried to train for more epochs and the losses still seem to be identical. $\endgroup$
    – tail
    Commented Nov 3, 2022 at 11:06
  • $\begingroup$ 50% accuracy is relative. On a 10 class problem, 50% is much better than on a 2 class problem. Also, loss is entirely relative and mostly meaningless without another loss for the same problem (and same loss function) to compare it to. Here we have only the validation loss for comparison. $\endgroup$
    – David Hoelzer
    Commented Nov 3, 2022 at 11:06
  • $\begingroup$ @tail Have you tried kfold split? askpython.com/python/examples/k-fold-cross-validation $\endgroup$
    – Skobo Do
    Commented Nov 3, 2022 at 11:09
  • $\begingroup$ @DavidHoelzer that's the point. It's relative. It does highly depend on the data chosen for training and validation. I can have easily 100% accuracy on a multiclass classification with arbitrary number of classes when choosing the validation data-set accordingly. $\endgroup$
    – Skobo Do
    Commented Nov 3, 2022 at 11:13
  • $\begingroup$ @DavidHoelzer "50% accuracy is relative. On a 10 class problem, 50% is much better than on a 2 class problem." That not generally true. It depends on whether your data is balanced or not. For balanced data, it would be true. $\endgroup$
    – Skobo Do
    Commented Nov 3, 2022 at 11:16

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .