1
$\begingroup$

I was training a CNN model on TensorFlow. After a while I came back and saw this loss curve:

training loss curve(green) and validation loss curve(gray)

The green curve is training loss and the gray one is validation loss. I know that before epoch 394 the model in heavily overfitted, but I have no idea what happened after that.

Also, this is accuracy curves if it helps:

accuracy curves

I'm using categorical cross-entropy and this is the model I am using: model architecture

and here is link to PhysioNet's challenge which I am working on: https://physionet.org/content/challenge-2017/1.0.0/

Improve this question
$\endgroup$
8
  • $\begingroup$ Maybe you should provide a little bit of context, i.e. which task you are trying to solve with CNNs (i.e. which dataset), which loss function you are using, maybe you should do plot_model(your_model) and report the architecture of your model. $\endgroup$
    – nbro
    Commented Dec 14, 2020 at 10:22
  • $\begingroup$ I'm trying to classify ECG signals in PhysioNet 2017 challenge. My output is four classes and I'm using categorical cross-entropy as loss function. $\endgroup$
    – Sepehr Golestanian
    Commented Dec 14, 2020 at 10:35
  • $\begingroup$ Please, edit your post to include these details (maybe also a link to the challenge). $\endgroup$
    – nbro
    Commented Dec 14, 2020 at 10:39
  • 1
    $\begingroup$ I've edited the question. $\endgroup$
    – Sepehr Golestanian
    Commented Dec 14, 2020 at 10:59
  • $\begingroup$ Why are you using a CNN to classify ECG signals? Aren't ECG signals just numerical time series? $\endgroup$
    – nbro
    Commented Dec 14, 2020 at 11:08

2 Answers 2

Reset to default
0
$\begingroup$

Follow this answer from StackOverflow, I think your problem related to the second cases where your loss gets Nan value.

Maybe you should try to use the larger datatype (for example float 16 -> float 32)

Improve this answer
$\endgroup$
11
  • $\begingroup$ As I'm using tensorflow's built-in categorical cross-entropy, I think tf's team knows they should take good care of NaN, values. By the way, can regularization help in this situation or should I put weight constraints? $\endgroup$
    – Sepehr Golestanian
    Commented Dec 14, 2020 at 10:45
  • $\begingroup$ We have to dig deeper into TF code to get the final conclusion. For the second question, I think yes, regularization is to solve the overfitting problem, here, before epoch 394, your validation accuracy is stable so I can say your model get overfit problem $\endgroup$
    – CuCaRot
    Commented Dec 14, 2020 at 11:28
  • $\begingroup$ Thanks very much, I'll try this and update the question! $\endgroup$
    – Sepehr Golestanian
    Commented Dec 14, 2020 at 12:51
  • $\begingroup$ @SepehrGolestanian So, why did you accept this answer? What was the problem in the end? $\endgroup$
    – nbro
    Commented Dec 15, 2020 at 17:51
  • $\begingroup$ @nbro, you look like the admin of this community, I saw you carefully edit nearly all the post and ask in very details :smile: $\endgroup$
    – CuCaRot
    Commented Dec 16, 2020 at 1:35
0
$\begingroup$

You have to keep the training loss value constant. Beyond that, your teaching technique or network model is not suitable for the job. I recommend that you are using MLP.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .