I am implementing an ANN whose training loss is in Figure:

training loss

As you can see training loss decreases very fast and it is approximately 3.2 at epochs 2, 3, ..., 8, ... 10, and so on. (batch learning)

The diagram is similar even though I increases MAX_EPOCHS to 2000.

Is it correct?

Note that my training set is a toy:

X = np.asmatrix([
        [2, 5, 3.2, 7.2],
        [1, 0.8, 1, 8.5],
        [2, 7, 3, 1.4]

targets = np.asarray([0,2,1])
targets = np.asmatrix(np.eye(np.max(targets) + 1)[targets])  # 1-hot

And I am using Gradient Descent with momentum

  • $\begingroup$ What is the architecture of your ANN? The dataset seems extremely small (with only 3 samples) so it is difficult to explain the behaviour of the ANN. It is possible that with the given architecture and this dataset, the loss cannot be reduced beyond 3.2. $\endgroup$ Commented Oct 3, 2022 at 21:18

1 Answer 1


You're loss is converging fast simply cause you're totally overfitting on you're toy input samples, which is good, at least it proves that your training script works.

  • $\begingroup$ Thank you. I was using the toy samples for that purposes, indeed. $\endgroup$
    – tail
    Commented Oct 4, 2022 at 10:47

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .