0
$\begingroup$

I am implementing an ANN whose training loss is in Figure:

training loss

As you can see training loss decreases very fast and it is approximately 3.2 at epochs 2, 3, ..., 8, ... 10, and so on. (batch learning)

The diagram is similar even though I increases MAX_EPOCHS to 2000.

Is it correct?

Note that my training set is a toy:

X = np.asmatrix([
        [2, 5, 3.2, 7.2],
        [1, 0.8, 1, 8.5],
        [2, 7, 3, 1.4]
    ])

targets = np.asarray([0,2,1])
targets = np.asmatrix(np.eye(np.max(targets) + 1)[targets])  # 1-hot

And I am using Gradient Descent with momentum

Improve this question
$\endgroup$
1
  • $\begingroup$ What is the architecture of your ANN? The dataset seems extremely small (with only 3 samples) so it is difficult to explain the behaviour of the ANN. It is possible that with the given architecture and this dataset, the loss cannot be reduced beyond 3.2. $\endgroup$
    – Ganesh Tata
    Oct 3, 2022 at 21:18

1 Answer 1

Reset to default
1
$\begingroup$

You're loss is converging fast simply cause you're totally overfitting on you're toy input samples, which is good, at least it proves that your training script works.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thank you. I was using the toy samples for that purposes, indeed. $\endgroup$
    – tail
    Oct 4, 2022 at 10:47

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .