0
$\begingroup$

I often heard people saying, "large set of training data is needed for producing an accurate AI".

But when I looked for articles explaining backpropagations online, it all seems like you could get the job done by "one single set of input", as long as you repeat the process enough times.

So what's the "large set of training data" for!?

After the optimized set of weights was calculated from the first input, plug in the second input and repeat the process again?

Won't that "screw up" the result from the first input since it was "tailored" from it?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
3
$\begingroup$

You are not training the model to give the optimum result for one input; You want the model to produce the minimum loss for the whole population of samples that model may be given. The inputs are only considered samples from that population. The more samples you have, if they are drawn i.i.d., the better they can represent that population.

That is, you do not want a cat detector that is good at identifying one cat; You want a cat detector that can identify all cats.

Also, data is often noisy. So, just relying on one sample won't lead you to the exact local minimum for the loss even for that particular sample. That is, if you collect that exact same sample again, the noise from the sensors may be different. So, the negative gradient of one sample may not point exactly towards the local minimum for the loss. By combining several samples, you can "average out" the errors and get closer to the actual minimum.

Moreover, even if the gradient was pointing in the correct direction, you can still overshoot. Again, combining several different samples can help you minimize loss..

I think, before reading about backpropagation, you need to understand the underlying theory. So, look up gradient descent and stochastic gradient descent. Preferably, you should have some understanding of calculus and statistics before you do that.

Improve this answer
$\endgroup$
2
  • $\begingroup$ So, if I understand it right, what you mean is to use "the variants of all errors" instead of "error of a single input"? $\endgroup$
    – Noob002
    Nov 17, 2021 at 16:46
  • $\begingroup$ @Noob002 that's one way to think about it. Another is that the optimum loss for "cats seen from the side"-classifier is probably not the same as for "cats seen from any angle"-classifier $\endgroup$
    – Avatrin
    Nov 17, 2021 at 17:58

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .