(Sorry, I'm not very experienced with ML and I apologize if these questions are vague, naive, or not written well. I also moved this from CV to here.) I think I understand that a MLP learns (via forward pass and backpropagation) by pulling each parameter in the opposite direction as its gradient of the loss for each training image. However, given that MLPs don't capture spatial relationships, I think I find it puzzling, without considering the existing empirical evidence, to understand why MLPs are able to learn image tasks like classification. I'm wondering, is there possibly a solid statistical/analytic reason we expect MLPs can learn tasks without primarily basing intuition on existing empirical evidence? Is there a tangible way to understand how iteratively updating the parameters to minimize the loss can translate to learning patterns such as learning digit "templates" with positive/negative regions to classify MNIST digits?

I'm also curious about what MLPs can and cannot do. Although not capturing spatial relationships seems to be a big limitation of MLPs, the universal approximation theorem and them being fully connected seem to give me some (subjective) sense that training large MLPs for advanced tasks might also be possible. I'm curious if there exists a database, kind of like this list of NN performances on MNIST, for the most advanced tasks people have gotten MLPs to learn? For example, has anyone ever successfully trained a cat or dog recognizer with a MLP? Or have MLPs really only worked for tasks that linear models have worked equally well for? Thank you

(For anyone interested I found this)



You must log in to answer this question.

Browse other questions tagged .