(Disclaimer: I don't know much about ML/AI, besides some basic ideas behind it all.)
It seems like ML/AI models can often be boiled down to statistics, where certain levers (weights) get fine-tuned based on the specific input of a large set of training data.
Clearly, ML/AI models don't only distinguish themselves in their training data alone, otherwise there would not be so many improvements happening in the field all the time. My question therefore is: What does distinguish different models of the same category?
If I have an AI that completes real-life pictures that have some missing parts, and an AI that completes a painting with missing parts, what key concepts separates the two?
If I have an AI detecting text in an image, and an AI detecting... trees in an image, what key concepts separates the two?
In other words, what is stopping me from "taking" an existing implementation of a certain AI category, and just feeding it my specific training set + rewards (i.e. judgement criteria for good vs bad output), in order to solve a specific task?
In yet again other words, if I wanted to use ML/AI to build a new model for a specific task, what concepts and topics would I need to pay extra attention to? (I guess you could say I'm trying to reverse engineer the learning process of the field here. I don't have the time to properly teach myself and become an "expert", but find it all very interesting and would still like to use some of the wonderful things people have done.)