From what I understand, there are 2 stages for deep learning: the first is training and the second is inference. The first is often done on GPUs because of their massive parallelism capabilities, among other things. The second, inference, while it can be done on GPUs, it's not used that much, because of power usage, and because the data presented while inferring are much less, so the full capabilities of GPUs won't be much needed. Instead, FPGAs and CPUs are often used for that.
My understanding also is that a complete DL system will have both, a training system and an inferring system.
My question is: are both systems required on the same application?
Let's assume an autonomous car or an application where visual and image recognition is done, will it have both a training system to be trained and an inference system to execute? Or it has only the inference system and will communicate with a distant system that is already trained and has built a database?
Also, if the application has both systems, will it have a big enough memory to store the training data? Given that it can be a small system and memory is ultimately limited.