When training a DNN on infinite samples, do ADAM or other popular optimization algorithms still work as intended?
I have an DNN training from an infinite stream of samples, that most likely won't repeat. So there is no real notion of "epoch".
Now I wonder if the math behind ADAM or other popular optimizers expect the repetition of data over the epochs?
If so, should I collect a limited amount of those samples and use them for training data and validation data, or would it be better to use all data available (even if the training data never repeats then)?