For real applications, concept drifts often exist, i.e., the relationship between the input and output changes overtime. Thus, we need our AI or machine learning system to quickly adapt to the environment.
What are the most common methods to enable neural networks to quickly adapt to the changing environment for supervised learning? Could somebody provide a link to a good review article?
For the vast majority of cases where you have a dynamic(and assumed non-linear) relationship between your input and output, you would not use modified architecture. You would simply retrain on the new data.
In some cases, based on domain knowledge or intuition, one might put a "weight" on the new data to increase or decrease its importance relative to previous data.
There are some attempts(mostly by those studying one-shot learning) to create NNs that quickly fit to new data effectively with only a few samples. However, most of these are not ready for anything resembling real-world problems(particularly on tabular data).
