# Will adding memory to a supervised learning system makes it into a Bayesian learning system?

Seung et.al recently published GameGAN paper, GameGAN learned and stored the whole Pacman game and was able to reproduce it without a game engine. The uniqueness of GameGAN is that it had added memory to its discriminator/generator which helped it to store the game states. In Bayesian interpretation, a supervised learning system learns by optimizing weights which maximizes a likelihood function.

$$\hat{\boldsymbol{\theta}} = \mathop{\mathrm{argmax}} _ {\boldsymbol{\theta}} P(X \mid \boldsymbol{\theta})$$

Will adding memory which can store prior information makes GameGAN a Bayesian learning system?

Can GameGAN or similar neural network with memory can be considered as a bayesian learning system. If yes, then which of these two equations(or something other) correctly explains this system (considering prior as memory)?

1. $$\mathop{\mathrm{argmax}} \frac{P(X \mid \boldsymbol{\theta})P(\boldsymbol{\theta})}{P(X)}$$

or

1. $$\mathop{\mathrm{argmax}} \frac{P(X_t \mid \boldsymbol{X^{t+1}})P(\boldsymbol{X^{t+1}})}{P(X_t)}$$

PS: I understand GAN's are unsupervised learning systems, but we can assume discriminator and generator models separately trying to find weights that maximize their individual likelihood function.

• If you're maximizing the likelihood (maximum likelihood estimation), that doesn't make the system Bayesian, in the sense that you just get a point estimate rather than a distribution and you are not specifying any prior. In Bayesian learning, you need to be able to specify a prior on the parameters and you often get a posterior distribution over them. So, your sentence "In Bayesian interpretation, a supervised learning system learns by optimizing weights which maximizes a likelihood function" is wrong, in the sense that's just a probabilistic interpretation (rather than a Bayesian one).
– nbro
Jun 13 '20 at 13:52
• You might also be interested in this post.
– nbro
Jan 17 at 12:17