I am reading thesis https://tel.archives-ouvertes.fr/tel-00850289v2 about use of mean field theory for the stochastic approximation of neural networks. There are lot of such research in arxiv cond-nn section, in condensed matter physics. Such research reveals that emergence of distinct behavior is possible in approxiamted neural networks. But what about full, non-approximated neural networs as researched in computer science? Does such non-approximated networks have some emergence? Sometimes I am thinking that attention mechanisms are something how emergence is manifested in non-approximated neural networks, is it so? Are there other kinds of emergence? Does neural machine translation can be considered as the emergent structure? Or maybe emergence has been detected only in the approximated neural networks as researched in condensed matter physics/complexity theory?