2

Information-theoretic view of Bayesian learning I once heard that the problem of approximating an unknown function can be modeled as a communication problem. How is this possible? Yes, this is indeed possible. More precisely, there is an information-theoretic view of Bayesian learning in neural networks, which can also be thought of as a communication ...


1

I think the wrong assumption here is that you've forgotten the cost of encoding the new features! MDL should be considered relative to the original or raw dataset. The idea is that you want to find an expression you could send to someone else that encodes the structure of the dataset in terms of the original variables. If you define new features, you need ...


Only top voted, non community-wiki answers of a minimum length are eligible