In the famous Information bottleneck paper by Tishby(https://arxiv.org/abs/1703.00810), the author proposed a framework that the neural network can compress information. And they computed the mutual information between different layers.
Now I have a question. We all know that the neural network for each layer is a deterministic function, which means given a specific X. We can get a new Z=f(X). So, we can get H(Z|Y) is 0. Because given any X, we can compute corresponding Z from layer function.
But actually, in this article, H(Z|Y) not equals 0, and we can compute H(Z|Y), or I(X, Z). Seems that we omit the deterministic relationship. Why?