I think the answer to your question is much more a rule of thumb than an appropriate analytical answer. First of all, I would like to remark that Batch Normalization [1] are applied most commonly to convolutional layer, constituting what is called a "convolutional block" (Convolution + Batch Normalization + Activation). Thus, for giving you an idea on where to put the normalization layers, I will analyze three papers that make use of Batch Normalization.
Unsupervised representation learning with deep convolutional generative adversarial networks [2]. In Section 3, Approach and Model Architecture, the authors make the following remark:
Third is Batch Normalization [1] which stabilizes learning by normalizing the
input to each unit to have zero mean and unit variance. This helps deal with training problems that
arise due to poor initialization and helps gradient flow in deeper models. This proved critical to get
deep generators to begin learning, preventing the generator from collapsing all samples to a single
point which is a common failure mode observed in GANs. Directly applying batchnorm to all layers
however, resulted in sample oscillation and model instability. This was avoided by not applying
batchnorm to the generator output layer and the discriminator input layer.
thus, the authors have experimented with Batch Normalization on all layers, but this approach resulted in model instability. They ultimately avoided Batch Normalization on the Generator's output layer, and Discriminator's input layer. These correspond to "raw image layers" (the output of the generator is an image, as well as the input of the discriminator). Using your notation, it would be something like:
$$G: z \rightarrow (C + BN + A)_{1} \rightarrow (C + A)_{2} \rightarrow O$$
$$D: I \rightarrow (C + A)_{1} \rightarrow (C + BN + A)_{2} \rightarrow h_{1} \rightarrow h_{2} \rightarrow O$$
where $G$ stands for Generator, $D$ for Discriminator, $C$ for convolution, $BN$ for batchnorm, $A$ for activation and $D_{i}$ for dense layers.
Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising [3]. In Figure 1., the authors show their proposed architecture that applies the same logic. Neither the first convolutional layer, neither the last use Batch Normalization. Although the authors make their point in justifying the usage of residual learning + Batch Normalization, the paper does not justify this architectural choice. The network would look something like this:
$$I \rightarrow (C + A)_{1} \rightarrow (C + BN + A)_{2} \rightarrow \cdots \rightarrow (C + BN + A)_{n} \rightarrow C \rightarrow O$$
Deep Residual Learning for Image Recognition [4]. This paper proposes a different scheme, as it applies Batch Normalization after each convolution operation on convolutional layers (thus including the input).
To sum up some papers use it only on the "hidden convolutional blocks", others on all of them. My advice is that, if you have the time, you should compare the two approaches. Remark: maybe I am unaware of some further development on the matter that gives a precise argument in favor of whether of these approaches.
References
[1] Ioffe, S., & Szegedy, C. (2015, June). Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International conference on machine learning (pp. 448-456). PMLR.
[2] Radford, A., Metz, L., & Chintala, S. (2015). Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:1511.06434.
[3] Zhang, K., Zuo, W., Chen, Y., Meng, D., & Zhang, L. (2017). Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising. IEEE transactions on image processing, 26(7), 3142-3155.
[4] He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770-778).
