# Are Neural Network layers resistent to noise?

Let's consider a classic feedforward neural network $$F$$ with input dimension $$d$$, output dimension $$k$$, $$L$$ layers $$l_i$$ with $$m$$ neurons each. ReLu activation.

This means that, given a point $$x \in R^d$$ its image $$F(x) \in R^k$$. Let's now assume i add some gaussian noise $$\eta_i$$ in EVERY hidden layer $$l_i(x)$$ at the same time, where the norm of this noise is 5% the norm of its layer computed on the point $$x$$. Let's call this new neural network $$F_*$$

I know that, empirically, neural networks are resistant to this kind of noise, especially on the first layers. How can i show this theoretically?

The question i'm trying to answer is the following:

After having injected this noise $$\eta_i$$ in every layer $$l_i(x)$$, how far the output $$F_{*}(x)$$ will be from the output of the original neural network $$F(x)$$?