# Is it possible to apply the associative property of the convolution operation when it is followed by a non-linearity?

The associative property of multidimensional discrete convolution says that:

$$Y=(x \circledast h_1) \circledast h_2=x\circledast(h_1\circledast h_2)$$

where $$h_1$$ and $$h_2$$ are the filters and $$x$$ is the input.

I was able to do exploit this property in Keras with Conv2D: first, I convolve $$h_1$$ and $$h_2$$, then I convolve the result with $$x$$ (i.e. the rightmost part of the equation above).

Up to this point, I don't have any problem, and I also understand that convolution is linear.

The problem is when two Conv2D layers have a non-linear activation function after the convolution. For example, consider the following two operations

$$Y_1=\text{ReLU}(x \circledast h_1)$$ $$Y_2=\text{ReLU}(Y_1\circledast h_2)$$

It is possible to apply the associative property if the first or both layers have a non-linear activation function (in the case above ReLU, but it could be any activation function)? I don't think so. Any idea or related paper or some kind of approach?

Also, $$ReLU(x)$$ is a linear mapping when $$x \geq 0$$, so if your input, weight, and biases are $$\geq 0$$, the net can be exactly modeled with a single layer network.