ReLU is non-linear by definition
In calculus and related areas, a linear function is a function whose graph is a straight line, that is a polynomial function of degree one or zero.
Since the graph of the ReLU function $f(x) = \max(0,x)$ is not a straight line (equivalently, it cannot be expressed in the form $f(x) = mx + c$), by definition it is not ...
Pooling has multiple benefits
Robust feature detection.
Makes it computationally feasible to have deeper CNNs
Robust Feature Detection
Think of max-pooling (most popular) for understanding this.
Consider a 2*2 box/unit in one layer which is mapped to only 1 box/unit in the next layer (Basically pooling). if the feature map (kernel) let's say detects a ...
I don't think there's a way of doing what you want, at least, I've never seen such a thing (and, currently, I am not seeing how this could be done in the general case).
The same neural network model but with different (or same) weights could have been trained with the same loss function or not. For example, although it may not be a good idea, you can train ...
Your solution is pretty much on spot. It corresponds to the YUV scheme used in television and designed to match human perception characteristics. As you already noticed, such an encoding wouldn't suffer from discontinuities.
From what I understand, don't bother with a CNN, you have essentially perfectly structured images.
You can hand code detectors to measure how much filled in a circle is.
Basically do template alignment and then search over the circles.
Ex a simple detector would measure the average blackness of the circle which you could then threshold.