ReLU stands for Rectified Linear Unit. Linear Unit, I understand, since the function is piecewise linear. But what does rectified mean?
I looked up the definition and it said:
denoting an electric current that has been converted from alternating current to direct current
But I don't quite see how this can connect to ReLU function. While it does have a bend at $x = 0$ indicating a change in direction, not sure how I can work this into the idea of current directions.