I think you are asking about Fully Input Convex Neural Networks as proposed in [1]. ReLU is in fact a convex function, and the sum of convex functions can only produce convex functions. However, unlike you said, composition of convex functions can produce non-convex functions, unless they are non-decreasing. With FICNNs you can only learn convex functions. For that all weights W must be non-negative for the activation function g

From [1] the interesting part is:

The function f is convex in y provided that all W_i(z)1:k are non-negative, and all functions g_i are convex and non-decreasing. The proof is simple and follows from the fact that non-negative sums of convex functions are also convex and that the composition of a convex and convex non-decreasing function is also convex (see e.g. Boyd & Vandenberghe (2004, 3.2.4)).

[1] Amos, Brandon, Lei Xu, and J. Zico Kolter. "Input convex neural networks." International Conference on Machine Learning. PMLR, 2017.

I think you are asking about Fully Input Convex Neural Networks as proposed in [1]. ReLU is in fact a convex function, and the sum of convex functions can only produce convex functions. However, unlike you said, composition of convex functions can produce non-convex functions, unless they are non-decreasing. With FICNNs you can only learn convex functions. For that all weights W must be non-negative for the activation function g

From [1] the interesting part is:

The function f is convex in y provided that all $W_i(z)_{1:k}$ are non-negative, and all functions $g_i$ are convex and non-decreasing. The proof is simple and follows from the fact that non-negative sums of convex functions are also convex and that the composition of a convex and convex non-decreasing function is also convex (see e.g. Boyd & Vandenberghe (2004, 3.2.4)).

[1] Amos, Brandon, Lei Xu, and J. Zico Kolter. "Input convex neural networks." International Conference on Machine Learning. PMLR, 2017.