I am training a deep neural network. There is a constraint on an output value of the network. (e.g. Output has to be between 0 and 180) I think some possible solutions are using sigmoid,tanh activation at the end of the layer. I wonder if there are better ways to put constraints on the output value of a neural network.
There are many ways of constraining the network's output.
Using an activation layer is a good one. If you sigmoid the output layer, the output is constrained between [0,1] and you can multiply that by 180 to adapt to your output. Since the layer is part of the optimization process, gradients will be learned correctly.
However, there are a few issues with it, namely sigmoid is not a linear curve, and is quite steep around 0. Maybe you can consider something like a ReLU, but with a maximum as well. This makes your output linearly between [0,180], and is also learned during optimization.
You could also just clip the output without any activation layers. This is simple and works, but the optimization process sees no difference between outputting 180 or 180000. This can lead to problems, depending on your problem.