I'm trying to train a neural network to do a multiple non-linear regression $y=f(x_i), i=1,2…N$. So far it works good (low MSE), but some predictions $y$ are “non-physical”, for instance for our application it is known from first principles that when $x_2$ increases, then $y$ also has to increase ($dy/dx_2>0$), but in some instances the neural network’s output doesn’t comply with this constraint. Another example is that $y + x_5 + x_7$ should be less than a constant $K$
I thought about adding a penalty term to the loss function to enforce these constraints, but I am wondering if there is a "harder" way to impose such a constraint (that is, to ensure that these constraints will always hold, no only that non-physical predictions will be penalized)