Imposing physical constraints (previous knowledge) in a neural network for regression

Question

I'm trying to train a neural network to do a multiple non-linear regression $y=f(x_i), i=1,2…N$. So far it works good (low MSE), but some predictions $y$ are “non-physical”, for instance for our application it is known from first principles that when $x_2$ increases, then $y$ also has to increase ($dy/dx_2>0$), but in some instances the neural network’s output doesn’t comply with this constraint. Another example is that $y + x_5 + x_7$ should be less than a constant $K$

I thought about adding a penalty term to the loss function to enforce these constraints, but I am wondering if there is a "harder" way to impose such a constraint (that is, to ensure that these constraints will always hold, no only that non-physical predictions will be penalized)

Answer 1

Some ideas out the top of my head:

1. In the case of $dy/dx_2>0$ you could compute the gradient using the chain rule and limit the weights so that the constrain holds

2. In the case of $y + x_5 + x_7 < K$ you could use a clipping function on the output layer?