# Imposing physical constraints (previous knowledge) in a neural network for regression

I'm trying to train a neural network to do a multiple non-linear regression $$y=f(x_i), i=1,2…N$$. So far it works good (low MSE), but some predictions $$y$$ are “non-physical”, for instance for our application it is known from first principles that when $$x_2$$ increases, then $$y$$ also has to increase ($$dy/dx_2>0$$), but in some instances the neural network’s output doesn’t comply with this constraint. Another example is that $$y + x_5 + x_7$$ should be less than a constant $$K$$

I thought about adding a penalty term to the loss function to enforce these constraints, but I am wondering if there is a "harder" way to impose such a constraint (that is, to ensure that these constraints will always hold, no only that non-physical predictions will be penalized)

• About the second one, is it x5+x7 <K or f(x5)+f(x7)<K? The former is not a constraint on the output but and error in the dataset. You should be able to weed them out without machine learning. – serali Nov 14 '19 at 11:05
• Hi serali, I had a typo in that equation, I meant the constaint y + x_5 + x_7 < K – Ken Grimes Nov 14 '19 at 11:15

1. In the case of $$dy/dx_2>0$$ you could compute the gradient using the chain rule and limit the weights so that the constrain holds
2. In the case of $$y + x_5 + x_7 < K$$ you could use a clipping function on the output layer?