2
$\begingroup$

I'm trying to train a neural network to do a multiple non-linear regression $y=f(x_i), i=1,2…N$. So far it works good (low MSE), but some predictions $y$ are “non-physical”, for instance for our application it is known from first principles that when $x_2$ increases, then $y$ also has to increase ($dy/dx_2>0$), but in some instances the neural network’s output doesn’t comply with this constraint. Another example is that $y + x_5 + x_7$ should be less than a constant $K$

I thought about adding a penalty term to the loss function to enforce these constraints, but I am wondering if there is a "harder" way to impose such a constraint (that is, to ensure that these constraints will always hold, no only that non-physical predictions will be penalized)

| improve this question | |
$\endgroup$
  • $\begingroup$ About the second one, is it x5+x7 <K or f(x5)+f(x7)<K? The former is not a constraint on the output but and error in the dataset. You should be able to weed them out without machine learning. $\endgroup$ – serali Nov 14 '19 at 11:05
  • $\begingroup$ Hi serali, I had a typo in that equation, I meant the constaint y + x_5 + x_7 < K $\endgroup$ – Ken Grimes Nov 14 '19 at 11:15
1
$\begingroup$

Some ideas out the top of my head:

  1. In the case of $dy/dx_2>0$ you could compute the gradient using the chain rule and limit the weights so that the constrain holds

  2. In the case of $y + x_5 + x_7 < K$ you could use a clipping function on the output layer?

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.