I would like to train a constrained neural network. I found a paper on this: https://papers.nips.cc/paper/4-constrained-differential-optimization.pdf. However, I don't really understand how to change my feedforward neural network. I don't understand exactly the role of the new output neurons, which serve as lagrangian multipliers. Can someone explain this to me in more detail?
Which steps in backpropagation do I have to change?