Can a neural network learn to avoid wrong decisions using backpropagation?

Question

I studied the articles on Neural Networks and Deep Learning from Michael Nielsen and developed a simple neural network based on his examples. I understand how backpropagation works and I already taught my neural network to not only play TicTacToe but also improve his own play by learning from his own successes using backpropagation.

Going forward with my experiments, I am facing the problem, that I won't always be able to show the network good moves to use for learning (maybe because I simply don't know what is correct in a certain situation), but I might be required to show it bad moves to avoid (because some of the bad moves are obvious). Teaching the network what to do using backpropagation is easy, but I haven't found a way to teach it what to avoid using similar techniques.

Is it possible to teach simple neural networks using negative examples like this or do I need other techniques? My gut feeling says, that it might be possible to "invert" gradient descent into gradient ascent to solve this problem. Or is it more complicated than this?

Answer 1

What you are describing is conceptually close to adversarial training. you should read more on adversarial examples and generative adversarial networks for more information.

The idea is that there is a discriminator network, whose job is to correctly discriminate between positive and negative examples. We also have a generative network, that learns to produce "adversarial examples" that "confuses" the discriminator network. By training these two networks side by side, both networks get better at their task. But it's usually the generator network that people are more interested in.

Intuitively, the naive implementation of the method you've described (gradient ascent on incorrect examples from a network in a clean/randomly-initialized state) shouldn't work. This is because negative examples don't form a "natural class" (all triangles have 3 edges, all things that are not triangles however....)