Drone training, how to train without training data?

I setupped a small drone simulator using PhysX, the time step is at 200 hz, while motors update like regular ESCs (at 50 Hz). I computed the inertia matrix, tweaked a bit mass of components to be real, air drag, gravity etc. After a first partial success in tuning a PID algorithm I got bored to find and hunt perfect values, and started thinking to tune it with a NN, but then I thinked, why using PID at all if NN can find better solutions?.

I started playing with NN, but then I realized I have no traning data.

Sure I could do

NN.Train(input,expectedOutput);


but what is actually the expected output? To be useful it has to be the thrust force of propellers tuned to keep input (position and orientation) stable in the desired place.

But actually I don't (Want to) know in advance the thrust that every single propeller has.

Since it is a simulation it is ok spending some computing time between each simulation step (eventually I'll try to optimize it later to fit it in a microcontroller)..

So, is it possible given a regular NN implementation where I can select number of:

• Input neurons
• Hidden neurons
• Output neurons

find a way to train my model live? I need a way to tell the NN

hei this time you performed X keep going or go the opposite way.