I setupped a small drone simulator using PhysX, the time step is at 200 hz, while motors update like regular ESCs (at 50 Hz). I computed the inertia matrix, tweaked a bit mass of components to be real, air drag, gravity etc. After a first partial success in tuning a PID algorithm I got bored to find and hunt perfect values, and started thinking to tune it with a NN, but then I thinked, why using PID at all if NN can find better solutions?.

I started playing with NN, but then I realized I have no traning data.

Sure I could do


but what is actually the expected output? To be useful it has to be the thrust force of propellers tuned to keep input (position and orientation) stable in the desired place.

But actually I don't (Want to) know in advance the thrust that every single propeller has.

Since it is a simulation it is ok spending some computing time between each simulation step (eventually I'll try to optimize it later to fit it in a microcontroller)..

So, is it possible given a regular NN implementation where I can select number of:

  • Input neurons
  • Hidden neurons
  • Output neurons

find a way to train my model live? I need a way to tell the NN

hei this time you performed X keep going or go the opposite way.


It's debatable whether neural networks can find a better solution than PID, if your goal is to simply keep the output around a certain reference point PID should do a perfect job pretty much. If you really want to use "intelligent" control with NN you can look into reinforcement learning.
Few interesting papers that directly adress your problem:

Autonomous helicopter flight via reinforcement learning
An Application of Reinforcement Learning to Aerobatic Helicopter Flight
Autonomous Helicopter Control Using Reinforcement Learning Policy Search Methods

You can also find more in the references of these papers.

  • 1
    $\begingroup$ Probably worth adding something that more directly answers the title question: e.g. Reinforcement Learning addresses the issue of no supervised learning training data, by generating label data (predicted utility) or gradients (changes to control policy) from observations of interactions with the environment. $\endgroup$ May 15 '19 at 10:35

Training is not possible without training data, but it is not necessary to have a data set before the project begins. One can be generated.

That initial training when no data is available is tenuous at best is why parents run alongside the bicycle when their children are first learning to ride and why the training wheels are elevated so the parents can know when balance has been achieved and they can be removed.

One approach is to use a joystick and fly around in the simulation, leaving an audit trail. The collision, crash, and successful flight data will be valuable as labels. The other more sophisticated approach is to use a flight evaluation model attached to the flight model. How you rate the flights in the evaluation will determine the results of training. Metrics such as jostle, fuel consumption, time of flight, and near collisions are common in evaluation.

Even if you use continuous learning such as reinforcement learning algorithms and libraries, you will need a flight model to train if you want anything close to optimal flight. The benefit of continuous learning in a real business context is that passengers can be paying customers as the safety and thrift of the flight business continues to improve. Reinforcement is only one kind of system that does that.

For instance, pilots don't actually use Markov chains and evaluate advantage of one option over another when piloting. It is a much more continuous type of control and there are attention mechanisms involving field of view and instrumentation that changes the control function under certain risk conditions. If you are interested in field-usable drone automation, these other aspects of flight control will need to be understood and included in your experimentation eventually. Better sooner than later if you are serious about the field.

  • $\begingroup$ I have far a smaller goal: not flight, just stability around a point (which later can become tuned into flight). I was not expecting that so difficult to achieve. The problem is that tuning a 4- propeller drone is affordable to be done manually, but when I start to use more propellers or place them into non-square shapes, tuning PID is becoming so difficult that I wanted to use a NN, and even now after some hours I had no success in having a NN doing that. It looked one of the problems that NN are good at, but maybe are not (or maybe I'm just implementing the algorithm in wrong way :/ ) $\endgroup$ May 15 '19 at 16:44
  • $\begingroup$ If there is no air current, all the loads are fixed, take off and landing are manual, and the prop speed is perfectly regulated, then it would be easy. You can set up your simulator like that, but it is unrealistic and too simple to require a neural network. All you would need is a constant engine speed. $\endgroup$
    – FelicityC
    May 17 '19 at 19:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.