0
$\begingroup$

I am trying to train an ANN to control a 7 Degrees-Of-Freedom arm. It should reach a target avoiding a single obstacle. Given my modeling of the situation, my input layer is composed of 12 nodes:

  • 5 nodes for the 5 joint states
  • 3 nodes for the cartesian coordinates of the target
  • 3 nodes for the cartesian coordinates of the obstacle
  • 1 node for the radius of the obstacle (it's a spherical object).

I have already tried training the ANN with DQN. I want to try neuroevolution (NEAT, in particular) and see how the results compare. I am using NEAT-python. As seen in this paper, this should be feasible.

However, I am having trouble choosing the best fitness function and also some other hyperparameters, namely the population size. (I am also puzzled by the extremely long training time for a single generation, but that's another story.)

So, the fitness function. I have tried to replicate what I have done with DQN. So, basically, my function evaluates a genome (so, an ANN) as follows (pseudocode):

counter = 0
reapeat for NUM_OF_EPISODES times:
    generate a random target
    generate an obstacle which lies about halfway from the end-effector to the target
    repeat for TIMEOUT times:
        use the ANN to decide the next_action and execute it
        if OBSTACLE_REACHED or TARGET_REACHED, stop 
    counter += 0.3 * relative_distance + 0.2 * relative_path + 0.5 * didntHitObstacle()
fitness = counter / NUM_OF_EPISODES

So, humanly speaking, for each ANN we try to execute NUM_OF_EPISODES times (how many times should be enough? 100 times seems ok but it get's really slow) a scenario. In this scenario, we use the ANN to search for the target, and if we reach it or the obstacle, we stop. Now for each one of these scenarios, we should "rank" how well the ANN performed (see the counter += ... part). But how should I do this? My idea (stolen from the paper above) was to compute something like this:

0.3 * relative_distance + 0.2 * relative_path + 0.5 * didntHitObstacle()

So, basically, we see how much we are closer to the target compared to when we started, how "short" the path was (compared to the ideal straight line start point-to-target), and whether we did or did not hit the target.

Does this function make sense? My concern is mainly about how we deal with the obstacle: 50% of the fitness. Is it correct? I am asking this because I am receiving poor results.

Another problem that I have is population size. Of course, the bigger the better, but this thing takes a lot to train. How big is ok, in your experience?

$\endgroup$
  • 1
    $\begingroup$ Do you have a link to a Jupyter notebook? If you aren't using Python, please don't convert just for me (haha). Was just looking to take a look at the code. Glancing at the paper I see that they are using a 6 DOF arm. Adding another degree of freedom probably exponential increases possibilities, so would be slower to train. $\endgroup$ – JakeJ Apr 18 at 14:16
  • $\begingroup$ I'm sorry, I don't @JakeJ. If you want I can add the fitness function code in the answer. $\endgroup$ – olinarr Apr 18 at 14:19
  • $\begingroup$ How about a github repo? Sure, fitness function would help. $\endgroup$ – JakeJ Apr 18 at 14:22
  • $\begingroup$ Are you physically running it on the arm for each iteration (I've seen someone who does that)? $\endgroup$ – JakeJ Apr 18 at 14:23
  • 1
    $\begingroup$ I'm not sure what NEAT implementations there are that are GPU enabled, but I should start keeping my eye open. In the meantime, I saw that you can speed up numpy with a BLAS: en.wikipedia.org/wiki/Basic_Linear_Algebra_Subprograms I also saw this: github.com/sean-dougherty/accneat (from the catalogue here: eplex.cs.ucf.edu/neat_software/#NEAT ). That's all the help I can provide as I've not used NEAT or trained neural nets for this application type. $\endgroup$ – JakeJ Apr 18 at 17:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.