I am trying to train an ANN to control a 7 Degrees-Of-Freedom arm. It should reach a target avoiding a single obstacle. Given my modeling of the situation, my input layer is composed of 12 nodes:
- 5 nodes for the 5 joint states
- 3 nodes for the cartesian coordinates of the target
- 3 nodes for the cartesian coordinates of the obstacle
- 1 node for the radius of the obstacle (it's a spherical object).
I have already tried training the ANN with DQN. I want to try neuroevolution (NEAT, in particular) and see how the results compare. I am using NEAT-python. As seen in this paper, this should be feasible.
However, I am having trouble choosing the best fitness function and also some other hyperparameters, namely the population size. (I am also puzzled by the extremely long training time for a single generation, but that's another story.)
So, the fitness function. I have tried to replicate what I have done with DQN. So, basically, my function evaluates a genome (so, an ANN) as follows (pseudocode):
counter = 0 reapeat for NUM_OF_EPISODES times: generate a random target generate an obstacle which lies about halfway from the end-effector to the target repeat for TIMEOUT times: use the ANN to decide the next_action and execute it if OBSTACLE_REACHED or TARGET_REACHED, stop counter += 0.3 * relative_distance + 0.2 * relative_path + 0.5 * didntHitObstacle() fitness = counter / NUM_OF_EPISODES
So, humanly speaking, for each ANN we try to execute
NUM_OF_EPISODES times (how many times should be enough? 100 times seems ok but it get's really slow) a scenario. In this scenario, we use the ANN to search for the target, and if we reach it or the obstacle, we stop. Now for each one of these scenarios, we should "rank" how well the ANN performed (see the counter += ... part). But how should I do this? My idea (stolen from the paper above) was to compute something like this:
0.3 * relative_distance + 0.2 * relative_path + 0.5 * didntHitObstacle()
So, basically, we see how much we are closer to the target compared to when we started, how "short" the path was (compared to the ideal straight line start point-to-target), and whether we did or did not hit the target.
Does this function make sense? My concern is mainly about how we deal with the obstacle: 50% of the fitness. Is it correct? I am asking this because I am receiving poor results.
Another problem that I have is population size. Of course, the bigger the better, but this thing takes a lot to train. How big is ok, in your experience?