The problem I am trying to attack is a predator-prey pursuit problem. There are multiple predators that pursue multiple preys and preys tried to evade predators. I am trying to solve a simplified version - one predator tries to catch a static prey on a plane. There is bunch of literature on the above problem when predators and preys are on the grid.

Can anybody suggest articles/code where such problem is solved on a continuous plane? I am looking at continuous state space, discrete action space (predator can turn left 10 degrees, go straight, turn right 10 degrees, runs at constant speed), and discrete time. MountainCar is one dimensional version (car is predator and flag is prey) and DQN works fine. However, when I tried DQN on two dimensional plane the training become very slow (I guess dimensionality curse).

The second question concerns the definition of states and reward. In my case I consider angle between predator heading vector and vector between the predator and prey positions. Reward is the change in distance between predator and prey, 10 when prey is captured, and -10 when predator gets too far from the prey. Is this reasonable? I already asked similar question before and with the help of @Neil Slater was able to refine reward and state.

The third question concerns when to update train network to target network. At each episode? Or only when prey is caught? Any ideas?

The last question I have is about the network structure: activation functions and regularization. Currently I am using two tanh hidden layers and linear output with l2 and dropout. Can anybody share some insights?

Thanks in advance!

  • $\begingroup$ In the literature, the predator prey scenario is usually described with differential equations. In contrast to piecewise-defined functions it's easier to find the minimum. $\endgroup$ – Manuel Rodriguez Jul 31 '19 at 20:52
  • $\begingroup$ Yep, but I wonder are there any articles/code that deals with the continuity of state space (it is not on the grid anymore but continuous plane). I still don't completely comprehend how different/similar are traditional predator-prey problems and predator-prey pursuit problems. There is no grass agent for sure... $\endgroup$ – TFbie Jul 31 '19 at 21:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.