# Deep Q-learning is not performing well when there are several enemies

I am playing with a deep Q-learning algorithm in my own environment. The network can perform well as long as there is only one enemy. My agent can perform the following actions:

1. do_nothing
2. prepare_for(e)
3. attack(e)

where e is some enemy.

In the case of two enemies, the action vector has 5 elements:

|   0       |      1          |      2      |        3         |     4      |
-----------------------------------------------------------------------------
|do_nothing | prepare_for(e1) |  attack(e1) |  prepare_for(e2) | attack(e2) |
-----------------------------------------------------------------------------


After a couple of episodes, the agent always starts picking the first do_nothing action, which is not desired. Changing reward for do_nothing action is not helping, even using significantly higher negative reward, than for other actions.

There is no problem with the environment with only one enemy. (Only using columns 0, 1, 2). I feel like my action encoding can be the issue, but I can't figure it out, how to fix it. Any suggestions?

• You have to provide a bit more details. What is the shape of your neural network? What are you using as input, state features or images? Are you providing state observation + action as the input or only state observation? How does the output look like, are there 5 nodes each for specific action or is it only a single output node for the specific action you provided as a part of the input. What are the rewards for your environment? Are you using exploratory policy? Are you using experience replay? Are you using separate networks? Are you providing enough training time? – Brale Feb 10 '19 at 10:59