I am trying to use reinforcement learning to let an agent learn simultaneously how to play a game and when to end a game.
The task is to find a single target in a grid of locations. At each time step, the agent needs to make a series of decisions:
- It believes the target is at the currently inspected location. End the trial and see whether the result is correct.
- It believes the target is not at the currently inspected location. It then needs to pick another location to check at the next timestep.
If the agent is choose decision #2, the environment will give some hints on where the target is, with some stochastic noise. The noise level depends on the distance between the true target location and currently inspected location. The shorter the distance, the lower the noise. The goal is to let the agent perform the task as fast and accurately as possible, so the agent needs to learn when to stop the trial, and how to select the next inspected location given the hints. The agent also has a internal memory so it won't select previously inspected locations. I would like to compare the agent's speed-accuracy trade off to human's.
In a previous simplified version of the task, the environment ends the trial once the agent hit the target location, so the agent only needs to learn how to choose the next location to inspect. I used a simple Q-network and it works well. I also found that the network should be a fully convolutional network because fully connected layers are not spatially shift-invariant.
Now how can I modify the existing convolutional network to satisfy the new task requirement? Or should I use a new network architecture?