# Training a reinforcement learning agent that can decide to continue or end the game

I am trying to use reinforcement learning to let an agent learn simultaneously how to play a game and when to end a game.

The task is to find a single target in a grid of locations. At each time step, the agent needs to make a series of decisions:

1. It believes the target is at the currently inspected location. End the trial and see whether the result is correct.
2. It believes the target is not at the currently inspected location. It then needs to pick another location to check at the next timestep.

If the agent is choose decision #2, the environment will give some hints on where the target is, with some stochastic noise. The noise level depends on the distance between the true target location and currently inspected location. The shorter the distance, the lower the noise. The goal is to let the agent perform the task as fast and accurately as possible, so the agent needs to learn when to stop the trial, and how to select the next inspected location given the hints. The agent also has a internal memory so it won't select previously inspected locations. I would like to compare the agent's speed-accuracy trade off to human's.

In a previous simplified version of the task, the environment ends the trial once the agent hit the target location, so the agent only needs to learn how to choose the next location to inspect. I used a simple Q-network and it works well. I also found that the network should be a fully convolutional network because fully connected layers are not spatially shift-invariant.

Now how can I modify the existing convolutional network to satisfy the new task requirement? Or should I use a new network architecture?

• If the agent chose action #1, but the target is not found, will the environment continue and the agent choose other areas or will the environment terminated? Oct 15, 2021 at 1:28
• @Sanyou The environment will terminate. Oct 15, 2021 at 10:34

Other choice is to pad the "inspect current location" action alongside the "location selecting" actions, #1 inspect current location, #$$2$$ until #($$N+1$$) visit next location, given $$N$$ is the number of possible locations to visit.