I'm building an agent to solve the Taxi environment. I've seen this problem solved with Q-Learning algorithms but my DQN consistently fails to learn anything. The environment has a discrete observation space, I one-hot encode the state before feeding it to the DQN. I also went ahead to implement Hindsight Experience Replay to help the learning process but the DQN still doesn't learn anything. What can I do to fix this?
I've heard that DQN doesn't excel at environments that require planning to succeed, if that's the case, which algorithms would work well for this environment?
EDIT
When I posted this question, my DQN was learning from only 2 step transitions, since this environment can go on for several timesteps without any positive reward, I updated the agent to use transitions of 200 steps. Since I'm using Hindsight Experience Replay, my agent is sure to receive rewards within 200 timesteps even if it didn't meet the goal. I tried this and my agent still hasn't improved, it continually performs worse than the random agent baseline. I checked the contents of the buffer, I observed transitions that do lead to several rewards because their goals have been modified during HER and yet the DQN agent doesn't learn anything.
Also, I'm using TensorFlow's tf_agents for my implementation. Here's a link to the code. I repurposed this example.
I hope this helps
