I'm trying to implement Deep Q Learning using Tensorflow. The input is a vectorized representation of the state, and the output is a vector whose length is the number of possible actions.
I've already experimented with using Tensorflow to train a Neural Network for other algorithms - but in those situations, the target output vector for a given state was all zeroes, with the "best" action set to 1. In this situation, I was able to simply provide the
model.fit function with the integer index of the action that should be chosen.
However, with DQN the target output for a given state is not a vector with all but one entry set to zero; every entry in the output vector should have a value (the Q-value of taking that action from the input state). My questions are thus:
- What is it called when the target output is a "one-hot" vector? What is it called when every entry of the output vector has a meaning?
- What do I need to do differently to make this work in Tensorflow?