I am using Q-learning to solve an engineering problem. The objective is to generate a Q-table associating state to Q-values.
I created a State vector
DS = [s1, s2, ..., sN] containing all "desired" states. So the Q-table has the form of
Q-table =[DS, Q-values].
On the other hand, my agent follows a trajectory. Playing action
a at state
s (which is a point of the trajectory) leads the agent to another state
s' (another point of the trajectory). However, I don't have the
s' state in the initially desired states vector
One solution is to add new states to the
DS vector while the Q-learning algorithm is running, but I do not want to add new states.
Any other ideas on how to handle this problem?