I am using Q-learning to solve an engineering problem. The objective is to generate a Q-table associating state to Q-values.
I created a State vector DS = [s1, s2, ..., sN]
containing all "desired" states. So the Q-table has the form of Q-table =[DS, Q-values]
.
On the other hand, my agent follows a trajectory. Playing action a
at state s
(which is a point of the trajectory) leads the agent to another state s'
(another point of the trajectory). However, I don't have the s'
state in the initially desired states vector DS
.
One solution is to add new states to the DS
vector while the Q-learning algorithm is running, but I do not want to add new states.
Any other ideas on how to handle this problem?