I have continual simulated data of million sentences of two simulated persons talking to each other in a room and I want to model one of the persons speech. Now, during this period things in the room can change. Let's say, one of them says "Where is the book?" The other one responds "I placed the book on the bookshelf". Now during time, the position of the book changes, so the question Where is the book? does not have stationary answer i.e the answer changes during time. However, in general the answer has to be "The book is at some_location" and not something else. Also, the mentioning that the book is placed on the bookshelf can be sometimes 10, 100 or 1000 sentences before the question "Where is the book?"
How do you approach this kind of problem? Since the window can be too large I can not split data into training samples of 10, 100 or 1000 sentences. My guess is that I should use BPTT + LSTM and train in one shot without shuffling the data. I am not sure this is feasible, so I will greatly appreciate your help! I have also my doubts what if "Where is the book?" appears 20 sentences after (instead of 10,100 and 1000) in the test set (which is not same as the training set)? Also, should I use Reinforcement Learning (since I can generate the data) or Supervised learning?
Thanks a lot!