I want to make a simple recommendation system based on reinforcement learning using
OpenAI Gym. My point is that I've already created an
Agent to learn the
Cartpole environment and now I want to set up my own environment to train against it.
DataFrame has the typical columns of
[user, item, relevance, timestamp] but I don't know how to deal with the sequences of acts to show to the
Agent and the reward to give to it.
My approach is to give the user id as the input to make an Embedding of it and the output as a softmax of the size of the number of items to make a ranking and multiply a number for the
ndcg of the number of ones (as it means the user has bought the item) minus the
ndcg of the number of ceros (as it means that the user has viewed the item and not buyed it).
The problem is that by this way the
Agent would learn to recommend the items that the user has already bought (not useful) and not to recommend the items that has already seen (useful). I hope to make a system that could recommend some items that the user is not going to buy (the less the better) hoping to maximize that the user buys a distinct item later (so the
timestamp is very important).
Another problem is the way of showing the data. It is clear that I have to show it ordered by timestamp but, is it relevant to order the data by user before? Where do I calculate the reward?
Has anyone any idea or critic to this approach? Can I do this with
kerasRL and a
Gym Environment? Has anyone a paper or a video useful to this issue?
I hope I expressed myself correctly and thank you in advance.