I want to make a simple recommendation system based on reinforcement learning using kerasRL and OpenAI Gym. My point is that I've already created an Agent to learn the Cartpole environment and now I want to set up my own environment to train against it.

My DataFrame has the typical columns of [user, item, relevance, timestamp] but I don't know how to deal with the sequences of acts to show to the Agent and the reward to give to it.

My approach is to give the user id as the input to make an Embedding of it and the output as a softmax of the size of the number of items to make a ranking and multiply a number for the ndcg of the number of ones (as it means the user has bought the item) minus the ndcg of the number of ceros (as it means that the user has viewed the item and not buyed it).

The problem is that by this way the Agent would learn to recommend the items that the user has already bought (not useful) and not to recommend the items that has already seen (useful). I hope to make a system that could recommend some items that the user is not going to buy (the less the better) hoping to maximize that the user buys a distinct item later (so the timestamp is very important).

Another problem is the way of showing the data. It is clear that I have to show it ordered by timestamp but, is it relevant to order the data by user before? Where do I calculate the reward?

Has anyone any idea or critic to this approach? Can I do this with kerasRL and a Gym Environment? Has anyone a paper or a video useful to this issue?

I hope I expressed myself correctly and thank you in advance.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.