I would like to build a model based on reinforcement learning (RL) for the following scenario

Recommend the best route (of cities listed for a given country) that satisfies the required criteria (museum, beaches, food, etc) for a total budget of $2000.

Based on the recommendation, the user will provide its feedback (as a reward), so the recommendations can be fine-tuned (by reinforcement learning) the next time. I modeled the system this way:

  • States = (c,cr), where $c$ is the city and $cr$ is the criteria (history, beach, food, etc)

  • Actions = (p) is the price of visiting the city

  • Reward: acceptance of the cities selected by end user as a route (1 or 0)

The objective is to decide which list of cities together satisfy the given budget.

Is this MDP model right and how can I implement this? May be the only option is using Monte Carlo methods and linear/dynamic programming.. Is there any other way?

  • $\begingroup$ If you can easily generate a user feedback for a recommendation given, maybe consider using supervised learning methods? (Not a comment as I have not enough reputation) $\endgroup$ – oleg.mosalov Jun 16 at 9:03
  • $\begingroup$ You say "delivers the required criteria (museum, beaches, food, etc)". Of course, no one will deliver "museums or beaches", so I suppose that there's a typo in your post. I suggest that you fix that typo! Also, I provisionally added a new title to your post. Change it to make it more descriptive of your question. $\endgroup$ – nbro Jun 16 at 10:16
  • $\begingroup$ @oleg.mosalov Thanks. Not sure how supervised learning could provide the list of cities that fulfill the criteria (budget and other parameters)? Could you please elaborate? $\endgroup$ – Cengiz Jun 16 at 12:29

I do not see how you came to choose prices as actions. Normally, actions are something like go left, go right, jump, stay etc. Analogously, I would say that in your case the actions are visiting a certain location, whereas locations are what you referred to as states. I'd go for something like that:

locations = {location1=(c1,cr1), location2=(c1,cr2), ...}
Actions = {Visit location1, Visit location2, ...}
States = {--set of all the possible paths the model can generate until the budget is possibly exhausted--}

The reward function could then be a combination of both the acceptance (vs. rejection) of a route/path by the user and the inverse of the cost associated with the suggested path (because you want the model to favor cheap paths in order to keep your own business costs low). How you balance these two terms is up to exploration.

For rapid prototyping, check out stable baselines, which offers a bunch of highly optimized RL algorithms.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.