0
$\begingroup$

I have have an environment with two models.

Model of the environment is stochastic. Given the price it returns the time when the next purchase will be made and how many items will be bought. Both of those values have some probability distribution. So in a given state the same action may return different results.

I need to train the other model - the model that will suggest prices that will maximize a profit over the course of 365 days of simulation.

So the loop goes like this, that the model that needs to be trained gives you the suggested price in a given state. And having this price, the environment returns the estimated time until next purchase and how many items will be bought at that time. The state is updated with let's say 3.4 days less and with total item count updated.

I'm confused about reinforcement learning terminology here - i'm not sure if this would be an example of model based or model free reinforcement learning. Also I'm not sure about the specific algorithms that I could use for such task.

I read this topic before, but i don't see how it translates to my situation. What's the difference between model-free and model-based reinforcement learning?

Feel free to ask extra questions if I didn't explain something precisely. I'm new to the subject of RL, so it totally can happen.

Improve this question
$\endgroup$
2
  • $\begingroup$ It might be a good idea to read about Markov decision processes and to understand if you can model your environment as a MDP. You need to answer the question: what would be states, actions and rewards in your example? $\endgroup$
    – nbro
    Commented Dec 11, 2022 at 11:42
  • $\begingroup$ Generally if you're unsure about the (stochastic) state transition function of the environment dynamics, just try model-free inductive RL. Financial markets under efficient market hypothesis usually lack mechanistic causality in the form of deterministic or indeterministic state transition functions. $\endgroup$
    – cinch
    Commented Dec 12, 2022 at 3:13

0

Reset to default

You must log in to answer this question.