Is there an RL approach/algorithm that would be suited for the following kind of problem?

  • There is a continuous action space with an action value $A_{a,t}$ for each action dimension $a$.
  • The objective function is a non-linear function of the satisfaction factors $S_{s}$ for each satisfaction dimension $s$ and some other random & independent factors. This objective function can be known to the agent.
  • Each satisfaction factor depends on an independent variable $\Delta^S_{s,t}$ and the effect $\delta^S_{a,s,t}$ of each action: $S_{s,t}=\Delta^S_{s,t} +\sum_a A_{a,t} * \delta^S_{a,s,t}$.
  • Each action can further have an effect $\delta^R_{a,r,t}$ on the inventory factors $I_{r,t}$ for each resource dimension $r$, with inventories being kept between time-steps and a factor $\Delta^R_{r,t}$ that is added or removed from the inventory at each step independent of the actions: $I_{r,t+1}=I_{r,t}+\Delta^R_{r,t} + \sum_a A_{a,t} * \delta^R_{a,r,t}$
  • The agent is constrained by each of these resources (i.e. the inventory has to remain positive).
  • The agent should be able to deal both with $\delta$ and $\Delta$ factors that are visible (states) and invisible (have to be learned).
  • A trained agent should be able to know how to adapt to changes of the $\delta$ and $\Delta$ factors, as well as the introduction or removal of activity dimensions.

EDIT: I have adapted the problem description after some feedback.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.