I am reading a research paper on the formulation of MDP problems to ICU treatment decision making: Treatment Recommendation in Critical Care: A Scalable and Interpretable Approach in Partially Observable Health States. The paper applies a Monte Carlo approach to approximate the value function. Below is a screenshot of the excerpt that I came across.

enter image description here

The last sentence of the excerpt reads "The approach is scalable for growing number of states variables and action variables".

What does it mean when the author says that the Monte Carlo approach is scalable for a growing number of states variables and action variables? Wouldn't the amount of data needed to approximate the value function increase with the higher dimensionality of states? Or does the Monte Carlo approach scale better in time complexity as compared to traditional Q-learning methods?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.