I would like to ask for a piece of advice with regard to Q-learning. I am studying RL and would like to do a basic project applied to life science and calculate the reward. I have been trying to get my head around how to define all possible states of the environment.

My states are $S = ( \text{health } (4 \text{ levels}), \text{shape } (3 \text{ levels}) \}$. My actions are $A=\{a_1, a_2, \dots, a_4 \}$. My possible states are $60=4 * 3 * 5$. Could you advise whether these are correct?

$(s_{w_0, sh_0}, a_1, s'_{w_1, sh_1})$ is a tuple of the initial state $s_{w_0, sh_0}$, the first action $a_1$ and the next state $s'_{w_1, sh_1}$, where $w$ is the health level, $sh$ is the shape of the tumor.

  • $\begingroup$ Hi and welcome to AI SE! What is "w0,sh0" and "w1,sh1"? Also, when you say that your states are a set composed of "health" and "shape", does this mean that all states have these components and that you have 48 possible states? $\endgroup$
    – nbro
    Feb 28 '20 at 1:40
  • $\begingroup$ S(w0,sh0) is the initial state. The 48 possible states are the 4X3X5 , health levels(w), shape of a malign tumor and actions of different doses of treatment. Reading some manuscripts the description of the states are very vague. In my case I have a tuple of variables and the variables also have different levels. The agent is supposed to find the best combination to cure/improve the disease(health and and reduction of the shape of the tumor). $\endgroup$
    – Aze
    Feb 28 '20 at 10:44
  • $\begingroup$ This is a fictitious scenario with only one patient to illustrate my understanding of Q-learning and compute the Q matrix manually. I want to be sure that I got the states correct before moving to use a neural network for complex real data. Any help or comments would be highly appreciated. $\endgroup$
    – Aze
    Feb 28 '20 at 10:44
  • $\begingroup$ It looks like you have 12 possible states not 48 (4*3*5 is 60 by the way). What do these 5 different doses of treatment mean? Are those different doses of actions $a_i$. For example does that mean there are 5 different doses of action $a_1$? If yes then that is an action and not part of the state so you would have 12 states (4*3) and 20 actions (4*5). $\endgroup$
    – Brale
    Feb 28 '20 at 21:19
  • $\begingroup$ Thanks Brale, yes these are different doses of a1 and effectively it was four no five. Thanks a lot for your help. $\endgroup$
    – Aze
    Feb 28 '20 at 22:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.