# How should I define the state space for this life science problem?

I would like to ask for a piece of advice with regard to Q-learning. I am studying RL and would like to do a basic project applied to life science and calculate the reward. I have been trying to get my head around how to define all possible states of the environment.

My states are $$S = ( \text{health } (4 \text{ levels}), \text{shape } (3 \text{ levels}) \}$$. My actions are $$A=\{a_1, a_2, \dots, a_4 \}$$. My possible states are $$60=4 * 3 * 5$$. Could you advise whether these are correct?

$$(s_{w_0, sh_0}, a_1, s'_{w_1, sh_1})$$ is a tuple of the initial state $$s_{w_0, sh_0}$$, the first action $$a_1$$ and the next state $$s'_{w_1, sh_1}$$, where $$w$$ is the health level, $$sh$$ is the shape of the tumor.

• Hi and welcome to AI SE! What is "w0,sh0" and "w1,sh1"? Also, when you say that your states are a set composed of "health" and "shape", does this mean that all states have these components and that you have 48 possible states?
– nbro
Feb 28 '20 at 1:40
• S(w0,sh0) is the initial state. The 48 possible states are the 4X3X5 , health levels(w), shape of a malign tumor and actions of different doses of treatment. Reading some manuscripts the description of the states are very vague. In my case I have a tuple of variables and the variables also have different levels. The agent is supposed to find the best combination to cure/improve the disease(health and and reduction of the shape of the tumor).
– Aze
Feb 28 '20 at 10:44
• This is a fictitious scenario with only one patient to illustrate my understanding of Q-learning and compute the Q matrix manually. I want to be sure that I got the states correct before moving to use a neural network for complex real data. Any help or comments would be highly appreciated.
– Aze
Feb 28 '20 at 10:44
• It looks like you have 12 possible states not 48 (4*3*5 is 60 by the way). What do these 5 different doses of treatment mean? Are those different doses of actions $a_i$. For example does that mean there are 5 different doses of action $a_1$? If yes then that is an action and not part of the state so you would have 12 states (4*3) and 20 actions (4*5). Feb 28 '20 at 21:19
• Thanks Brale, yes these are different doses of a1 and effectively it was four no five. Thanks a lot for your help.
– Aze
Feb 28 '20 at 22:52