# In the cross-entropy method, should I select state-action pairs by their immediate reward or by the episode reward?

I am trying to understand the code mechanics when selecting the elite states and elite actions. It appears clear to me that they are those that appear in the episodes with the rewards bigger than the threshold.

My question is: should I select state-action pairs by their immediate reward or by the episode reward?

I am applying the method to a craft environment interesting to me and I have been studying an example applying the OpenAI's Gym taxi environment, but I do not fully understand the code.

• Hello. I've edited your post in order to clarify what your question is by putting it in the title. Please, make sure that the current version of your post is still consistent with the original one. If not, feel free to edit your post again to fix that.
– nbro
Jul 22, 2021 at 10:39
• Although the edition is Ok, I am not sure if monte-carlo-methods as tag is correct. Jul 22, 2021 at 11:58
• It is Monte Carlo in the general sense of simulating outcomes by sampling from random variables (in this case only the policy). I would not have used monte-carlo-methods here, CEM is not a variant of Monte Carlo control. Although you could consider REINFORCE algorithm strongly related to CEM, and an "upgrade" of it. Jul 22, 2021 at 12:36
• It may be a good idea to provide a reference/link to the CEM you're referring to, just to make sure we're talking about the same thing.
– nbro
Jul 22, 2021 at 14:47
• @NeilSlater I am refining my skills related to Reinforcement Learning after doing some online courses in Coursera. I intend to apply these knowledge area to the adaptation of livestock farmers to climate variability. As an excercise, I am trying to deep my expertise in the different kind of RL and related algorithms. For that, initially I must master the environment construction under different perspectives, tabular and approximation methods, discrete and continuous state and action spaces, and solidify my familiarity with the discipline Jul 22, 2021 at 16:56