I am trying to implement safe exploration technique in [Ref.1]. I am using Soft Actor-Critic algorithm to teach an agent to introduce a bias between 0 and 1 to a specific state of interest in my environment.

I would like to ask for your help in order to modify the critic update equation -which is originally based on the return or the RL cost function- from this:

enter image description here

which is based on the return functions:

enter image description here =

enter image description here

to be based on the following cost function to make the RL objective risk-sensitive and avoid large-valued actions (bias values) at the start of the agent's learning,

enter image description here

How can I include the second part of the cost function - in which the variance of the reward is evaluated- in the update equation?

[Ref.1] Heger, M., Consideration of risk in reinforcement learning. 11th International Machine Learning Conference (1994)

  • $\begingroup$ what is $c_t$ here? $\endgroup$ Sep 3 '20 at 19:17
  • $\begingroup$ It’s the return at instant t. When compared to the $R_t$ equation, $c_t $ ~ $r_{t+k+1} $\endgroup$ Sep 3 '20 at 19:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.