on-policy (left) vs. off-policy (right) actor critic

Does this figure correctly represent the overall general idea about actor-critic methods for on-policy (left) and off-policy (right) case?

I am a bit confused about the off-policy case (right figure). Why does the right figure represent the off-policy actor-critic methods?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.