Hot answers tagged

3

What I want to know is whether I can add expert data to the replay buffer, given that DDPG is an off-policy algorithm? You certainly can, that is indeed one of the advantages of off-policy learning algorithms; they're still "correct", regardless of which policy generated the data that you're learning from (and a human expert providing the ...


2

generally the approach is to have a separate head. For example, imagine you have latent vector $z_k$, you would output two values: $h(z_k)$ and $f(z_k)$ where $0 \leq h \leq 1$ and $b_0 \leq f \leq b_1$ where $b_0$ and $b_1$ are your bounds. In thios setup, during inference you would check $h_k$ and if its greater than some threshold (usually .5), youd ...


1

Straight theoretical answer: In theory, yes, it is possible to model this problem as a Reinforcement Learning. But in practice, RL is not the most suitable approach for a simple linear maximization with a boundary. For instance, you could use a Lagrangian. Practical analysis on your specific problem In this specific example, you have 1 single constrain: $\...


1

You are right, it is sloppy notation by the authors. However, the target network is not necessarily linked to the behaviour policy $\beta$ either. Essentially when they take the expectation with respect to $\rho^\beta$ they are taking expectation with respect to a state distribution induced by some policy $\beta$ that is not necessarily the same as our ...


1

I would recommend doing is allowing your network to output any real number and then clipping the output. For instance, I was working with an agent that had to learn an angle between $[0, 2\pi]$ and $[0, 1]$. If the network outputted e.g. 10 in the first dimension then this would just be clipped to $2\pi$. This way the agent only learns about actions within ...


1

First, is it even possible to use DDPG for multi-dimensional continuous action spaces? Yes, DDPG was primarily developed to deal with continuous action space you can find out more here, here and here. I have not found any code examples to learn from and many of the papers I have read are near the limit of my understanding in this area. You can check ...


Only top voted, non community-wiki answers of a minimum length are eligible