# Tag Info

3

What I want to know is whether I can add expert data to the replay buffer, given that DDPG is an off-policy algorithm? You certainly can, that is indeed one of the advantages of off-policy learning algorithms; they're still "correct", regardless of which policy generated the data that you're learning from (and a human expert providing the ...

2

generally the approach is to have a separate head. For example, imagine you have latent vector $z_k$, you would output two values: $h(z_k)$ and $f(z_k)$ where $0 \leq h \leq 1$ and $b_0 \leq f \leq b_1$ where $b_0$ and $b_1$ are your bounds. In thios setup, during inference you would check $h_k$ and if its greater than some threshold (usually .5), youd ...

1

Straight theoretical answer: In theory, yes, it is possible to model this problem as a Reinforcement Learning. But in practice, RL is not the most suitable approach for a simple linear maximization with a boundary. For instance, you could use a Lagrangian. Practical analysis on your specific problem In this specific example, you have 1 single constrain: $\... 1 You are right, it is sloppy notation by the authors. However, the target network is not necessarily linked to the behaviour policy$\beta$either. Essentially when they take the expectation with respect to$\rho^\beta$they are taking expectation with respect to a state distribution induced by some policy$\beta$that is not necessarily the same as our ... 1 I would recommend doing is allowing your network to output any real number and then clipping the output. For instance, I was working with an agent that had to learn an angle between$[0, 2\pi]$and$[0, 1]$. If the network outputted e.g. 10 in the first dimension then this would just be clipped to$2\pi\$. This way the agent only learns about actions within ...

1

First, is it even possible to use DDPG for multi-dimensional continuous action spaces? Yes, DDPG was primarily developed to deal with continuous action space you can find out more here, here and here. I have not found any code examples to learn from and many of the papers I have read are near the limit of my understanding in this area. You can check ...

Only top voted, non community-wiki answers of a minimum length are eligible