I am working on a problem where I need to generate a list of 0 - 8 different prices as the action space where generating 0 prices represents doing nothing

action_space = [i for i in range(NUM_PRICES + 1)]  # Include 0 as an option

# Sample an action from the action space
action = np.random.choice(action_space)

if action == 0:
    prices = []
    prices = np.random.uniform(-10.0, 200, action)

I have something like this where prices would be the generated action. Would something like this work with the A2C algorithm? I'm unfamiliar with A2C and have only worked with DQN and QLearning So I'm worried maybe the action space is too complex


1 Answer 1


A2C is a policy gradient method, that learns a policy $\pi(a\mid s)$ directly.

This means that the actions are sampled from a probability distribution. In practice, you can model a discrete action space with a Categorical distribution or, in case you experience gradient issues, with a Gumbel-Softmax (also called Concrete) distribution which is its continuous relaxation.

Lastly, unless NUM_PRICES is very large you shouldn't experience issue about the action space size.


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .