Questions tagged [action-spaces]

For questions about action spaces in the context of reinforcement learning and other AI sub-fields.

Filter by
Sorted by
Tagged with
2
votes
1answer
41 views

What should the input and output of the Q-network be in the case of an ordinal action space?

I recently started looking into implementations of the DQN algorithm (e.g. TensorFlow) in some more detail. All the implementations that I found use a network that gives an output for each possible ...
3
votes
1answer
49 views

How to use DQN when the action space can be different at different time steps?

I would like to employ DQN to solve a constrained MDP problem. The problem has constraints on action space. At different time steps till the end, the available actions are different. It has different ...
4
votes
1answer
79 views

How should I define the action space for a card game like Magic: The Gathering?

I'm trying to learn about reinforcement learning techniques. I have little background in machine learning from university, but never more than using a CNN on the MNIST database. My first project was ...
0
votes
0answers
40 views

How to implement RL model with increasing dimensions of state space and action space?

I've read in this discussion that "reinforcement learning is a way of finding the value function of a Markov Decision Process". I want to implement an RL model, whose state space and action ...
1
vote
1answer
59 views

Why does each component of the tuple that represents an action have a categorical distribution in the TRPO paper?

I was going through the TRPO paper, and there was a line under Appendix D "Approximating Factored Policies with Neural Networks" in the last paragraph which I am unable to understand The ...
2
votes
0answers
43 views

When to do discretization to decrease the state/action space in RL?

When to do discretization to decrease the state/action space in RL? Can you give me some references that such a technique is used?
1
vote
2answers
57 views

What is meant by a multi-dimensional continuous action space?

In the context of Reinforcement Learning, what does it mean to have a multi-dimensional continuous action space? I came across the following in the COBRA Paper A method for learning a distribution ...
5
votes
1answer
61 views

Is the agent aware of a possible different set of actions for each state?

I have a use case where the set of actions is different for different states. Is the agent aware of what actions are valid for each state, or is the agent only aware of the entire action space (in ...
6
votes
1answer
105 views

Are there RL techniques to deal with incremental action spaces?

Let's say we have a problem that can be solved by some RL algorithms (DQN, for example, because we have discrete action space). At first, the action space is fixed (the number of actions is $n_1$), ...
1
vote
1answer
119 views

Should I ignore the actions RIGHTFIRE and LEFTFIRE in the SpaceInvaders environment? [closed]

I'm trying to replicate the DeepMind DQN paper. I'm using OpenAI's Gym. I'm trying to get a decent score with Space Invaders (using SpaceInvaders-v4 environment). I ...
13
votes
1answer
3k views

How to deal with a huge action space, where, at every step, there is a variable number of legal actions?

I am working on creating an RL-based AI for a certain board game. Just as a general overview of the game so that you understand what it's all about: It's a discrete turn-based game with a board of ...
3
votes
1answer
452 views

Why does Deep Q Network outputs multiple Q values?

I am learning Deep RL following this tutorial: https://medium.freecodecamp.org/an-introduction-to-deep-q-learning-lets-play-doom-54d02d8017d8 I understand everything but one detail: This image shows ...
3
votes
2answers
316 views

How can I design a reinforcement learning model for a game with multiple complex actions taken at a time?

I have a steady hex-map and turn-based wargame featuring WWII carrier battles. On a given turn, a player may choose to perform a large number of actions. Actions can be of many different types, and ...
16
votes
2answers
6k views

Any papers regarding different/inconsistent action space in Reinforcement Learning? [closed]

This question is regarding Reinforcement Learning and different/inconsistent action space for every/some states. What do I mean by different/inconsistent action space? Let say you have an MDP where ...
13
votes
3answers
4k views

How to implement a variable action space in Proximal Policy Optimization?

I'm coding a Proximal Policy Optimization (PPO) agent with the Tensorforce library (which is built on top of TensorFlow). The first environment was very simple. Now, I'm diving into a more complex ...
1
vote
1answer
282 views

How can I incorporate domain knowledge to choose actions in the case of large action spaces in multi-armed bandits?

Suppose one is using a multi-armed bandit, and one has relatively few "pulls" (i.e. timesteps) relative to the action set. For example, maybe there are 200 timesteps and 100 possible actions....
5
votes
1answer
2k views

How to deal with different actions for different states of the environment?

I'm new to this AI/Machine Learning and was playing around with OpenAI Gym a bit. When looking through the environments, I came across the Blackjack-v0 environment, ...
9
votes
3answers
14k views

What do the different actions of the OpenAI gym's environment of 'Pong-v0' represent? [closed]

Printing action_space for Pong-v0 gives Discrete(6) as output, i.e. $0, 1, 2, 3, 4, 5$ are actions defined in the environment as ...