3
The game of TIC-TAC-TOE can be modelled as a non-deterministic Markov decision process (MDP) if, and only if:
The opponent is considered part of the environment. This is a reasonable approach when the goal is to solve playing against a specific opponent.
The opponent is using a stochastic policy. Stochastic policies are a generalisation that include ...
3
Depends on perspective.
On one hand, you have an agent playing in an environment with another agent also evolving. This falls under the definition of Multi-Agent Learning, as can be seen with works such as
Michael Bowling and Manuela Veloso. Multiagent learning using a variable learning
rate. Artificial Intelligence, 136(2):215 – 250, 2002.
Michael Bowling....
2
I think there is an intersection. There are problems that are in reinforcement learning and in learning in multi-agent systems. There are problems in reinforcement learning, but not exactly in multi-agent systems. And there is learning in multi-agent systems that is not through reinforcement learning. For sort you can say: multi-agent reinforcement learning. ...
2
it can be either. If you consider the lack of reward as "penalty" then getting 0 reward is bad.
if you use a value estimator through a neural network, the range of rewards will dictate the squashing function you use for the output layer
2
Let's do the code, so all the details are down.
Encoding dictionary:
codes, i = {}, 0
for nSquares in range(1,8):
for direction in ["N", "NE", "E", "SE", "S", "SW", "W", "NW"]:
codes[(nSquares,direction)] = i
i += 1
You'll see that the codes dictionary will ...
1
This is mostly an implementation architecture problem, and the thing is that basically you can implement anything in the traditional setting. To do so instead of having Env<->Agent1<->Agent2, you should have Agent1<->SuperEnv<->Agent2 where SuperEnv contains Env, and simply uses the reward given to SuperEnv by Agent1 and passes it to ...
1
You might be able to glean what you want from Chapter 13 or Sutton & Barto's Reinforcement Learning: An Introduction, which deals with policy gradient algorithms, and includes pseudocode for a variety of agents based on linear approximation using softmax regression. From your description, you appear to be using - or should consider - softmax regression ...
1
I depends on your overall model architecture (and problem specification). As I understand it, you take the observations of all agents together and feed it into one model, a central controller, which then predicts the action per available agent.
I believe that this varying number of applicable observations (depending on the number of currently present agents) ...
1
In the context of reinforcement learning, the idea of modeling your goal-oriented problem as a hierarchy of multiple sub-problems is called hierarchical reinforcement learning, which gives rise to concepts such as semi-Markov decision processes and options (aka macro actions). The article The Promise of Hierarchical Reinforcement Learning presents and ...
Only top voted, non community-wiki answers of a minimum length are eligible
