Return to Answer

Simple suggestion: Tabular Q-Learning

Why?

Let's say we interpret MNIST's label as a Hello World of Supervised Learning to mean something showing the basic "syntax" of a Supervised Learning algorithm: Create a model, load the data, then train.

If that interpretation is not far off, we can say the basic "syntax" of Reinforcement Learning (RL) would focus on showing a working Markov Decision Process (MDP) which is the backbone of the RL decision making process. As such, this minimal working syntax would involve: Observing the world, selecting an action, as shown in this loop:

This picture is missing two important steps in an RL algorithm learning loop:

1. Estimating the rewards or Fitting the model
2. Improving how you select actions. (Updating your policy)

How we decide to update the policy, or fit the model is what makes difference in the RL algorithm most of the time.

So a simple hello world RL algorithm would be one that helps you see the MDP in action, while keeping steps 1 and 2 simple enough so that you understand how the agent learns. Tabular Q-Learning seems simple enough for this because it uses a Q-table represented as a 2D array to do the two steps.

You will be unable to use it anywhere else than in a toy environment though, typically Frozen-Lake and CartPole. An improvement would be using a neural network instead of a table to estimate Q values.

Here are a few useful resources:

1. Q-Learning with Tables (Guide)
2. Q-learning jupyter notebook (Code ~25 lines)
3. Q-Learning with Frozen-Lake and Taxi (Code)
4. Reinforcement Learning with Q-Learning (Guide)

A multi-armed bandit would also be great in introducing you to exploration-exploitation trade-off (which Q-learning does too), though it wouldn't be considered a full RL algorithm since it has no context.

While there's no simple Hello World problem of RL, if your aim is to understand the basic working of Reinforcement Learning and see it at play while using as few moving parts as possible, a simple suggestion would be using Tabular Q-Learning in a toy environment (like your suggested Cart-Pole Env).

Here's the reasoning behind this suggestion

Let's say we interpret MNIST's label as a Hello World of Supervised Learning to mean something showing the basic steps of doing Supervised Learning: Create a model, load the data, then train.

If that interpretation is not far off, we can say a simple introductory problem to Reinforcement Learning (RL) should focus on easily demonstrating a working Markov Decision Process (MDP) which is the backbone of the RL decision making process. As such, this minimal working would involve: Observing the world, selecting an action, as shown in this loop:

This picture is missing two important steps in an RL algorithm learning loop:

1. Estimating the rewards or Fitting the model
2. Improving how you select actions. (Updating your policy)

How we decide to update the policy, or fit the model is what makes difference in the RL algorithm most of the time.

So a suggested first problem would be one that helps you see the MDP in action, while keeping steps 1 and 2 simple enough so that you understand how the agent learns. Tabular Q-Learning seems clear enough for this because it uses a Q-table represented as a 2D array to do the two steps. This should not suggest Q-learning is a "Hello World" RL algorithm because of the said relative ease in understanding it :)

You will be unable to use it's Tabular version anywhere else than in a toy environment though, typically Frozen-Lake and CartPole. An improvement would be using a neural network instead of a table to estimate Q values.

Here are a few useful resources:

1. Q-Learning with Tables (Guide)
2. Q-learning jupyter notebook (Code ~25 lines)
3. Q-Learning with Frozen-Lake and Taxi (Code)
4. Reinforcement Learning with Q-Learning (Guide)

A multi-armed bandit would also be great in introducing you to exploration-exploitation trade-off (which Q-learning does too), though it wouldn't be considered a full RL algorithm since it has no context.

Simple suggestion: Tabular Q-Learning

Why?

Let's say we interpret MNIST's label as a Hello World of Supervised Learning to mean something showing the basic "syntax" of a Supervised Learning algorithm: Create a model, load the data, then train.

If that interpretation is not far off, we can say the basic "syntax" of Reinforcement Learning (RL) would focus on showing a working Markov Decision Process (MDP) which is the backbone of the RL decision making process. As such, this minimal working syntax would involve: Observing the world, selecting an action, as shown in this loop:

This picture is missing two important steps in an RL algorithm learning loop:

1. Estimating the rewards or Fitting the model
2. Improving how you select actions. (Updating your policy)

How we decide to update the policy, or fit the model is what makes difference in the RL algorithm most of the time.

So a simple hello world RL algorithm would be one that helps you see the MDP in action, while keeping steps 1 and 2 simple enough so that you understand how the agent learns. Tabular Q-Learning seems simple enough for this because it uses a Q-table represented as a 2D array to do the two steps.

You will be unable to use it anywhere else than in a toy environment though, typically Frozen-Lake and CartPole. An improvement would be using a neural network instead of a table to estimate Q values.

Here are a few useful resources:

1. Q-Learning with Tables (Guide)
2. Q-learning jupyter notebook (Code ~25 lines)
3. Q-Learning with Frozen-Lake and Taxi (Code)
4. Reinforcement Learning with Q-Learning (Guide)

A multi-armed bandit would also be great in introducing you to exploration-exploitation trade-off, though it wouldn't be considered a full RL algorithm since it has no context.

Simple suggestion: Tabular Q-Learning

Why?

Let's say we interpret MNIST's label as a Hello World of Supervised Learning to mean something showing the basic "syntax" of a Supervised Learning algorithm: Create a model, load the data, then train.

If that interpretation is not far off, we can say the basic "syntax" of Reinforcement Learning (RL) would focus on showing a working Markov Decision Process (MDP) which is the backbone of the RL decision making process. As such, this minimal working syntax would involve: Observing the world, selecting an action, as shown in this loop:

This picture is missing two important steps in an RL algorithm learning loop:

1. Estimating the rewards or Fitting the model
2. Improving how you select actions. (Updating your policy)

How we decide to update the policy, or fit the model is what makes difference in the RL algorithm most of the time.

So a simple hello world RL algorithm would be one that helps you see the MDP in action, while keeping steps 1 and 2 simple enough so that you understand how the agent learns. Tabular Q-Learning seems simple enough for this because it uses a Q-table represented as a 2D array to do the two steps.

You will be unable to use it anywhere else than in a toy environment though, typically Frozen-Lake and CartPole. An improvement would be using a neural network instead of a table to estimate Q values.

Here are a few useful resources:

1. Q-Learning with Tables (Guide)
2. Q-learning jupyter notebook (Code ~25 lines)
3. Q-Learning with Frozen-Lake and Taxi (Code)
4. Reinforcement Learning with Q-Learning (Guide)

A multi-armed bandit would also be great in introducing you to exploration-exploitation trade-off (which Q-learning does too), though it wouldn't be considered a full RL algorithm since it has no context.

Simple suggestion: Tabular Q-Learning

Why?

Let's say we interprete MNIST's label as a Hello World of Supervised Learning as something showing the basic "syntax" of a Supervised Learning algorithm: Create a model, load the data, then train.

If that interpration is not far off, we can say the basic "syntax" of Reinforcement Learning (RL) would focus on showing a working Markov Decision Process (MDP) which is the backbone of the RL decision making process. As such, this minimal working syntax would involve: Observing the world, selecting an action, as shown in this loop:

This picture is missing two important steps in an RL algorithm learning loop:

1. Estimating the rewards or Fitting the model
2. Improving how you select actions. (Updating your policy)

How we decide to update the policy, or fit the model is what makes difference in the RL algorithm most of the time.

So a simple hello world RL algorithm would be one that helps you see the MDP in action, while keeping steps 1 and 2 simple enough so that you understand how the agent learns. Tabular Q-Learning seems simple enough for this because it uses a Q-table represented as a 2D array to do the two steps.

You will be unable to use it anywhere else than in a toy environment though, typically Frozen-Lake and CartPole.

Here is are a few useful resources:

1. Q-Learning with Tables (Guide)
2. Q-learning jupyter notebook (25lines Code)
3. Q-Learning with Frozen-Lake and Taxi (Code)
4. Reinforcement Learning with Q-Learning (Guide)

A multi-armed bandit would also be great in introducing you to exploration-exploitation trade-off, though it wouldn't be considered a full RL algorithm since it has no context.

Simple suggestion: Tabular Q-Learning

Why?

Let's say we interpret MNIST's label as a Hello World of Supervised Learning to mean something showing the basic "syntax" of a Supervised Learning algorithm: Create a model, load the data, then train.

If that interpretation is not far off, we can say the basic "syntax" of Reinforcement Learning (RL) would focus on showing a working Markov Decision Process (MDP) which is the backbone of the RL decision making process. As such, this minimal working syntax would involve: Observing the world, selecting an action, as shown in this loop:

This picture is missing two important steps in an RL algorithm learning loop:

1. Estimating the rewards or Fitting the model
2. Improving how you select actions. (Updating your policy)

How we decide to update the policy, or fit the model is what makes difference in the RL algorithm most of the time.

So a simple hello world RL algorithm would be one that helps you see the MDP in action, while keeping steps 1 and 2 simple enough so that you understand how the agent learns. Tabular Q-Learning seems simple enough for this because it uses a Q-table represented as a 2D array to do the two steps.

You will be unable to use it anywhere else than in a toy environment though, typically Frozen-Lake and CartPole. An improvement would be using a neural network instead of a table to estimate Q values.

Here are a few useful resources:

1. Q-Learning with Tables (Guide)
2. Q-learning jupyter notebook (Code ~25 lines)
3. Q-Learning with Frozen-Lake and Taxi (Code)
4. Reinforcement Learning with Q-Learning (Guide)

A multi-armed bandit would also be great in introducing you to exploration-exploitation trade-off, though it wouldn't be considered a full RL algorithm since it has no context.