2
$\begingroup$

What is the difference between reinforcement learning (RL) and evolutionary algorithms (EA)?

For some problems, you could presumably co-evolve two "species" populations using evolutionary algorithms to achieve similar results to RL, although you would have the problem of calculating species1 * species2 at each epoch and it is obviously less direct than an RL would be?

I am trying to understand the basics of RL but do not have practical experience of using them, I know slightly more about EAs, thus the reason for requesting a comparison between them.

$\endgroup$
  • $\begingroup$ Hi and welcome to AI SE! I would replace your first sentence "Is reinforcement learning (RL) more powerful than an evolutionary algorithm (EA) in some way?" with the less provocative question that you have in the title, i.e. "What is the difference between reinforcement learning and evolutionary algorithms?", also because "powerful" is not well defined. $\endgroup$ – nbro Jun 2 at 16:31
  • $\begingroup$ What do you mean by "calculating species1 * species2 at each epoch"? $\endgroup$ – nbro Jun 2 at 16:35
  • $\begingroup$ Calculating the fitness of each of species1 population against the fitness of each of species2 population (ignoring that there may be quicker ways of doing this instead of naive O(nm)). $\endgroup$ – Single Malt Jun 2 at 16:38
2
$\begingroup$

Evolutionary algorithms (EAs) are a family of algorithms inspired by the biological evolution that can be used to solve (constrained or not) optimization problems where the function that needs to be optimized does not necessarily need to be differentiable (or satisfy any strong constraint). In EAs, you typically only need to define

  • an encoding of the solution (aka chromosome or individual)
  • a fitness function that determines the relative quality of each solution
  • operations that stochastically change or combine solutions (e.g. the cross-over or the mutation operators, in genetic algorithms)

There are other parameters that you need to defined (such as the number of solutions to consider at each generation or the number of generations to run the algorithms for), but these are three most important things to take into account when attempting to solve an optimization problem with EAs (in particular, GAs).

Reinforcement learning (RL) is the field that studies how agents can sequentially take actions in a certain environment in order to maximize some notion of long-term reward (aka return). The strategy that determines the behavior of the agent (i.e. which actions the agent takes) is called the policy. So, the goal of RL is to find a policy that maximizes the (expected) return, which depends on the reward function of the environment. For example, in the case of chess, a reward function may be any function that gives you a positive number if you win the game or a negative number if you lose it. The RL algorithms typically assume that the agent is able to interact with the environment in order to understand its dynamics.

RL is thus concerned with a specific type of optimization problem, i.e. finding policies (strategies) that maximize the return, while an agent interacts with an environment in time steps. On the other hand, EAs can be applied to any optimization problem where you can encode solutions, define a fitness function that compares solutions and you can stochastically change those solutions. Essentially, EAs can be applied to almost any optimization problem. In principle, you could use EAs to find policies, as long as you're able to compare them with a fitness function (e.g. the amount of reward that you obtain by following these policies).

Of course, this does not mean that EAs are the most efficient and appropriate approach to solve all optimization problems! You typically use EAs when you need to solve certain problems where better approaches do not exist. For example, when your objective function is not differentiable, then you cannot apply gradient-based solutions, so, in that case, EAs may be a viable option (but there are also other alternatives to EAs, such as simulated annealing).

| improve this answer | |
$\endgroup$
  • $\begingroup$ This is useful. On the chess example, the feedback of a win or loss comes at the end of the game - is a more immediate feedback of say capturing a piece also just as valid? In terms of the word “reinforcement”, is it the agent that is being “reinforced”? If so, how is it being reinforced? Finally, in a given implementation are there often (or mainly in two player game contexts) two agents, one to create policy and one to evaluate the created policy? $\endgroup$ – Single Malt Jun 3 at 7:54
  • $\begingroup$ @SingleMalt My example of chess is really simplified and just to give you the intuition, so, yes, you could definitely have rewards after each action. Please, ask the other questions in their separate post (one question per post). If you want, then I will answer them. $\endgroup$ – nbro Jun 3 at 9:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.