What introductory books to reinforcement learning do you know, and how do they approach this topic?

Question

Currently, I'm only going through these two books

• Reinforcement Learning: An Introduction, by Sutton and Barto: RL explained on an engineering level (mathematical, but readable for a non-mathematician). Elementary notions from probability and statistics are required (conditional probability, total probability theorem, total expectation theorem, and similar. The MIT RES.6-012 "Introduction to Probability" course is a great source of information for these topics.).

• Deep Reinforcement Learning, by Miguel Morales: this book introduces the main elements of reinforcement learning in a less formal way than Sutton and Barto (derivations for some equations are not given), using examples to describe the math.

What other introductory books to reinforcement learning do you know, and how do they approach this topic?

Answer 1

In addition to the ones you mentioned, I would add Algorithms of Reinforcement Learning by Csaba Szepesvári. There is a number of professors who use it as a reference in their RL teaching materials (for example this one).

It generally follows the same outline as Sutton & Barto's book (except the part on bandits, it is included in the Chapter on Control). In fact, it may be considered as a condensed version of Sutton & Barto (about 100 pages). In addition, it's freely available online.

I like the author's justification as to why he wrote this book, so I'm just going to quote it:

Why did I write this book? Good question! There exist a good number of really great books on Reinforcement Learning. So why a new book? I had selfish reasons: I wanted a short book, which nevertheless contained the major ideas underlying state-of-the-art RL algorithms (back in 2010), a discussion of their relative strengths and weaknesses, with hints on what is known (and not known, but would be good to know) about these algorithms.

Answer 2

The (draft) book Reinforcement Learning: Theory and Algorithms, by Sham M. Kakade (who published a natural policy gradient algorithm and other important research) and others, introduces RL in a mathematical/formal way. It seems to me that this is a reliable book, but a bit advanced for "regular people". Yes, I know the question was about introductory books on RL, but this may be suitable for people that have a solid knowledge of math and would like a hardcore intro to RL. For example, the book starts with a non-trivial (in my view) proof that there exists an optimal stationary and deterministic policy for an MDP.

Answer 3

Foundations of Deep Reinforcement Learning: Theory and Practice in Python (Addison-Wesley Data & Analytics Series) 1st Edition

This book does not give a detailed background information on Markov Decision Processes, different Bellman equations and relationships between the value function and action-value function, etc. It focuses on Deep Reinforcement Learning and goes straight to Policy and Value - based algorithms using neural networks. It might be good for someone trying to quickly understand what Deep RL algorithms are out there and apply them.