Reinforcement Learning an introduction by Sutton & Barto

What's this all about?

This repo is my playground for diving deep into the world of Reinforcement Learning (RL). I'm following the legendary book "Reinforcement Learning: An Introduction" by Richard S. Sutton and Andrew G. Barto.

How's it organized?

I'm breaking down each chapter of the book into three Jupyter Notebooks:

1. Notions: This is where I summarize the key concepts, definitions, and algorithms presented in the chapter. Think of it as my own condensed version of the book's text.
2. Exercises: Here's where I tackle the exercises at the end of each chapter. I try to provide clear explanations and code implementations for the solutions.
3. Quiz: This is for me to test my own understanding. I come up with my own questions to quiz myself on the chapter's material.

Dependencies:

Create a virtual env with python and install:

pip install numpy gymnasium plotly nbformat numba

Disclaimer:

This repo is still under construction. Expect updates, improvements, and maybe even some bugs along the way. If you find some mistakes, please open an issue, I'll be glad to fix it.

Table of Contents

1. Introduction
   • What is reinforcement learning ?
   • How reinforcement learning differ from other type of machine learning ?
   • Elements of Reinforcement Learning
2. Multi-Armed Bandits
3. Finite Markov Decision Processes

Who is this for?

• Fellow RL enthusiasts: Feel free to explore, learn, and maybe even contribute!
• Future me: A handy reference for when I need to revisit a specific concept or algorithm.

Why Notebooks?

Notebooks are awesome for RL! They let me:

• Experiment: Easily try out different hyperparameters, algorithms, and environments.
• Visualize: Plot results, visualize policies, and gain insights into the learning process.
• Document: Keep track of my thoughts, code, and the learning journey itself.

Let's learn some RL!

I hope this repo is helpful for anyone looking to learn more about Reinforcement Learning. Feel free to reach out if you have any questions or suggestions.

More Ressources

Also, if you are realy interested into reinforcement learning, you should consider these ressources:

books

• Reinforcement Learning An Introduction second edition
  by Richard S. Sutton and Andrew G. Barto.
• Grokking Deep Reinforcement Learning
  by Miguel Morales
• Multi-Agent Reinforcement Learning: Foundations and Modern Approaches
  by Stefano V. Albrecht, Filippos Christianos, Lukas Schäfer

Websites

• OpenAI Spinnning Up
• Berkeley’s Deep RL Bootcamp
• Berkeley’s Deep Reinforcement Learning course
• David Silver's Course
• Denny Britz's Github
• Lilian's blog
• Pytorch's tutorial
• Pytorch lightning's tutorial