What Helped Me Learn AI: A list of Online Resources

When I first started training a computer program to drive in Trackmania 5 years ago, I didn’t know much about AI. So over the years, I’ve had to continually teach myself about the field. A few months ago, I shared a post with some book recommendations that I found particularly helpful for learning about Machine Learning and Reinforcement Learning.

But in my experience, books aren’t everything: their content can sometimes be too broad or theoretical. With a specific project in mind (training an AI to drive in a racing game), I often found more useful insights in online resources like blog posts, research papers, and videos. This was especially true when working with reinforcement learning: the field is highly empirical, so looking at similar projects and experiments can be extremely valuable.

So, as promised, here’s a follow-up to my first recommendation post: a non-exhaustive list of online (and free!) resources :)

* Python Plays GTA V by Sentdex (https://www.youtube.com/watch?v=ks4MPfMq8aQ&list=PLQVvvaa0QuDeETZEOy4VdocT7TOjfSA8a)
In this video series, YouTuber Sentdex builds an AI to drive in the game GTA V using Supervised Learning. He goes over several practical details, like how to connect a Python program to a game, for example. This was especially useful to me when I trained my first AI in Trackmania, which also used Supervised Learning on screen captures of the game. Later on, I realized that Reinforcement Learning (RL) was more appropriate for my project, which is why all the following resources focus on RL.

* MarIQ by SethBling (https://www.youtube.com/watch?v=Tnu4O_xEmVk)
This is another YouTube video where an AI is trained to drive in a racing game (Mario Kart) using RL. It’s just one example among many, but I found it helpful for understanding RL and it was also great inspiration for my first videos.

* Two great talks about RL
These two talks helped me a lot in understanding good practices when working with RL:
- Nuts and Bolts of Deep RL experimentation - John Schulman (https://www.youtube.com/watch?v=8EcdaCk9KaQ)
- RL in practice: tips & tricks and practical session with stable-baselines3 - Antonin Raffin (first hour) (https://www.youtube.com/watch?v=Ikngt0_DXJg)

* RL course by Sergey Levine (CS 285 at UC Berkeley) (https://www.youtube.com/watch?v=SupFHGbytvA&list=PL_iWQOsE6TfVYGEGiAOMaOzzv41Jfm_Ps)
A full free university-level course that goes deep into RL concepts, theory, and practice. It’s not beginner-friendly, but it's valuable once you have the basics down.

* Research articles
Over the years, I’ve read a ton of RL papers, too many to list here ahah! If you’re just starting out, I recommend beginning with a few of the key papers in this field. You can find a good list here: https://spinningup.openai.com/en/latest/spinningup/keypapers.html
Then, you can continue with more specific stuff depending on your specific interests / problems. For example, here are two papers I found particularly useful to train an AI in a racing game:
-Super-human performance in Gran Turismo sport using deep reinforcement learning (Fuchs et al. 2019) https://ieeexplore.ieee.org/abstract/document/9372847
-Outracing champion Gran Turismo drivers with deep reinforcement learning (Wurman et al. 2022) https://www.nature.com/articles/s41586-021-04357-7

* RL libraries
To implement RL algorithms, and to understand the subtle differences between them, I found some popular RL libraries (and their documentation) really helpful. In particular:
- cleanrl https://docs.cleanrl.dev/
- stable-baseline 3 https://stable-baselines3.readthedocs.io/en/master/

* An interesting blog post about RL:
Lessons Learned Reproducing a Deep Reinforcement Learning Paper https://amid.fish/reproducing-deep-rl

Again, this list is far from exhaustive, but I hope it’ll be useful for some of you :)

And.. concerning the next video, it’s 80% done! I might have a few teasers to show you in the coming weeks..