StarAi: Deep Reinforcement Learning Course

 

Welcome to the StarAi Deep Reinforcement Learning course

 

The goal of this course is two fold:

  • Most RL courses come at the material from a highly mathematical approach. We aim to explain essential Reinforcement Learning concepts such as value based methods using a fundamentally human tool - stories.
  • We believe what you cannot create, you do not understand. We have provided easy to use exercises, with answers, to reinforce your learning.

 

We have built the course in such a way that each weeks content builds upon the next, eventually culminating with Deepmind’s PySC2 Starcraft 2 interface. The Starcraft content can be found in Lecture 8.

 

Maximise Your Learning.

To maximise your learning, we recommend you form a study group(s) in your local area and cover the course contents over a period of 6 weeks.

For inspiration, check out what we did last year here.

 

Exercise Technology.

All exercises are executed in Google Colaboratory and run “in browser” making them hardware independent. All you need is a Google account and either Firefox or Google Chrome Browsers.

 

How to do the exercises.

Watch the lecture material, then have an attempt at the exercises.

We highly recommend you only look at the answers after spending significant time trying to solve the exercise material.

 

En Taro Adun!

Way back in April 2018! -an actual epoch in the machine learning world!- we set out to create this course to help speed up the development of advanced machine learning techniques through a fun platform - mainly - Starcraft 2, hence the name “StarAi”.

For a long time Starcraft has been considered within the machine learning community to be the next “Grand Challenge” for Artificial Intelligence due to several properties of the game including very high state & action spaces, partial observability & multi agent gameplay.

This year Deepmind made significant progress with regards to solving the game. For all intents and purposes, removing the “human limitations” restrictions currently emposed on Deepmind’s system, the game can be said to be solved and am sure we will see even further progress towards the end of the year from Deepmind’s Starcraft team akin to AlphaGo Zero.

Why release the course then? We believe that the way we taught the content - through stories - is unique, and a fun way to learn advanced machine learning & reinforcement learning techniques in a “human” way. We would also be honoured if someone out there takes inspiration from this course that eventually leads to new breakthroughs!

In addition, we believe that RL will have massive impact in the real world. That is the subject of the next section!

For those of you only interested in the Starcraft element of StarAi please refer to the content of week 6 by clicking this link.

 

Reinforcement Learning in the Wild.

Outside of games however, we are beginning to see more use cases of Reinforcement Learning in the wild.

Including web store product recommendations, news recommendation, in the finance industry , Automated Architecture search for Machine Learning using RL ,End to End Machine Learning for Self Driving Cars, and last and most excitingly the field of Robotics where Deep Reinforcement Learning is exploding!

We believe that RL will start to play an ever increasing role where automated decision making is required in an end to end manner.

 

Additional Learning Material

For those technically inclined, or if you would like to dive deeper we have provided links to relevant chapters in Sutton & Bartos’s excellent text book, “Reinforcement Learning: An Introduction” for each week’s material.

 

StarAi is Beta

StarAi was entirely created by volunteer experts of their respective fields here in Sydney, Australia.

Whilst we did our best to ensure the content is as polished as it could be, we admit bugs exist both logical and otherwise.

It has also been some time since StarAi was run and some of the exercise content might be out of date (tensorflow 2.0 etc). We would welcome and feedback good or bad. Potential updates to our exercises & codebase would also be much appreciated.

If you would like to provide feedback or help in any way please reach out at contact@starai.io

 

Disclaimer

Your use of the StarAi Reinforcement Learning course acknowledges that you have read this Disclaimer.

The course is provided “as is” without warranty of any kind. We assume no responsibility for the use of this course material and we therefore exclude all liability.

StarAi is a non-commericial entity. StarAi is not associated associated with Blizzard Entertainment, Google Inc., Microsoft Pty Ltd, Deepmind Technologies Limited, OpenAI LP or any other legal entity mentioned though out the duration of the StarAi course. Any associated trademarks are the property of their respective owners.

All rights in and to intellectual property rights, including without limitation, copyright or other proprietary rights, belong to their respective owners. Except where expressly stated otherwise or if such use is expressly licensed, these Terms of Use are governed by Australian law and through international treaties, applicable law in other countries. You hereby acknowledge that you are solely responsible for compliance with those terms, rules, regulations, and notices pertaining to the use of StarAi and the operation of the services provided thereon.

This Disclaimer was written, in part, by GPT2 :)

 

Credits

Opening Cinematic in part created by Upheaval ArtsGaming