Reinforcement Learning and Optimal Control (Only offered in the Fall)

Important: this course is only offered in the Fall. Next: Fall 2021

This course (IFT6760C) is intended for advanced graduate students with a good background in machine learning, mathematics, operations research or statistics. You can register to IFT6760C on Synchro if your affiliation is with UdeM, or via the CREPUQ if you are from another institution. Due to the research-oriented nature of this class, you need to be comfortable with a teaching format involving open-ended questions and assignments. You will be required to think critically and adopt an open mindset. My teaching goal with this course is for all the participants to build their own understanding of reinforcement learning in relation to their primary research area while sharing their unique perspective and insights with the entire class.

Sujets/Topics

Processus de décision markovien, formulation sous forme de programme linéaire, forme lisse des équations de Bellman, équations de Bellman projettées, analyse des algorithmes de type TD, estimation de dérivées, commande optimale en temps continu et discret, principe du maximum de Pontryagin, Hamiltonien en temps discret et en temps continu, méthode par état adjoint et méthode variationelle pour le calcul de sensibilité, méthodes de contrôle directes et indirectes, apprentissage par renforcement inversé, et plus!

Markov Decision Processes, LP formulation, occupation measure, smooth bellman equations, projected bellman equations, analysis of TD algorithms, derivative estimation, discrete and continuous optimal control, Pontryagin maximum principle, discrete and continuous time Hamiltonian, adjoint and forward sensitivity equations, single shooting, multiple shooting, collocation methods, inverse reinforcement learning, and more!

Evaluation

There is no mandatory textbook. I will however be referencing content from:

Wellbeing