RL & Control

IFT6162 · Fall 2026 · Final project

Pick a real system worth controlling

Take a system from the engineering literature, implement the controller its operators would use, then show where learning-based control does better. You present it at a poster session on December 3, with a small website that lets people play with your results.

Find my project Browse all topics
disturbances w: weather, prices, demand Real system x⁺ = f(x, u, w) state x action u learned model, value or policy data Your controller baseline, then learning
Every project closes this loop on a real system.

The project in one minute

  1. Pick a real system. Start from a paper in the control or domain literature that gives you a model or data, and numbers to compare against. No Atari, MuJoCo, Gym or toy robotics.

  2. Write it as a control problem. Specify what the policy observes, what it can change and when, and how actions affect later states or information: xt+1 = f(xt, ut, wt). Define a horizon objective and constraints. Explain why solving each decision independently would miss something.

  3. Implement a baseline. The published controller, a PID, a rule, or a myopic policy. Compare policies with the same observations and constraints. A perfect-foresight schedule is a hindsight bound.

  4. Let learning do real work. A learned model inside MPC, a value function, a policy amortized from a solver, offline RL from logs, a forecast trained for the decision. Evaluate the decisions it changes over full trajectories; prediction accuracy or a one-off ranking alone is not enough.

  5. Show we can trust your numbers. Check units, limiting cases and a reproduced result, then compare horizon cost and constraint violations on unseen conditions. Logged outcomes alone cannot score actions that were never taken: use a justified action-response simulator or defensible off-policy evaluation, and state the assumptions.

Team
Exactly three students; any exception is arranged with the instructor. Mix backgrounds, and pick a field you don't already work in. Once you're set, send us your team and topic.
Proposal
One page by email, ungraded, any time before the poster session. The earlier you send it, the more feedback you get.
Poster
Thursday, December 3, 13:30–15:30 at Mila. In person; it cannot be done late.
Hand in
A code repository with a README and tests, a one-page project website with at least one interactive figure, the poster, and an oral defense at the poster.
Grading
30% of the course grade.
Real-world grounding 25Formulation 25Technical quality 25Creativity 15Communication: poster, website, pitch 10
AI tools
Allowed. Every member must be able to defend every claim, line of code and plot.

Problems that matter come first. We have a strong preference for climate mitigation, energy, industry, water, food and health. For the big picture, read Tackling Climate Change with Machine Learning (Rolnick et al., ACM Computing Surveys, 2022).

Poster checklist
  • One sentence states your main result; everything on the poster supports it.
  • Blocks for the problem, the model, the method, results with verification, and limitations.
  • One system diagram showing states, controls, disturbances and constraints.
  • Two to four plots, including your method against the baseline, each with a caption that states the conclusion.
  • 400 to 700 words in total. Above 800 you are writing a paper.
  • A QR code to your project website.
  • If your topic is marked Startup potential, say who would pay for your controller and roughly what it would save them.
  • Print a draft on Letter paper. If it looks like a wall of text, cut.
Project website
  • One page, published from your repository (GitHub Pages is enough) and linked from a QR code on the poster.
  • At least one interactive figure that explains your core idea: let the visitor change a disturbance, a horizon or a cost weight and watch the closed loop respond.
  • Drive it with your real results. Precomputed traces from your runs are fine; a cartoon is not.
  • Show the baseline next to your method, and how you checked the numbers.
  • Keep it small: a few days of work. Content beats polish, and you must be able to explain every number it shows.
  • For the genre, see the demos in the book, such as the thermoacoustic pull-down in Iterative Trajectory Optimization or the aircraft replay in Model Predictive Path Integral Control.
What last year's teams built
  • Ship routing in ice-covered waters with RL and control
  • Incentive-based demand response for residential heating
  • Wind energy with battery storage
  • Hybrid RL and MPC for building HVAC in BOPTEST
  • Crane motion for container loading
  • Energy deployment in Formula 1 for lap time
  • Conservation planning for California condor recovery
  • Koopman models and convex MPC for portfolio rebalancing

Find your project

Five questions about what you care about and what you want to learn. Each answer is a measurement; when the loop closes, you get three matches and one detour.

Five measurements to close the loop.

All topics

Each topic spells out the state, decisions, consequences over time, objective and constraints, with papers, starting resources and a baseline. Some require a model extension, described in their project scope. Topics marked Startup potential suggest an operator and a practical value to test. Bring your own system if it meets the brief.

Loading topics…

Tell us your team and project

Once your team has settled on a topic, send this short email so we know who is working on what. It is not the proposal; that comes later, once you have something running.

Team
1
2
3
Write the email in
To
pierre-luc.bacon@mila.quebec
Cc
Subject

          

            
            
            
            
          

        

        
Paste it into a new email to pierre-luc.bacon@mila.quebec with your teammates in copy, or open it in your mail app. What you type here stays in this browser.

      

    

  







  
Topic