Receding-Horizon Control

An open-loop trajectory specifies controls from an initial state to a final time. During execution, the control $\mathbf u_t$ depends only on the clock, reading precomputed values or interpolating between them. The MPPI examples already used a different execution rule: apply one action, observe the new state, and update the remaining plan. Receding-horizon control applies this rule to any suitable finite-horizon optimizer, including the transcription methods developed in the preceding chapters.

Real systems face modeling errors, external disturbances, and measurement noise that accumulate over time. A precomputed trajectory becomes increasingly irrelevant as these perturbations push the actual system state away from the predicted path. The solution is to incorporate feedback, making control decisions that respond to the current state rather than blindly following a predetermined schedule. While dynamic programming provides the theoretical framework for deriving feedback policies through value functions and Bellman equations, there exists a more direct approach that leverages the trajectory optimization methods already developed.

Can the finite-horizon planner itself become a feedback controller by solving again whenever a new state is measured?

Closing the Loop by Replanning¶

Which action from each finite-horizon solution should be applied before the state is measured and the horizon is shifted forward?

Model Predictive Control creates a feedback controller by repeatedly solving trajectory optimization problems. Rather than computing a single trajectory for the entire task duration, MPC solves a finite-horizon problem at each time step, starting from the current measured state. The controller then applies only the first control action from this solution before repeating the entire process. This strategy transforms any trajectory optimization method into a feedback controller.

The battery example in Model Interfaces used a transparent current governor: it tested a local voltage and thermal envelope after updating one resistance parameter. MPC retains that structured model but optimizes an entire future current sequence and repeats the optimization as measurements arrive.

The Receding Horizon Principle¶

The defining characteristic of MPC is its receding horizon strategy. At each time step, the controller solves an optimization problem looking a fixed duration into the future, but this prediction window constantly moves forward in time. The horizon “recedes” because it always starts from the current time and extends forward by the same amount.

Consider the discrete-time optimal control problem in Bolza form:

This is the Bolza problem of Finite-Horizon Optimal Control Problems, with its index counted from the current time rather than from the start of the task. If the state is measured at time $t$, then $\mathbf x_0=\mathbf x_{\text{current}}$ is that measurement, $\mathbf x_k$ is the state predicted for time $t+k$, and the $N$ controls $\mathbf u_0,\ldots,\mathbf u_{N-1}$ span the prediction horizon, as in the MPPI algorithm. For a time-varying task, $c_k$, $\mathbf f_k$, and $\mathbf g_k$ are the stage cost, dynamics, and constraints of time $t+k$. The terminal cost $c_N$ is charged at the end of the prediction horizon, not at the end of the task.

At time step $t$, this problem optimizes over the interval $[t, t+N]$. At the next time step $t+1$, the horizon shifts to $[t+1, t+N+1]$. What makes this work is that only the first control $\mathbf{u}_0^\star$ from each optimization is applied. The remaining controls $\mathbf{u}_1^\star, \ldots, \mathbf{u}_{N-1}^\star$ are discarded, though they may initialize the next optimization through warm-starting.

This receding horizon principle enables feedback without computing an explicit policy. By constantly updating predictions based on current measurements, MPC naturally corrects for disturbances and model errors. The apparent waste of computing but not using most of the trajectory is actually the mechanism that provides robustness.

Horizon Selection and Problem Formulation¶

The choice of prediction horizon depends on the control objective. We distinguish between three cases, each requiring different mathematical formulations.

Infinite-Horizon Regulation¶

For stabilization problems where the system must operate indefinitely around an equilibrium, the dynamics, constraints, and stage cost are usually time-invariant, so we drop their index: $\mathbf f_k=\mathbf f$, $\mathbf g_k=\mathbf g$, and $c_k=c$. The true objective is:

Since this cannot be solved directly, MPC approximates it with:

The terminal cost $c_N(\mathbf{x}_N)$ approximates $\sum_{k=N}^{\infty} c(\mathbf{x}_k, \mathbf{u}_k)$, the cost-to-go beyond the horizon. The terminal constraint $\mathbf{x}_N \in \mathcal{X}_f$ ensures the state reaches a region where a known stabilizing controller exists. Without these terminal ingredients, the finite-horizon approximation may produce unstable behavior, as the controller ignores consequences beyond the horizon.

Finite-Duration Tasks¶

For a task ending at time $T$, the true objective from the current time $t$ is the remaining part of the Bolza objective, written in absolute time:

The MPC formulation must adapt as time progresses. While the end of the task lies beyond the prediction horizon, the subproblem keeps the surrogate terminal cost $c_N$. Once $t+N\geq T$, the horizon shrinks to the $T-t$ remaining steps:

In the second case, $\mathbf{x}_{T-t}$ is the prediction of the final state $\mathbf{x}_T$, so the task’s own terminal cost applies. As the task approaches completion, the horizon shrinks and the terminal cost switches from the approximation $c_N$ to the true final cost $c_T$. This prevents the controller from optimizing beyond task completion, which would produce meaningless or aggressive control actions.

Periodic Tasks¶

Some systems operate on repeating cycles where the optimal behavior depends on the time of day, week, or season. Consider a commercial building where heating costs are higher at night, electricity prices vary hourly, and occupancy patterns repeat daily. The MPC controller must account for these periodic patterns while planning over a finite horizon.

For tasks with period $T_p$, such as daily building operations, the formulation accounts for transitions across period boundaries:

The stage cost $c_k$ changes with the phase $\varphi_k$ of the predicted time $t+k$ within the period. Constraints may similarly depend on the phase, reflecting different operational requirements at different times.

The MPC Algorithm¶

The complete MPC procedure implements the receding horizon principle through repeated optimization:

Successive Linearization and Quadratic Approximations¶

The iLQR and DDP chapter develops successive approximation as a way to improve one finite-horizon plan. Such a solver can also be used inside MPC: solve from the newly observed state, execute the first action, and repeat. Its backward-pass feedback gains and the outer decision to replan are distinct operations.

For many regulation and tracking problems, the nonlinear dynamics and costs we encounter can be approximated locally by linear and quadratic functions. The basic idea is to linearize the system around the current operating point and approximate the cost with a quadratic form. This reduces each MPC subproblem to a quadratic program (QP), which can be solved reliably and very quickly using standard solvers.

Suppose the true dynamics are nonlinear,

Around a nominal trajectory $(\bar{\mathbf{x}}_k,\bar{\mathbf{u}}_k)$, we take a first-order expansion:

with the same Jacobians as in the iLQR derivation,

Similarly, if the stage cost is nonlinear,

we approximate it quadratically near the nominal point:

with positive semidefinite weighting matrices $Q_k$ and $R_k$, where $\|\mathbf{v}\|_{Q}^2=\mathbf{v}^\top Q\,\mathbf{v}$.

The resulting MPC subproblem has the form

where $\mathbf{d}_k = \mathbf{f}_k(\bar{\mathbf{x}}_k,\bar{\mathbf{u}}_k) - A_k \bar{\mathbf{x}}_k - B_k \bar{\mathbf{u}}_k$ captures the local affine offset. The terminal weight $P$ plays the role of the terminal cost $c_N$.

Because the dynamics are now linear and the cost quadratic, this optimization problem is a convex quadratic program. Quadratic programs are attractive in practice: they can be solved at kilohertz rates with mature numerical methods, making them the backbone of many real-time MPC implementations.

At each MPC step, the controller updates its linearization around the new operating point, constructs the local QP, and solves it. The process repeats, with the linear model and quadratic cost refreshed at every reoptimization. Despite the approximation, this yields a closed-loop controller that inherits the fast computation of QPs while retaining the ability to track trajectories of the underlying nonlinear system.

Theoretical Guarantees¶

Repeated optimization creates feedback, but which terminal ingredients make feasibility persist and the closed-loop state converge?

The finite-horizon approximation in MPC brings a new challenge: the controller cannot see consequences beyond the horizon. Without proper design, this myopia can destabilize even simple systems. The solution is to carefully encode information about the infinite-horizon problem into the finite-horizon optimization through its terminal conditions.

Before diving into the mathematics, we should first establish what “stability” means and which tasks these theoretical guarantees address, as the notion of stability varies significantly across different control objectives.

Stability Notions Across Control Tasks¶

The terminal conditions provide different types of guarantees depending on the control objective. For regulation problems, where the task is to drive the state to a fixed equilibrium $(\mathbf{x}_\mathrm{eq}, \mathbf{u}_\mathrm{eq})$ (often shifted to the origin), the stability guarantee is asymptotic stability: starting sufficiently close to the equilibrium, the closed-loop state satisfies $\mathbf{x}_t \to \mathbf{x}_\mathrm{eq}$ as $t\to\infty$ while constraints remain satisfied throughout the trajectory (recursive feasibility). This requires the stage cost $c(\mathbf{x},\mathbf{u})$ to be positive definite in the deviation from equilibrium.

When tracking a constant setpoint, the task becomes following a constant reference $(\mathbf{x}^\mathrm{ref},\mathbf{u}^\mathrm{ref})$ that solves the steady-state equations. This problem is handled by working in error coordinates $\tilde{\mathbf{x}}=\mathbf{x}-\mathbf{x}^\mathrm{ref}$ and $\tilde{\mathbf{u}}=\mathbf{u}-\mathbf{u}^\mathrm{ref}$, transforming the tracking problem into a regulation problem for the error system. The stability guarantee becomes asymptotic tracking, meaning $\tilde{\mathbf{x}}_t \to \mathbf{0}$, again with recursive feasibility.

The terminal conditions we discuss below primarily address regulation and constant reference tracking. Time-varying tracking and economic MPC require additional techniques such as tube MPC and dissipativity theory.

MPC with Stability Guarantees¶

To provide theoretical guarantees, the finite-horizon MPC problem is augmented with three interconnected components. The terminal cost $c_N(\mathbf{x})$ approximates the cost-to-go beyond the horizon, providing a surrogate for the infinite-horizon tail that cannot be explicitly optimized. The terminal constraint set $\mathcal{X}_f$ defines a region where we have local knowledge of how to stabilize the system. Finally, the terminal controller $\pi_f(\mathbf{x})$ is a local feedback policy that stabilizes the system within $\mathcal{X}_f$.

These components must satisfy specific compatibility conditions to provide theoretical guarantees:

Suboptimality Bounds¶

The finite-horizon MPC value $V_N(\mathbf{x})$ provides an upper bound approximation of the true infinite-horizon value $V_\infty(\mathbf{x})$. Understanding how close this approximation can be tells us about the effectiveness of short-horizon MPC.

The upper bound $V_N(\mathbf{x}) \geq V_\infty(\mathbf{x})$ follows immediately from the fact that MPC considers fewer control choices. The infinite-horizon controller can choose any sequence $(\mathbf{u}_0, \mathbf{u}_1, \mathbf{u}_2, \ldots)$, while the $N$-horizon controller is restricted to sequences of the form $(\mathbf{u}_0, \ldots, \mathbf{u}_{N-1}, \pi_f(\mathbf{x}_N), \pi_f(\mathbf{x}_{N+1}), \ldots)$ where the tail follows the fixed terminal controller. Since the infinite-horizon problem optimizes over a larger feasible set, its optimal value cannot exceed that of the finite-horizon problem.

Deriving the Approximation Error¶

The interesting question is bounding the approximation error $\varepsilon_N = V_N(\mathbf{x}) - V_\infty(\mathbf{x})$. This error represents the cost of being forced to use $\pi_f$ beyond the horizon rather than continuing to optimize.

Let $(\mathbf{u}_0^\star, \mathbf{u}_1^\star, \ldots)$ denote the infinite-horizon optimal control sequence with corresponding state trajectory $(\mathbf{x}_0^\star, \mathbf{x}_1^\star, \ldots)$ where $\mathbf{x}_0^\star = \mathbf{x}$. The infinite-horizon cost is:

Now consider what happens when we truncate this optimal sequence at horizon $N$ and continue with the terminal controller. The cost becomes:

where $c_N(\mathbf{x}_N^\star)$ approximates the tail cost $\sum_{k=N}^{\infty} c(\mathbf{x}_k^\star, \mathbf{u}_k^\star)$.

Since $V_N(\mathbf{x})$ is the optimal $N$-horizon cost (which may do better than this particular truncated sequence), we have $V_N(\mathbf{x}) \leq \tilde{V}_N(\mathbf{x})$. The approximation error therefore satisfies:

This bound shows that the approximation error depends on how well the terminal cost $c_N$ approximates the true tail cost along the infinite-horizon optimal trajectory.

Summary and Outlook¶

Receding-horizon control turns a finite-horizon optimizer into feedback by reinitializing it from each measured state and applying only the first planned action. Terminal costs, terminal sets, and invariant local controllers connect the truncated problem to recursive feasibility and stability.

The basic loop leaves several design choices unresolved. How should the same replanning mechanism represent tracking, economic objectives, uncertainty, hybrid decisions, solver failures, and hard real-time deadlines? MPC variants and reliable operation organize those choices.

Self-checks¶