Applying T to every value function at once
Each dot is one value function v = (v(1), v(2)) of a two-state MDP. One press applies the Bellman optimality operator T to all of them. The dashed box is the smallest square holding the cloud: its side is the cloud's diameter in the sup norm.
Value-function plane, iteration k = 0
u
w
★ fixed point v*
faint dots: previous iterate
Distances after k applications (log scale)
cloud diameter
‖u − w‖∞
γk × start
Cloud diameter, now / start1.000
γk1.000
‖u − w‖∞–
‖u − v*‖∞–