Section 2.4.1 line of reasoning

[nlyu1]

I fail to see how (the Bellman optimality operator $\mathcal J^*$ is a $\gamma$-contraction map) follows from (the Bellman consistency operator $\mathcal J^\pi$ is a $\gamma$-contraction map).

The two operators have different expressions, and there's no reduction (maybe I'm wrong here) along the lines of $\mathcal J^*(v) = \mathcal J^{\pi(g(v))}$ for some $\pi$.

The convergence of VI is still guaranteed by a separate proof.

[adzcai]

Note that the Bellman optimality operator is equivalent to the Bellman consistency operator for the optimal policy that selects

$$\pi^\star(s) = \arg\max_a r(s, a) + \gamma \mathbb E[V^\star(s')].$$

Since the Bellman consistency equations hold for any policy, it also holds for the policy above, which gives the more explicit form in that expression.