算法1：求解HJI方程的多智能体策略迭代

要求：设 $u_{i k}^{0}, \forall i = 1, \dots, N$ 为任何稳定的初始控制策略。

$p = 0, 1, \dots$ ，给定 $u_{i k}^{p}, \forall i = 1, \dots, N$

重复p

$l = 0, 1, \dots$ ，给定 $w_{i k} = 0$

重复l

求解 $V_{i} (ε_{i (k + 1)}^{(p, l)})$ 使用零和贝尔曼方程

$\begin{array}{l} {(\frac{\partial V_{i}^{(p, l)} (ε_{i (k + 1)})}{\partial ε_{i (k + 1)}})}^{T} (A ε_{i k} + (d_{i} + g_{i}) B_{i} u_{i k}^{p} - \sum_{j \in N_{i}} a_{i j} B_{j} u_{j k} + (d_{i} + g_{i}) D_{i} w_{i k}^{l} - \sum_{j \in N_{i}} a_{i j} D_{j} w_{j k}) \\ + \frac{1}{2} (ε_{i k}^{T} Q_{i i} ε_{i k} + {(u_{i k}^{p})}^{T} R_{i i} u_{i k}^{p} + \sum_{j \in N_{i}} u_{j k}^{T} R_{i j} u_{j k} - γ^{2} {(w_{i k}^{l})}^{T} T_{i i} w_{i k}^{l} - γ^{2} \sum_{j \in N_{i}} w_{j k}^{T} T_{i j} w_{j k}) = 0 \end{array}$ (33)

更新 $w_{i k}^{l + 1}$

$w_{i k}^{l + 1} = \frac{1}{γ^{2}} (d_{i} + g_{i}) T_{i i}^{- 1} D_{i}^{T} \frac{\partial V_{i}^{(p, l)} (ε_{i (k + 1)})}{\partial ε_{i (k + 1)}}$ (34)

直到 $‖ V_{i} (ε_{i (k + 1)}^{(p, l + 1)}) - V_{i} (ε_{i (k + 1)}^{(p, l)}) ‖ \leq ε$ 收敛

结束

更新 $u_{i k}^{p + 1}$

$u_{i k}^{p + 1} = - (d_{i} + g_{i}) R_{i i}^{- 1} B_{i}^{T} \frac{\partial V_{i}^{(p, l)} (ε_{i (k + 1)})}{\partial ε_{i (k + 1)}}$ (35)

直到 $‖ V_{i} (ε_{i (k + 1)}^{(p + 1, l)}) - V_{i} (ε_{i (k + 1)}^{(p, l)}) ‖ \leq ε$ 收敛

结束