算法2：随机逼近Q-learning算法

1) 初始化矩阵使 $Q_{2} (0) = I_{d}$

2) for $t = 0, T$ :

3) $Q_{2} (t + 1) = Q_{2} (t) + α (N_{t + 1} + γ Λ_{t + 1}^{T} Π (Q_{2} (t)) Λ_{t + 1} - Q_{2} (t))$

4) $K_{0} (t + 1) = \frac{1}{1 - γ} Γ_{0} (Q_{2} (t + 1), 0)$

5) $t = t + 1$

6) end for