https://github.com/alecwangcq/KFAC-Pytorch/blob/25e6dbe14752348d4f6030697b4b7f553ead2e92/optimizers/kfac.py#L137 The KL clipping should be $vFv^T$, here it seems to calculate $vF^{-1}v^T$.
KFAC-Pytorch/optimizers/kfac.py
Line 137 in 25e6dbe
The KL clipping should be$vFv^T$ , here it seems to calculate $vF^{-1}v^T$ .