On Adversarial RL [R]

Zhang et al. paper's introducing the SA-MDP framework (2020) (state adversarial MDP) argues that an attack using the critic network (V(s)) is expected and supposed to produce a weaker attack than an attack using the actor network (pi(s)) itself to generate perturbation on agent observations. A claim supported by their empirical results using different single-agent simulation environment. However, I'm consistently finding the opposite when comparing both attacks on multi-agent PPO policies trained on some sc