Scaling RL with verl on AMD Instinct MI355X: Async Walkthrough and Sync Benchmark

Reinforcement learning (RL) for large language models (LLMs) alternates between two phases: generation (rollout), where the current policy produces responses, and training, where those responses are used to update the policy. In verl, the key design choices are when these phases run relative to each other (synchronously or with overlap) and where they run (colocated on the same GPUs or on separate GPU pools). This blog first explains the differences between the two modes and when to use each.