verl: An Open Source Large Scale LLM RL Framework for Agentic Tasks - Yuxuan Tong, Bytedance
Recent advances in reinforcement learning significantly boosts the reasoning capabilities of LLMs. Models such as OpenAI o3, DeepSeek r1, etc,. demonstrates magnificent performance in STEM and coding tasks. Yet, training such models requires complex infrastructures.
In this talk, we present verl (https://github.com/volcengine/verl), a comprehensive framework that utilizes HybridFlow programming abstraction to achieve both flexibility to implement various algorithms and high performance. verl has been adopted by various universities and companies for RL training, and is contributed by 100+ contributors from the community.
Through this talk, audiences will gain i) a basic understanding of various RL algorithms including GRPO; ii) best practices to implement tool calling and multi-turn rollout for agentic tasks, as well vision language model reasoning; iii) latest large scale performance optimization techniques for RL with MOE models such as DeepSeek v3.