Optimizing Multi-Stage AI Pipelines: Prefix Caching for Faster Autoregressive Inference // TRAIN BRAIN

Optimizing Multi-Stage AI Pipelines: Prefix Caching for Faster Autoregressive Inference

At PyTorch Conference North America, Ricardo Noriega de Soto, Tech Lead for the vLLM Omni team and Alexander Brooks, Principal Machine Learning Engineer at Red Hat, will demonstrate how extending vLLM’s prefix caching mechanism to multistage pipelines boosts inference speeds while reducing GPU memory overhead.
Join us in San Jose on October 20th to learn practical strategies for optimizing complex AI workloads: https://hubs.la/Q04v4SL60

PyTorch

Welcome to the official PyTorch YouTube Channel. Learn about the latest PyTorch tutorials, new, and more. PyTorch is an open source machine learning framework that is used by both researchers and developers to build, train, and deploy ML systems that so...

Scaling ExecuTorch & Disaggregating LLM Serving

A Tiered Cross-Repository CI Relay for Out-of-Tree Backends

Retrieval as Inference: Building a Production-Scale Torch-Native Retrieval Engine with Dhritiman Das

Federated Learning for Medical Imaging at PyTorch Conference 2026

PyTorch Con: Where Open Innovation in AI Happens

Empowering developers through collaboration, choice and open contributions

PyTorch can extend language modeling beyond classification to practical time-to-event prediction

Optimizing LLM Inference: Disaggregated Serving, PD Protocol, & KV Pinning

AI Agents for LLM Inference Runtimes on Edge Hardware

Optimizing Multi-Stage AI Pipelines: Prefix Caching for Faster Autoregressive Inference

Why PyTorch Foundation Ambassador Arun Bhandari thinks you should attend PyTorch Conference NA 2026

Building high-performance recommendation inference systems using PyTorch

Learn how PyreFly enables end-to-end static shape coverage for PyTorch models.

Learn how to grow a local PyTorch community at PyTorch Conference North America 2026.

Richard Liaw (Ray) PyTorch Conference North America 2025

Primus Tuning Hybrid Projection and Agentic Search for Distributed Training

PyTorch Conferences are where the open source AI community gathers to decide what's next

PyTorch Conference Europe 2026, Niles Burbank (AMD)

Meet the people behind the open source tools you use every day at PyTorchCon North America

Efficient Pretraining of LLMs in NVFP4 at PyTorchCon

Tackle the systems challenges of Agentic RL at PyTorchCon North America

PyTorch Conference 2025: Christian Jacobi (IBM)

PyTorch Conference 2025: Simon Mo (vLLM, Inferact)

PyTorch Conference 2025 Jana van Greunen (Meta)

Learn how to train more with less at PyTorch Conference North America 2026 this October

PyTorch Conference Europe 2026 – Official Highlights | Paris

PyTorch Conference brings together so many of the smartest people in the world

PyTorch Conference 2025 Tristan Rice (Meta)

PyTorch Conference North America 2025

What PyTorch Conference Europe 2026 Was Really Like – Official PyTorchCon EU Highlights | Paris

Why WideEP Inference Needs Data-Parallel-Aware Scheduling - Maroon Ayoub & Tyler Michael Smith

Lightning Talk: How DeepInverse Is Solving Imaging in Science and H... Andrew Wang & Minh Hai Nguyen

Write Once, Run Everywhere with Pytorch Transformers - Pedro Cuenca, Hugging Face

The Science and Practice of Open and Scalable LLM Evaluations - Grzegorz Chlebus, NVIDIA

Lightning Talk: Deep Learning in the Wild: Embedded PyTorch for... Taraqur Rahman & Owen O'Donnell

The Token Slice: Implementing Preemptive Scheduling Via Chunked Decod... Maroon Ayoub & Kellen Swain

Enabling State-of-the-art Asynchronous Execution in Torch.compile With CUDA Streams - Michael Lazos

Lightning Talk: ExecuTorch on Microcontrollers: Deploying PyTorch To... RJ Ascani & Matthias Cremon

Lightning Talk: TerraKit: Standardising AI-Ready Geospatial Data... Rosie Lickorish & Romeo Kienzler

Lightning Talk: Bayesian Neural Networks With Variational Inference in PyTorch - Lars Heyen

Lightning Talk: Why Your Forecasting Transformer Isn’t Working (And How To Fix It... Rosheen Naeem

Build PyTorch to Understand PyTorch - Vijay Janapa Reddi & Andrea Mattia Garavagno

Lightning Talk: Jigsaw: Domain and Tensor Parallelism for High-Resolution Inp... Deifilia Kieckhefen

Optimizing PyTorch on CPU-GPU Coherent Platforms - Matthias Jouanneaux, Nvidia

torch.compile and Diffusers: A Hands-On Guide to Peak Performance - Sayak Paul, Hugging Face

PyTorch Foundation Spotlight: Matt White

PyTorch Foundation Spotlight: Prabha Ganapathy (AWS)

PyTorch Foundation Spotlight: Christian Jacobi

PyTorch Foundation Spotlight: Simon Mo

PyTorch Foundation Spotlight: Richard Liaw

PyTorch Foundation Spotlight: Zhipeng Wang

PyTorch Foundation Spotlight: Joe Spisak

PyTorch Foundation Spotlight: Ankit Patel

PyTorch Foundation Spotlight: Red Hat

PyTorch Foundation Spotlight: Zesheng Zong PyTorch Ambassador

PyTorch Foundation Spotlight: Niles Burbank, AMD

PyTorch Foundation Welcomes Snowflake as a Premier Member

PyTorch Foundation Spotlight: Ankita Guha, PyTorch Ambassador

PyTorch Foundation Spotlight - Luca Antiga, Lightning AI

PyTorch Conference 2025 Recap

Behind Every AI SBOM Lies a Minefield: Here's How to Not Step... - Matthew Crawford & Jimmy Ahlberg

SPDX: Tackling System Risk in Modern Supply Chains - Kate Stewart & Gary O'Neall

The OpenMDW License Agreement: Simple, Permissive Terms for AI Model Materials - Steve Winslow

Sponsored Session: Lightning Talk: Accelerating Experimentation... - Niall Lyons & Luca Antiga

Using LlamaIndex to Build an Open-Source NotebookLM Alternative - Tuana Çelik, LlamaIndex

Our Journey With TorchTitan - Linsong Chu & Garrett Goon, IBM Research

Panel Discussion: The Future of Agents and Human-Agent Collaboration

PyTorch-Native Stack for Agents - Allen Wang & Davide Testuggine, Meta

Sponsored Session: Architecting AI: Efficient Models, Everywhere - Sharbani Roy, Arm

Sponsored Session: Lightning Talk: Optimizing Model Inference with PyTorch 2.0 - Devansh Ghatak

Keynote: Building an Open Agentic Future - Samuel Colvin, Founder, Pydantic

Open Source Model Performance Optimization With SGLang - Yineng Zhang, Together AI

Sponsored Session: Fine Tune Your AI Toolbox with GitHub Copilot, Azure AI Foundry... - Sarah Kaiser

Sponsored Session: Lightning Talk: Build and Deploy AI Flows with an Agent Factory - Arjun Upadhyay

Generating State-of-the-Art GEMMs for Heterogeneous Hardware with... - Michael Lazos & Henry Tsang

torch compile and Diffusers - A Hands On Guide to Peak Performance - PyTorch Compiler Series

verl: An Open Source Large Scale LLM RL Framework for Agentic Tasks - Yuxuan Tong, Bytedance

SGLang: An Efficient Open-Source Framework for Large-Scale LLM Serving - Liangsheng Yin

torch.accelerator: A Unified, Device-Agnostic Runtime API for Stream-Based Accelerators - Yu Guangye

vLLM: Easy, Fast, and Cheap LLM Serving for Everyone - Kaichao You, Tsinghua University