Sayak Paul introduces Diffusers, a Python library for state-of-the-art video, image, and audio generation, highlighting its optimization with torch.compile for performance benefits like offloading, LoRA, and quantization. Sayak Paul and Animesh Jain discuss the strategic application of torch.compile to avoid graph breaks, advocating for regional compilation to reduce initial compilation time significantly while maintaining speedups. The main talking points include discussions on dynamic shapes and `mark_dynamic` for optimizing compilation with varying resolutions, handling LoRA adapters without recompilations, and ensuring memory optimization with quantization and offloading while maintaining torch.compile compatibility.
Links:
https://pytorch.org/blog/torch-compile-and-diffusers-a-hands-on-guide-to-peak-performance/
https://torchcompile-guards.hashnode.dev/inside-torchcompile-guards-how-they-work-what-they-cost-and-ways-to-optimize
https://docs.pytorch.org/tutorials/recipes/regional_compilation.html
Speaker: Sayak Paul
Sayak works on diffusion models at Hugging Face. Sayak has worked in different areas of machine learning (such as training and deploying BERT-sized language models1, scalable MLOps architectures2, model robustness3, etc.).
PyTorch Compiler Series
In this video series, watch the PyTorch Compiler team share tips and tricks that help you get the max out of torch.compile, torch.export, and related technologies, while enjoying a glimpse into all the cool engineering work that goes on behind the scenes.