PyTorch 2.13 introduces updates across attention, compilation, distributed training, memory efficiency, Python support, and accelerator platforms. Highlights include FlexAttention support on Apple Silicon with up to approximately 12x speedup over SDPA on sparse patterns, the CuTeDSL "Native DSL" backend for key GPU operations, and nn.LinearCrossEntropyLoss to reduce peak GPU memory by up to 4x during large-vocabulary language model training. On Wednesday, July 22, 2026, at 11 a.m. PT, PyTorch maintainers and contributors will provide a brief overview of the PyTorch 2.13 release and answer questions from the community live. Topics will include: - FlexAttention support on Apple Silicon and deterministic backward computation on CUDA - The CuTeDSL "Native DSL" backend for Inductor - nn.LinearCrossEntropyLoss for reducing peak GPU memory - torchcomms for large-cluster training - FSDP2 communication overlap improvements - Torch wheel support for Python 3.15 on Linux, including free-threaded 3.15t builds - Expanded ROCm, Arm, and Intel XPU platform support The live Q&A will feature expert panelists Alban Desmaison, Andrey Talman, and Piotr Bialecki, with Chris Gottbrath moderating. PyTorch 2.13 includes 3,328 commits from 526 contributors since PyTorch 2.12. Register for the live Q&A today!

What was PyTorch Conference Europe 2026, held on April 7–8 in Paris, France, really like? Hear from attendees about their experiences. PyTorchCon EU featured inspiring keynotes, cutting-edge technical sessions, hands-on workshops, and vibrant discussions shaping the future of AI and deep learning. From major announcements and deep technical insights to powerful community moments and the unforgettable energy of Paris, here are the highlights from the first PyTorch Conference in Europe. Whether you joined us in person or are catching up now, we hope these moments inspire your next breakthrough. ► View all the session recordings here https://www.youtube.com/playlist?list=PL_lsbAsL_o2DZbCiISSjbucDOW0QOxwBF #PyTorchCon See you at PyTorch Conference China and PyTorch Conference North America later this year!

PyTorch 2.12 includes major updates across compilation, distributed systems, export, graph capture, and accelerator support. Highlights include a new device-agnostic torch.accelerator.Graph API, up to 100x faster batched eigenvalue decomposition on CUDA, support for microscaling quantization formats in torch.export.save, and expanded CUDA, ROCm, XPU, MPS, and Arm platform support. Join us on Wednesday, May 20 at 10:00 AM PT for a live Q&A with panelists Andrey Talman, Alban Desmaison, and Joe Spisak, moderated by Chris Gottbrath. The panel will provide a brief overview of the release and answer your questions live. Register today! Topics include: -Device-Agnostic Accelerator Graph Capture -ProcessGroup Support in Custom Ops -torch.export.save Support for Microscaling Quantization Formats -Fused Adagrad Optimizer Support -FlightRecorder Updates -Multi-GPU and Multi-Node Profiling Improvements -Updated Backend Selection for torch.linalg.eigh on CUDA -Expanded CUDA, ROCm, XPU, MPS, and Arm Platform Support Register today. Panelists: Andrey Talman is a Software Engineer at Meta, primarily focused on open source releases for PyTorch and its ecosystem libraries. He works on release management, continuous integration, and process improvements, ensuring high-quality and timely delivery of PyTorch and related projects. Alban Desmaison is a Research Engineer at Meta and the Lead Core Maintainer of PyTorch. Joe Spisak is Vice President of Product and Head of Open Source at Reflection AI. He is a PyTorch core maintainer, serves on the PyTorch Foundation Governing Board, and previously worked at Meta. Moderator: Chris Gottbrath is a Group Technical Program Manager supporting PyTorch at Meta and Chair of the PyTorch Foundation Marketing Committee.

The 2026 PyTorch Docathon will take place May 5 to May 19, a hackathon-style community event focused on improving PyTorch documentation. Whether you're new to open source or an experienced contributor, the Docathon offers opportunities to improve tutorials, guides, examples, and website content. Many issues are beginner-friendly, with tasks labeled by skill level and support available through the PyTorch Discord. Contributors can make an immediate impact while learning more about PyTorch modules, tutorials, and workflows. Schedule May 5 Kickoff and Q&A, 10:00 AM PT May 6 to May 15 Submissions and feedback May 16 to May 18 Final reviews May 20 Winner announcements RSVP to receive updates, participation details, and event instructions.

Lightning Talk: How DeepInverse Is Solving Imaging in Science and Healthcare With PyTorch - Andrew Wang, DeepInverse; Minh Hai Nguyen, Université de Toulouse Deep learning has revolutionised imaging, a foundation of science and healthcare. DeepInverse is the PyTorch library for solving imaging problems, unifying deep learning methods (e.g. diffusion models), physics (medical, optics) and modern tooling. In this talk, we’ll show how the PyTorch community can get involved in this exciting yet accessible application of open-source AI. AI methods in imaging must model the imaging physics, leading to interesting engineering problems e.g. efficient differentiable ops, physics-informed losses. We’ll show notebooks on real use-cases: accelerating brain MRI, reducing radiation in CT scans, imaging black holes. PyTorch enthusiasts at any level/background can contribute - from training infra for scientific data to high-level generative modelling frameworks - their AI engineering skills can directly impact imaging across multiple fields. DeepInverse is supported by a growing international user community and proudly rooted in Paris. We’ve joined the PyTorch Ecosystem and received the Prix Science Ouverte in 2024. We’re excited to join the PyTorch Conf to celebrate the vibrant French developer community!

Why WideEP Inference Needs Data-Parallel-Aware Scheduling - Maroon Ayoub, IBM; Tyler Michael Smith, Red Hat WideEP—wide expert parallelism fails not because experts are expensive, but because routing ignores where state already lives. In PyTorch LLM serving with vLLM, WideEP fans tokens across many experts while KV caches accumulate unevenly across data-parallel replicas. When routing is unaware of KV placement and per-replica load, requests land on replicas that cannot reuse cache or make progress efficiently and latency spikes as expert fan-out grows. The fix is not reshaping expert parallelism, but making routing data-parallel aware using signals vLLM already exposes. In this talk, we show how llm-d extends its router to leverage KV-cache locality and load awareness when routing WideEP flows. Rather than treating replicas as interchangeable, the router prefers replicas with warm KV state and available capacity, aligning routing decisions with vLLM’s execution reality and reducing cache fragmentation. This session walks through how KV-aware, data-parallel routing changes WideEP inference in practice: which signals matter, how routing behavior evolves, and where the gains come from. Attendees leave with a clear mental model for when KV- and load-aware routing unlocks higher throughput.

Write Once, Run Everywhere with Pytorch Transformers - Pedro Cuenca, Hugging Face The Hugging Face transformers library is built on pure PyTorch and can be succinctly described as a model-definition framework. It provides an unified, familiar, clear and concise interface to multiple machine learning architectures across modalities. Serving and inference optimizations are not its focus. However, transformers model definitions become the de-facto reference implementations multiple other projects use. This includes training libraries, fast deployment engines such as vLLM and SGLang, and on-device libraries like MLX and llama.cpp. This session describes the path towards increasingly simpler downstream integration of transformers models into inference and deployment libraries, and how transformers and PyTorch core features enable the ecosystem to enjoy newly-released models as soon as they are released. We'll go through the journey towards easier modeling, which implies easier downstream porting and adaptation. The end-game is pure interoperability, where no code changes are required! This is now possible with vLLM and SGLang, and we'll show how. We'll end up discussing our ideas on upcoming interop features with MLX and llama.cpp.

Build PyTorch to Understand PyTorch - Vijay Janapa Reddi, Harvard University; Andrea Mattia Garavagno, University of Genoa PyTorch's success depends on more than users—it needs engineers who understand what's inside. Engineers who can debug framework issues, optimize at the systems level, contribute upstream, and build what comes next. But ML education today produces practitioners who call APIs without understanding them. They train models without knowing why Adam needs 3× the memory of SGD, or what happens when they call loss.backward(). TinyTorch is a 20-module open-source curriculum that closes this gap. Students construct PyTorch's core components—tensors, autograd, optimizers, CNNs, transformers—in pure Python, building a complete framework where every operation is code they wrote. By the final module, they don't just use PyTorch; they understand how to build it. The curriculum uses progressive disclosure, systems-first profiling from Module 01, and build-to-validate milestones—recreating ML breakthroughs from Perceptron (1958) through Transformers (2017), culminating in MLPerf-style benchmarking. TinyTorch is how we grow the next generation of PyTorch contributors and the engineers who will build what comes after. Open source: mlsysbook.ai/tinytorch

Lightning Talk: Why Your Forecasting Transformer Isn’t Working (And How To Fix It in Python) - Rosheen Naeem, Open Climate Fix Renewable energy is clean — but it’s also inherently variable. Solar PV generation can change dramatically within minutes due to cloud cover and weather conditions, making accurate short-term forecasts essential for grid stability, energy trading, and smart-home optimisation. Open Climate Fix builds open and high-impact forecasting tools to accelerate the transition to a low-carbon energy system. One of these projects is Open Quartz Solar Forecast: an open-source model that uses public PV generation data, site metadata, and numerical weather prediction variables to forecast solar power for any location. In this talk, I’ll present a real case study from my Google Summer of Code project where I implemented and trained a Temporal Fusion Transformer for multi-horizon solar forecasting. I’ll cover the practical engineering challenges behind making transformer forecasting work in Python: building continuous training windows, aligning weather forecast steps with observations, separating static vs time-varying features, and stabilising training using PyTorch Forecasting and PyTorch Lightning. Attendees will leave with reusable patterns for real-world time-series forecasting pipelines.

Lightning Talk: Bayesian Neural Networks With Variational Inference in PyTorch - Lars Heyen, Karlsruhe Instute of Technology, Scientific Computing Center Uncertainty quantification is becoming more and more important as neural networks are used for increasingly critical tasks. Bayesian neural networks (BNNs) inherently provide a measure of their own uncertainty, but can be either hard to implement or inflexible if one uses common frameworks. In this session I discuss how to efficiently implement BNNs using Variational Inference within PyTorch and present torch_blue, a light-weight open source library that implements these methods with the goal of being easy to pick up, yet flexible enough for research on BNNs.

Lightning Talk: TerraKit: Standardising AI-Ready Geospatial Data Preparation for the TorchGeo Ecosystem - Rosie Lickorish & Romeo Kienzler, IBM With the advent of geospatial foundation models, unexplored use cases are emerging that require well-curated datasets. Currently, no standardised approach exists for creating such AI-ready geospatial datasets. In this session, we introduce TerraKit: a comprehensive open-source Python library for retrieving, and processing geospatial data, that seamlessly integrates with upstream geospatial model training libraries such as TorchGeo or TerraTorch. From raster/vector annotations, TerraKit will match, download, process, align and split the requested data source (e.g., EarthData, CDSE, Planetary Computer) based on user specifications provided by a simple configuration file. TerraKit also supports spatial train/val splits and exports datasets in standard formats such as TACO datasets. TerraKit streamlines the pipeline from raw EO data to AI-ready datasets, accelerating the development of custom geospatial applications, and ensuring query and processing pipelines are reproducible. By lowering the barrier to entry, a wider community of TorchGeo and TerraTorch users are empowered to leverage foundation models for Earth observation.

Lightning Talk: ExecuTorch on Microcontrollers: Deploying PyTorch To the Smallest Edge - RJ Ascani & Matthias Cremon, Meta ExecuTorch extends PyTorch's reach to the most resource-constrained devices: microcontrollers, DSPs, and specialized neural processing units powering always-on sensors, wearables, and embedded systems. In this talk, we'll share the current state and roadmap for running ExecuTorch on platforms where every kilobyte of memory and milliwatt of power matters. What you'll learn: - How ExecuTorch's design enables deployment from ultra-low-power MCUs to DSP and NPU accelerators, all from a single PyTorch workflow - The state of backend support for Cadence DSPs, ARM Ethos-U and Cortex-M - Practical considerations for deploying models with sub-megabyte footprints and milliwatt power budgets - Case studies spanning always-on audio, embedded vision, and TinyML applications

Enabling State-of-the-art Asynchronous Execution in Torch.compile With CUDA Streams - Michael Lazos, Meta CUDA streams are a widely-used method for parallelizing GPU computation on NVIDIA GPUs. They have long been requested by our users and enable multiple key capabilities - overlapping communication and compute kernels, training on multiple batches in parallel and parallelizing kernels, all of which are needed for achieving SOTA training performance. Another key capability is activation offloading - this can be applied to any model to prevent OOMs by asynchronously storing activations in cpu memory until they are needed by the model. Before this work, torch.compile previously would graph break on CUDA stream contexts, which can be costly for models that utilize streams. Although workarounds exist (e.g. wrapping stream manipulation into custom ops), these solutions add complexity and create friction in the user experience. By enabling seamless CUDA stream support in PT2, we allow our users to leverage the familiar eager APIs for stream assignment and synchronization directly within torch.compile. This not only simplifies the workflow but also ensures that models using custom streaming patterns can run efficiently out-of-the-box without manual intervention or code restructuring.

The Token Slice: Implementing Preemptive Scheduling Via Chunked Decoding - Maroon Ayoub, IBM & Kellen Swain, Google Production LLM serving faces a critical trade-off: while continuous batching maximizes throughput, it often sacrifices SLAs due to Head-of-Line (HoL) blocking. When long-context requests hijack the engine, tail latencies spike. Without fine-grained preemption, guaranteeing priority or fairness remains nearly impossible. We propose a solution: Chunked Decoding. By treating a fixed number of tokens as a "time slice," we bring 50 years of OS scheduling wisdom to inference. This technique decouples generation from completion, enabling a preemptive multitasking environment for LLMs. In this talk, we present a sidecar implementation for PyTorch-based servers (like vLLM) that orchestrates decoding in manageable chunks. This allows the system to pause, hold, or swap requests mid-stream without discarding the KV cache. We will share early evaluation results, discussing how varying chunk sizes impact priority handling and tail latency. Attendees will learn how a sidecar approach enables sophisticated scheduling while keeping the core engine lean—offering a blueprint for integrating preemptive scheduling into the next generation of model servers.

Lightning Talk: Deep Learning in the Wild: Embedded PyTorch for Real-World Conservation Bioacoustics - Taraqur Rahman & Owen O'Donnell, OWL Integrations Passive acoustic monitoring is a powerful tool for wildlife conservation, but deploying deep learning models in remote rainforest environments introduces strict constraints on power, memory, and compute. In this talk, we present an end-to-end PyTorch-based pipeline for detecting and analyzing the endangered three-wattled bellbird using embedded deep learning systems. We cover the full lifecycle from audio preprocessing and model training in PyTorch to optimization and deployment on resource-constrained embedded devices. Topics include model architectures for sparse bioacoustic event detection, handling extreme class imbalance, model compression and quantization, and practical trade-offs between accuracy, latency, and power consumption. The session emphasizes real-world lessons learned deploying machine learning at the edge, where unreliable connectivity, noisy signals, and limited hardware define success more than benchmark metrics. Attendees will gain practical patterns for building and deploying PyTorch models for embedded and edge AI applications with real environmental impact.

The Science and Practice of Open and Scalable LLM Evaluations - Grzegorz Chlebus, NVIDIA Rapid advances in AI have expanded the range of capabilities required for successful real-world deployment. Understanding where we are in this multi-dimensional frontier is essential for accelerating innovation through effective quality assurance. Rigorous evaluation is increasingly difficult to scale as development requires testing many checkpoints across numerous benchmarks. Model comparison is further complicated by limited transparency of reported results. This talk explores challenges, best practices, and open-source tools that elevate evaluation to a core component of LLM development, delivering continuous signals across the model lifecycle. We discuss principles for standardizing evaluation methods and improving consistency through practical patterns and anti-patterns, and examples of integrating the science of evaluation directly into model development. Using Nemo-Evaluator, an open-source scalable evaluation tool, we demonstrate modular architectures that enable transparent, reproducible measurement. Finally, we show how Nemo-Evaluator supports reproducible evaluation for the Nemotron model family, helping enable one of the most open development processes in modern AI.