Serving large video diffusion models to multiple concurrent users sounds challenging till you partition a GPU correctly.

This talk is a deep technical exploration of running large-scale video generation inference on modern GPUs across Hopper and Blackwell with Multi-Instance GPU (MIG) isolation.

We'll explore:

1. GPU MIG topology: Memory hierarchy, interconnect partitioning, and leveraging high-bandwidth memory effectively.
2. Memory profiling for inference: Tracking GPU memory allocation across the generation pipeline
3. MIG profile selection: Choosing partition sizes—when isolation beats raw throughput
4. Request scheduling: Fair queuing for heterogeneous workloads and batch sizes
5. Failure modes: OOM recovery, MIG instance health checks, and graceful degradation strategies
6. Monitoring at scale: Per-instance GPU metrics and detecting performance bottlenecks

Whether you're building a multi-tenant inference platform, optimizing GPU utilization for your team, or exploring how to serve video diffusion models cost-effectively, this talk provides practical configurations for your AI workloads.