nvidia-smi reports 100% utilization, but your workload underperforms. What's missing?

Relying only on nvidia-smi is like measuring highway usage by checking if any car is present, not how many lanes are full.

This talk reveals the metrics nvidia-smi doesn't show and introduces open source tools that expose actual GPU efficiency metrics.

We'll cover:

1. Why GPU Utilization is not same as GPU Efficiency.
2. Deep dive into relevant key metrics: SM Active, SM Occupancy, and Tensor Core utilization explained.
3. Steps for practical gpu profiling and active monitoring setup.
4. Identifying bottlenecks in inference workloads.

Attendees will leave understanding how to identify underutilized GPU and discover real optimization opportunities across inference workloads.