A GPU allocated to a pod is a GPU allocated to that pod. Kubernetes treats accelerators as indivisible extended resources, so a notebook idling at 4% utilisation holds an entire card, and the next job queues behind it. On CPU nobody would accept this. On hardware that costs an order of magnitude more per hour, most clusters accept it by default.
NVIDIA gives you two ways out, and they are not interchangeable. Choosing between them is really a question about who your tenants are.
Time slicing
Time slicing advertises one physical GPU as several allocatable replicas. The scheduler places multiple pods on it and the driver interleaves their kernels. From the workload's point of view it has a GPU; in reality it has a share of one, on a rota.
It is a configuration change, not a hardware feature. That is its main appeal — it works on cards that have no partitioning support at all, which in practice means most of the fleet outside the datacentre-class parts.
What you give up is isolation, and it is worth being precise about which kind. There is no memory partition. Two pods sharing a card share its memory, so one workload allocating aggressively can push another into an out-of-memory failure it did nothing to cause. There is no compute guarantee either — a neighbour saturating the card slows you down with no signal that anything is wrong.
MIG
Multi-Instance GPU partitions the card in hardware into instances with their own memory, cache and compute slices. A pod bound to an instance cannot see or starve its neighbours. The failure isolation is genuine, not cooperative.
The costs are that it only exists on supported hardware, the partition layout is fixed at configuration time rather than negotiated per workload, and a job that needs the whole card cannot have it while the partitioning stands. You are trading flexibility for a guarantee.
How to choose
The question that decides it is not utilisation. It is whether your tenants can hurt each other, and whether that matters.
- Development, experimentation and internal inference, run by people who share a Slack channel: time slicing. The workloads are bursty, mostly idle, and a noisy neighbour is a conversation rather than an incident.
- Multiple teams with independent SLOs, or anything customer-facing: MIG, where the hardware supports it. A guarantee you can point at beats a convention you have to police.
- Untrusted or externally-submitted workloads: MIG, or separate nodes. Shared memory is a shared blast radius.
On the EKS platform I built, the instance family in play was g4dn — T4 class, with no MIG support at all. So the choice was made for me by the hardware, which is often how it goes. What still mattered was recording it: the next person to read that cluster config will otherwise assume an isolation boundary that is not there.
The part people skip
Whichever you pick, you cannot tell whether it is working without GPU-level telemetry. Standard Kubernetes metrics report that a pod holds a GPU resource. They say nothing about whether the silicon is busy.
DCGM exporter into Prometheus closes that gap, and the two signals worth alerting on are utilisation — is sharing actually being used, or are you paying for idle capacity — and memory pressure, which under time slicing is your only early warning that tenants are about to collide.
Sharing a GPU without GPU metrics is not capacity management. It is hoping.