How containers use GPUs

A container reaches a physical graphics processing unit (GPU) through resources exposed by the host. The ownership path is:

The host owns the hardware and kernel driver

The machine has the physical GPU and an installed NVIDIA driver that talks to the hardware.

The container commonly carries the application, framework, and compatible CUDA userspace libraries.

Container tooling exposes selected GPU devices and required host capabilities into the container.

This is why GPU containers still have host compatibility requirements. Containerization does not make an arbitrary host driver/GPU combination disappear.

Kubernetes schedules a resource; the runtime executes on it

A Kubernetes node can advertise GPU resources, commonly through NVIDIA’s device integration.

A Pod can request something conceptually like:

resources:
  limits:
    nvidia.com/gpu: 4

Kubernetes uses that resource request for placement.

Once the container starts, the inference runtime uses the exposed GPUs through CUDA and related libraries.

So:

Kubernetes scheduler → which node/GPU resources are assigned?

inference runtime → how is model computation executed on those GPUs?

Multi-GPU communication adds another layer

If one distributed process uses several GPUs, a collective-communication library such as the NVIDIA Collective Communications Library (NCCL) can coordinate data exchange. The physical path can involve NVLink/NVSwitch, PCIe, or network fabrics across nodes.

That is a performance/topology issue on top of basic device access.

The next chapter moves from one container host to cluster orchestration: Kubernetes.