Analysis of GPU Memory Read Path on NVIDIA RTX 4090

Tracing a Global Load Instruction

The analysis investigates the path a critical SASS instruction, a global load (LDG.E), takes through GPU hardware. This specific examination uses an NVIDIA RTX 4090 to understand how memory reads are processed. The goal is to detail the journey of data from global memory into a register for a vector-add kernel.

The Vector-Add Kernel Example

The CUDA kernel under investigation is a simple vector-add: `c[i] = a[i] + b[i]`. The compiled SASS instructions for this kernel include `IMAD.WIDE` and `LDG.E`. The `LDG.E` instruction is responsible for loading elements of vector `b` from global memory into a register, where they can then be added to elements of vector `a`.

Hardware Journey of a Memory Request

A single `LDG.E` instruction requests four bytes across 32 lanes. Fulfilling this request involves several hardware stages: four 32-byte sectors, one cache line, an address translation, a crossbar crossing, and one of 36 L2 slices. If the data is not found in caches, it necessitates an activate and four column reads at a DRAM chip. The analysis aims to follow this entire path.

Undocumented Details and Experimental Approach

Many of the specific details regarding this hardware path are not publicly documented by NVIDIA. To overcome this, the analysis relies on timing experiments conducted directly on the hardware. This empirical approach allows for the inference of the internal workings and stages involved in GPU memory access.