QWIX fixes
To preserve gradient flow during backpropagation, Qwix QAT mode does not actually cast weights to low-bit integers; instead, it does this to simulate lower-bit math while staying in 32-bit float registers.
What is fake quantization?
This famous memory-efficient mechanism avoids materializing the massive O(L2) attention matrix by tiling and computing online softmax in blocks.
What is FlashAttention (or SplashAttention)?
On a roofline plot, performance is plotted on the y-axis against this ratio on the x-axis.
What is Arithmetic Intensity (FLOPs per Byte)?
This JAX operation automatically maps a function over array axes, eliminating the need for explicit, inefficient Python loops.
What is vmap?
First introduced as a fully programmable unit in Viperfish, this core offloads embedding lookups and sparse updates from the dense processing units.
What is SparseCore?
One of the main performance benefits of weight-only quantization is reducing this major hardware bottleneck, even if the actual matrix multiplication computation is still performed in floating-point.
What is Memory Bandwidth (or Memory Footprint / I/O)?
To turn idle wait-time into peak FLOPS, custom Pallas kernels use this technique to fetch the next block of data from slow HBM into VMEM while simultaneously crunching math on the current block.
What is double buffering?
Developed to avoid FP16 underflow, this 16-bit floating-point format retains FP32’s 8-bit exponent range, eliminating the need for dynamic loss scaling.
What is bfloat16?
Starting with Pufferfish, this feature allows two TensorCores sharing the same HBM package to work together transparently as a single logical unit.
What is Megacore?
In LLM performance modeling, this execution phase is typically compute-bound due to parallel matmuls over prompt tokens, whereas the subsequent phase is memory-bandwidth bound due to KV-cache loading.
What is the prefill phase?
To enable loading data efficiently on the vector cores, XLA typically pads the minor dimensions of TPU tensors to these two numbers.
To smooth out the quantization difficulty between dynamic activations and static weights, Qwix supports this algorithm which applies a mathematical "smoothing factor" across channels before clipping.
What is SmoothQuant?
In distributed Tokamax kernels like Fused MoE, engineers hide the extreme latency of All-to-All Expert Routing by fusing the communication step with local computations and invoking this specific primitive.
What is Asynchronous Remote Copy (or explicit DMAs)?
To trade compute for a lower memory footprint, this technique drops intermediate forward-pass activations and recomputes them on the fly during the backward pass.
What is activation checkpointing (or gradient rematerialization)?
Handling tasks too complex for independent, isolated VPU lanes, this specialized subsystem is responsible for operations such as data shuffles and transposes.
What is the Cross-Lane Unit (XLU)?
You can write perfectly parallel code for all 128 VPU lanes, but if your algorithm asks two different sublanes within a lane to access different rows of data on the same clock cycle, the chip will instantly stall due to this performance-killing hardware violation.
What is a bank conflict?