Understanding MLA Decoding Materialize vs Absorb

MLA keeps a small cache per token. How do you compute attention from it? There are two algorithms, and modern engines ship both. This post walks through each (including values, not just keys), then uses interactive charts to show when one beats the other.

1.Prefill and decode are the same computation

People treat prefill and decode as different, but for attention they are the same operation with one number changed:

decode   =  new_tokens is 1        (one fresh token attends to the whole cache)
prefill  =  new_tokens is large    (many new tokens attend to the growing context)

So instead of two stories, we analyze one cost as new_tokens grows from 1 upward. The crossover between the two algorithms will fall somewhere on that axis.

One modeling note: the formulas below treat new_tokens and cached_tokens as independent. That matches decode exactly (1 new token, large cache). Prefill is messier — the prompt is both the queries and the keys being written — but the same two algorithms still apply, and the crossover still lands in the same ballpark.

2.The two algorithms

Both start from the same cache entry. Name the pieces once (same as Part 1):

The two algorithms differ in where the content projections (W_K, W_V) are applied. Rope is handled the same way in both — RoPE, then concat. Only content moves.

The single most important difference is the warm (coral) step:

• Materialize rebuilds keys and values for every cached token, so its projection work grows with cached_tokens.
• Absorb only transforms the new queries, so its projection work grows with new_tokens.

At decode, cached_tokens is huge and new_tokens is 1. So Materialize does a huge rebuild while Absorb does almost nothing. That is the whole intuition — the charts below make it exact.

3.What “absorb” actually means

Materialize is easy: turn every cached latent into a full key and value, then run ordinary attention. Absorb looks like a trick until you see that both paths compute the same numbers. We build it in four stages: keys, then many heads, then values, then what refuses to absorb.

3aKeys: same score, two factorizations

Start with one head, one new query, three cached tokens. Content path only — rope comes in stage 3d. Latent width 3, nope width 2 (same pattern as 512 vs 128):

Q_nope = [1, 0]                    # width 2

W_K = [[1, 0],                     # (3 × 2): latent → nope
       [0, 2],
       [1, 1]]

cached latents c:                  # each width 3  (content only)
  c0 = [1, 0, 0]
  c1 = [0, 1, 0]
  c2 = [0, 0, 1]

K0 = c0 · W_K = [1, 0]             # width 2
K1 = c1 · W_K = [0, 2]
K2 = c2 · W_K = [1, 1]

score0 = Q_nope · K0 = 1
score1 = Q_nope · K1 = 0
score2 = Q_nope · K2 = 1

Q_absorbed = Q_nope · W_Kᵀ = [1, 0, 1]   # width 3 = latent

score0 = Q_absorbed · c0 = 1
score1 = Q_absorbed · c1 = 0
score2 = Q_absorbed · c2 = 1

for each cached c:
    K = c · W_K          # → nope
    score = Q_nope · K

work ∝ cached_tokens
score space = nope_dim

Q_abs = Q_nope · W_Kᵀ    # → latent, once
for each cached c:
    score = Q_abs · c

work ∝ new_tokens
score space = latent_dim

3bMany heads, one latent

The toy used one head. The real model has many. The packed weight looks like latent → (heads × nope), but it is really one small matrix per head, each different:

Absorb applies each matrix to the matching query head — heads are never flattened into one giant vector and projected away:

q head 0  →  W_K for head 0  →  Q_absorbed[0]  ·  shared latent c
q head 1  →  W_K for head 1  →  Q_absorbed[1]  ·  shared latent c
q head 2  →  W_K for head 2  →  Q_absorbed[2]  ·  shared latent c
...

After absorb you still have num_heads queries, each of width latent_dim, all dotted against the one shared latent. That is the MLA bargain again: MQA-sized cache, MHA-style per-head views. In code this is a bmm over the head axis (w_kc in MiniCPM3).

3cValues: the same trick after softmax

QKᵀ only produces attention weights of shape (new_tokens × cached_tokens). V’s width is a separate choice. You could up-project every cached latent to V before mixing — that is materialize. Absorb delays it.

Add a value matrix and toy weights w = [0.5, 0.0, 0.5] (illustrative, not a real softmax of the scores above):

W_V = [[1, 0],                     # (3 × 2): latent → value
       [0, 1],
       [0, 1]]

# Materialize: build every V, then mix
V0 = c0 · W_V = [1, 0]
V1 = c1 · W_V = [0, 1]
V2 = c2 · W_V = [0, 1]
out = 0.5·V0 + 0.0·V1 + 0.5·V2 = [0.5, 0.5]

# Absorb: mix latents first, then one W_V
c_mixed = 0.5·c0 + 0.0·c1 + 0.5·c2 = [0.5, 0.0, 0.5]
out     = c_mixed · W_V             = [0.5, 0.5]   # same

Materialize does a latent → value matmul for every cached token. Absorb does one matmul on a short result. At decode (new_tokens = 1), that is the difference between touching the whole cache and touching almost nothing. V is not skipped — it is absorbed after attention (w_vc in MiniCPM3 / SGLang), and again there is one \(W_V^{(h)}\) per head.

3dWhat does not absorb: rope

The regrouping needs a matrix identical for every cached token. RoPE breaks that. The position key at cached position n is rotated by an angle that depends on n:

score_rope at n  =  Q_rope · (R_n · k_rope)

R_0, R_1, R_2, … are all different
→ no single matrix you can move onto Q_rope

So in both algorithms:

• K_rope is stored as-is (width 64), RoPE’d, never up-projected, never absorbed.
• Q_rope is RoPE’d and concatenated beside the content half — only Q_nope is folded into latent space.
• RMSNorm touches the latent only, never the rope half.

Final absorb tensors (DeepSeek-V3 widths):

Q = [ Q_nope @ W_Kᵀ  |  RoPE(Q_rope) ]     # (heads, 512 + 64)
K = [ RMSNorm(c)     |  RoPE(K_rope) ]     # (1,     512 + 64)   shared
V =   RMSNorm(c)                           # (1,     512)        shared
→ after attn:  (heads, 512) @ W_V → (heads, 128)

So absorb’s attention width is latent + rope because the two are concatenated for scoring — not because rope lives inside the latent. Content can be absorbed; rope cannot.

4.Cost of each step

The one rule: a matmul of shape (rows × inner) × (inner × cols) costs rows × cols × inner multiply-adds. Applying it to every step, per layer. These counts cover the content path and the attention that follows; they omit the cheap shared-rope broadcast (same on both sides) and the final output projection back to hidden size (also the same on both sides).

Materialize — rebuild the cache

Absorb — fold into the query

Put the two projection terms next to each other. They are the same work per token — latent_dim × (nope_dim + value_dim) — but Materialize multiplies it by cached_tokens (rebuild the whole cache) while Absorb multiplies it by new_tokens (only the fresh queries). That single swap is the entire trade-off. Absorb pays for it with a wider attention term (latent_dim in place of nope_dim / value_dim).

5.The trade-off, interactively

The chart plots both totals as new_tokens grows from 1 (decode) to thousands (prefill). Pick a model and drag the context length. Where the lines cross is where the winner changes.

Notice the shape: the two lines are nearly parallel on the log scale, and they cross once. Left of the cross (few new tokens = decode) Absorb is far below. Right of the cross (many new tokens = prefill) Materialize dips under. Increasing the context slides the crossover only slightly — it is close to a fixed number of tokens, not a fixed fraction of context.

6.The crossover point

Set the two totals equal and solve for new_tokens. Fully expanded:

crossover new_tokens =
        cached_tokens × latent_dim × (nope_dim + value_dim)
   ───────────────────────────────────────────────────────────────────
   latent_dim × (nope_dim + value_dim)
   + cached_tokens × ( 2 × latent_dim − nope_dim − value_dim )

For long context the first term in the denominator becomes negligible, and it simplifies to a value that does not depend on context length at all:

crossover new_tokens  ≈   latent_dim × (nope_dim + value_dim)
                          ───────────────────────────────────
                          2 × latent_dim − nope_dim − value_dim

For DeepSeek-V3 that is about 171 tokens; Kimi K2.5 lands in the same place (same latent / nope / value widths — only the head count changes, and it cancels in the ratio). GLM-5.2’s wider heads (nope 192, value 256) push the long-context limit to about 228 tokens. At a realistic context of 4,096 the exact formula is a little lower — the live value for your current settings is in the box above. The takeaway: the boundary sits at a couple hundred query tokens, essentially independent of how long the context is.

7.The cache-size win (why we bother)

All of this exists to keep the KV cache tiny. Storing one shared latent instead of full per-head keys and values is the reason MLA is worth the extra attention width. Per token, per layer:

DeepSeek-V3 and Kimi K2.5 both store 576 values per token per layer (512 + 64); the full-cache side differs because Kimi has half as many heads (20,480 vs 40,960). GLM-5.2 keeps the same 576-wide MLA entry but a wider full head (256 + 256), so the full cache is 32,768 — still about 57× larger. Pick a model above to see the exact numbers.

Absorb is what makes that small cache usable at decode: you never expand every past token into full K_nope / V just to attend. Materialize can still expand the latent when there are enough new queries — but K_rope is never expanded either way, and the stored cache stays [latent | K_rope]. (The Hugging Face reference path is different: it caches the expanded tensors. Serving engines keep the latent + rope.)

8.Which algorithm runs when

This is exactly why a serving engine ships both paths. Decode lives far below the crossover, so it folds the projections into the query (Absorb) and keeps the cache tiny. Prefill lives far above it, so it rebuilds keys and values once and runs the cheaper narrow attention (Materialize). Neither wins everywhere, and the crossover formula above is the dividing line.

References

• DeepSeek-V2 / V3 papers — MLA and the two attention formulations.
• vLLM model_executor/layers/attention/mla_attention.py — the "compute-friendly" (materialize) and "data-movement-friendly" (absorb) paths, documented in the file header.
• SGLang forward_mla.py and minicpm3.py — the absorb decode path.
• MLA Part 1 — the compression and nope/rope split.

How to cite this post

Dong, S. (2026). Understanding MLA Decoding (Materialize vs Absorb).
https://simondong1.github.io/mla-decoding.html