Clover-35B Hits 208 tok/s on a 4090 — in llama.cpp's Slow Path

llama.cpp runs DiffuSpace's Clover-35B-A3B at 202 to 211 tokens per second on one RTX 4090 — in autoregressive mode only, not its diffusion fast path.

That gap is the whole story. Clover's speed case rests on filling many token slots at once, and the engine that just shipped support for it does not do that yet.

What the test actually measured

The official Q4_0 GGUF was loaded on a single RTX 4090 running stock llama.cpp v0.9.0. After a small file fix, one request returned 202 to 211 tokens per second, with the first token back in 0.13 to 0.31 seconds. The full 128K context window — prompt plus answer held in memory — peaked at 21,924 MiB.

That peak is the figure to plan around. A full-length context leaves very little headroom on a single consumer card, which matters more for capacity planning than the throughput number does.

Why "supported" is the slow path

Clover is a diffusion language model. Instead of emitting one token and then starting the next, it drafts a block of 32 slots, commits every slot where its confidence reaches 95% in a single pass, and revisits the others.

PR #31306 landed on 2 November and the v0.9.0 release carried it on 4 November, with four words attached: "in AR mode only." It runs Clover one token per step. A faster self-speculative mode — draft a block in parallel, then verify it the normal way — was left for a later pull request, and none was found open. So 208 tokens per second measures the typist, not the diffusion trick.

Quoted figures vs the measured one

Setup	Throughput	Standing
llama.cpp v0.9.0, AR mode, 1x RTX 4090, Q4_0	202-211 tok/s	Measured on the card
SGLang PR #46927, 1x H200, one request at a time	453.59 tok/s	The PR's own test; awaiting review
1x H200, uncompressed	515 tok/s	DiffuSpace's quote; no engine or mode named
GB300, uncompressed	680 tok/s	DiffuSpace's quote; no engine or mode named

DiffuSpace's own Python code wants about 69 GB for the weights alone, so the vendor figures describe a different class of machine than one consumer card. The draft-and-verify path that would narrow the gap sits in Zirui Wu's SGLang pull request, still waiting on review.

The trap before you book a GPU

Out of the box, v0.9.0 rejects the official file:

key not found in model: clover_moe.expert_shared_count

A file that loads is not a file that works. Budget time for that step before scheduling benchmark hardware, and treat any vendor throughput figure that names neither engine nor mode as unverifiable against your own setup.

What to watch

llama.cpp shipped 1.0.0 on 13 November 2026, removing 35 deprecated functions and promising breaking changes only on a major-version bump. Clover's parallel decode path is not in that release. Watch for a self-speculative pull request against llama.cpp, and for SGLang PR #46927 to clear review. Until one of them lands, Clover on consumer hardware is a 208 tok/s autoregressive model, whatever the spec sheet quotes.