Nemotron 3.5 Open Weights: Which Size Fits Your GPU?

NVIDIA has published a new batch of open-weight Nemotron 3.5 models on Hugging Face, topped by a 120-billion-parameter multimodal model aimed at agents.

The weights are free. The hardware to serve them is not, and that is the part worth doing arithmetic on before you clear a slot in your cluster.

What shipped

The release covers three lines, per the dev.to write-up of the launch:

Model	Size	Built for
Nemotron 3.5 Super VL	120B, multimodal	Specialized agent tasks
Nemotron 3.5 Bolt	74B, plus a 4B efficient variant (E4B)	Enterprise work; edge and local deployment
Nemotron 3.5 Transcribe	30B mixture-of-experts	Transcription, translation, speaker recognition

Transcribe covers 40 languages and was trained on a 463,062-hour audio dataset. All of the above are published on Hugging Face for agentic, enterprise, edge and multilingual audio workloads.

On performance, the write-up relays reporting rather than measurement: it says coverage from VentureBeat and Interesting Engineering notes Super VL "outperforms rivals such as Claude Opus 5 and GPT-6.1 Sol on specific agent benchmarks and can complete tasks up to five times faster." Treat that as a claim from secondary coverage, not a result anyone has reproduced in public. No independent benchmark run accompanies the release post.

The number that actually decides it

Open weights move the question from "can I afford the tokens" to "does it fit on the card". Those are not the same question, and the second one has a harder answer.

A separate deployment write-up from InferenceKey, serving a different MoE model — Groot-19B-A3B — on one RTX 5090, lays out the budget cleanly: "GPU memory pays for two things. The weights are a fixed cost. The KV cache is the model's notepad for the conversation so far, and it grows with every token of context you allow."

Their measured run is a useful reference point for anyone sizing a Nemotron deployment. The weights took 19.7 GiB. At --gpu-memory-utilization 0.92, vLLM had 6.8 GiB left for cache — room for 307,024 tokens. Time to first token on a 205,525-token prompt was 34.6 seconds, with the complete answer at 40.3.

The failure case is the instructive one. Capped to a 24 GB budget with 65,536 tokens requested, the weights ate 86% of the budget and vLLM refused to start, reporting an estimated maximum model length of 9,648 tokens against the 128K minimum the model's authors recommend. Their conclusion for that build: budget 32 GB.

What this means for the 4B variant

Mixture-of-experts sizing trips people up because two different numbers both look like "the size". In the Groot example, roughly 3 billion of 19 billion parameters are active per token, so it generates fast — but the card still has to hold all 19 billion.

That is the lens to apply to Nemotron 3.5. Transcribe is described as a 30B MoE, so its active-parameter count will understate what the card must hold. Bolt E4B at 4B is the only one in the family explicitly positioned for edge devices and local deployment, and it is the realistic starting point if you are evaluating on a single consumer GPU rather than a datacenter node.

Before you pull the weights

Read the startup log, not the spec sheet — the server tells you how many cache tokens survived after the weights loaded, and that figure is the one that decides whether your context target is real. Measure peak memory on your longest prompt, and check time to first token at full context before you promise anyone a latency number.