2026-11-21 18:21 UTC
DANGMUAAI & Developer Tools, Decoded
BackAI Models

Gemini 4 Neon: 1M-Token Answers Still Stop at 64K Per Hop

Google's new Gemini 4 Neon can write 1M tokens in one answer, but each request caps at 65,536 and the SDK's input count overstates your context.

DangMua EditorialNov 21, 20264 min read
Gemini 4 Neon: 1M-Token Answers Still Stop at 64K Per Hop

Google released Gemini 4 Neon on 18 November with answers of up to 1M tokens — but any single request still stops at 65,536.

Switching the model ID alone will not get you the long answers. The 65,536 cap covers thinking plus output, and getting past it changes how you wire the request, and how you count tokens.

Neon (gemini-4-neon) sits between Hydrogen and Carbon in the Gemini 4 family and is aimed at complex coding, knowledge work and science. The launch post leads with benchmark rankings; the part that changes how you wire a chat UI is quieter.

The pricing and the window

From Google's launch post and API docs, as read on 21 November: $3.00 per 1M input tokens and $15.00 per 1M output tokens, thinking included, flat across the window. Cached input is $0.15 per 1M, a 95% discount from implicit caching. Unlike Hydrogen, there is no free tier — Neon is paid-tier only.

Input, thinking and output all share one 1,048,576-token window. Thinking levels are low, medium, high (the default) and xhigh; minimal is rejected.

How answers past 64K actually work

Generation runs in hops of up to 64K tokens. When a hop hits that cap and budget remains, the response ends with finishReason: CONTINUATION and an encrypted continuationToken. You resend the same request with that token and do not append the partial output to contents — the token carries the state. Once the full max_output_tokens budget is spent, the response ends with MAX_TOKENS instead.

Google is direct about when this is worth the complexity: chat, Q&A and summaries finish well under 32K tokens and do not need it. It is for hard reasoning and "write the whole thing" jobs such as a full migration spec. Treating long decode as a per-request toggle rather than a default follows from that.

SDK support landed days before the model

On the Vercel AI SDK side, @ai-sdk/google 4.0.123 (20 November) added the gemini-4-neon ID with xhigh. Long decode itself arrived earlier, in 4.0.95: the provider runs the CONTINUATION hops for you, sums usage across them and exposes initialPromptTokenCount. The writeup's wiring test ran on ai@7.0.179 and @ai-sdk/google@4.0.124, setting maxOutputTokens to 262,144 for long requests and leaving it undefined otherwise.

That test came with a stated caveat worth repeating: the team had no Gemini API key and drove the route against a local mock of the documented streamGenerateContent stream. Read it as a wiring test, not a quality review of the model.

The token-meter trap

The finding most likely to bite a production chat app is that usage.inputTokens is not your context size. Across a three-hop run it came back as 199,608 — the sum of each hop's input (1,000 + 66,536 + 132,072), because every hop re-reads everything generated so far. The real context was 162,072 tokens.

If you drive a context meter off that field, it will overstate how full the window is and overstate it more on every extra hop. Track context separately from billed input: initialPromptTokenCount plus output tokens is the figure that reflects the conversation. Every hop is billed, including thinking-only hops — the first hop in that run was all thinking with empty text, which the docs call normal. Repeated input does get the 95% cache discount, but a single answer that consumes the whole 262,144-token budget still runs roughly $3.90 in output.

Is it worth enabling?

For a chat product, probably not by default: Google's own guidance puts ordinary chat under 32K, and the hop machinery adds latency, a resumable token to persist, and a billing line that no longer matches your context display. Enable it where a single answer genuinely has to be enormous — a migration spec, a long structured document — and wire the meter to initialPromptTokenCount before shipping, not after the first invoice.

More from DangMua