TokenPrint Is a 3D Debugger for What Happens Inside an LLM

A developer has open-sourced TokenPrint, an interactive 3D environment that walks a single token through every operation inside a transformer. The part worth paying attention to is not the 3D: it is that every number on screen is labelled with where it came from.

What it actually shows

Instead of collapsing inference into x → attention → MLP → x, TokenPrint lets you move layer by layer and inspect the individual operations: token embeddings, positional information, RMSNorm, Q/K/V projections, Grouped-Query Attention, RoPE, attention scores, softmax, weighted value aggregation, the output projection, SwiGLU MLP projections, residual streams, and the final logits and predictions.

An attention block is broken out as the sequence it is — Q = XW_Q, K = XW_K, V = XW_V, then A = softmax(QKᵀ / √d_k + M) and C = AV — rather than one opaque box. For a model using Grouped-Query Attention, the author says the tool can also expose the relationship between query heads and the shared key/value groups.

The stated reason for 3D is practical rather than aesthetic: a transformer is already a computational graph, and a flat diagram becomes hard to read once it has to show many layers, branches and intermediate tensors at once. "The intention isn't to make the model look futuristic," the author writes. "The intention is to make the structure easier to understand."

The provenance labels are the real feature

Selecting a component opens an inspection view with a plain-language explanation, the equation, input and output dimensions, the number of learned parameters, the actual tensor path, and a provenance tag. Those tags are the design decision that separates this from a diagram:

• REAL — directly obtained from the model or runtime.
• DERIVED — calculated from real model information.
• CONCEPTUAL — an educational representation of a model concept.
• SIMULATION — intentionally simulated behavior.

The author's justification is blunt, and it is the right instinct for any explainer tool: "an attractive visualization is not useful if it quietly invents model internals."

The tensor inspection workflow carries the same specificity. Selecting a projection can surface a path such as model.layers.3.self_attn.v_proj.weight together with Shape: 128 × 896, Dtype: float32, roughly 114.7K parameters, Layer 3, and a runtime of hf_local. That ties the picture to a real checkpoint rather than a stylised stand-in.

Where it is today

TokenPrint runs in the browser at tokenprint.in, with source on GitHub. It is early: the author describes it as visualizing one local model today, with a roadmap covering broader Hugging Face model support, better model capability detection, richer execution traces, remote inference, experiment workflows, and shareable traces that can be explored without reproducing the original runtime. Longer-term targets named in the post include activation analysis, residual stream analysis, layer and head experiments, ablations, activation patching, trace replay and model comparisons — a mechanistic-interpretability workbench, if it gets there.

Treat the roadmap as intent, not shipped capability. What exists now is narrower and still useful.

Who should open it

If you are teaching transformer internals, debugging a fine-tune that drifts at a specific layer, or trying to explain attention to someone who has read the diagram three times without it clicking, this is a cheaper starting point than instrumenting a runtime yourself. If you need repeatable measurement across models, the roadmap items are the ones to wait for. Either way, the provenance-tagging idea deserves to spread to every LLM visualization that currently blurs the line between a measurement and an illustration.