Laya-MLX Claims 7-14ms Typed Decisions on Apple Silicon

A new Python package argues you are paying an 8B generative model to answer yes-or-no questions — and that an Apple Silicon Mac can do it in single-digit milliseconds instead.

laya-mlx, published November 2, is a native Apple MLX runtime for Laya typed decision models. Its author claims it executes "strict structured decisions in just 7–14 ms on an M3 Max—with zero PyTorch bloat, no hallucinated JSON, and completely offline execution."

The pitch

The argument is about the wrong tool for the job. Per the project's write-up, "most modern agent pipelines waste massive compute using 8B+ generative LLMs just to classify intents, trigger functions, or make boolean decisions." Routing a query, checking input safety or picking a branch does not need token-by-token autoregression — it needs a discrete, typed decision.

The claimed advantages, all from the author:

• 7–14 ms per decision on Apple Silicon M3 Max chips.
• No PyTorch dependency — "built ground-up on Apple's native MLX framework, cutting container and bundle sizes drastically."
• Typed output straight from model weights, with "no brittle JSON schema extraction or regex parsing required."
• Fully offline: an "air-gapped runtime" with no per-token API costs.

What using it looks like

Installation is pip install laya-mlx. The documented call loads a pretrained router and returns a typed decision object:

from laya_mlx import LayaDecisionEngine

engine = LayaDecisionEngine.from_pretrained("laya/intent-router-v1")
decision = engine.decide({"user_prompt": "Cancel my current subscription and export my workspace data."})
# decision.action_type -> 'SUBSCRIPTION_CANCEL'
# decision.confidence  -> 0.98
# decision.latency_ms  -> ~8.42

That 8.42 ms figure is the author's sample output, not an independent measurement.

Who it is aimed at

Three use cases are named: agent architects swapping "slow LLM-based intent routers for microsecond-scale decision layers before passing tasks to heavier models"; local AI and edge developers building macOS background tools that "react instantly without battery drain"; and guardrail engineers running "local, real-time safety gating and policy checks on user inputs before hitting external APIs."

Note the internal inconsistency: the same post describes the layer as both 7–14 ms and "microsecond-scale." Those differ by three orders of magnitude, and only the millisecond figure appears in the benchmark line.

Is it worth your afternoon?

Treat every number here as a vendor claim. There is no third-party benchmark, no accuracy comparison against the 8B routers it proposes to replace, and no published hardware matrix beyond the M3 Max — nothing in the post says what an M1, an M4 or a base-model chip does. The write-up also carries sponsorship and affiliate-style sections, so read the framing accordingly.

The underlying idea is sound and predates this package: classification does not need a generative model, and a typed classifier avoids JSON parsing failures by construction. If you already run an intent router on a Mac and can measure your current p50 latency, the test is cheap — install it, point it at your own labelled prompts, and compare both latency and accuracy. Skip it if you are not on a Mac: the runtime is built on Apple's own MLX framework, so none of it transfers.

What to watch for: an independent latency benchmark on more than one chip, and any accuracy numbers against the generative routers this is meant to replace. Until one of those lands, this is a promising shortcut with no scoreboard.