2026-11-17 18:21 UTC
DANGMUAAI & Developer Tools, Decoded
BackDev Tools

AWS Emulator: 200+ Services Local, 56.6% of Tests Passing

Amazon shipped a free local emulator for 200+ AWS services that AWS says agents wrote entirely. The published pass rate is 56.6% — here is what it buys.

DangMua EditorialNov 17, 20264 min read
AWS Emulator: 200+ Services Local, 56.6% of Tests Passing

Amazon Web Services released AWS Emulator on November 11, 2026: a free local copy of more than 200 AWS services that AWS says autonomous agents wrote end to end.

What actually shipped

Point the AWS CLI or any SDK at 127.0.0.1:9297 and you can create queues, spin up resources, run queries and get back the same validation errors the real cloud returns — with no network connection, no AWS account and no bill. The write-up, citing GeekWire and AI News Bites, describes a Rust codebase built from 438 public API models in Smithy format, the same models that feed the AWS SDKs, released under CC0 with full commit history on GitHub and roughly 29,700 commits by November 12.

AWS is not hedging the authorship claim. Senior Principal Engineer Matt Goldberg wrote in the launch post: "The emulator was written entirely by autonomous agents. Our team hasn't written or reviewed any of the code." Recent commits are attributed to agents named IssueFinderAgent and IssueFixerAgent.

The number that decides adoption: 56.6%

The published scores put the pass rate at 56.6% — 24,243 of 42,857 scored tests in v1.0.322. Treat that as a service-by-service question rather than a verdict on the whole tool: the project ships a scores.json, and the only figure that matters for your pipeline is the one covering the services your tests actually call.

Where it breaks

  • No IAM evaluation. It does not check IAM or resource-based policies the way AWS does, so a request the real cloud would deny can succeed locally. AWS says plainly: do not use it for security testing.
  • Unsupported operations return HTTP 501.
  • Single account only — no cross-account or cross-region behaviour.
  • No failure simulation: no throttling, no outages, no regional differences, and it does not sandbox the code it runs.
  • Releases move fast and sometimes break. Versions went from v1.0.265 to v1.0.322 in a few days, so pin a version in CI.

How the agents built it

It began as a weekend experiment: Goldberg prompted the Kiro coding agent to build a passable Amazon SQS emulator, and it took about 15 minutes. AWS then scaled that into three agent teams running on Amazon Bedrock AgentCore. Test writers harvest scenarios from the public Smithy models and verify them against real AWS. Issue finders replay those scenarios against the emulator and log every failure. Fixers each take a failure and patch the code, and every fix passes a merge gate that checks compilation, coding standards, security problems and fidelity regressions. Known gaps sit in a GitHub folder called things-that-do-not-work; fixed ones move to things-that-now-work.

Note the condition that made this work, because it is a limit rather than a template: the task had a machine-readable spec and a live answer key, so the agents could grade their own output without a human in the loop. Codebases without either one do not get the same treatment.

Is it worth wiring into CI?

That part is a judgement call, not something the launch post answers. The sensible shape: use it as a fast local layer for the services whose scores are high, keep your existing integration suite pointed at real AWS for anything that touches permissions, cross-region behaviour or failure handling, and never let a green local run stand in for a security review. With 56.6% of scored tests passing, a failing emulator run tells you something real, while a passing one is not proof that the real service agrees.

Installation is one command — aws agent-toolkit add-skill --skill-name aws-emulator --region us-east-1, which needs AWS CLI 2.35.0 or later — then point your SDK at the local endpoint and run your suite.

The thing to watch is the pass rate between releases. If 56.6% climbs quickly across the services you depend on, the agents' fix loop is working at the rate AWS claims; if it stalls, the published number is the ceiling you are buying into.

More from DangMua