How OpenAI Scaled PostgreSQL to 800 Million ChatGPT Users

Nearly 50 read replicas feed off one PostgreSQL primary to handle millions of queries per second for roughly 800 million ChatGPT users, according to OpenAI's engineering team. In a recent write-up, the company describes scaling a single-primary relational database instead of sharding it — and says PostgreSQL load has grown by more than 10x over the past year and "continues to rise quickly." That is a growth curve most infrastructure teams never have to plan for, let alone survive with one write node.

The central question for any engineer reading this: how far can a single-primary Postgres design actually scale before sharding becomes unavoidable? OpenAI's answer, at least so far, is further than conventional wisdom suggests — through a stack of targeted fixes rather than a single silver bullet.

Why a single primary keeps breaking under load

OpenAI's team writes that PostgreSQL has been "one of the most critical, under-the-hood data systems powering core products like ChatGPT and OpenAI's API" for years. The post describes a recurring failure pattern behind the company's worst incidents: an upstream issue causes a sudden spike in database load — widespread cache misses from a caching-layer failure, a surge of expensive multi-way joins saturating CPU, or a write storm from a new feature launch.

Postgres's own storage engine compounds the problem. Under MVCC (multi-version concurrency control), updating even a single field on a row copies the entire row to create a new version. The post explains that under heavy write loads, this results in significant write amplification, and it increases read amplification too, since queries must scan through dead tuples to find the latest version. A single-primary architecture inherits every one of those costs on one machine.

Callout: the 12-table join that kept paging engineers

OpenAI's team says it once identified "an extremely costly query that joined 12 tables," and that spikes in this query were responsible for past high-severity incidents (SEVs). Their stated conclusion is blunt: avoid complex multi-table joins whenever possible. For teams debugging their own Postgres SEVs, this is a useful diagnostic starting point — check for join complexity before reaching for more hardware.

Connection pooling: the 50ms-to-5ms fix

Every managed Postgres instance caps how many connections it can hold — 5,000 on Azure PostgreSQL, per the post — and OpenAI says it has previously had incidents caused by connection storms that exhausted all available connections. Its fix is PgBouncer, run in statement or transaction pooling mode so many client connections share a small pool of real database connections.

The reported payoff is specific: OpenAI says average connection time dropped from 50 milliseconds to 5 milliseconds in its own benchmarks after adopting this pooling mode — a 10x reduction the team attributes directly to reusing connections instead of opening new ones per request. The post describes each read replica running its own Kubernetes deployment of multiple PgBouncer pods, with several deployments sitting behind one Kubernetes Service that load-balances traffic across them.

Cache locking to stop the thundering herd

Read replicas help, but only if the cache in front of them is doing its job. OpenAI describes a cache locking (and leasing) mechanism designed so that when a value falls out of cache, only one reader fetches it from PostgreSQL. When multiple requests miss the same cache key simultaneously, the post says, only one request acquires the lock and repopulates the cache — the rest wait rather than all hammering the primary at once.

Nearly 50 replicas, and a plan for a hundred more

OpenAI currently runs close to 50 read replicas spread across multiple geographic regions to keep latency down, according to the post. The catch: in the current architecture, the primary has to stream write-ahead log (WAL) data to every one of those replicas directly, which puts a ceiling on how far the fan-out can grow before the primary itself becomes the bottleneck.

The team describes moving toward a cascading replication approach — replicas feeding other replicas rather than all of them pulling from the primary directly — that it says "allows us to scale to potentially over a hundred replicas without overwhelming the primary." The tradeoff, OpenAI acknowledges, is "additional operational complexity, particularly around failover management": more replication hops means more places a failover can go wrong.

Guardrails: schema changes on a leash

At this scale, an unbounded schema migration can lock a table long enough to cascade into an outage. OpenAI's post says only lightweight schema changes are permitted in production — adding or removing certain columns that don't trigger a full table rewrite — and that the team enforces a strict five-second timeout on schema changes. Anything riskier has to go through a different, presumably more controlled, path.

What the numbers add up to

Put together, OpenAI reports the following results for the current architecture. All figures below are the company's own reporting on its own system, with no independent benchmark cited in the post.

Technique	Problem it addresses	Reported impact
Avoiding multi-way joins	CPU-saturating queries causing SEVs	Eliminated a recurring class of incidents tied to a 12-table join
PgBouncer pooling	Connection storms hitting the 5,000-connection cap	Average connection time fell from 50ms to 5ms
Cache locking/leasing	Thundering-herd reads on cache misses	Only one request per key fetches from Postgres on a miss
Nearly 50 read replicas	Read scaling across regions	Millions of queries per second served, per the post
Cascading replication (planned)	Primary overwhelmed by direct WAL fan-out	Path to 100+ replicas without new primary load
Schema-change guardrails	Long-running migrations locking tables	Lightweight changes only, 5-second timeout enforced

OpenAI says the combined result is low double-digit millisecond p99 client-side latency and five-nines availability in production, with only one SEV-0 PostgreSQL incident over the past 12 months — a period that included, by the post's own account, a write-traffic surge of more than 10x when ChatGPT's image generation feature drove over 100 million new signups within a week.

Sharding vs. read replicas: the actual decision

None of this makes sharding wrong. It makes single-primary-plus-replicas viable for longer than most teams assume, provided the workload is read-heavy and the write path can be kept lean.

The pattern worth borrowing is the order of operations: fix the query patterns and connection handling first, then scale reads horizontally, and only reach for sharding once a single write node is the actual bottleneck — not before. Teams whose writes are growing as fast as OpenAI's reads, or whose data doesn't fit a single logical primary for compliance or geographic reasons, are the ones for whom this playbook runs out first. Everyone else weighing the two options should read OpenAI's replica count as a floor to test against, not a ceiling to copy: "over a hundred replicas" is the company's own stated next step, not its final answer.

What to watch

• Whether OpenAI's cascading replication rollout actually ships without a failover incident of its own — the team flagged that operational risk itself.
• Whether the "avoid multi-way joins" rule survives contact with new product features that need cross-entity queries.
• Whether write growth from image and agent features outpaces what read-replica scaling alone can absorb, forcing an actual sharding decision.