text-embedding-3-small vs large: Which to Start With

If you are picking an embedding model for a multi-tenant RAG chatbot, start with text-embedding-3-small and move up only on evidence. A guide published October 24 puts concrete numbers on that call: 1,536-dimensional vectors at a stated price of $0.02 per 1M tokens, which the author frames as a workable quality-cost balance for semantic search.

The larger option, text-embedding-3-large, targets higher retrieval quality and costs 5x more in the same supplied figures. Both numbers come from the guide's own model facts rather than an independent benchmark, so treat them as the starting point for a quote, not a price sheet.

When the upgrade is worth it

The trade is reasonable where missed evidence is more damaging than indexing cost — legal discovery, incident corpora, anything where a retrieval miss is the failure. It is a poor default when the product requirement is per-tenant cost visibility and most tenants hold ordinary support or product documentation.

The guide's decision rule is narrow and worth copying: start small, then promote only the tenants or collections whose retrieval evaluation shows a meaningful miss rate. That means you need a miss-rate measurement before the upgrade conversation, not after.

The boundary matters more than the model

The design choice that actually drives the bill is the split between indexing, retrieval, and answer generation. At ingestion, chunk a tenant's knowledge base and send the chunks through the embeddings endpoint, storing vectors alongside tenant and document identifiers. At chat time, embed the query, search within the permitted tenant scope, then pass retrieved context to the chat model.

That separation gives operations a clean ledger. A re-index is an ingestion cost. A question carries a query-embedding cost, vector search work, and an answer-generation cost. Record each event with tenant ID, document or conversation ID, model, token count, and a request identifier — and do not infer tenant ownership from whichever worker happened to process the job. Get this wrong and a shared knowledge base looks cheap while one customer quietly subsidizes another.

Two things to get right on day one

The index is also a security boundary. The tenant filter belongs in the retrieval query and in the authorization check around the source documents. The guide points at OWASP's LLM guidance for the reason: retrieved content is untrusted input, not an instruction channel. A relevant paragraph from the wrong tenant is a production incident regardless of how good the answer reads.

Second, keep the model choice in the index metadata. Vectors from models with different dimensions should not be mixed in one index without an explicit migration plan — which is why this decision happens before the index is built, not after it has a year of documents in it.

One category error worth avoiding: Pinecone and Weaviate are vector databases in this design. They solve storage and search. They do not choose the embedding model for you, and swapping one in does not retire the migration problem above.

What to do this week

Commit to text-embedding-3-small for the first production version, write the model name into index metadata, and stand up per-tenant logging before the second tenant arrives. Revisit the large model when you have a measured miss rate to argue with — and price the re-index into that decision, because changing dimensions later is a backfill, not a config change.