Embedding Cost Calculator

Understanding the Embedding Cost Calculator

An embedding cost calculator multiplies documents by average tokens and applies a per-million-token price. Embedding is usually the cheapest part of a retrieval system, and the costs that actually dominate sit elsewhere, which is worth knowing before budgeting.

How it actually works

Enter document count, average tokens per document, and a model tier. The calculator multiplies for total tokens and applies a price per million. One thousand documents at 1,000 tokens each on the mid tier gives 1,000,000 tokens and $0.02.

The deeper context most people miss

Embedding a million tokens for two cents is genuinely cheap, and the same corpus queried a thousand times a day with retrieved context passed to a generation model will cost orders of magnitude more in inference than it ever did in embedding. Budgeting a retrieval system from embedding cost alone misses where the money goes.

What embeddings are and why chunking matters more than model choice

An embedding maps text to a vector of numbers positioned so that semantically similar text sits nearby, which is what allows retrieval by meaning rather than keyword. The practical quality of a retrieval system depends less on which embedding model is used, since the leading options perform broadly similarly on general text, than on how documents are split before embedding. Chunking is the decision that determines what can be retrieved. Chunks that are too large dilute the embedding, since a vector representing 2,000 tokens covering several topics sits in an average position that matches nothing precisely. Chunks that are too small lose the context needed to be useful, retrieving a sentence whose meaning depended on the paragraph around it. Common approaches include fixed-size chunking with overlap, which is simple and splits mid-thought; recursive splitting on structural boundaries such as sections, paragraphs, then sentences, which respects document structure better; and semantic chunking, which splits where embedding similarity drops, indicating a topic change. Adding surrounding context to each chunk, whether the document title, section heading, or a generated summary, materially improves retrieval and costs little. Overlap between adjacent chunks reduces the chance that an answer falls across a boundary. None of this is about the embedding model, and teams frequently switch models hoping to fix retrieval quality when the chunking strategy is the actual problem.

A worked example: costing a real system

Embedding a thousand documents costs two cents, which is not a budget item. Now consider running it. A vector database hosting a million vectors costs somewhere from nothing on a self-hosted open source option to tens or hundreds of dollars monthly on a managed service, depending on dimensions, index type, and query volume, and this is recurring rather than one-off. Each user query must itself be embedded, so a system handling ten thousand queries daily embeds ten thousand short texts daily, which remains cheap. The retrieved chunks are then passed as context to a generation model, and this is where cost concentrates: passing five chunks of 500 tokens each plus a system prompt and the question means perhaps 3,000 input tokens per query, and at ten thousand queries daily that is 30 million input tokens a day, which at typical generation model pricing costs orders of magnitude more than the entire corpus embedding. Reprocessing matters too: any change to chunking strategy, or switching embedding models, requires re-embedding the whole corpus, which is cheap at this scale and considerably less so at millions of documents. Storage of the original documents alongside vectors adds cost. The practical budgeting lesson is that embedding cost is a rounding error and query-time generation cost is the number to model.

Deciding whether retrieval is the right architecture

Retrieval augmented generation is the default answer to grounding a model in specific documents, and several alternatives suit particular situations better. Long-context models can now accept very large inputs, and for a corpus that fits, passing the whole thing avoids retrieval entirely along with its failure modes, at the cost of higher per-query token consumption and, in practice, degraded attention to material in the middle of very long contexts. Prompt caching, offered by several providers, makes repeatedly passing a large fixed context substantially cheaper, which shifts the calculation further toward long context for stable corpora. Fine-tuning teaches a model style, format, and task behaviour rather than facts, and it is frequently misapplied as a way to inject knowledge, where retrieval works better and updates without retraining. Structured queries against a database beat semantic retrieval whenever the question is actually structured, and a system answering questions about numeric data should query the data rather than retrieve text about it. Hybrid search combining semantic similarity with keyword matching, typically BM25, consistently outperforms either alone, particularly for queries containing specific identifiers, product codes, or names where exact matching matters and embeddings perform poorly. Reranking retrieved candidates with a cross-encoder improves precision substantially at modest cost and is one of the higher-value additions to a basic retrieval pipeline.

Dimensions, storage, and the practical constraints

Embedding dimensionality affects both quality and cost. Higher-dimensional vectors capture more nuance and consume proportionally more storage and memory, and increase query latency in exhaustive search. Several current models support dimensionality reduction, allowing a shorter vector to be truncated from a longer one with modest quality loss, which is a genuine cost lever at scale. Storage arithmetic is straightforward: a million vectors at 1,536 dimensions in 32-bit floats is roughly 6 gigabytes before index overhead, which matters for memory-resident indexes. Quantisation reduces this substantially, with binary and scalar quantisation cutting memory by a large factor at some accuracy cost, and it is standard practice at scale. Index type governs the speed-accuracy trade-off: exact search guarantees the true nearest neighbours and scales poorly, while approximate methods including HNSW and IVF return near-optimal results far faster, with tunable parameters trading recall against latency. For small corpora, exact search over a simple array is entirely adequate and avoids the operational complexity of a vector database entirely, which is worth remembering since many projects adopt heavyweight infrastructure for datasets that fit comfortably in memory. Metadata filtering, restricting search to documents matching structured criteria, is frequently essential and is implemented very differently across vector stores, so it is worth checking before choosing one.

Variations: model tiers, self-hosting, and multimodal embeddings

Provider pricing varies by model, with smaller and older embedding models costing less per million tokens and newer ones offering better quality or dimensionality flexibility. Open source embedding models can be self-hosted, eliminating per-token cost in exchange for infrastructure, and several perform competitively on standard benchmarks, with the MTEB leaderboard being the common reference for comparison. Self-hosting makes sense at high volume or where data cannot leave an environment, and adds operational burden. Multilingual models matter for non-English corpora, where English-optimised models perform poorly. Domain-specific models exist for code, legal, biomedical, and scientific text and frequently outperform general models within their domain. Multimodal embeddings map images and text into a shared space, enabling cross-modal retrieval. For query-time, the same model must be used for queries and documents, since vectors from different models are not comparable, which constrains model switching to a full re-embedding. Batch APIs from some providers offer reduced pricing for asynchronous embedding of large corpora, which suits initial ingestion where latency does not matter.

Budgeting and building a retrieval system

Model generation cost rather than embedding cost, since embedding a corpus is typically a rounding error while query-time context passed to a generation model dominates the recurring bill. Include vector database hosting, which is recurring and frequently exceeds the one-off embedding cost within weeks. Invest effort in chunking strategy rather than embedding model selection, since leading models perform similarly on general text while chunking determines what can be retrieved at all. Add context to chunks, such as document title and section heading, which materially improves retrieval for negligible cost. Use hybrid search combining semantic and keyword matching, which consistently outperforms either alone, particularly for queries containing names, codes, or identifiers. Consider reranking retrieved candidates, which improves precision substantially at modest cost. Check whether your corpus simply fits in a long context window with prompt caching, which avoids retrieval complexity entirely. And remember that switching embedding models requires re-embedding everything, since vectors from different models are not comparable.

What people get wrong

• Budgeting a retrieval system from embedding cost, when query-time generation calls typically cost orders of magnitude more and are recurring.
• Switching embedding models to fix poor retrieval, when chunking strategy is far more often the actual problem and leading models perform similarly on general text.
• Using semantic search alone for queries containing names, product codes, or identifiers, where keyword matching performs better and hybrid search outperforms either.
• Adopting a vector database for a small corpus, when exact search over an in-memory array is adequate below a substantial scale and avoids the operational complexity.

Where the math comes from

Total Tokens = Documents × Average Tokens per Document. Cost = Total Tokens / 1,000,000 × Price per Million, where the price is the figure you enter. The default of $0.02 per million reflects small-model embedding pricing as of September 2026; larger models and third-party providers have run around $0.10 per million, so check the pricing page you are actually billing against. This covers initial corpus embedding only and excludes vector database hosting, query-time embedding, reranking, and the generation model calls that typically dominate a retrieval system's running cost.

Questions and answers

Related calculators

AI ROI · GPU Memory Required · RAG System Cost · Fine-Tuning Cost · AI Latency