Self-Hosted vs API LLM Calculator

Understanding the Self-Hosted vs API LLM Calculator

A hosting comparison calculator contrasts monthly API token spend against a fixed server cost and finds the break-even volume. The break-even is real and it is only half the comparison, because a self-hosted server costs the same whether it runs at full utilisation or idle.

How it actually works

Enter monthly requests, tokens per request, API cost per million tokens, and monthly server cost. The calculator computes API spend, compares it to the server, and finds the request volume where they meet. A hundred thousand requests at a thousand tokens each costs $200 against a $600 server.

The deeper context most people miss

API pricing is purely variable and self-hosting is largely fixed, which is why the comparison is a break-even rather than a simple price difference. Below the crossover the API wins on cost and above it self-hosting does, and utilisation is what determines which side you are on.

What the server cost figure has to cover

A monthly server figure understates self-hosting substantially unless it is built carefully. GPU instance cost is the visible part, and hardware suitable for serving a mid-sized model runs from moderate to very expensive depending on the model and required throughput, with VRAM being the binding constraint since a model and its key-value cache must fit in memory. Redundancy multiplies it, since a single instance means an outage takes the service down entirely, and any production deployment needs at least failover. Utilisation is the crux: an instance reserved continuously costs the same at 5% load as at 95%, so a service with peaky traffic pays for capacity it uses briefly, and autoscaling GPU instances is slower and clumsier than scaling stateless web servers because model loading takes time. Engineering time is frequently the largest line and the most often omitted: setting up inference serving, batching, quantisation, monitoring, and deployment pipelines is real work, and keeping current with a fast-moving ecosystem is ongoing. Operational burden includes on-call responsibility for a service that was previously somebody else's problem. Model updates require re-evaluation rather than arriving automatically. Against all this, self-hosting brings genuine benefits that a cost calculation misses entirely: data never leaves your environment, which is decisive for some regulatory positions; latency can be lower and more predictable; there is no rate limiting or dependency on a provider's availability; and the model does not change underneath you, which matters for reproducibility.

A worked example: where the crossover actually sits

At $200 of API spend against a $600 server, the API is clearly cheaper and the break-even sits around 300 million tokens monthly. Reaching that requires sustained volume, and the more useful question is what the load profile looks like rather than the total. A service running continuously near capacity approaches the theoretical break-even. A service with traffic concentrated in business hours in one timezone uses perhaps a third of the reserved capacity, so its effective break-even is roughly three times higher. Batch workloads that can be queued and run when convenient use hardware efficiently and reach break-even sooner, which is why offline processing self-hosts more readily than interactive serving. Throughput per instance depends heavily on choices: batching multiple requests together amortises the weight reads that dominate inference cost and can improve throughput several-fold, quantisation reduces memory and increases speed with some quality cost, and serving frameworks including vLLM and similar implement continuous batching and paged attention that substantially outperform naive serving. A poorly configured self-hosted deployment can cost more per token than an API while providing worse latency, which is a common outcome for teams that measure the hardware cost and not the achieved throughput. Measuring tokens per second per dollar on your actual workload is the only reliable comparison.

Deciding which model fits your situation

The decision is rarely purely financial. API suits variable or uncertain volume, small teams without infrastructure capacity, workloads needing frontier capability that open models do not match, and anything where speed to build matters more than unit cost. Self-hosting suits sustained high volume, data that cannot leave an environment for regulatory or contractual reasons, workloads needing a specific fine-tuned model, latency-sensitive applications where network round trips matter, and situations where provider dependency is unacceptable. Hybrid arrangements are common and frequently the right answer: routing bulk simple work to a self-hosted small model while sending complex requests to an API captures most of the cost saving without needing to match frontier capability locally. Rented GPU sits between owned hardware and API, avoiding capital cost while retaining control, with hourly rates making the utilisation question explicit. Before committing either way, several intermediate steps usually reduce cost more than the hosting decision does: routing to smaller models, prompt caching, trimming context, and capping output length frequently cut API spend by more than half, which can move a workload from above the break-even to well below it. Doing that optimisation first means the hosting comparison is made against an efficient baseline rather than a wasteful one.

What open-weight models can and cannot do

The capability gap between open-weight and frontier proprietary models has narrowed considerably and has not closed, and the honest position is that it varies by task. For many practical applications including classification, extraction, summarisation, and routine generation, well-chosen open models perform comparably at a fraction of the cost, and this is where self-hosting is most defensible. For tasks requiring the strongest reasoning, long-context handling, or breadth of knowledge, frontier models retain an advantage that matters. The gap also moves quickly in both directions as new models release, so any assessment ages fast and should be re-tested rather than assumed. Model licensing varies and deserves attention: open weights does not mean open source in the conventional sense, and several prominent model licences carry restrictions on commercial use above certain thresholds, on training other models, or on specific applications, so reading the licence before building on a model matters. Fine-tuning open models is where self-hosting frequently earns its place, since a small model fine-tuned on a specific task can outperform a much larger general model on that task at far lower serving cost. Evaluation on your own data rather than public benchmarks is essential, since benchmark performance correlates imperfectly with task performance and benchmark contamination is a recognised problem.

Variations: serving frameworks, quantisation, and deployment options

Serving frameworks differ substantially in throughput, with implementations offering continuous batching, paged attention, and optimised kernels achieving several times the throughput of naive approaches on identical hardware. Quantisation reduces weights to lower precision, commonly 8-bit or 4-bit, cutting memory requirements substantially and increasing throughput with a quality cost that varies by model and method, and it frequently determines whether a model fits available hardware at all. Speculative decoding uses a small model to draft tokens verified by a larger one, improving latency. Model parallelism splits large models across multiple GPUs. On deployment, managed inference services sit between API and self-hosting, running open models on your behalf without infrastructure work, at a price between the two. Serverless GPU offerings reduce the idle cost problem with cold start latency as the trade-off. On-premises deployment serves regulatory requirements. Edge deployment on local hardware suits privacy-sensitive and offline applications, with small models now running usefully on consumer devices. For any of these, benchmarking on representative traffic is what distinguishes them, since throughput claims rarely transfer between workloads.

Comparing self-hosting against an API

Build the server figure properly, including redundancy, engineering time to set up and maintain, and the on-call burden, since a bare instance price substantially understates it. Account for utilisation rather than capacity, since a reserved instance costs the same idle and a service running a third of the time has an effective break-even three times higher. Optimise API usage before comparing, since routing to smaller models, prompt caching, and trimming context frequently halve spend and can move a workload below the break-even entirely. Measure tokens per second per dollar on your actual workload, since a poorly configured self-hosted deployment can cost more per token than an API while performing worse. Use a serving framework with continuous batching rather than naive inference, which can multiply throughput several-fold on identical hardware. Consider a hybrid, routing bulk simple work locally and complex requests to an API, which captures most of the saving without matching frontier capability. Read the model licence, since open weights carries restrictions in several prominent cases. And evaluate on your own data rather than public benchmarks.

What people get wrong

• Comparing the API bill against a bare GPU instance price, when redundancy, engineering time, and operational burden frequently exceed the hardware cost.
• Calculating break-even from capacity rather than utilisation, when a reserved instance costs the same idle and peaky traffic raises the effective crossover substantially.
• Self-hosting before optimising API usage, when routing, caching, and context trimming frequently halve spend and can move a workload below the break-even entirely.
• Assuming open weights means unrestricted use, when several prominent model licences restrict commercial use above thresholds, training other models, or specific applications.

Where the math comes from

API Monthly Cost = Monthly Requests × Tokens per Request / 1,000,000 × Cost per Million. Break-Even Requests = (Server Monthly Cost / Cost per Million × 1,000,000) / Tokens per Request. This treats the server cost as fully utilised; at lower utilisation the effective break-even rises proportionally, since a reserved instance costs the same whether busy or idle.

Questions and answers

Related calculators

Brick Count · Rent Increase · Laundry Load · Noise Level · Clothing Size Converter