Kimi K3 Cost Model: Hosted API Versus Self-Hosting

Finance asks for one number: at what monthly volume does self-hosting Kimi K3 beat the hosted API?

Part four explained why the downloadable checkpoint occupies 1.561 TB. That storage floor now becomes one line in a larger ownership decision.

The API price table looks simple:

$0.30 per million cached input tokens
$3.00 per million uncached input tokens
$15.00 per million output tokens

Self-hosting looks simple too: find a GPU price, multiply it by eight, and compare the result with the API bill.

Both shortcuts hide the variables that decide the answer.

Use a transport mental model. A hosted API is metered travel: you pay for each trip. Self-hosting is owning the fleet: you pay while vehicles wait, and you own maintenance and recovery.

The API bill depends on prefix-cache reuse, reusing computation for an identical prompt beginning, and completion tokens, the generated reasoning plus visible answer. The cluster bill depends on a topology, the GPU count plus links between those devices, large enough to hold a 1.561 TB checkpoint. A validated task is one completed request whose output passes the predeclared correctness check. The cluster bill also depends on measured throughput, validated tasks completed per second; latency, time per task; paid idle capacity; redundancy; storage; networking; and engineering.

Serving capacity is the measured useful work the deployment can sustain. Utilization is the share of that capacity useful work actually consumes. Availability is the fraction of time the service can accept work.

There is no defensible universal Kimi K3 break-even token count. Break-even is a surface across workload, cache behavior, throughput, utilization, and availability.

The post uses four evidence labels:

• Official fact: a dated price, rate limit, license, artifact, or recipe.
• Derived calculation: arithmetic from official or measured inputs.
• Planning estimate: an explicit workload or ownership assumption.
• Hypothesis: an unmeasured belief about cache behavior, throughput, or demand.

Before using the calculator, enter observed API usage first. Then enter an all-in topology quote and throughput measured on the same prompt, output, reasoning, latency, quality, and availability target. Leave unknown fields blank; a missing measurement is more useful than invented precision.

After the calculator, read the result in this order: quality parity, capacity feasibility, then cost. A cheap configuration that fails validators or cannot serve the volume has no valid break-even point.

The calculator uses cost per validated task. I rechecked the three token prices and the published rate-limit tiers on August 3; the prices had not changed.

The hosted API side is exact only for a dated price sheet

Official fact — dated price sheet. The official Kimi K3 pricing page for the Global Kimi Open Platform (platform.kimi.ai, API endpoint api.moonshot.ai) lists:

This is a USD, pay-as-you-go snapshot checked on August 3, 2026. Prices are per 1,000,000 tokens and exclude applicable taxes. The account's cumulative top-up tier changes concurrency and rate limits; the public page does not list different K3 token prices by tier. Cache-hit and cache-miss input are separate billing categories.

Derived calculation — request cost. For one request:

API cost =
  uncached_input_tokens × $3.00 / 1,000,000
+ cached_input_tokens   × $0.30 / 1,000,000
+ completion_tokens     × $15.00 / 1,000,000

If:

• I is total prompt tokens;
• H is cached prompt tokens;
• C is completion tokens;

then:

cost = ((I - H) × 3 + H × 0.3 + C × 15) / 1,000,000

The API response's usage object is the authoritative source for actual prompt, cached, and completion tokens. Forecasts help with planning. Billing should be reconciled against observed usage.

Recheck the price sheet, platform region, tax treatment, cache policy, and rate-limit rules before using this model for a publication or budget decision.

K3 always reasons. Moonshot documents low, high, and max effort, but it does not publish a deterministic completion-token multiplier for each level. Avoid fixed ratios such as:

high = 2 × low
max = 4 × low

unless your own workload measurements support those ratios.

The cost calculator therefore asks for completion tokens as one observed or planned value. It does not invent a hidden-reasoning coefficient.

Prefix caching changes input cost by 10×

Cached input is 0.30/M. Uncached input is 3/M. That 10× difference can matter more than small provider price gaps.

Take an 8,000-token prompt and a 1,000-token completion.

No cache hit

input  = 8,000 × $3 / 1M  = $0.024
output = 1,000 × $15 / 1M = $0.015
 
total = $0.039/request

90% of input tokens cached

cached input   = 7,200 × $0.30 / 1M = $0.00216
uncached input =   800 × $3.00 / 1M = $0.00240
output         = 1,000 × $15.00 / 1M = $0.01500
 
total = $0.01956/request

At 100,000 requests:

These are arithmetic scenarios, not a claim that your application will reach a 90% hit rate.

Worked decision scenario. Suppose those 100,000 requests are your measured monthly workload. The API side is now a derived $1,956 at 90% cached input. If the all-in cluster quote or validated local throughput is still blank, the self-host side is unknown—not zero. The decision is to keep measuring, not to announce a break-even volume.

That is the junior stop rule: no topology quote plus no workload-matched throughput means no break-even answer. The next sections show what those two inputs must include.

Optional finance and architecture depth: the decision table near the end is the fast path. The sections between build the auditable rate-limit, ownership, capacity, and break-even model.

Official reported result, not a local measurement. Moonshot says its official API exceeds a 90% cache-hit rate in coding workloads in the K3 launch post. It does not publish enough workload detail to use that figure as a calculator default.

The context-caching documentation gives the conditions you can model:

• caching is automatic;
• the previous prompt must exceed 256 tokens;
• identical leading context can be reused;
• appending preserves earlier prefix eligibility;
• modifying earlier history invalidates reuse after the change;
• changing reasoning effort resets cache reuse;
• usage.cached_tokens exposes the result.

The correct production input is:

observed cache fraction =
sum(cached_tokens) / sum(prompt_tokens)

Calculate it by request class. A coding agent with a stable repository prefix can behave differently from a short consumer chat.

Completion tokens often dominate the bill

In the 90%-cached example:

input cost  = $0.00456
output cost = $0.01500

The output is 76.7% of total request cost even though the prompt contains eight times more tokens.

K3's completion budget covers reasoning_content plus visible answer. Use the response's usage.completion_tokens as the generated-output quantity in the cost equation and reconcile it with billing records. The reasoning documentation does not expose a separate billed reasoning price.

This creates a product trade-off:

• stable long prefixes make input cheap;
• long reasoning can still make the request expensive;
• reducing visible answer length does not guarantee a short completion;
• changing reasoning effort can save output tokens but reset prefix caching.

Measure the combination, not each field in isolation.

A useful telemetry row contains:

request class
reasoning effort
prompt tokens
cached tokens
completion tokens
latency
finish reason
tool steps
success/failure outcome

Without task success, cost per token is a vanity metric. The useful unit is cost per completed task at an acceptable quality and latency.

Rate limits can block the API before price does

Moonshot's rate-limit table defines concurrency, requests in flight at once; requests per minute (RPM); tokens per minute (TPM); and daily tokens by recharge tier.

Admission uses prompt tokens plus max_completion_tokens, not the completion length you hope the model will produce. The platform introduction documents this distinction.

Derived admission example. Suppose a request has:

8,000 prompt tokens
16,384 max completion tokens

It reserves roughly 24,384 tokens for token-per-minute admission. Leaving the 131,072 default would reserve about 139,072 instead.

The bill can remain small because actual generation ends early while admission still fails because the declared maximum is large.

For an average request:

reserved tokens = prompt tokens + max completion tokens

The maximum request rate is bounded by the smallest of:

RPM / 60
TPM / reserved_tokens / 60
concurrency / average_service_seconds

Add the daily token limit where the tier has one.

The result is a feasibility constraint, not a billing equation. A workload can be affordable and still exceed the hosted service's current rate limits.

Self-hosting starts with a 1.561 TB placement problem

Part one established the released hardware floor:

• a 1.390 TB flat four-bit logical lower bound;
• 1,560,936,091,448 bytes across the 96 visible weight shards;
• at least eight GB300 or MI350X/MI355X GPUs in the vLLM recipe;
• 32 H100s, 16 H200s, or 16 B200s in published SGLang shapes.

The SGLang cookbook still marked its recipe cells Final Verification In Progress on July 30. A documented launch shape is evidence that the topology was being brought up, not a promise that your workload meets its throughput or reliability target.

A public listing such as “H100: $X/GPU-hour” does not prove that the provider offers:

• 32 devices in one usable deployment;
• the required NVLink or RDMA fabric, the high-speed links that carry cross-GPU traffic;
• enough CPU RAM and local storage;
• the needed engine image and driver versions;
• capacity in your region;
• a service-level objective (SLO), a stated latency or availability target;
• the same price for reserved capacity or a failover deployment.

The calculator therefore asks for an all-in cluster-hour quote. If your contract is per GPU, multiply by GPU count once. If the quote is already per node or cluster, do not multiply it again.

The self-host bill has fixed and variable parts

For one month:

self-host cost =
  GPU or cluster compute
+ storage
+ orchestration/control plane
+ network and data transfer
+ monitoring
+ engineering and on-call
+ support or license cost
+ genuinely variable request cost

The calculator groups non-compute fixed costs into one explicit input because the categories vary by organization. Keeping the field visible is more honest than silently setting every category to 10% of GPU spend.

Convert staffing into the same monthly period:

monthly ownership cost =
loaded annual engineer cost × allocated FTE / 12

Full-time equivalent (FTE) is the share of one engineer's working capacity assigned to the service. The allocation includes engine upgrades, security review, observability, capacity planning, incident response, and on-call coverage. If those duties already belong to a platform team, use the fraction of that team's real capacity. Do not enter zero because payroll sits in another budget.

A standby deployment is paid capacity kept ready to take traffic after a failure. Compute is:

cluster hourly price
× billed hours
× active and paid standby deployments

For a continuously allocated month, the calculator uses 730 hours. If your contract bills differently, replace the period.

Idle time is already inside billed compute. Do not add an “idle cost” line on top and count the same hours twice. Report idle capacity as an allocation:

idle allocation = billed compute × (1 - achieved utilization)

This shows why utilization matters without changing the invoice.

Throughput must be measured on the same workload

The launch reports publish impressive K3 serving numbers, but not every number can enter a cost model.

The vLLM release post reports single-user decode rates with and without speculative decoding. The SGLang report reports batch-one decode and a large-scale prefill/decode frontier.

Prefill processes the input prompt. Decode generates new tokens one at a time. Their token rates are different measurements and neither automatically equals completed tasks per second.

These metrics answer different questions:

• output tokens per second for one user;
• aggregate tokens per second across a deployment;
• prefill input throughput;
• decode output throughput;
• tokens per GPU;
• requests per second at a latency target.

Do not convert single-user decode tokens per second into completed requests per second.

The self-host calculator requires:

measured validated successful tasks / second

for the same:

• Kimi K3 revision;
• inference-engine revision and image;
• hardware topology;
• precision and cache settings;
• input and output distributions;
• reasoning effort;
• prefix-cache state;
• tool pattern;
• concurrency;
• latency objective;
• success criteria.

If your benchmark uses 8K random input and 1K output, it supports that workload. It does not support a claim about 500K-token repository agents.

If your evaluation runner reports raw completions, convert them first:

validated task throughput =
attempted requests/second × task pass rate

A faster local engine that fails more validators is not cheaper. It is doing less useful work.

Match quality and reliability before comparing invoices

This is gate zero. If the two options do not meet the same task-quality, latency, and reliability target, stop comparing their invoices.

The hosted API and the released checkpoint are related products, not automatically identical services. A fair comparison pins or records:

• hosted model ID, date, reasoning effort, tool schemas, and response format;
• self-hosted model revision, engine image, parser, cache precision, and speculative-decoding settings;
• the same prompt/output distribution and deterministic task validators;
• p50 latency, the median request time, and p99 latency, the threshold within which 99% of requests complete, plus timeout policy and retry budget;
• the same availability target and recovery objective.

Use cost per validated task, the bill divided by tasks that pass the agreed validator, as the primary unit. Report cost per token only as a diagnostic.

Reliability also needs symmetry. A managed API may still have overloads and rate limits, but comparing it with one self-hosted cluster and no failover prices different failure envelopes. If the target needs another failure domain, include a paid standby or second active deployment, storage copies, health checks, and tested recovery time.

Capacity turns self-hosting into a step function

Let:

• q be measured validated successful tasks per second for one deployment;
• D be active deployments;
• u be achieved fraction of benchmark capacity;
• a be serving availability;
• S be seconds in the billing period.

Then:

monthly request capacity = q × D × u × a × S

If volume exceeds one deployment's capacity, the next unit is not a fraction of a GPU. It can be another eight-, 16-, or 32-GPU deployment.

required deployments =
ceil(required requests / capacity per deployment)

That produces a staircase:

one deployment  → fixed cost F
two deployments → fixed cost 2F
three           → 3F

A smooth self-host “cost per token” line hides these boundaries. Cost drops as the deployment fills, then jumps when another full topology is required.

Break-even is a formula with conditions

Let:

• A be weighted API cost per validated task;
• V be self-host variable cost per validated task;
• F be fixed self-host cost for the period.

Then:

break-even validated tasks = F / (A - V)

The equation only applies when:

1. A > V;
2. the break-even volume fits deployment capacity;
3. both sides run the same workload and quality settings;
4. the API rate limits can serve the comparison volume;
5. currency, taxes, and billing periods match.

If A <= V, fixed self-host costs can never be recovered through request volume.

If break-even exceeds measured capacity, the calculation must add another deployment and run again.

Another useful form asks how much compute you can afford:

maximum cluster-hour price =
(validated volume × (API cost/task - self variable cost/task)
 - non-compute fixed cost)
/ billed deployment hours

A negative result means self-hosting cannot break even at that volume even with free compute.

Use scenarios, not one forecast

Planning estimates, not observations. Every uncertain input should have at least three values:

Do not call the high case “best” if it assumes both perfect cache reuse and maximum cluster utilization. Correlated assumptions matter. Large contexts can improve cache value while reducing throughput.

The useful output is a small matrix:

API monthly cost by cache/output scenario
self-host monthly cost by price/utilization scenario
capacity status
break-even requests
largest input sensitivity

The calculator on this page intentionally refuses to prefill cluster price or throughput. Those are the two values most likely to turn a source-backed tool into fabricated precision.

The license can change self-host economics

Kimi K3 uses a custom Kimi K3 License.

It broadly permits use, modification, distribution, deployment, and fine-tuning, but adds commercial conditions:

• a model-as-a-service business whose licensee-plus-affiliate revenue exceeds $20M over any consecutive 12 months must obtain a separate Moonshot agreement before commercial use;
• products above 100M monthly active users or $20M monthly revenue must prominently display “Kimi K3”;
• internal use and use through Moonshot or certified partners receive stated exceptions.

This post is an engineering cost model, not legal advice. The operational point is simple: license review can produce a real cost or deployment constraint. Do not keep it outside the spreadsheet because the weight download is public. These terms were checked on August 3, 2026. Re-read the current license and obtain qualified legal review before a commercial deployment or publication.

Choose the hosted API when flexibility is worth more than utilization

The default should not be ideological. Open weights do not require self-hosting, and a convenient API does not remove platform risk.

Modeled failure cases expose bad comparisons

These are planning scenarios, not incident reports. Each one invalidates an input assumption and should trigger a fresh calculation.

• Prefix-cache savings disappear. A timestamp, reordered tool definition, or reasoning-effort change invalidates reuse.
• Completion tokens grow under long reasoning. Input becomes cheap while output dominates the bill.
• A large max_completion_tokens causes 429s. Admission reserves more tokens than the response actually emits.
• A GPU quote cannot supply the topology. Per-GPU price is meaningless without fabric and capacity.
• Single-user decode speed is used as request throughput. The self-host capacity estimate becomes fictional.
• Standby capacity is omitted. The cost model compares a managed API surface with one fragile cluster.
• Storage and cold starts are ignored. The 1.561 TB artifact must be downloaded, verified, staged, and recovered.
• Engineering cost is set to zero by accident. Someone still owns engine updates, parsers, kernel regressions, capacity, and incidents.
• Demand crosses a deployment boundary. Fixed cost jumps instead of scaling smoothly.
• License conditions are discovered after launch. Commercial constraints can change the viable serving model.

The right answer can change month to month as cache behavior, output length, provider limits, and serving engines improve. Store the inputs and verification date with the result.

A break-even number without workload equivalence is not a decision. It is a spreadsheet-shaped guess.

Series takeaway

Kimi K3's five operational truths now line up:

1. the released checkpoint cannot fit on eight H100s;
2. Kimi Delta Attention (KDA) uses fixed recurrent state, Multi-Head Latent Attention (MLA) keeps a token-growing key-value (KV) cache, Attention Residuals (AttnRes) select across depth, and LatentMoE uses mixture-of-experts (MoE) routing, assigning each token to selected experts; each mechanism moves a different bottleneck;
3. the API requires exact replay of reasoning and tool state;
4. 1.56 TB follows from mixed MXFP4, grouped 4-bit floating-point storage, scales, and higher-precision modules;
5. API versus self-hosting depends on measured tasks, not token prices alone.

The full reading path lives on the Kimi K3 series page.

The Kimi K3 Lab turns the architecture and serving claims into draft experiments with explicit conclusion limits.

Sources and References

Hosted API

• Kimi K3 token prices and context: Kimi K3 pricing
• Automatic prefix caching: Context-caching guide
• Reasoning effort: Reasoning-effort guide
• Rate limits: Recharge and rate limits
• Token admission behavior: Kimi API introduction

Self-hosting

• Checkpoint size: Pinned Hugging Face model API
• Minimum vLLM topology: vLLM Kimi K3 recipe
• Published SGLang topologies: SGLang K3 cookbook
• vLLM performance context: vLLM Kimi K3 release
• SGLang performance context: SGLang Kimi K3 report

Commercial use

• Kimi K3 commercial conditions: Kimi K3 License