~/blog

An 80% token price cut is not an 80% smaller bill

published

#llm#cost#api

Two shapes on a dark background: a short neon green downward arrow on the left, and on the right a tall dark grey column with only a small pink slice removed from its top.

TL;DR

On 2026-07-30 OpenAI cut the price of its two cheaper GPT-5.6 tiers — the cheapest by 80%, the mid tier by 20% — and left the flagship alone. A headline cut on the input side moves your invoice far less than it sounds, because output tokens are billed at 6× input across the whole lineup, and because the things that actually inflate an LLM bill (resent conversation history, retries, and JSON you asked for and then discarded) are billed at full price either way. Price per million tokens is an input to your cost model. It is not your cost.

The problem

The published numbers, per million tokens:

ModelInputOutputOutput ÷ input
GPT-5.6 Luna$0.20$1.20
GPT-5.6 Terra$2.00$12.00
GPT-5.6 Sol$5.00$30.00

Two things fall out of that table immediately, and neither is in the headline.

The ratio is constant. Every tier charges 6× for output. So “which model” changes your bill by a scale factor, but what you ask it to produce changes the shape of it. Moving Sol → Luna is 25× cheaper on input; if your workload is output-heavy, you feel that as 25× on output too — but only on the tokens you actually generate.

The cheapest tier got the biggest cut, which is not generosity. An 80% cut on the tier already priced at pennies is the tier where volume lives — classification, extraction, routing. The flagship, where the margin is, did not move.

Why your bill doesn’t follow the headline

The unit price is the smallest term in the equation most people are actually solving. The real one:

cost_per_request
  = (prompt_tokens + context_tokens) × input_price
  + output_tokens                    × output_price   ← 6× the input rate
  
cost_per_task
  = cost_per_request × (1 + retry_rate)

Three multipliers hide in there, and none of them are affected by a price cut:

1. Conversation history is re-billed on every turn. The API is stateless. A 10-turn chat does not send one conversation — it sends turn 1 again at turn 2, turns 1–2 again at turn 3, and so on. Your input tokens grow roughly quadratically with turn count while the user thinks they are having one conversation.

2. Retries are billed. A malformed JSON response you reject and re-request costs you twice: the failed generation is billed in full. A 20% retry rate is a 20% surcharge on everything, invisible on the pricing page.

3. Output you throw away costs the same as output you keep. Asking for a full explanation and then regexing one number out of it means you paid the 6× rate for the prose you deleted.

What to do

1. Measure output/input ratio before you pick a tier. Log both counts for a representative sample. If your workload is 10:1 input-heavy (classification, extraction), the input price cut is genuinely most of your bill and Luna is a real win. If it is output-heavy (drafting, code generation), you are mostly paying the 6× rate and the input cut barely registers.

// Log the split, not just the total — the ratio is what picks your tier.
const r = await client.responses.create({ model: 'gpt-5.6-luna', input });
const { input_tokens, output_tokens } = r.usage;
console.log({ input_tokens, output_tokens, ratio: output_tokens / input_tokens });

2. Cap output explicitly. max_output_tokens is a cost control, not just a safety rail. Combine it with a schema so the model cannot pad:

const r = await client.responses.create({
  model: 'gpt-5.6-luna',
  input,
  max_output_tokens: 200,
  text: { format: { type: 'json_schema', name: 'verdict', schema: {
    type: 'object',
    properties: { label: { type: 'string' }, confidence: { type: 'number' } },
    required: ['label', 'confidence'],
    additionalProperties: false,
  }, strict: true } },
});

Structured output pays for itself twice: the model stops emitting preamble you discard, and strict schemas remove most of the retry rate.

3. Stop resending what has not changed. Summarise old turns, or cache the stable prefix. Both major providers bill cached input at a steep discount — check the current rate rather than trusting a number in a blog post, including this one.

4. Price the task, not the token. Convert to cost per unit of work you care about — per support ticket, per document, per PR reviewed. That number survives a pricing change; “$0.20 per million” does not, and it is the only figure you can put next to what the task is worth to you.

Caveats

References