> ## Content Index
> Fetch the complete content index at: https://bullquill.com/llms.txt
> Use this file to discover other available public pages before exploring further.

# How to reduce LLM API costs: caching, batching and model routing
- URL: https://bullquill.com/how-to-cut-llm-api-costs/
- Published: 2026-10-23T07:17:30.000Z
- Updated: 2026-10-23T23:45:53.000Z
- Description: Log input, cached, output and thinking tokens first, then apply four techniques in Python. Cache reads cost 0.1x the normal input price or less at OpenAI and Anthropic, and batch APIs charge 50% of standard prices.
- Author: Rowan Ashby
- Tags: AI, Tutorial, How-To, OpenAI, Anthropic, Gemini, Prompt Caching, Batch API, #lang-en, #evergreen, #tr-how-to-cut-llm-api-costs

## Key takeaways

- To reduce LLM API costs, measure first: log input, cached, output and thinking tokens for every request, then price them per feature.
- Prompt caching is the biggest lever for apps with long, repeated prompts. Cache reads cost 0.1x the normal input price or less at OpenAI and Anthropic, according to their documentation.
- Batch APIs at OpenAI, Anthropic and Google charge 50% of standard prices for work that can wait up to 24 hours.
- Routing easy tasks to a small model and capping output length often saves more than any single provider discount.
- In our worked example (prices as of Oct. 23, 2026), the four steps cut a $1,900 monthly bill to about $198, a fall of roughly 90%. Your result will depend on traffic and prompt shape.

You can reduce LLM API costs by stacking four techniques: prompt caching, batch APIs, model routing and output limits. In a worked example, they cut a $1,900 monthly bill to about $198, using official OpenAI, Anthropic and Google prices as of Oct. 23, 2026.

The examples use the official Python SDKs (`openai`, `anthropic` and `google-genai`). The ideas carry over to any provider. Prices change often, and Google's Gemini rates are promotional until the end of 2026, so check each pricing page before you rely on the numbers below.

## Where LLM API costs come from: input vs output tokens

Every provider bills per million tokens, and each request has several parts that are priced differently:

- **Uncached input**: the system prompt, documents, chat history and user message, billed at the base input price.
- **Cache writes**: the first time a long prefix is stored. Anthropic charges 1.25x the base input price for a 5-minute cache and 2x for a 1-hour cache. On OpenAI's GPT-5.6 and later models, a write costs 1.25x with the default 30-minute lifetime and 2.5x with the 12-hour option.
- **Cache reads**: later requests that reuse the stored prefix, billed at a fraction of the input price.
- **Output**: the generated text, which usually costs four to six times as much per token as input. Reasoning or "thinking" tokens are billed as output too, even when you only see a summary of them.

Here are the published list prices for the models used in this guide:

| Model                                                  | Input | Cached input (read) | Output | Batch input / output |
| ------------------------------------------------------ | ----- | ------------------- | ------ | -------------------- |
| OpenAI gpt-6.1-sol                                     | $2.00 | $0.10               | $10.00 | $1.00 / $5.00        |
| OpenAI gpt-6-luna                                      | $0.10 | $0.01               | $0.50  | $0.05 / $0.25        |
| Anthropic Claude Sonnet 5.5                            | $2.00 | $0.10               | $10.00 | $1.00 / $5.00        |
| Anthropic Claude Haiku 5.5 (prompts up to 100k tokens) | $0.10 | $0.01               | $0.50  | $0.05 / $0.25        |
| Google Gemini 3.8 Flash (through Dec. 31, 2026)        | $0.75 | $0.075              | $3.75  | $0.375 / $1.875      |
| Google Gemini 4 Argon (through Nov. 30, 2026)          | $2.00 | $0.10               | $10.00 | $1.00 / $5.00        |

*USD per 1 million tokens, standard short-context rates. Data as of 2026-10-23 07:30 UTC. Sources: OpenAI API pricing, Anthropic pricing, Google Gemini API pricing (links below). Gemini 3.8 Flash doubles to $1.50 / $7.50 from Jan. 1, 2027, and Gemini 4 Argon to $4 / $20 from Dec. 1, 2026.*

Two details catch people out. Anthropic's Claude Haiku 5.5 rises to $0.50 input and $2.50 output for prompts over 100,000 tokens. OpenAI charges more for prompts over 272,000 tokens; gpt-6-astra, for example, goes from $10 to $20 per million input tokens. Long prompts cost more per token as well as in total. For a wider model comparison, see our [Gemini 4 Argon benchmarks and pricing comparison](https://bullquill.com/gemini-4-argon-benchmarks-pricing-compared/).

## Step 1: Measure token spend per request (Python)

You cannot cut what you have not measured. The helper below turns each provider's usage object into the same four numbers, prices them, and appends one line per call to a JSONL file. Tag each call with a feature name so you can see which part of your product is expensive.

```
import json, time

# USD per 1M tokens, from the official pricing pages (as of Oct. 23, 2026)
PRICES = {
    "claude-sonnet-5-5": {"input": 2.00, "write": 2.50,  "read": 0.10,  "output": 10.00},
    "claude-haiku-5-5":  {"input": 0.10, "write": 0.125, "read": 0.01,  "output": 0.50},
    "gpt-6.1-sol":       {"input": 2.00, "write": 2.50,  "read": 0.10,  "output": 10.00},
    "gpt-6-luna":        {"input": 0.10, "write": 0.125, "read": 0.01,  "output": 0.50},
    "gemini-3.8-flash":  {"input": 0.75, "write": 0.75,  "read": 0.075, "output": 3.75},
}

def cost_usd(model, uncached, written, read, output):
    p = PRICES[model]
    return (uncached * p["input"] + written * p["write"]
            + read * p["read"] + output * p["output"]) / 1_000_000

def from_anthropic(resp):
    u = resp.usage  # total input = input_tokens + cache_creation + cache_read
    return dict(uncached=u.input_tokens,
                written=u.cache_creation_input_tokens or 0,
                read=u.cache_read_input_tokens or 0,
                output=u.output_tokens)

def from_openai(resp):
    u = resp.usage
    d = u.input_tokens_details
    read = d.cached_tokens or 0
    written = getattr(d, "cache_write_tokens", 0) or 0
    return dict(uncached=u.input_tokens - read - written,
                written=written, read=read,
                output=u.output_tokens)  # includes reasoning tokens

def from_gemini(interaction):
    u = interaction.usage
    read = u.total_cached_tokens or 0
    return dict(uncached=u.total_input_tokens - read, written=0, read=read,
                output=(u.total_output_tokens or 0) + (u.total_thought_tokens or 0))

def log_call(model, feature, tokens, path="llm_costs.jsonl"):
    row = {"ts": time.time(), "model": model, "feature": feature, **tokens,
           "usd": round(cost_usd(model, **tokens), 6)}
    with open(path, "a") as f:
        f.write(json.dumps(row) + "\n")
    return row

```

Check the helper against your provider's billing dashboard for a day before trusting it. The field names come from each provider's documentation. Anthropic says total input equals `input_tokens` plus `cache_creation_input_tokens` plus `cache_read_input_tokens`. OpenAI reports cache reads in `input_tokens_details.cached_tokens` and, on GPT-5.6 and later, writes in `cache_write_tokens`. Google's Interactions API reports `total_cached_tokens` and `total_thought_tokens`. Our [Gemini 4 Argon API tutorial](https://bullquill.com/how-to-use-gemini-4-argon-api-tutorial/) explains why thinking tokens count as output.

After a few days of logs, group by `feature` and `model`. Most teams find that one or two features account for most of the bill. Start with those.

## Step 2: Prompt caching at OpenAI, Anthropic and Google

Prompt caching stores the processed form of a prompt prefix so later requests that start with exactly the same text are cheaper. The rule is the same at all three providers: **put stable content first and variable content last**. System instructions, tool definitions, policy documents and few-shot examples go at the top; the user's message goes at the end. If anything early in the prompt changes, even a timestamp, the cache cannot be used for anything after it.

| Provider      | How it switches on                                                       | Minimum prompt                                | Cache read price                                       | Cache lifetime                                    |
| ------------- | ------------------------------------------------------------------------ | --------------------------------------------- | ------------------------------------------------------ | ------------------------------------------------- |
| OpenAI        | Automatic on supported models; optional explicit breakpoints on GPT-5.6+ | 1,024 tokens (GPT-5.6 and later)              | 0.1x input (0.05x on GPT-6.1 Sol)                      | 30 minutes by default, or 12 hours (GPT-5.6+)     |
| Anthropic     | cache\_control at request level (automatic) or on content blocks         | 512 tokens on Opus 5.5, Sonnet 5.5, Haiku 5.5 | 0.1x on most models; 0.05x on Opus 5.5 and Sonnet 5.5  | 5 minutes, refreshed on each hit; 1 hour optional |
| Google Gemini | Implicit caching on by default for Gemini 2.5 and newer                  | 4,096 tokens (Gemini 4 Argon, 3.8 Flash)      | 10% of input on Gemini 3.8 Flash; 5% on Gemini 4 Argon | Managed by Google for implicit caching            |

*Data as of 2026-10-23 07:30 UTC. Sources: OpenAI prompt caching guide, Anthropic prompt caching documentation, Google Gemini context caching and pricing pages.*

### Anthropic: mark the stable block

```
import anthropic

client = anthropic.Anthropic()
POLICY = open("support_policy.md").read()  # ~6,000 tokens, identical on every call

def answer(ticket_text):
    resp = client.messages.create(
        model="claude-sonnet-5-5",
        max_tokens=400,
        system=[
            {"type": "text", "text": "You are a support assistant. Answer in at most 4 sentences."},
            {"type": "text", "text": POLICY, "cache_control": {"type": "ephemeral"}},
        ],
        messages=[{"role": "user", "content": ticket_text}],
    )
    log_call("claude-sonnet-5-5", "support", from_anthropic(resp))
    return resp.content[0].text

```

The first call writes the cache at 1.25x the input price. Calls within the next five minutes read it at a fraction of that price, and each hit resets the five-minute timer at no extra cost. Anthropic allows up to four explicit breakpoints per request. If traffic is sparse, the documented `"ttl": "1h"` option keeps the entry longer, but each write then costs 2x the base input price.

### OpenAI: keep the prefix identical

```
from openai import OpenAI

client = OpenAI()

resp = client.responses.create(
    model="gpt-6.1-sol",
    instructions=STATIC_INSTRUCTIONS + POLICY,  # stable prefix first
    input=ticket_text,                          # variable part last
    max_output_tokens=400,
    prompt_cache_key="support-bot",             # optional: separate cache accounting per feature
    # prompt_cache_options={"ttl": "12h"},      # longer life, 2.5x write price
)
log_call("gpt-6.1-sol", "support", from_openai(resp))

```

OpenAI caches automatically, so the main work is keeping the prefix byte-for-byte identical. On GPT-5.6 and later, the docs say `prompt_cache_key` is optional and is used for separate cache accounting, for example per customer. Cached tokens still count toward tokens-per-minute rate limits.

### Google Gemini: let implicit caching work

Gemini 4 Argon supports only implicit caching, and Google's caching page says explicit caching is not available for it or through the Interactions API. Savings apply automatically once a repeated prefix passes 4,096 tokens. Check `usage.total_cached_tokens` to confirm hits. Explicit caches for older Gemini 3 and 2.5 models add an hourly storage fee; Google lists $0.50 per million tokens per hour for Gemini 3.8 Flash through Dec. 31, 2026, so they pay off only when a large context is reused often.

## Step 3: Batch APIs, when 50% off is worth the wait

All three providers sell asynchronous batch processing at half the standard price. You give up an instant answer. In return you get a lower bill and, at OpenAI, a separate rate-limit pool.

- **OpenAI Batch API**: a JSONL file of up to 50,000 requests (200 MB), each with `custom_id`, `method`, `url` and `body`. Completion window is 24 hours.
- **Anthropic Message Batches**: up to 100,000 requests or 256 MB. Anthropic says most batches finish within an hour; unfinished ones expire after 24 hours, and results stay downloadable for 29 days.
- **Gemini Batch API**: priced at 50% of the interactive cost, with a 24-hour target turnaround. Inline requests must stay under 20 MB; input files can be up to 2 GB.

Good candidates are nightly summaries, document tagging, evaluation runs, embedding backfills and anything a person will not wait on. Here is a complete Anthropic batch job that also uses a one-hour cache, which Anthropic recommends for batches that share context because processing can take longer than five minutes:

```
import time
import anthropic

client = anthropic.Anthropic()

batch = client.messages.batches.create(requests=[
    {"custom_id": f"doc-{i}",
     "params": {
         "model": "claude-sonnet-5-5",
         "max_tokens": 300,
         "system": [{"type": "text", "text": POLICY,
                     "cache_control": {"type": "ephemeral", "ttl": "1h"}}],
         "messages": [{"role": "user", "content": doc}],
     }}
    for i, doc in enumerate(docs)
])

while client.messages.batches.retrieve(batch.id).processing_status != "ended":
    time.sleep(60)

for r in client.messages.batches.results(batch.id):
    if r.result.type == "succeeded":
        print(r.custom_id, r.result.message.content[0].text)
    else:
        print("retry later:", r.custom_id, r.result.type)

```

Anthropic's pricing page says batch and prompt-caching discounts stack. OpenAI's batch price list also includes discounted cached-input rates. The OpenAI version of the job writes the same requests to a `.jsonl` file, uploads it with `purpose="batch"` and calls `client.batches.create(input_file_id=..., endpoint="/v1/responses", completion_window="24h")`. Results can come back in any order, so always match on `custom_id`.

## Step 4: Model routing, sending easy tasks to small models

The price gap between tiers is large. Claude Haiku 5.5 lists at $0.10 per million input tokens against $2 for Sonnet 5.5, and gpt-6-luna at $0.10 against $2 for gpt-6.1-sol. Classification, extraction, tagging and short summaries rarely need a frontier model.

A router does not need to be clever. Start with rules, and escalate only when the cheap answer fails a check:

```
SMALL, LARGE = "claude-haiku-5-5", "claude-sonnet-5-5"
EASY_TASKS = {"classify", "extract", "tag", "short_summary"}

def pick_model(task, text):
    if task in EASY_TASKS and len(text) < 20_000:
        return SMALL
    return LARGE

def run(task, text, validate):
    model = pick_model(task, text)
    out = call_model(model, task, text)      # your wrapper around messages.create
    if model == SMALL and not validate(out):
        out = call_model(LARGE, task, text)  # escalate only on failure
    return out

```

The `validate` function is where the savings are kept safe. It can check that the output parses as JSON, that a label is in the allowed set, or that required fields are present. Log how often escalation happens. If the small model fails more than about one time in ten on a task, move that task to the large tier, because paying twice for the same request cancels the saving. For work that can stay entirely on your own hardware, a model served through [Ollama](https://bullquill.com/how-to-run-local-llm-with-ollama/) removes the per-token bill, though not the hardware and power costs.

## Step 5: Trim context and cap output length

Output tokens are the most expensive tokens, so set a hard limit on every call: `max_tokens` for Anthropic, `max_output_tokens` for OpenAI's Responses API, and `max_output_tokens` inside `generation_config` for Gemini. Back the limit up with instructions such as "answer in at most four sentences" or a fixed JSON schema. A cap alone can cut answers off mid-sentence. On reasoning models, thinking counts toward the limit, so a cap that is too low can still bill you for thinking and return no usable answer. Our [prompt engineering guide](https://bullquill.com/prompt-engineering-techniques-guide/) covers ways to get short, structured replies.

On the input side, chat history and tool results grow with every turn. In agent loops like the one in our [tool-calling agent tutorial](https://bullquill.com/how-to-build-ai-agent-tool-calling-python/), they often become the largest part of the bill. Trim from the middle, not the front, so the cached prefix survives:

```
def trim_history(messages, keep_last=6):
    """Keep the opening task message and the most recent turns."""
    if len(messages) <= keep_last + 1:
        return messages
    return messages[:1] + messages[-keep_last:]

```

Check that the trimmed list still alternates user and assistant turns, which the Anthropic Messages API requires. Other easy wins: send retrieved passages instead of whole documents, shorten bulky tool outputs before returning them to the model, and keep prompts under the long-context price thresholds mentioned above.

## Worked example: before/after cost table

Take a support assistant that handles 100,000 requests a month on Claude Sonnet 5.5\. Each request carries a 6,000-token policy prefix that never changes, 500 tokens of user input, and about 600 tokens of output. Our assumptions: 2% of requests write the cache and 98% read it; output caps bring the average answer down to 350 tokens; 60% of tickets are simple enough for Claude Haiku 5.5; and 30% of the total workload, such as overnight ticket tagging, can move to the Batch API at the same cache hit rate. Prices are Anthropic's list prices as of 2026-10-23 07:30 UTC.

| Step              | What changes                                                   | Monthly cost (USD) | Change vs baseline |
| ----------------- | -------------------------------------------------------------- | ------------------ | ------------------ |
| Baseline          | 6,500 input tokens at $2/M, 600 output tokens at $10/M         | $1,900.00          | n/a                |
| \+ Prompt caching | 6,000-token prefix: 2% writes at $2.50/M, 98% reads at $0.10/M | $788.80            | \-58.5%            |
| \+ Output cap     | Average output 600 to 350 tokens                               | $538.80            | \-71.6%            |
| \+ Model routing  | 60% of requests on Haiku 5.5 ($0.10 / $0.50 per M)             | $233.45            | \-87.7%            |
| \+ Batch API      | 30% of workload at 50% off                                     | $198.43            | \-89.6%            |

*Bullquill calculation from Anthropic list prices (data as of 2026-10-23 07:30 UTC). Usage assumptions are illustrative. Routing assumes no escalations, and the batch row ignores the higher 2x price of one-hour cache writes.*

The order matters. Caching did the most because the prompt was mostly a repeated prefix. Routing came second because the gap between Sonnet 5.5 and Haiku 5.5 is twentyfold. Batching added the least here because it applied to only part of the traffic. An app with short, unique prompts would see little from caching and more from routing and output caps, which is why Step 1 comes first.

## What it means for developers

List prices are falling. Anthropic cut Opus pricing with version 5.5, and Google is running introductory Gemini rates through the end of 2026\. Even so, the way an app is built still matters more than which provider it uses. A badly structured prompt on a cheap model can cost more than a well-cached prompt on an expensive one. The habits that last are the ones in this guide: log every call, keep prompts stable at the front, push non-urgent work into batches, use the smallest model that passes your checks, and cap output.

Two cautions. First, Google's promotional Gemini prices double on Dec. 1, 2026 (Gemini 4 Argon) and Jan. 1, 2027 (Gemini 3.8 Flash), so budgets built on today's rates need a review date. Second, discounts have conditions: minimum prompt lengths, cache lifetimes, and batch jobs that can expire. Re-run your cost logs after every model or pricing change rather than assuming the savings hold.

## Frequently asked questions

### What is the fastest way to reduce LLM API costs?

Log token usage per request first, then fix the biggest item. Apps that resend a long, unchanging system prompt or document usually save most from prompt caching; apps with short unique prompts save more by routing simple tasks to a smaller model and capping output length.

### How much does prompt caching save?

Cache reads are billed at a fraction of the normal input price: 0.1x on most OpenAI GPT-5.6+ and Anthropic models, 0.05x on GPT-6.1 Sol, Claude Opus 5.5 and Claude Sonnet 5.5, according to the providers' documentation as of Oct. 23, 2026\. Writing the cache costs extra (1.25x for short lifetimes), so savings depend on how often the same prefix is reused.

### Is the Batch API really 50% cheaper?

Yes. OpenAI, Anthropic and Google all price batch requests at 50% of their standard rates. The trade-off is latency: OpenAI uses a 24-hour completion window, Anthropic batches expire if not finished within 24 hours, and Google targets a 24-hour turnaround.

### Does prompt caching work with the Batch API?

Anthropic's pricing page says batch and prompt-caching discounts can be combined, and recommends a one-hour cache for batches that share context. OpenAI's batch price list also includes discounted cached-input rates.

## Sources

- [OpenAI: Prompt caching guide](https://developers.openai.com/api/docs/guides/prompt-caching?ref=bullquill.com)
- [OpenAI: API pricing](https://developers.openai.com/api/docs/pricing?ref=bullquill.com)
- [OpenAI: Batch API guide](https://developers.openai.com/api/docs/guides/batch?ref=bullquill.com)
- [Anthropic: Prompt caching](https://platform.claude.com/docs/en/build-with-claude/prompt-caching?ref=bullquill.com)
- [Anthropic: Pricing](https://platform.claude.com/docs/en/about-claude/pricing?ref=bullquill.com)
- [Anthropic: Batch processing (Message Batches API)](https://platform.claude.com/docs/en/build-with-claude/batch-processing?ref=bullquill.com)
- [Google: Gemini API context caching](https://ai.google.dev/gemini-api/docs/caching?ref=bullquill.com)
- [Google: Gemini Developer API pricing](https://ai.google.dev/gemini-api/docs/pricing?ref=bullquill.com)
- [Google: Gemini Batch API](https://ai.google.dev/gemini-api/docs/batch-api?ref=bullquill.com)
- [Google: Interactions API reference](https://ai.google.dev/api/interactions-api?ref=bullquill.com)