How to get structured JSON output from an LLM in Python

Key takeaways

• Define the shape you want once as a Pydantic model, then reuse it as the JSON Schema you send and as the validator you run on the reply.
• OpenAI (responses.parse with text_format), Anthropic (messages.parse with output_format) and Ollama (the format parameter) can all constrain output to a JSON Schema.
• Schema-valid JSON is not the same as correct data: keep Pydantic validation, add business rules, and retry with the error message when a check fails.
• Plan for edge cases the schema cannot cover: refusals, truncated output when the token limit is hit, and providers or deployments that do not support schema-constrained output.

The reliable way to get structured JSON output from an LLM in Python is to define a Pydantic model and pass its JSON Schema to the provider's native structured-output mode. OpenAI, Anthropic and Ollama all document such a mode as of 25 October 2026.

This guide builds one small extraction task, pulling fields out of an invoice email, and runs it against all three providers. It then adds the part most tutorials skip: validating the result and retrying when the model returns something that parses but is still wrong.

Why "please reply in JSON" is not enough

Asking for JSON in the prompt works most of the time, which is exactly the problem. A model that answers correctly 98 times out of 100 will still eventually wrap the object in a Markdown code fence, add a friendly sentence before it, rename a key, or return the number "1,200.50" as a string. Each of those breaks json.loads or a downstream database insert.

Providers solved the first half of this with "JSON mode", which guarantees syntactically valid JSON but not a particular shape. OpenAI's documentation says plainly that both JSON mode and Structured Outputs produce valid JSON, but only Structured Outputs ensure schema adherence, and it recommends Structured Outputs whenever possible. Anthropic and Ollama describe their schema modes in similar terms: the model's sampling is constrained so that the output matches the schema you supply.

Step 1: Install the libraries

You need Python 3.10 or later for the str | None syntax used below. Install only the clients you plan to use:

pip install -U pydantic openai anthropic ollama

Set OPENAI_API_KEY and ANTHROPIC_API_KEY as environment variables for the hosted APIs. For Ollama, install the local runtime and pull a model first (for example ollama pull llama3.1); Bullquill's guide to running a local LLM with Ollama covers that setup step by step.

Step 2: Define the output as a Pydantic model

The Pydantic model is the single source of truth. Field descriptions end up in the JSON Schema, so they double as instructions to the model.

from typing import Literal
from pydantic import BaseModel, Field, field_validator


class Invoice(BaseModel):
    vendor: str = Field(description="Company that issued the invoice")
    invoice_number: str
    total: float = Field(description="Grand total as a number, without currency symbol")
    currency: Literal["USD", "EUR", "GBP", "IDR"]
    due_date: str | None = Field(description="Due date in ISO 8601 (YYYY-MM-DD), or null if not stated")
    line_items: list[str]

    @field_validator("total")
    @classmethod
    def total_not_negative(cls, value: float) -> float:
        if value < 0:
            raise ValueError("total must be zero or positive")
        return value


print(Invoice.model_json_schema())

Three design choices matter here:

• Every field is required. OpenAI's strict mode requires all fields to be listed as required. To express "optional", the field stays required but its type allows null, which is what str | None without a default produces.
• Enums use Literal. A closed list of allowed values becomes an enum in the schema, which all three providers support.
• Business rules live in validators, not the schema. Anthropic's documentation lists numeric constraints such as minimum and string constraints such as minLength as unsupported, so a rule like "total must not be negative" is safer as a Pydantic validator that runs in your code.

According to the Pydantic documentation, model_json_schema() returns a JSON-serialisable dict and defaults to 'validation' mode; each Field(description=...) appears as a description key on that property.

Step 3: Structured output with OpenAI

The OpenAI Python SDK accepts the Pydantic class directly through responses.parse and returns a typed object in output_parsed. This mirrors the example in OpenAI's Structured Outputs guide:

from openai import OpenAI

client = OpenAI()
invoice_text = open("invoice_email.txt", encoding="utf-8").read()

response = client.responses.parse(
    model="gpt-6-astra",  # use any current model that supports Structured Outputs
    input=[
        {"role": "system", "content": "Extract the invoice fields. Use null when a field is missing."},
        {"role": "user", "content": invoice_text},
    ],
    text_format=Invoice,
)

invoice = response.output_parsed
print(invoice)

Two failure modes are documented. If the model declines the request, the response carries a refusal item instead of parsed content, and OpenAI notes that a refusal does not necessarily follow your schema. If the response stops early because of the output-token limit, its status is "incomplete" with the reason max_output_tokens. Check for both before trusting output_parsed.

Step 4: Structured output with Anthropic (Claude)

Anthropic's structured outputs are generally available and, according to its documentation, need no beta header when you use the output_config.format parameter. The Python SDK's recommended helper is client.messages.parse(), which converts the Pydantic model, sends it as output_config.format, validates the reply and returns it in parsed_output:

import anthropic

client = anthropic.Anthropic()

response = client.messages.parse(
    model="claude-opus-5-5",
    max_tokens=1024,
    messages=[{"role": "user", "content": f"Extract the invoice fields:\n\n{invoice_text}"}],
    output_format=Invoice,
)

if response.stop_reason in ("refusal", "max_tokens"):
    raise RuntimeError(f"Output may not match the schema: stop_reason={response.stop_reason}")

invoice = response.parsed_output
print(invoice)

The stop-reason check follows Anthropic's own warnings: a refusal still returns HTTP 200 and is billed, and a max_tokens stop may leave the JSON incomplete. Anthropic's docs also note the older output_format request parameter with the structured-outputs-2025-11-13 beta header is deprecated; the output_format= argument above belongs to the SDK's parse() helper, which sends the current parameter for you.

Alternative: forced tool use

Before native JSON outputs existed, the standard Claude pattern was to define a single tool whose input schema is your model, then force the model to call it. It still works and is useful when you are already running a tool-calling agent:

response = client.messages.create(
    model="claude-opus-5-5",
    max_tokens=1024,
    tools=[{
        "name": "record_invoice",
        "description": "Record the fields extracted from an invoice email",
        "input_schema": Invoice.model_json_schema(),
    }],
    tool_choice={"type": "tool", "name": "record_invoice"},
    messages=[{"role": "user", "content": invoice_text}],
)

tool_call = next(block for block in response.content if block.type == "tool_use")
invoice = Invoice.model_validate(tool_call.input)

Adding "strict": True to the tool definition turns on what Anthropic calls grammar-constrained sampling, guaranteeing that the tool input matches the schema. Strict mode uses the same restricted JSON Schema subset as JSON outputs, including additionalProperties: false on every object. For more on the agent loop around tool calls, see Bullquill's tutorial on building an AI agent with tool calling in Python.

Step 5: Structured output with Ollama (local models)

Ollama added structured outputs in December 2024. You pass the schema to the format parameter and validate the text reply with the same Pydantic model:

from ollama import chat

response = chat(
    model="llama3.1",
    messages=[{"role": "user", "content": f"Extract the invoice fields. Return as JSON.\n\n{invoice_text}"}],
    format=Invoice.model_json_schema(),
    options={"temperature": 0},
)

invoice = Invoice.model_validate_json(response.message.content)
print(invoice)

Ollama's own tips are reflected above: set the temperature to 0 for more deterministic output, and say "return as JSON" in the prompt to help the model. Passing format="json" instead of a schema gives you plain JSON mode without a fixed shape. Note that Ollama's documentation states its cloud service does not currently support structured outputs, so this pattern is for locally run models. The same schema also works through Ollama's OpenAI-compatible endpoint via response_format.

Step 6: Validate, then retry with the error

Constrained decoding guarantees shape, not truth. A model can return "currency": "USD" for a euro invoice, or a negative total that the schema could not forbid. The fix is a short loop that validates every reply and, on failure, sends the validation error back so the model can correct itself:

from ollama import chat
from pydantic import ValidationError


def extract_invoice(text: str, max_attempts: int = 3) -> Invoice:
    messages = [{"role": "user", "content": f"Extract the invoice fields. Return as JSON.\n\n{text}"}]
    for attempt in range(1, max_attempts + 1):
        response = chat(
            model="llama3.1",
            messages=messages,
            format=Invoice.model_json_schema(),
            options={"temperature": 0},
        )
        raw = response.message.content
        try:
            return Invoice.model_validate_json(raw)
        except ValidationError as err:
            messages.append({"role": "assistant", "content": raw})
            messages.append({
                "role": "user",
                "content": f"That JSON failed validation:\n{err}\nReturn corrected JSON only.",
            })
    raise RuntimeError(f"No valid invoice after {max_attempts} attempts")

The same structure works with the hosted SDKs: wrap the call in try, catch ValidationError (the Anthropic and OpenAI helpers validate with Pydantic too), append the error, and try again. Keep the attempt limit low. If three tries fail, the input is usually ambiguous and should go to a human queue, not a fourth retry. To measure how often that happens across a test set, pair this loop with the approach in Bullquill's guide to evaluating an LLM app in Python.

OpenAI vs Anthropic vs Ollama: structured output compared

Sources: OpenAI Structured Outputs guide; Anthropic structured outputs and strict tool use documentation; Ollama structured outputs documentation and blog. Checked 25 October 2026.

Common mistakes to avoid

• Optional fields with defaults. A field such as due_date: str | None = None is not required in the generated schema, which OpenAI's strict mode rejects. Keep the field required and nullable.
• Relying on schema keywords the provider ignores. Constraints like minimum or maxLength may be stripped or refused. Anthropic's docs say its SDK helpers send a simplified schema but still enforce every constraint in your code after the response arrives.
• Huge or deeply nested schemas. Flatten where you can. Large schemas cost tokens on every call and are closer to documented limits.
• Parsing before checking for refusals or truncation. Both can produce output that does not match the schema even in strict modes.
• Sending sensitive data to cloud APIs by default. If the documents contain personal data, the local Ollama route keeps them on your machine; the same applies to retrieval pipelines such as the one in Bullquill's local RAG app tutorial.

What it means

Structured output turns an LLM from a text generator into a component you can wire into ordinary software: the result fits a typed object, so it can go straight into a database row, an API response or the next step of an agent. All three providers covered here now support schema-constrained output, which means the prompt no longer has to carry the burden of "format exactly like this". Clear field names and descriptions still help, and Bullquill's prompt engineering techniques guide covers how to write them.

The remaining risk sits in the data itself. A schema can force a valid currency code but cannot know whether it is the right one. Treat the Pydantic model as a contract, keep business rules in validators, retry with error feedback a small number of times, and log every failure so you can see which inputs your extraction still gets wrong.

Sources

• OpenAI: Structured Outputs guide
• Anthropic: Structured outputs
• Anthropic: Strict tool use
• Ollama: Structured outputs documentation
• Ollama blog: Structured outputs (6 December 2024)
• Pydantic: JSON Schema
• JSON Schema: Understanding JSON Schema