> ## Content Index
> Fetch the complete content index at: https://bullquill.com/llms.txt
> Use this file to discover other available public pages before exploring further.

# How to get structured JSON output from an LLM in Python
- URL: https://bullquill.com/how-to-get-structured-json-output-llm-python/
- Published: 2026-10-25T09:14:29.000Z
- Updated: 2026-10-25T09:25:41.000Z
- Description: One Pydantic model, three providers: send its JSON Schema to OpenAI, Claude or a local Ollama model, check for refusals and truncation, and retry with the validation error when the data is still wrong.
- Author: Rowan Ashby
- Tags: AI, Tutorial, How-To, Structured Outputs, OpenAI, Anthropic, Ollama, Pydantic, #lang-en, #evergreen, #tr-how-to-get-structured-json-output-llm-python

## Key takeaways

- Define the shape you want once as a Pydantic model, then reuse it as the JSON Schema you send and as the validator you run on the reply.
- OpenAI (`responses.parse` with `text_format`), Anthropic (`messages.parse` with `output_format`) and Ollama (the `format` parameter) can all constrain output to a JSON Schema.
- Schema-valid JSON is not the same as correct data: keep Pydantic validation, add business rules, and retry with the error message when a check fails.
- Plan for edge cases the schema cannot cover: refusals, truncated output when the token limit is hit, and providers or deployments that do not support schema-constrained output.

The reliable way to get **structured JSON output from an LLM in Python** is to define a Pydantic model and pass its JSON Schema to the provider's native structured-output mode. OpenAI, Anthropic and Ollama all document such a mode as of 25 October 2026.

This guide builds one small extraction task, pulling fields out of an invoice email, and runs it against all three providers. It then adds the part most tutorials skip: validating the result and retrying when the model returns something that parses but is still wrong.

## Why "please reply in JSON" is not enough

Asking for JSON in the prompt works most of the time, which is exactly the problem. A model that answers correctly 98 times out of 100 will still eventually wrap the object in a Markdown code fence, add a friendly sentence before it, rename a key, or return the number `"1,200.50"` as a string. Each of those breaks `json.loads` or a downstream database insert.

Providers solved the first half of this with "JSON mode", which guarantees syntactically valid JSON but not a particular shape. OpenAI's documentation says plainly that both JSON mode and Structured Outputs produce valid JSON, but only Structured Outputs ensure schema adherence, and it recommends Structured Outputs whenever possible. Anthropic and Ollama describe their schema modes in similar terms: the model's sampling is constrained so that the output matches the schema you supply.

## Step 1: Install the libraries

You need Python 3.10 or later for the `str | None` syntax used below. Install only the clients you plan to use:

```
pip install -U pydantic openai anthropic ollama
```

Set `OPENAI_API_KEY` and `ANTHROPIC_API_KEY` as environment variables for the hosted APIs. For Ollama, install the local runtime and pull a model first (for example `ollama pull llama3.1`); Bullquill's [guide to running a local LLM with Ollama](https://bullquill.com/how-to-run-local-llm-with-ollama/) covers that setup step by step.

## Step 2: Define the output as a Pydantic model

The Pydantic model is the single source of truth. Field descriptions end up in the JSON Schema, so they double as instructions to the model.

```
from typing import Literal
from pydantic import BaseModel, Field, field_validator

class Invoice(BaseModel):
    vendor: str = Field(description="Company that issued the invoice")
    invoice_number: str
    total: float = Field(description="Grand total as a number, without currency symbol")
    currency: Literal["USD", "EUR", "GBP", "IDR"]
    due_date: str | None = Field(description="Due date in ISO 8601 (YYYY-MM-DD), or null if not stated")
    line_items: list[str]

    @field_validator("total")
    @classmethod
    def total_not_negative(cls, value: float) -> float:
        if value < 0:
            raise ValueError("total must be zero or positive")
        return value

print(Invoice.model_json_schema())
```

Three design choices matter here:

- **Every field is required.** OpenAI's strict mode requires all fields to be listed as required. To express "optional", the field stays required but its type allows `null`, which is what `str | None` without a default produces.
- **Enums use `Literal`.** A closed list of allowed values becomes an `enum` in the schema, which all three providers support.
- **Business rules live in validators, not the schema.** Anthropic's documentation lists numeric constraints such as `minimum` and string constraints such as `minLength` as unsupported, so a rule like "total must not be negative" is safer as a Pydantic validator that runs in your code.

According to the Pydantic documentation, `model_json_schema()` returns a JSON-serialisable dict and defaults to `'validation'` mode; each `Field(description=...)` appears as a `description` key on that property.

## Step 3: Structured output with OpenAI

The OpenAI Python SDK accepts the Pydantic class directly through `responses.parse` and returns a typed object in `output_parsed`. This mirrors the example in OpenAI's Structured Outputs guide:

```
from openai import OpenAI

client = OpenAI()
invoice_text = open("invoice_email.txt", encoding="utf-8").read()

response = client.responses.parse(
    model="gpt-6-astra",  # use any current model that supports Structured Outputs
    input=[
        {"role": "system", "content": "Extract the invoice fields. Use null when a field is missing."},
        {"role": "user", "content": invoice_text},
    ],
    text_format=Invoice,
)

invoice = response.output_parsed
print(invoice)
```

Two failure modes are documented. If the model declines the request, the response carries a `refusal` item instead of parsed content, and OpenAI notes that a refusal does not necessarily follow your schema. If the response stops early because of the output-token limit, its status is `"incomplete"` with the reason `max_output_tokens`. Check for both before trusting `output_parsed`.

## Step 4: Structured output with Anthropic (Claude)

Anthropic's structured outputs are generally available and, according to its documentation, need no beta header when you use the `output_config.format` parameter. The Python SDK's recommended helper is `client.messages.parse()`, which converts the Pydantic model, sends it as `output_config.format`, validates the reply and returns it in `parsed_output`:

```
import anthropic

client = anthropic.Anthropic()

response = client.messages.parse(
    model="claude-opus-5-5",
    max_tokens=1024,
    messages=[{"role": "user", "content": f"Extract the invoice fields:\n\n{invoice_text}"}],
    output_format=Invoice,
)

if response.stop_reason in ("refusal", "max_tokens"):
    raise RuntimeError(f"Output may not match the schema: stop_reason={response.stop_reason}")

invoice = response.parsed_output
print(invoice)
```

The stop-reason check follows Anthropic's own warnings: a `refusal` still returns HTTP 200 and is billed, and a `max_tokens` stop may leave the JSON incomplete. Anthropic's docs also note the older `output_format` request parameter with the `structured-outputs-2025-11-13` beta header is deprecated; the `output_format=` argument above belongs to the SDK's `parse()` helper, which sends the current parameter for you.

### Alternative: forced tool use

Before native JSON outputs existed, the standard Claude pattern was to define a single tool whose input schema is your model, then force the model to call it. It still works and is useful when you are already running a tool-calling agent:

```
response = client.messages.create(
    model="claude-opus-5-5",
    max_tokens=1024,
    tools=[{
        "name": "record_invoice",
        "description": "Record the fields extracted from an invoice email",
        "input_schema": Invoice.model_json_schema(),
    }],
    tool_choice={"type": "tool", "name": "record_invoice"},
    messages=[{"role": "user", "content": invoice_text}],
)

tool_call = next(block for block in response.content if block.type == "tool_use")
invoice = Invoice.model_validate(tool_call.input)
```

Adding `"strict": True` to the tool definition turns on what Anthropic calls grammar-constrained sampling, guaranteeing that the tool input matches the schema. Strict mode uses the same restricted JSON Schema subset as JSON outputs, including `additionalProperties: false` on every object. For more on the agent loop around tool calls, see Bullquill's [tutorial on building an AI agent with tool calling in Python](https://bullquill.com/how-to-build-ai-agent-tool-calling-python/).

## Step 5: Structured output with Ollama (local models)

Ollama added structured outputs in December 2024\. You pass the schema to the `format` parameter and validate the text reply with the same Pydantic model:

```
from ollama import chat

response = chat(
    model="llama3.1",
    messages=[{"role": "user", "content": f"Extract the invoice fields. Return as JSON.\n\n{invoice_text}"}],
    format=Invoice.model_json_schema(),
    options={"temperature": 0},
)

invoice = Invoice.model_validate_json(response.message.content)
print(invoice)
```

Ollama's own tips are reflected above: set the temperature to 0 for more deterministic output, and say "return as JSON" in the prompt to help the model. Passing `format="json"` instead of a schema gives you plain JSON mode without a fixed shape. Note that Ollama's documentation states its cloud service does not currently support structured outputs, so this pattern is for locally run models. The same schema also works through Ollama's OpenAI-compatible endpoint via `response_format`.

## Step 6: Validate, then retry with the error

Constrained decoding guarantees shape, not truth. A model can return `"currency": "USD"` for a euro invoice, or a negative total that the schema could not forbid. The fix is a short loop that validates every reply and, on failure, sends the validation error back so the model can correct itself:

```
from ollama import chat
from pydantic import ValidationError

def extract_invoice(text: str, max_attempts: int = 3) -> Invoice:
    messages = [{"role": "user", "content": f"Extract the invoice fields. Return as JSON.\n\n{text}"}]
    for attempt in range(1, max_attempts + 1):
        response = chat(
            model="llama3.1",
            messages=messages,
            format=Invoice.model_json_schema(),
            options={"temperature": 0},
        )
        raw = response.message.content
        try:
            return Invoice.model_validate_json(raw)
        except ValidationError as err:
            messages.append({"role": "assistant", "content": raw})
            messages.append({
                "role": "user",
                "content": f"That JSON failed validation:\n{err}\nReturn corrected JSON only.",
            })
    raise RuntimeError(f"No valid invoice after {max_attempts} attempts")
```

The same structure works with the hosted SDKs: wrap the call in `try`, catch `ValidationError` (the Anthropic and OpenAI helpers validate with Pydantic too), append the error, and try again. Keep the attempt limit low. If three tries fail, the input is usually ambiguous and should go to a human queue, not a fourth retry. To measure how often that happens across a test set, pair this loop with the approach in Bullquill's [guide to evaluating an LLM app in Python](https://bullquill.com/how-to-evaluate-llm-app-python/).

## OpenAI vs Anthropic vs Ollama: structured output compared

| Feature                  | OpenAI                                                                                   | Anthropic (Claude)                                                                             | Ollama                                     |
| ------------------------ | ---------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------- | ------------------------------------------ |
| Python helper            | client.responses.parse(text\_format=Model)                                               | client.messages.parse(output\_format=Model)                                                    | chat(format=Model.model\_json\_schema())   |
| Parsed result            | response.output\_parsed                                                                  | response.parsed\_output                                                                        | Validate response.message.content yourself |
| Raw API parameter        | json\_schema format with strict: true                                                    | output\_config.format with type: "json\_schema"                                                | format (schema or "json")                  |
| Tool-based alternative   | Function calling with strict                                                             | Tool with strict: true, forced via tool\_choice                                                | Tool calling (no schema guarantee stated)  |
| Documented schema limits | Up to 5,000 object properties, 10 nesting levels, 1,000 enum values; all fields required | additionalProperties must be false; no recursive schemas, numeric or string-length constraints | Not supported on Ollama's cloud            |
| Failure signals          | refusal item; status incomplete                                                          | stop\_reason refusal or max\_tokens                                                            | ValidationError on parse                   |

*Sources: OpenAI Structured Outputs guide; Anthropic structured outputs and strict tool use documentation; Ollama structured outputs documentation and blog. Checked 25 October 2026.*

## Common mistakes to avoid

- **Optional fields with defaults.** A field such as `due_date: str | None = None` is not required in the generated schema, which OpenAI's strict mode rejects. Keep the field required and nullable.
- **Relying on schema keywords the provider ignores.** Constraints like `minimum` or `maxLength` may be stripped or refused. Anthropic's docs say its SDK helpers send a simplified schema but still enforce every constraint in your code after the response arrives.
- **Huge or deeply nested schemas.** Flatten where you can. Large schemas cost tokens on every call and are closer to documented limits.
- **Parsing before checking for refusals or truncation.** Both can produce output that does not match the schema even in strict modes.
- **Sending sensitive data to cloud APIs by default.** If the documents contain personal data, the local Ollama route keeps them on your machine; the same applies to retrieval pipelines such as the one in Bullquill's [local RAG app tutorial](https://bullquill.com/how-to-build-rag-app-python-local/).

## What it means

Structured output turns an LLM from a text generator into a component you can wire into ordinary software: the result fits a typed object, so it can go straight into a database row, an API response or the next step of an agent. All three providers covered here now support schema-constrained output, which means the prompt no longer has to carry the burden of "format exactly like this". Clear field names and descriptions still help, and Bullquill's [prompt engineering techniques guide](https://bullquill.com/prompt-engineering-techniques-guide/) covers how to write them.

The remaining risk sits in the data itself. A schema can force a valid currency code but cannot know whether it is the right one. Treat the Pydantic model as a contract, keep business rules in validators, retry with error feedback a small number of times, and log every failure so you can see which inputs your extraction still gets wrong.

## Frequently asked questions

### What is the difference between JSON mode and structured outputs?

JSON mode only guarantees that the reply is syntactically valid JSON. Structured outputs constrain the reply to a JSON Schema you supply, so required keys, types and enum values match. OpenAI recommends Structured Outputs over JSON mode whenever possible.

### How do I make an LLM return JSON that matches a Pydantic model?

Pass the model to the provider's helper: client.responses.parse(text\_format=Model) for OpenAI, client.messages.parse(output\_format=Model) for Anthropic, or format=Model.model\_json\_schema() for Ollama, then validate the reply with the same Pydantic model.

### Can structured output still be wrong?

Yes. Constrained decoding guarantees the shape of the JSON, not the accuracy of the values. Refusals and replies cut off by the token limit can also break the schema, so check for them, run Pydantic validators for business rules, and retry with the error message.

### Does Ollama support structured outputs?

Yes, for locally run models since December 2024: pass a JSON Schema to the format parameter. Ollama's documentation states that its cloud service does not currently support structured outputs.

## Sources

- [OpenAI: Structured Outputs guide](https://developers.openai.com/api/docs/guides/structured-outputs?ref=bullquill.com)
- [Anthropic: Structured outputs](https://platform.claude.com/docs/en/build-with-claude/structured-outputs?ref=bullquill.com)
- [Anthropic: Strict tool use](https://platform.claude.com/docs/en/agents-and-tools/tool-use/strict-tool-use?ref=bullquill.com)
- [Ollama: Structured outputs documentation](https://docs.ollama.com/capabilities/structured-outputs?ref=bullquill.com)
- [Ollama blog: Structured outputs (6 December 2024)](https://ollama.com/blog/structured-outputs?ref=bullquill.com)
- [Pydantic: JSON Schema](https://pydantic.dev/docs/validation/latest/concepts/json%5Fschema/?ref=bullquill.com)
- [JSON Schema: Understanding JSON Schema](https://json-schema.org/understanding-json-schema?ref=bullquill.com)