Ollama Structured Outputs in Practice — Getting Type-Safe JSON from Local LLMs with Pydantic

json.loads(response) fails at a certain point. You told the model "return JSON only," but it added a ```json markdown code fence around everything. A quick regex strips it — until that regex hits an edge case, and that edge case blows up in production. Since Ollama 0.3.0, passing a JSON schema to the format parameter eliminates this problem at the root. The model's inference itself is constrained by the schema, so no code fences, no explanatory text, no mid-thought artifacts. Just parseable JSON. I ran these tests locally with Gemma4 and Ollama 0.30.7 to see how well it holds up in practice. Why LLM Response Parsing Is Tricky The most common problem when running Ollama locally — without a cloud LLM API — is JSON parsing. Two reasons. First, text generation models are trained toward "natural text." Even if you ask for JSON only, they'll often wrap it in json ... blocks or prepend "Of course! Here is the JSON you requested:" style text. Here's what I reproduced directly: json Input: 'Give me 3 Python tips as JSON with keys: tips (array), difficulty (1-5)' Model output (no format parameter): ```json { "tips": [ "Master the fundamentals first...", ... ] } JSON parse: FAILED Python ' s `json.loads()` can ' t handle the markdown wrapper . The " JSON only " instruction is unreliable in production . Second , speed . I measured the same query both ways : 32 seconds without structured output , 5 seconds with it . More on why below . ## How the Ollama format Parameter Works Ollama ' s `/api/generate` endpoint has a `format` field. Pass a JSON schema object and Ollama applies **constrained decoding** during inference. python import json import urllib.request def ollama_structured(prompt, schema, model="gemma4:e4b"): payload = { "model": model, "prompt": prompt, "format": schema, # ← pass JSON schema object directly "stream": False, "options": {"temperature": 0} } data = json.dumps(payload).encode() req = urllib.request.Request( " http://localhost:11434/api/generate ", data=data