Home › Guides › Why LLM JSON breaks (and how to recover it)

Why LLM JSON breaks (and how to recover it)

Direct answer: an LLM writes JSON as prose, one token at a time, with no parser in the loop. So its JSON fails in predictable shapes: it gets cut off at the token limit, wrapped in conversation and code fences, written in the loose syntax of its training examples (single quotes, trailing commas, True/None), and it drifts on long documents. Each shape has a matching recovery — the tool below handles the syntax family; truncation has its own stricter tool.

Try it on an LLM-style sample — or your own output

Processed in your browser, so big pastes don't freeze the page. Nothing is uploaded — check your browser's network tab if you'd like to verify.

The four failure shapes

1. Truncation — the token limit cut

The LLM was mid-document when the output budget ended. The JSON stops anywhere: mid-array, mid-key, mid-string, halfway through a number. This is not a syntax error to correct but missing data to account for. Use Truncated Recovery: it returns the largest complete prefix and lists what was dropped, instead of completing a half-written value with a plausible invention. The prevention is an output budget sized to the document, response pagination, or streaming with accumulation — and never assuming a long response arrived whole.

2. Wrapping — prose and fences around the payload

"Sure! Here's the JSON:" … a fenced code block … "Hope that helps!" The JSON inside is often perfect; the envelope breaks the parser. The repair tool strips markdown fences and surrounding prose as its first, listed step. At the source: ask for the document only, or use the API's JSON/structured mode, which skips the envelope entirely.

3. Imitated loose syntax

LLMs have read oceans of JavaScript, Python, and hand-written config, and they imitate it: single-quoted strings, unquoted keys, trailing commas, True/False/None, the occasional comment. All deterministic to fix — convert, quote, remove, map — and every conversion belongs in a report you can inspect, because a silent converter can also silently change meaning (a Python None becoming the string "None" is a classic).

4. Long-document drift

The longer the generation, the more its grip on the exact schema loosens: a key renamed halfway, a number that becomes a string, nesting that flattens. Syntax repair cannot fix semantic drift — validate against your schema after repair, and prefer several small responses over one enormous one when the shape matters.

A sane pipeline for LLM JSON

  1. Constrain at the source: structured outputs / JSON mode with a schema where the provider offers it.
  2. Size the budget: token limits that fit the document, or pagination by design.
  3. Validate on receipt: strict parse first; a failure is data, not an exception to swallow.
  4. Fall back deterministically: strip and repair the syntax family; recover — never complete — the truncated family.
  5. Log the report: when a repair happens in production, the list of changes is the difference between a recovered payload and a mystery.

Frequently asked questions

Why does LLM JSON break so often?

Because an LLM generates JSON as text, token by token, without a parser checking its work. Long outputs get cut at token limits; it wraps answers in conversation and markdown fences; it imitates the loosely-typed examples it has seen (single quotes, trailing commas, Python literals); and the longer the document, the more chances for a slip. None of this is a syntax checker failing — there was never a syntax checker in the loop.

What is the most reliable way to get valid JSON from an LLM?

Use the provider's structured-output / JSON mode with a schema when available, keep outputs short or paginate them, and always validate on receipt with a repair/recovery fallback for the failures that remain. Prompting "respond with valid JSON only" helps but is not a guarantee.

Should I repair LLM JSON or just ask again?

If the call is cheap and reproducible, asking again (ideally with a tighter output budget) gets you real data. Repair when the output is long, expensive, or non-reproducible — and prefer recovery over completion for truncated output, so a cut value is never silently "finished" by a guess.