What this usually means
LangChain chains combine prompts, LLMs, parsers, and memory. An output error typically means the data flow broke at some point: the LLM returned something the parser can't handle, a key in the output dictionary is missing because a previous step didn't produce it, or memory corrupted the context. The non-obvious part is that LangChain's abstraction hides the raw LLM response, so you see a parser error without the actual response. You need to intercept that raw output to understand what the LLM actually sent.
The first ten minutes — establish facts before touching code.
- 1Run chain with verbose=True: chain.run(input, verbose=True) — this prints the LLM prompt and raw response.
- 2Set environment variable LANGCHAIN_VERBOSE=true before running your script.
- 3Wrap the chain call in try/except and print the exception's 'response' attribute (if OutputParserException).
- 4Use langchain.debug=True to get detailed logging of each step.
- 5Test the prompt directly by calling llm(prompt) to see the LLM's raw output without parsing.
The specific files, logs, configs, and dashboards that usually own this bug.
- searchPrompt template: Check for placeholder mismatches (e.g., {input} vs {text})
- searchOutput parser: Look at parse() method—regex, JSON schema, or custom logic
- searchChain definition: Verify the order of steps and input/output keys
- searchMemory object: Check that memory keys match chain input variables
- searchLLM response: Intercept via callback or by calling llm.invoke directly
- searchEnvironment logs: stdout/stderr for verbose output
- searchModel provider dashboard: OpenAI or Anthropic logs for token usage and errors
Practical causes, not theory. These are the things you will actually find.
- warningPrompt template has extra or missing curly braces causing placeholder mismatch
- warningOutput parser expects JSON but LLM returns markdown code block with backticks
- warningMemory variable name collides with chain input variable (e.g., both use 'history')
- warningLLM returns a non-answer due to content filter or token limit (e.g., 'I cannot answer that')
- warningChain step produces a dictionary with wrong key name (typo like 'answer' vs 'ans')
- warningCustom output parser's parse() method raises exception due to unexpected input format
Concrete fix directions. Pick the one that matches your root cause.
- buildAdd a 'stop' sequence to the LLM to prevent it from producing extra text
- buildUse OutputFixingParser from langchain.output_parsers to retry with corrected prompt
- buildSanitize LLM output in a custom parser: strip code fences, fix JSON commas
- buildExplicitly map output keys using a simple transformation step (e.g., map_reduce)
- buildSet model temperature=0 for deterministic output, especially for JSON
- buildUse prompt + output parser in a try/except and fallback to a simpler parser
A fix you cannot prove is a guess. Close the loop.
- verifiedRun chain with multiple varied inputs and confirm output structure matches
- verifiedUnit test the output parser directly with sample LLM responses
- verifiedEnable verbose mode and check raw LLM output matches parser expectations
- verifiedTest with a mock LLM that returns perfect output to isolate parser issues
- verifiedCheck memory state after run: print(memory.load_memory_variables({}))
Things that make this bug worse or harder to find.
- warningDo not ignore the raw LLM output — always log it before the parser
- warningDo not assume the LLM will follow instructions; always validate output format
- warningDo not use complex output parsers without fallback for malformed responses
- warningDo not forget to clear memory between test runs; stale state causes intermittent errors
- warningDo not hardcode prompt templates without testing for edge cases (empty strings, special chars)
JSON Output Parser Fails on OpenAI Response with Code Fence
Timeline
- 09:15Deploy new chain to production for generating structured reports
- 10:30PagerDuty alert: 500 errors on /generate-report endpoint
- 10:35Check logs: OutputParserException: Could not parse LLM output
- 10:40Enable verbose mode locally, reproduce with same input
- 10:45Raw LLM output: JSON wrapped in ```json ... ```
- 10:50Realize parser expects pure JSON, not code fence
- 10:55Implement custom parser to strip markdown code fences
- 11:00Deploy fix, monitor error rate drops to zero
We had a chain that took a user query, passed it to GPT-4 with a prompt asking for a JSON response with keys 'summary' and 'recommendations'. The output went through a PydanticOutputParser. Everything worked in staging with GPT-3.5. After switching to GPT-4 and deploying to prod, we got 500 errors immediately.
I ssh'd into a box and manually ran the chain with the same input that failed in prod. With verbose=True, I saw the raw LLM output: a markdown code block with ```json ... ```. The parser was trying to parse that as literal JSON and failing. GPT-4 had decided to be helpful by formatting the JSON in a code block, which the parser didn't handle.
I wrote a custom output parser that first strips leading/trailing backticks and 'json' label, then falls back to the original parser. I also added a fix prompt instruction asking the LLM to return ONLY raw JSON. Deployed the fix and added a unit test that simulates the code fence scenario. Error rate went from 15% to 0%.
Root cause
LLM output included markdown code fences around JSON, which the PydanticOutputParser could not parse.
The fix
Custom output parser with regex to strip markdown code blocks before parsing.
The lesson
Always inspect the raw LLM output before the parser. LLMs, especially newer ones, may add formatting that breaks strict parsers. Include explicit formatting instructions in the prompt and have a fallback parser.
To debug chain output errors, you need to see what the LLM actually sends. LangChain's verbose mode prints the prompt and response to stderr. Enable it with chain.run(input, verbose=True) or set LANGCHAIN_VERBOSE=true in your environment.
If you can't use verbose (e.g., in production), add a callback handler that captures the LLM response. Here's a minimal example: class RawOutputCallback(BaseCallbackHandler): def on_llm_end(self, response, **kwargs): print(response.generations[0][0].text). Attach it to your chain via callbacks=[RawOutputCallback()].
The most common parser failure is when the LLM wraps JSON in markdown code fences. GPT-4 and Claude often do this. Fix: write a custom parser that strips ```json and ``` before feeding to the actual parser.
Another pitfall: the LLM might add a trailing comma in a JSON array. Use json.loads with strict=False or preprocess with regex. For structured output, consider using the new LangChain 'with_structured_output' method that forces JSON mode in supported models.
Chain output errors can also come from memory state. If a chain step expects a variable from memory but the key name is wrong, you get a KeyError. Check memory keys with memory.load_memory_variables({}).
A common issue is using the same variable name in both memory and chain input (e.g., 'history'). This causes a collision. Rename one of them. Also, clear memory between test runs to avoid stale state affecting results.
LangChain provides an OutputFixingParser that takes a failed parse attempt and passes it back to the LLM with a prompt to fix it. This is a quick fix but increases latency and cost.
Example: from langchain.output_parsers import OutputFixingParser; fix_parser = OutputFixingParser.from_llm(parser=original_parser, llm=llm). Use it as a fallback only when the initial parse fails.
Frequently asked questions
Why does my LangChain chain return None?
This usually means the chain's final step did not return a value. Check that the last step has an output key and that it's set. Also, if using a SequentialChain, ensure the output_variables list includes the expected key.
How do I get the raw LLM output in production?
Use a custom callback handler that logs the response. Attach it to your chain's callbacks. Be careful not to log sensitive data. Alternatively, set the environment variable LANGCHAIN_VERBOSE=true but that logs to stderr which may be captured.
Why does the same input sometimes work, sometimes fail?
LLMs are non-deterministic unless you set temperature=0. Even then, subtle differences in sampling can produce slightly different outputs. Also, memory state might differ between runs. Set temperature=0 and seed if possible.
How do I debug a chain that works in local but fails in production?
Differences in environment: Python version, LangChain version, model provider (maybe staging uses GPT-3.5, prod GPT-4). Also check if production has a different prompt template or parser. Replicate the exact production environment locally.
What is the difference between chain.run() and chain()?
chain.run() is a convenience method that only works for chains with a single input and output key. chain() (or chain.__call__) takes a dictionary and returns a dictionary. If you use run() and your chain expects multiple inputs, you'll get an error.