Introduction/ Issue:
One of the biggest challenges when working with Large Language Models (LLMs) is obtaining reliable structured output.
Even if you explicitly ask an LLM to return JSON, the response isn’t always in the format your application expects. Sometimes it wraps the JSON inside Markdown code blocks, includes additional explanations, misses required fields, or returns values with incorrect data types.
These issues are easy to overlook during experimentation, but they can quickly become a problem in production applications where APIs, databases, or downstream services expect data in a consistent structure.
A common solution is to validate the response after receiving it. However, instead of immediately throwing an exception when validation fails, we can give the LLM an opportunity to correct its own output.
That’s exactly what we’ll build in this article.
We’ll create a reusable Structured Output Fixer using LangChain and Pydantic. This utility validates the LLM’s response against a predefined schema and, whenever the response is invalid, automatically sends the validation error back to the model. The LLM then attempts to generate a corrected response, making the entire process more reliable without adding extra complexity to your application.
Since the component accepts any LangChain chat model, you can use it with Groq, OpenAI, Anthropic, Google Gemini, Ollama, or any other compatible provider.
Since the component accepts any LangChain chat model, you can use it with Groq, OpenAI, Anthropic, Google Gemini, Ollama, or any other compatible provider.
Why we need to do / Cause of the issue:
Instead of implementing validation and retry logic in every project, you can build this component once and reuse it across all your LangChain applications.
This component:
- Works with any LangChain chat model.
- Validates responses using Pydantic.
- Removes Markdown code fences from LLM responses.
- Automatically retries when validation fails.
- Returns strongly typed Python objects.
- Keeps your application code clean, reusable, and easier to maintain.
By centralizing this logic into a reusable component, you improve reliability, reduce duplicate code, and make your AI applications more production-ready.
How do we solve:
- Send the user’s request along with the expected JSON schema to the LLM.
- Extract the JSON object from the response.
- Validate it using a Pydantic model.
- If the validation succeeds, return the structured object.
- If validation fails, send the validation error back to the LLM and ask it to fix the response.
- Repeat the process until the output is valid or the retry limit is reached.
Instead of failing immediately, the model gets an opportunity to correct its own mistakes, resulting in much more reliable structured output.
Step 1: Create a project folder and Install the required packages in the terminal using the below command.
pip install langchain langchain-core langchain-groq pydantic python-dotenv
structured-output-fixer/
│
├── structured_output_fixer.py
└── .env
Step 2: Create a environment file using .env using and paste your LLM API Key
Step -3 :Create a file named structured_output_fixer.py and paste the complete code from this article into it.
“””
Structured Output Fixer
A reusable utility that validates LLM responses against a Pydantic schema.
If the generated response is invalid, the validation error is sent back to
the model, allowing it to correct its own output automatically.
“””
from __future__ import annotations
import json
import re
from typing import Generic, Type, TypeVar
from pydantic import BaseModel, ValidationError
from langchain_core.language_models.chat_models import BaseChatModel
from langchain_core.messages import HumanMessage, SystemMessage
T = TypeVar(“T”, bound=BaseModel)
class StructuredOutputError(Exception):
“””Raised when a valid structured response cannot be generated.”””
def __init__(self, message: str, last_raw_output: str, last_error: str):
super().__init__(message)
self.last_raw_output = last_raw_output
self.last_error = last_error
class StructuredOutputFixer(Generic[T]):
“””
Ensures that LLM responses conform to a Pydantic schema by
validating the generated JSON and automatically requesting
corrections whenever validation fails.
“””
def __init__(
self,
llm: BaseChatModel,
schema: Type[T],
max_retries: int = 3,
verbose: bool = False,
):
self.llm = llm
self.schema = schema
self.max_retries = max_retries
self.verbose = verbose
def _log(self, message: str):
“””
Prints debug messages when verbose mode is enabled.
“””
if self.verbose:
print(f”[StructuredOutputFixer] {message}”)
def _system_prompt(self) -> str:
“””
Builds the system prompt containing the JSON schema that
the LLM must follow.
“””
schema_json = json.dumps(
self.schema.model_json_schema(),
indent=2,
)
return (
“You are an information extraction assistant.\n\n”
“Return ONLY a valid JSON object.\n”
“Do not include markdown, explanations or extra text.\n\n”
f”JSON Schema:\n{schema_json}”
)
def _initial_messages(self, user_input: str):
“””
Creates the initial conversation sent to the LLM.
“””
return [
SystemMessage(content=self._system_prompt()),
HumanMessage(content=user_input),
]
def _retry_messages(
self,
user_input: str,
previous_output: str,
validation_error: str,
):
“””
Creates a retry prompt by including the previous response
and the corresponding validation error.
“””
return [
SystemMessage(content=self._system_prompt()),
HumanMessage(
content=f”””
Original Request
{user_input}
Your previous response
{previous_output}
Validation Error
{validation_error}
Please correct your previous response.
Return ONLY the corrected JSON object.
“””
),
]
@staticmethod
def _extract_json(text: str) -> str:
“””Extracts the JSON object from the LLM response.”””
text = text.strip()
text = re.sub(
r”^“`(?:json)?\s*|\s*“`$”,
“”,
text,
flags=re.MULTILINE,
)
match = re.search(r”\{.*\}”, text, re.DOTALL)
return match.group(0) if match else text
def run(self, user_input: str) -> T:
“””
Generates, validates and returns structured output.
“””
messages = self._initial_messages(user_input)
last_raw_output = “”
last_error = “”
for attempt in range(1, self.max_retries + 1):
self._log(f”Attempt {attempt}/{self.max_retries}”)
response = self.llm.invoke(messages)
last_raw_output = response.content
cleaned_json = self._extract_json(last_raw_output)
try:
parsed = json.loads(cleaned_json)
validated = self.schema.model_validate(parsed)
self._log(“Validation successful.”)
return validated
except (json.JSONDecodeError, ValidationError) as ex:
last_error = str(ex)
self._log(“Validation failed.”)
self._log(last_error)
messages = self._retry_messages(
user_input=user_input,
previous_output=last_raw_output,
validation_error=last_error,
)
raise StructuredOutputError(
message=(
f”Failed to generate a valid ”
f”{self.schema.__name__} ”
f”after {self.max_retries} attempts.”
),
last_raw_output=last_raw_output,
last_error=last_error,
)
# ———————————————————————-
# Example Usage
# ———————————————————————-
if __name__ == “__main__”:
from dotenv import load_dotenv
from langchain_groq import ChatGroq
load_dotenv()
class Invoice(BaseModel):
“””
Represents the structured invoice information extracted from
unstructured text.
“””
invoice_number: str
vendor_name: str
invoice_date: str
total_amount: float
currency: str
llm = ChatGroq(
model=”llama-3.3-70b-versatile”,
temperature=0,
)
fixer = StructuredOutputFixer(
llm=llm,
schema=Invoice,
max_retries=3,
verbose=True,
)
invoice = fixer.run(
“””
Extract invoice details.
Invoice Number: INV-2026-1045
Vendor: ABC Technologies Pvt Ltd
Invoice Date: 30 July 2026
Amount Due: USD 1589.50
“””
)
print(“\nValidated Output\n”)
print(invoice)
Step 5: Run the file
python structured_output_fixer.py
Expected Output
You should see logs similar to the following:
[StructuredOutputFixer] Attempt 1/3
[StructuredOutputFixer] Validation successful.
Conclusion:
Structured output is a common requirement in modern AI applications, but relying on an LLM to always return perfectly formatted JSON is risky. Even well-crafted prompts can produce responses that are missing fields, contain invalid JSON, or include unwanted formatting.
By introducing a reusable Structured Output Fixer, you can validate every response against a Pydantic schema and automatically allow the model to correct its own mistakes before the data reaches your application.
Because the component is model-agnostic, it can be used with any LangChain-supported chat model, making it easy to integrate into existing and future projects. More importantly, it encapsulates validation, correction, and retry logic into a single reusable utility, resulting in cleaner application code and more reliable AI workflows.
I hope this component helps simplify your LangChain projects and serves as a useful building block for creating robust, production-ready GenAI applications.
