AzureGroundednessMiddleware

AgentMiddleware that evaluates groundedness of model outputs.

Groundedness detection analyses language model outputs to determine whether they are factually aligned with user-provided information or contain fictional/hallucinated content.

The middleware runs as an after_model hook — it evaluates every model response immediately after generation. The behaviour when ungrounded content is detected depends on exit_behavior:

• "error" (default) – raises :exc:ContentSafetyViolationError, halting the graph. The exception carries the evaluation details.
• "continue" – annotates the state with a groundedness_evaluation key containing the evaluation results and lets execution proceed.

In both modes the state is annotated with the evaluation result so callers can inspect it.

Grounding sources are collected automatically from the chat history by default:

• SystemMessage content – the system prompt often contains the authoritative context the model should stay grounded in.
• ToolMessage content – tool / function-call results such as RAG retrieval chunks, web-search snippets, or database lookups.
• AIMessage annotation titles – citation metadata attached to model responses (e.g. url_citation annotations from web-search grounding).

You can override the extraction logic by supplying a context_extractor callable. It receives the current graph state and the LangGraph :class:~langchain.agents.middleware.Runtime execution context, and must return a :class:GroundednessInput (or None to skip evaluation entirely) containing the answer to evaluate, the grounding sources, and (for task="QnA") the question::

from langchain_azure_ai.agents.middleware import (
    AzureGroundednessMiddleware,
    GroundednessInput,
)

def my_extractor(state, runtime):
    # ``runtime`` is the LangGraph Runtime object — use it to access
    # the user-defined context, memory store, stream writer, etc.
    return GroundednessInput(
        answer=state["custom_answer"],
        sources=state["retrieved_chunks"],
        question=state.get("user_question"),
    )

middleware = AzureGroundednessMiddleware(
    context_extractor=my_extractor,
    task="QnA",
)

After the model runs (in continue mode), the state will contain a groundedness_evaluation key with the evaluation results::

{
    "is_grounded": True,
    "ungrounded_percentage": 0.0,
    "details": []
}