content_safety

Extract content-safety annotation payloads from a message.

Inspects the message's content for non_standard_annotation blocks whose value contains provider == "azure_content_safety".

Print a formatted summary of content-safety annotations on a message.

Useful for troubleshooting and inspecting middleware results in notebooks or the console. Extracts all non_standard content blocks produced by Azure Content Safety middleware and renders them in a human-readable, colour-free table format.

Usage::

result = agent.invoke({"messages": [HumanMessage(content="...")]})
print_content_safety_annotations(
    result["messages"][-1]
)

A harm-category evaluation from text or image content analysis.

Value payload stored inside a NonStandardAnnotation for violations.

Base class for all content-safety violations.

Raised when content safety violations with exit_behavior='error'.

AgentMiddleware that evaluates groundedness of model outputs.

Groundedness detection analyses language model outputs to determine whether they are factually aligned with user-provided information or contain fictional/hallucinated content.

The middleware runs as an after_model hook — it evaluates every model response immediately after generation. The behaviour when ungrounded content is detected depends on exit_behavior:

• "error" (default) – raises :exc:ContentSafetyViolationError, halting the graph. The exception carries the evaluation details.
• "continue" – annotates the state with a groundedness_evaluation key containing the evaluation results and lets execution proceed.

In both modes the state is annotated with the evaluation result so callers can inspect it.

Grounding sources are collected automatically from the chat history by default:

• SystemMessage content – the system prompt often contains the authoritative context the model should stay grounded in.
• ToolMessage content – tool / function-call results such as RAG retrieval chunks, web-search snippets, or database lookups.
• AIMessage annotation titles – citation metadata attached to model responses (e.g. url_citation annotations from web-search grounding).

You can override the extraction logic by supplying a context_extractor callable. It receives the current graph state and the LangGraph :class:~langchain.agents.middleware.Runtime execution context, and must return a :class:GroundednessInput (or None to skip evaluation entirely) containing the answer to evaluate, the grounding sources, and (for task="QnA") the question::

from langchain_azure_ai.agents.middleware import (
    AzureGroundednessMiddleware,
    GroundednessInput,
)

def my_extractor(state, runtime):
    # ``runtime`` is the LangGraph Runtime object — use it to access
    # the user-defined context, memory store, stream writer, etc.
    return GroundednessInput(
        answer=state["custom_answer"],
        sources=state["retrieved_chunks"],
        question=state.get("user_question"),
    )

middleware = AzureGroundednessMiddleware(
    context_extractor=my_extractor,
    task="QnA",
)

After the model runs (in continue mode), the state will contain a groundedness_evaluation key with the evaluation results::

{
    "is_grounded": True,
    "ungrounded_percentage": 0.0,
    "details": []
}

A groundedness evaluation.

Inputs extracted from an agent state for groundedness evaluation.

This is the return type for a context_extractor callable passed to :class:~langchain_azure_ai.agents.middleware.content_safety.AzureGroundednessMiddleware.

AgentMiddleware that screens image content with Azure AI Content Safety.

Use this middleware alongside :class:AzureContentModerationMiddleware when your agent handles visual content. Because image analysis uses a separate API endpoint (analyze_image) and different category enumerations, a dedicated class keeps each concern focused and composable.

The middleware extracts images from the most recent HumanMessage (input) and, optionally, from the most recent AIMessage (output). It supports:

• Base64 data URLs – data:image/png;base64,<data>
• HTTP(S) URLs – publicly accessible image URLs

Content is analyzed using the Azure AI Content Safety image analysis API. When violations are detected, the middleware takes one of two actions:

• "error" – raises :exc:ContentSafetyViolationError, halting the graph.
• "continue" – replaces the offending message with a violation notice (either a service-derived description or a custom violation_message) and lets execution proceed.

Both synchronous (before_agent / after_agent) and asynchronous (abefore_agent / aafter_agent) hooks are implemented.

Input extracted from an agent state for image content moderation.

This is the return type for a context_extractor callable passed to :class:~langchain_azure_ai.agents.middleware.content_safety.AzureContentModerationForImagesMiddleware.

AgentMiddleware that detects prompt injection using Azure AI Content Safety.

Prompt shield protects agents from adversarial inputs designed to hijack the agent's behavior. Two types of injection are detected:

• Direct prompt injection – malicious instructions in the user's own prompt (user_prompt in the API).
• Indirect prompt injection – malicious instructions embedded in external documents fed back to the agent (documents in the API), such as web search results, retrieved knowledge-base chunks, or email bodies.

The middleware extracts the last HumanMessage as the user prompt. Any ToolMessage items in the state (tool/function outputs) are forwarded to the API as documents so indirect injection via tool results is also caught.

When an injection attack is detected, the middleware takes one of two actions:

• "error" – raises :exc:ContentSafetyViolationError, halting the graph.
• "continue" – replaces the offending message with a violation notice (either a service-derived description or a custom violation_message) and lets execution proceed.

A prompt-injection evaluation.

Input extracted from an agent state for prompt shield evaluation.

This is the return type for a context_extractor callable passed to :class:~langchain_azure_ai.agents.middleware.content_safety.AzurePromptShieldMiddleware.

AgentMiddleware that detects protected material using Azure AI Content Safety.

Protected material detection checks whether text contains copyright-protected content such as song lyrics, news articles, book passages, or other protected intellectual property. Use this middleware to prevent agents from reproducing or accepting copyrighted content.

When protected material is detected, the middleware takes one of two actions:

• "error" – raises :exc:ContentSafetyViolationError, halting the graph.
• "continue" – replaces the offending message with a violation notice (either a service-derived description or a custom violation_message) and lets execution proceed.

Both synchronous (before_agent / after_agent) and asynchronous (abefore_agent / aafter_agent) hooks are implemented.

A protected-material evaluation.

AgentMiddleware that screens text messages with Azure AI Content Safety.

Pass this class (or multiple instances) in the middleware parameter of any LangChain create_agent call:

from langchain.agents import create_agent
from langchain_azure_ai.agents.middleware import (
    AzureContentModerationMiddleware
)

agent = create_agent(
    model="azure_ai:gpt-4.1",
    middleware=[
        # Screen both input and output text for all harm categories
        AzureContentModerationMiddleware(
            endpoint="https://my-resource.cognitiveservices.azure.com/",
            exit_behavior="error",
        ),
    ],
)

You can compose multiple instances with different configurations:

agent = create_agent(
    model="azure_ai:gpt-4.1",
    middleware=[
        # Raise on hate/violence on input only
        AzureContentModerationMiddleware(
            categories=["Hate", "Violence"],
            exit_behavior="error",
            apply_to_input=True,
            apply_to_output=False,
            name="input_safety",
        ),
        # Replace self-harm content in model output and continue
        AzureContentModerationMiddleware(
            categories=["SelfHarm"],
            exit_behavior="continue",
            apply_to_input=False,
            apply_to_output=True,
            name="output_safety",
        ),
    ],
)

The middleware analyses text content using the Azure AI Content Safety API and takes one of three actions when violations are detected:

• "error" – raises :exc:ContentSafetyViolationError, halting the graph.
• "replace" – replaces the offending message with a violation notice (either a service-derived description or a custom violation_message) and lets execution proceed.
• "continue" – ignores the violation and lets execution proceed by adding annotations to the message metadata with details of the violation(s).

Both synchronous (before_agent / after_agent) and asynchronous (abefore_agent / aafter_agent) hooks are implemented.

By default the middleware extracts the last HumanMessage (input) or AIMessage (output) and submits its text to the service. You can override this behaviour by supplying a context_extractor callable::

from langchain_azure_ai.agents.middleware import (
    AzureContentModerationMiddleware,
    TextModerationInput,
)

def my_extractor(state, runtime):
    # Return None to skip moderation for this call
    messages = state.get("messages", [])
    text = " ".join(m.content for m in messages if hasattr(m, "content"))
    return TextModerationInput(text=text) if text else None

middleware = AzureContentModerationMiddleware(
    context_extractor=my_extractor,
)

A blocklist-match evaluation from text content analysis.

Input extracted from an agent state for text content moderation.

This is the return type for a context_extractor callable passed to :class:~langchain_azure_ai.agents.middleware.content_safety.AzureContentModerationMiddleware or :class:~langchain_azure_ai.agents.middleware.content_safety.AzureProtectedMaterialMiddleware.