ChatNVIDIA subclass that injects nvext.agent_hints into requests
for Dynamo KV cache routing optimization.
A unique prefix_id is auto-generated for every request.
Model information.
Client to NVIDIA embeddings models.
LangChain LLM that uses the Completions API with NVIDIA NIMs.
NVIDIA chat model.
Set inference priority for all LLM calls within scope.
Lower number = higher priority (priority=1 is most urgent).
Works as both a decorator and a context manager::
# decorator — depriori
Callback Handler that tracks OpenAI info.
LangChain Document Compressor that uses the NVIDIA NeMo Retriever Reranking API.
Base exception for NVIDIA RAG retriever errors.
Raised when the RAG server endpoint is unreachable.
Common causes:
Raised when the RAG server returns an error response.
Includes the HTTP status code and response body for debugging.
Raised when the request payload is invalid (e.g. 422 Unprocessable Entity).
LangChain retriever that queries the NVIDIA RAG Blueprint /v1/search endpoint.
Targets containerized RAG deployments where the rag-server exposes the search API. Supports all DocumentSearch parameter
Register a model as a known model.
Must be done at the beginning of a program, at least before the model is used or available models are listed.
For instance:
from langchain_nvidia_ai_end
Lookup a model by name, using only the table of known models.
The name is either:
None)Callers can check to see if the name was an al
Determine the model to use based on a name, using only the table of known models.
Raise a warning if the model is found to be an alias of a known model.
If the model is not found, return None.
Convert a LangChain message to a dictionary.
Parse thinking content from text.
This function handles multiple formats by trying to find the reasoning content
Return the active inference priority, or None if unset.
Return an explicit constructor setting or the default-on environment setting.
Map a public client class to the approved coarse operation category.
Return a canonical allowlisted NIM ID and approved model-family bucket.
Extract only aggregate token counters from a known usage object.
Keep the latest cumulative usage counters observed in a stream.
Map an error to an approved coarse category without retaining its text.
Map a response status to the approved coarse status-class bucket.
Record one logical request without retaining request or response content.
Flush the current in-memory aggregate, primarily for controlled shutdown.
Standardize the model name to a format that can be used in the OpenAI API.
Get the cost in USD for a given model and number of tokens.
Get the OpenAI callback handler in a context manager.
Exposes token and cost information.