ChatOutlines()Pydantic model, JSON Schema, or callable (function signature) for structured JSON generation.
Outlines can generate JSON output that follows a specified structure, which is useful for:
You can provide:
The generated JSON will adhere to the specified structure.
For more details, see: https://dottxt-ai.github.io/outlines/reference/generation/json/
Outlines chat model integration.
Setup:
pip install outlines
Key init args — client params: backend: Literal["llamacpp", "transformers", "transformers_vision", "vllm", "mlxlm"] = "transformers" Specifies the backend to use for the model.
Key init args — completion params: model: str Identifier for the model to use with Outlines. max_tokens: int = 256 The maximum number of tokens to generate. stop: Optional[List[str]] = None A list of strings to stop generation when encountered. streaming: bool = True Whether to stream the results, token by token.
See full list of supported init args and their descriptions in the params section.
Instantiate:
from langchain_community.chat_models import ChatOutlines chat = ChatOutlines(model="meta-llama/Llama-2-7b-chat-hf")
Invoke:
chat.invoke([HumanMessage(content="Say foo:")])
Stream:
for chunk in chat.stream([HumanMessage(content="Count to 10:")]): print(chunk.content, end="", flush=True)