Construct a Spark SQL agent from an LLM and tools.
This agent can execute arbitrary SQL against your Spark environment.
By default, the agent is allowed to generate SQL strings and run them via the underlying connection. This is powerful, but it also means the agent can generate expensive or dangerous queries (e.g., long-running queries, large scans/joins, or locking queries depending on your environment and permissions).
create_spark_sql_agent returns a langchain_classic AgentExecutor.
AgentExecutor is an agent abstraction that has long been considered legacy
and is not actively supported as the recommended foundation for new production
applications.
For production-grade agent development, prefer building with Deep Agents: https://github.com/langchain-ai/deepagents
If you use this in production, coordinate with your security/DB teams and apply server-side controls:
Client-side timeouts do not always guarantee that a running statement is cancelled on the server.
create_spark_sql_agent(
llm: BaseLanguageModel,
toolkit: SparkSQLToolkit,
callback_manager: Optional[BaseCallbackManager] = None,
callbacks: Callbacks = None,
prefix: str = SQL_PREFIX,
suffix: str = SQL_SUFFIX,
format_instructions: Optional[str] = None,
input_variables: Optional[List[str]] = None,
top_k: int = 10,
max_iterations: Optional[int] = 15,
max_execution_time: Optional[float] = None,
early_stopping_method: str = 'force',
verbose: bool = False,
agent_executor_kwargs: Optional[Dict[str, Any]] = None,
**kwargs: Any = {}
) -> AgentExecutor| Name | Type | Description |
|---|---|---|
llm* | BaseLanguageModel | The language model to use. |
toolkit* | SparkSQLToolkit | The Spark SQL toolkit. |
callback_manager | Optional[BaseCallbackManager] | Default: NoneOptional. The callback manager. Default is None. |
callbacks | Callbacks | Default: NoneOptional. The callbacks. Default is None. |
prefix | str | Default: SQL_PREFIXOptional. The prefix for the prompt. Default is SQL_PREFIX. |
suffix | str | Default: SQL_SUFFIXOptional. The suffix for the prompt. Default is SQL_SUFFIX. |
format_instructions | Optional[str] | Default: NoneOptional. The format instructions for the prompt. Default is None. |
input_variables | Optional[List[str]] | Default: NoneOptional. The input variables for the prompt. Default is None. |
top_k | int | Default: 10Optional. The top k for the prompt. Default is 10. |
max_iterations | Optional[int] | Default: 15Optional. The maximum iterations to run. Default is 15. |
max_execution_time | Optional[float] | Default: NoneOptional. The maximum execution time. Default is None. |
early_stopping_method | str | Default: 'force'Optional. The early stopping method. Default is "force". |
verbose | bool | Default: FalseOptional. Whether to print verbose output. Default is False. |
agent_executor_kwargs | Optional[Dict[str, Any]] | Default: NoneOptional. The agent executor kwargs. Default is None. |
kwargs | Any | Default: {}Any. Additional keyword arguments. |