Class TrajectoryEvalChain

A chain for evaluating ReAct style agents.

This chain is used to evaluate ReAct style agents by reasoning about the sequence of actions taken and their outcomes.

Hierarchy

AgentTrajectoryEvaluator
- TrajectoryEvalChain

Constructors

constructor

new TrajectoryEvalChain(
fields: LLMChainInput<EvalOutputType, BaseLanguageModelInterface>,
): TrajectoryEvalChain
Parameters
- fields: LLMChainInput<EvalOutputType, BaseLanguageModelInterface>
Returns TrajectoryEvalChain
Inherited from AgentTrajectoryEvaluator.constructor
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/chains/llm_chain.ts:123

Properties

`Optional`criterionName

criterionName?: string

`Optional`evaluationName

evaluationName?: string = ...

The name of the evaluation.

lc_serializable

lc_serializable: boolean = true

llm

llm: BaseLanguageModelInterface

LLM Wrapper to use

`Optional`llmKwargs

llmKwargs?: any

Kwargs to pass to LLM

`Optional`memory

memory?: any

outputKey

outputKey: string = "text"

Key to use for output, defaults to text

outputParser

outputParser: TrajectoryOutputParser = ...

prompt

prompt: BasePromptTemplate

Prompt object to use

requiresInput

requiresInput: boolean = true

requiresReference

requiresReference: boolean = false

`Optional`skipInputWarning

skipInputWarning?: string = ...

`Optional`skipReferenceWarning

skipReferenceWarning?: string = ...

Accessors

inputKeys

get inputKeys(): any
Returns any
Inherited from QAEvalChain.inputKeys
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/chains/llm_chain.ts:115

lc_namespace

get lc_namespace(): string[]
Returns string[]
Inherited from QAEvalChain.lc_namespace
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/chains/base.ts:41

outputKeys

get outputKeys(): string[]
Returns string[]
Inherited from QAEvalChain.outputKeys
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/chains/llm_chain.ts:119

Methods

_chainType

_chainType(): "llm"
Return the string type key uniquely identifying this class of chain.

Returns "llm"
Inherited from AgentTrajectoryEvaluator._chainType
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/chains/llm_chain.ts:254

_evaluateAgentTrajectory

_evaluateAgentTrajectory(
    args: LLMTrajectoryEvaluatorArgs,
    callOptions: unknown,
    config?: any,
): Promise<ChainValues>
Evaluate a trajectory.
Parameters
- args: LLMTrajectoryEvaluatorArgs
- callOptions: unknown
- Optionalconfig: any
Returns Promise<ChainValues>
The evaluation result.
Overrides AgentTrajectoryEvaluator._evaluateAgentTrajectory
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/evaluation/agents/trajectory.ts:195

`Protected`_formatValues

_formatValues(values: any): Promise<any>
Parameters
- values: any
Returns Promise<any>
Inherited from AgentTrajectoryEvaluator._formatValues
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/chains/base.ts:217

_getNumTokens

_getNumTokens(text: string): Promise<number>
Parameters
- text: string
Returns Promise<number>
Inherited from AgentTrajectoryEvaluator._getNumTokens
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/chains/llm_chain.ts:284

_prepareOutput

_prepareOutput(result: ChainValues): ChainValues
Parameters
- result: ChainValues
Returns ChainValues
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/evaluation/agents/trajectory.ts:154

apply

apply(inputs: ChainValues[], config?: any[]): Promise<ChainValues[]>
Parameters
- inputs: ChainValues[]
- Optionalconfig: any[]
Returns Promise<ChainValues[]>
Deprecated
Use .batch() instead. Will be removed in 0.2.0.

Call the chain on all inputs in the list
Inherited from AgentTrajectoryEvaluator.apply
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/chains/base.ts:258

call

call(values: any, config?: any): Promise<ChainValues>
Run the core logic of this chain and add to output if desired.

Wraps _call and handles memory.
Parameters
- values: any
- Optionalconfig: any
Returns Promise<ChainValues>
Inherited from AgentTrajectoryEvaluator.call
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/chains/llm_chain.ts:180

checkEvaluationArgs

checkEvaluationArgs(reference?: string, input?: string): void
Check if the evaluation arguments are valid.
Parameters
- Optionalreference: string
  The reference label.
- Optionalinput: string
  The input string.
Returns void
Throws
If the evaluator requires an input string but none is provided, or if the evaluator requires a reference label but none is provided.
Inherited from AgentTrajectoryEvaluator.checkEvaluationArgs
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/evaluation/base.ts:62

evaluateAgentTrajectory

evaluateAgentTrajectory(
    args: LLMTrajectoryEvaluatorArgs,
    callOptions?: unknown,
    config?: any,
): Promise<ChainValues>
Evaluate a trajectory.
Parameters
- args: LLMTrajectoryEvaluatorArgs
- OptionalcallOptions: unknown
- Optionalconfig: any
Returns Promise<ChainValues>
The evaluation result.
Inherited from AgentTrajectoryEvaluator.evaluateAgentTrajectory
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/evaluation/base.ts:351

formatReference

formatReference(reference?: string): string
Parameters
- Optionalreference: string
Returns string
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/evaluation/agents/trajectory.ts:183

getAgentTrajectory

getAgentTrajectory(steps: AgentStep[]): string
Get the agent trajectory as a formatted string.
Parameters
- steps: AgentStep[]
  The agent trajectory.
Returns string
The formatted agent trajectory.
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/evaluation/agents/trajectory.ts:168

invoke

invoke(input: ChainValues, options?: any): Promise<ChainValues>
Invoke the chain with the provided input and returns the output.
Parameters
- input: ChainValues
  Input values for the chain run.
- Optionaloptions: any
Returns Promise<ChainValues>
Promise that resolves with the output of the chain run.
Inherited from AgentTrajectoryEvaluator.invoke
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/chains/base.ts:86

predict

predict(values: any, callbackManager?: any): Promise<EvalOutputType>
Format prompt with values and pass to LLM
Parameters
- values: any
  keys to pass to prompt template
- OptionalcallbackManager: any
  CallbackManager to use
Returns Promise<EvalOutputType>
Completion from LLM.
Example
```
llm.predict({ adjective: "funny" })
```
Inherited from AgentTrajectoryEvaluator.predict
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/chains/llm_chain.ts:246

prepOutputs

prepOutputs(
    inputs: Record<string, unknown>,
    outputs: Record<string, unknown>,
    returnOnlyOutputs?: boolean,
): Promise<Record<string, unknown>>
Parameters
- inputs: Record<string, unknown>
- outputs: Record<string, unknown>
- returnOnlyOutputs: boolean = false
Returns Promise<Record<string, unknown>>
Inherited from AgentTrajectoryEvaluator.prepOutputs
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/chains/base.ts:149

run

run(input: any, config?: any): Promise<string>
Parameters
- input: any
- Optionalconfig: any
Returns Promise<string>
Deprecated
Use .invoke() instead. Will be removed in 0.2.0.
Inherited from AgentTrajectoryEvaluator.run
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/chains/base.ts:190

serialize

serialize(): SerializedLLMChain
Returns SerializedLLMChain
Deprecated
Inherited from AgentTrajectoryEvaluator.serialize
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/chains/llm_chain.ts:274

`Static`deserialize

deserialize(data: SerializedLLMChain): Promise<LLMChain<string, any>>
Load a chain from a json-like object describing it.
Parameters
- data: SerializedLLMChain
Returns Promise<LLMChain<string, any>>
Inherited from AgentTrajectoryEvaluator.deserialize
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/chains/llm_chain.ts:258

`Static`fromLLM

fromLLM(
    llm: BaseChatModel,
    agentTools?: StructuredToolInterface[],
    chainOptions?: Partial<
        Omit<
            LLMEvalChainInput<EvalOutputType, BaseLanguageModelInterface>,
            "llm",
        >,
    >,
): Promise<TrajectoryEvalChain>
Create a new TrajectoryEvalChain.
Parameters
- llm: BaseChatModel
- OptionalagentTools: StructuredToolInterface[]
  The tools used by the agent.
- OptionalchainOptions: Partial<
      Omit<
          LLMEvalChainInput<EvalOutputType, BaseLanguageModelInterface>,
          "llm",
      >,
  >
  The options for the chain.
Returns Promise<TrajectoryEvalChain>
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/evaluation/agents/trajectory.ts:130

`Static`lc_name

lc_name(): string
Returns string
Overrides AgentTrajectoryEvaluator.lc_name
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/evaluation/agents/trajectory.ts:80

`Static`resolveTrajectoryPrompt

resolveTrajectoryPrompt(
prompt?: any,
agentTools?: StructuredToolInterface[],
): any
Parameters
- Optionalprompt: any
- OptionalagentTools: StructuredToolInterface[]
Returns any
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/evaluation/agents/trajectory.ts:94

`Static`toolsDescription

toolsDescription(agentTools: StructuredToolInterface[]): string
Get the description of the agent tools.
Parameters
- agentTools: StructuredToolInterface[]
Returns string
The description of the agent tools.
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/evaluation/agents/trajectory.ts:115

Class TrajectoryEvalChain

Hierarchy

Index

Constructors

Properties

Accessors

Methods

Constructors

constructor

Parameters

Returns TrajectoryEvalChain

Properties

OptionalcriterionName

OptionalevaluationName

lc_serializable

llm

OptionalllmKwargs

Optionalmemory

outputKey

outputParser

prompt

requiresInput

requiresReference

OptionalskipInputWarning

OptionalskipReferenceWarning

Accessors

inputKeys

Returns any

lc_namespace

Returns string[]

outputKeys

Returns string[]

Methods

_chainType

Returns "llm"

_evaluateAgentTrajectory

Parameters

Returns Promise<ChainValues>

Protected_formatValues

Parameters

Returns Promise<any>

_getNumTokens

Parameters

Returns Promise<number>

_prepareOutput

Parameters

Returns ChainValues

apply

Parameters

Returns Promise<ChainValues[]>

Deprecated

call

Parameters

Returns Promise<ChainValues>

checkEvaluationArgs

Parameters

Returns void

Throws

evaluateAgentTrajectory

Parameters

Returns Promise<ChainValues>

formatReference

Parameters

Returns string

getAgentTrajectory

Parameters

Returns string

invoke

Parameters

Returns Promise<ChainValues>

predict

Parameters

Returns Promise<EvalOutputType>

Example

prepOutputs

Parameters

Returns Promise<Record<string, unknown>>

run

Parameters

Returns Promise<string>

`Optional`criterionName

`Optional`evaluationName

`Optional`llmKwargs

`Optional`memory

`Optional`skipInputWarning

`Optional`skipReferenceWarning

`Protected`_formatValues

`Static`deserialize

`Static`fromLLM

`Static`lc_name

`Static`resolveTrajectoryPrompt

`Static`toolsDescription