Class CriteriaEvalChain

Hierarchy (View Summary)

LLMStringEvaluator
- CriteriaEvalChain
  - LabeledCriteriaEvalChain

Constructors

constructor

new CriteriaEvalChain(
fields: LLMChainInput<EvalOutputType, BaseLanguageModelInterface>,
): CriteriaEvalChain
Parameters
- fields: LLMChainInput<EvalOutputType, BaseLanguageModelInterface>
Returns CriteriaEvalChain
Inherited from LLMStringEvaluator.constructor
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/chains/llm_chain.ts:123

Properties

`Optional`criterionName

criterionName?: string

`Optional`evaluationName

evaluationName?: string = ...

The name of the evaluation.

lc_serializable

lc_serializable: boolean = true

llm

llm: BaseLanguageModelInterface

LLM Wrapper to use

`Optional`llmKwargs

llmKwargs?: any

Kwargs to pass to LLM

`Optional`memory

memory?: any

outputKey

outputKey: string = "text"

Key to use for output, defaults to text

outputParser

outputParser: BaseLLMOutputParser<EvalOutputType> = ...

prompt

prompt: BasePromptTemplate

Prompt object to use

requiresInput

requiresInput: boolean = true

requiresReference

requiresReference: boolean = false

`Optional`skipInputWarning

skipInputWarning?: string = ...

skipReferenceWarning

skipReferenceWarning: string = ...

Accessors

inputKeys

get inputKeys(): any
Returns any
Inherited from QAEvalChain.inputKeys
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/chains/llm_chain.ts:115

lc_namespace

get lc_namespace(): string[]
Returns string[]
Inherited from QAEvalChain.lc_namespace
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/chains/base.ts:41

outputKeys

get outputKeys(): string[]
Returns string[]
Inherited from QAEvalChain.outputKeys
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/chains/llm_chain.ts:119

Methods

_chainType

_chainType(): "llm"
Return the string type key uniquely identifying this class of chain.

Returns "llm"
Inherited from LLMStringEvaluator._chainType
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/chains/llm_chain.ts:254

_evaluateStrings

_evaluateStrings(args: StringEvaluatorArgs, config?: any): Promise<ChainValues>
Evaluate Chain or LLM output, based on optional input and label.
Parameters
- args: StringEvaluatorArgs
- Optionalconfig: any
Returns Promise<ChainValues>
The evaluation results containing the score or value. It is recommended that the dictionary contain the following keys:
- score: the score of the evaluation, if applicable.
- value: the string value of the evaluation, if applicable.
- reasoning: the reasoning for the evaluation, if applicable.
Overrides LLMStringEvaluator._evaluateStrings
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/evaluation/criteria/criteria.ts:272

`Protected`_formatValues

_formatValues(values: any): Promise<any>
Parameters
- values: any
Returns Promise<any>
Inherited from LLMStringEvaluator._formatValues
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/chains/base.ts:217

_getNumTokens

_getNumTokens(text: string): Promise<number>
Parameters
- text: string
Returns Promise<number>
Inherited from LLMStringEvaluator._getNumTokens
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/chains/llm_chain.ts:284

_prepareOutput

_prepareOutput(result: ChainValues): ChainValues
Prepare the output of the evaluation.
Parameters
- result: ChainValues
Returns ChainValues
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/evaluation/criteria/criteria.ts:264

apply

apply(inputs: ChainValues[], config?: any[]): Promise<ChainValues[]>
Parameters
- inputs: ChainValues[]
- Optionalconfig: any[]
Returns Promise<ChainValues[]>
Deprecated
Use .batch() instead. Will be removed in 0.2.0.

Call the chain on all inputs in the list
Inherited from LLMStringEvaluator.apply
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/chains/base.ts:258

call

call(values: any, config?: any): Promise<ChainValues>
Run the core logic of this chain and add to output if desired.

Wraps _call and handles memory.
Parameters
- values: any
- Optionalconfig: any
Returns Promise<ChainValues>
Inherited from LLMStringEvaluator.call
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/chains/llm_chain.ts:180

checkEvaluationArgs

checkEvaluationArgs(reference?: string, input?: string): void
Check if the evaluation arguments are valid.
Parameters
- Optionalreference: string
  The reference label.
- Optionalinput: string
  The input string.
Returns void
Throws
If the evaluator requires an input string but none is provided, or if the evaluator requires a reference label but none is provided.
Inherited from LLMStringEvaluator.checkEvaluationArgs
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/evaluation/base.ts:62

evaluateStrings

evaluateStrings(args: StringEvaluatorArgs, config?: any): Promise<ChainValues>
Evaluate Chain or LLM output, based on optional input and label.
Parameters
- args: StringEvaluatorArgs
- Optionalconfig: any
Returns Promise<ChainValues>
The evaluation results containing the score or value. It is recommended that the dictionary contain the following keys:
- score: the score of the evaluation, if applicable.
- value: the string value of the evaluation, if applicable.
- reasoning: the reasoning for the evaluation, if applicable.
Inherited from LLMStringEvaluator.evaluateStrings
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/evaluation/base.ts:196

getEvalInput

getEvalInput(__namedParameters: StringEvaluatorArgs): CriteriaEvalInput
Parameters
- __namedParameters: StringEvaluatorArgs
Returns CriteriaEvalInput
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/evaluation/criteria/criteria.ts:245

invoke

invoke(input: ChainValues, options?: any): Promise<ChainValues>
Invoke the chain with the provided input and returns the output.
Parameters
- input: ChainValues
  Input values for the chain run.
- Optionaloptions: any
Returns Promise<ChainValues>
Promise that resolves with the output of the chain run.
Inherited from LLMStringEvaluator.invoke
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/chains/base.ts:86

predict

predict(values: any, callbackManager?: any): Promise<EvalOutputType>
Format prompt with values and pass to LLM
Parameters
- values: any
  keys to pass to prompt template
- OptionalcallbackManager: any
  CallbackManager to use
Returns Promise<EvalOutputType>
Completion from LLM.
Example
```
llm.predict({ adjective: "funny" })
```
Inherited from LLMStringEvaluator.predict
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/chains/llm_chain.ts:246

prepOutputs

prepOutputs(
    inputs: Record<string, unknown>,
    outputs: Record<string, unknown>,
    returnOnlyOutputs?: boolean,
): Promise<Record<string, unknown>>
Parameters
- inputs: Record<string, unknown>
- outputs: Record<string, unknown>
- returnOnlyOutputs: boolean = false
Returns Promise<Record<string, unknown>>
Inherited from LLMStringEvaluator.prepOutputs
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/chains/base.ts:149

run

run(input: any, config?: any): Promise<string>
Parameters
- input: any
- Optionalconfig: any
Returns Promise<string>
Deprecated
Use .invoke() instead. Will be removed in 0.2.0.
Inherited from LLMStringEvaluator.run
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/chains/base.ts:190

serialize

serialize(): SerializedLLMChain
Returns SerializedLLMChain
Deprecated
Inherited from LLMStringEvaluator.serialize
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/chains/llm_chain.ts:274

`Static`deserialize

deserialize(data: SerializedLLMChain): Promise<LLMChain<string, any>>
Load a chain from a json-like object describing it.
Parameters
- data: SerializedLLMChain
Returns Promise<LLMChain<string, any>>
Inherited from LLMStringEvaluator.deserialize
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/chains/llm_chain.ts:258

`Static`fromLLM

fromLLM(
    llm: BaseLanguageModelInterface,
    criteria?: CriteriaLike,
    chainOptions?: Partial<
        Omit<
            LLMEvalChainInput<EvalOutputType, BaseLanguageModelInterface>,
            "llm",
        >,
    >,
): Promise<CriteriaEvalChain>
Create a new instance of the CriteriaEvalChain.
Parameters
- llm: BaseLanguageModelInterface
- Optionalcriteria: CriteriaLike
- OptionalchainOptions: Partial<
      Omit<
          LLMEvalChainInput<EvalOutputType, BaseLanguageModelInterface>,
          "llm",
      >,
  >
  Options to pass to the constructor of the LLMChain.
Returns Promise<CriteriaEvalChain>
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/evaluation/criteria/criteria.ts:209

`Static`lc_name

lc_name(): string
Returns string
Overrides LLMStringEvaluator.lc_name
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/evaluation/criteria/criteria.ts:123

`Static`resolveCriteria

resolveCriteria(criteria?: CriteriaLike): Record<string, string>
Resolve the criteria to evaluate.
Parameters
- Optionalcriteria: CriteriaLike
  The criteria to evaluate the runs against. It can be: - a mapping of a criterion name to its description - a single criterion name present in one of the default criteria - a single ConstitutionalPrinciple instance
Returns Record<string, string>
A dictionary mapping criterion names to descriptions.
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/evaluation/criteria/criteria.ts:150

`Static`resolvePrompt

resolvePrompt(prompt?: any): any
Resolve the prompt to use for the evaluation.
Parameters
- Optionalprompt: any
Returns any
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/evaluation/criteria/criteria.ts:183

Class CriteriaEvalChain

Hierarchy (View Summary)

Index

Constructors

Properties

Accessors

Methods

Constructors

constructor

Parameters

Returns CriteriaEvalChain

Properties

OptionalcriterionName

OptionalevaluationName

lc_serializable

llm

OptionalllmKwargs

Optionalmemory

outputKey

outputParser

prompt

requiresInput

requiresReference

OptionalskipInputWarning

skipReferenceWarning

Accessors

inputKeys

Returns any

lc_namespace

Returns string[]

outputKeys

Returns string[]

Methods

_chainType

Returns "llm"

_evaluateStrings

Parameters

Returns Promise<ChainValues>

Protected_formatValues

Parameters

Returns Promise<any>

_getNumTokens

Parameters

Returns Promise<number>

_prepareOutput

Parameters

Returns ChainValues

apply

Parameters

Returns Promise<ChainValues[]>

Deprecated

call

Parameters

Returns Promise<ChainValues>

checkEvaluationArgs

Parameters

Returns void

Throws

evaluateStrings

Parameters

Returns Promise<ChainValues>

getEvalInput

Parameters

Returns CriteriaEvalInput

invoke

Parameters

Returns Promise<ChainValues>

predict

Parameters

Returns Promise<EvalOutputType>

Example

prepOutputs

Parameters

Returns Promise<Record<string, unknown>>

run

Parameters

Returns Promise<string>

Deprecated

serialize

Returns SerializedLLMChain

`Optional`criterionName

`Optional`evaluationName

`Optional`llmKwargs

`Optional`memory

`Optional`skipInputWarning

`Protected`_formatValues

`Static`deserialize

`Static`fromLLM

`Static`lc_name

`Static`resolveCriteria

`Static`resolvePrompt