Class LabeledPairwiseStringEvalChain

A chain for comparing two outputs, such as the outputs of two models, prompts, or outputs of a single model on similar inputs, with labeled preferences.

Hierarchy (View Summary)

PairwiseStringEvalChain
- LabeledPairwiseStringEvalChain

Constructors

constructor

new LabeledPairwiseStringEvalChain(
fields: LLMChainInput<EvalOutputType, BaseLanguageModelInterface>,
): LabeledPairwiseStringEvalChain
Parameters
- fields: LLMChainInput<EvalOutputType, BaseLanguageModelInterface>
Returns LabeledPairwiseStringEvalChain
Inherited from PairwiseStringEvalChain.constructor
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/chains/llm_chain.ts:123

Properties

`Optional`criterionName

criterionName?: string

`Optional`evaluationName

evaluationName?: string = ...

The name of the evaluation.

lc_serializable

lc_serializable: boolean = true

llm

llm: BaseLanguageModelInterface

LLM Wrapper to use

`Optional`llmKwargs

llmKwargs?: any

Kwargs to pass to LLM

`Optional`memory

memory?: any

outputKey

outputKey: string = "text"

Key to use for output, defaults to text

outputParser

outputParser: PairwiseStringResultOutputParser = ...

prompt

prompt: BasePromptTemplate

Prompt object to use

requiresInput

requiresInput: boolean = true

requiresReference

requiresReference: boolean = true

`Optional`skipInputWarning

skipInputWarning?: string = ...

skipReferenceWarning

skipReferenceWarning: string = ...

Accessors

inputKeys

get inputKeys(): any
Returns any
Inherited from QAEvalChain.inputKeys
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/chains/llm_chain.ts:115

lc_namespace

get lc_namespace(): string[]
Returns string[]
Inherited from QAEvalChain.lc_namespace
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/chains/base.ts:41

outputKeys

get outputKeys(): string[]
Returns string[]
Inherited from QAEvalChain.outputKeys
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/chains/llm_chain.ts:119

Methods

_chainType

_chainType(): "llm"
Return the string type key uniquely identifying this class of chain.

Returns "llm"
Inherited from PairwiseStringEvalChain._chainType
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/chains/llm_chain.ts:254

_evaluateStringPairs

_evaluateStringPairs(
    args: LLMPairwiseStringEvaluatorArgs,
    callOptions: unknown,
    config?: any,
): Promise<ChainValues>
Evaluate the output string pairs.
Parameters
- args: LLMPairwiseStringEvaluatorArgs
- callOptions: unknown
- Optionalconfig: any
Returns Promise<ChainValues>
A dictionary containing the preference, scores, and/or other information.
Inherited from PairwiseStringEvalChain._evaluateStringPairs
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/evaluation/comparison/pairwise.ts:219

`Protected`_formatValues

_formatValues(values: any): Promise<any>
Parameters
- values: any
Returns Promise<any>
Inherited from PairwiseStringEvalChain._formatValues
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/chains/base.ts:217

_getNumTokens

_getNumTokens(text: string): Promise<number>
Parameters
- text: string
Returns Promise<number>
Inherited from PairwiseStringEvalChain._getNumTokens
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/chains/llm_chain.ts:284

_prepareOutput

_prepareOutput(result: ChainValues): ChainValues
Parameters
- result: ChainValues
Returns ChainValues
Inherited from PairwiseStringEvalChain._prepareOutput
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/evaluation/comparison/pairwise.ts:211

apply

apply(inputs: ChainValues[], config?: any[]): Promise<ChainValues[]>
Parameters
- inputs: ChainValues[]
- Optionalconfig: any[]
Returns Promise<ChainValues[]>
Deprecated
Use .batch() instead. Will be removed in 0.2.0.

Call the chain on all inputs in the list
Inherited from PairwiseStringEvalChain.apply
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/chains/base.ts:258

call

call(values: any, config?: any): Promise<ChainValues>
Run the core logic of this chain and add to output if desired.

Wraps _call and handles memory.
Parameters
- values: any
- Optionalconfig: any
Returns Promise<ChainValues>
Inherited from PairwiseStringEvalChain.call
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/chains/llm_chain.ts:180

checkEvaluationArgs

checkEvaluationArgs(reference?: string, input?: string): void
Check if the evaluation arguments are valid.
Parameters
- Optionalreference: string
  The reference label.
- Optionalinput: string
  The input string.
Returns void
Throws
If the evaluator requires an input string but none is provided, or if the evaluator requires a reference label but none is provided.
Inherited from PairwiseStringEvalChain.checkEvaluationArgs
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/evaluation/base.ts:62

evaluateStringPairs

evaluateStringPairs(
    args: LLMPairwiseStringEvaluatorArgs,
    callOptions?: unknown,
    config?: any,
): Promise<ChainValues>
Evaluate the output string pairs.
Parameters
- args: LLMPairwiseStringEvaluatorArgs
- OptionalcallOptions: unknown
- Optionalconfig: any
Returns Promise<ChainValues>
A dictionary containing the preference, scores, and/or other information.
Inherited from PairwiseStringEvalChain.evaluateStringPairs
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/evaluation/base.ts:310

invoke

invoke(input: ChainValues, options?: any): Promise<ChainValues>
Invoke the chain with the provided input and returns the output.
Parameters
- input: ChainValues
  Input values for the chain run.
- Optionaloptions: any
Returns Promise<ChainValues>
Promise that resolves with the output of the chain run.
Inherited from PairwiseStringEvalChain.invoke
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/chains/base.ts:86

predict

predict(values: any, callbackManager?: any): Promise<EvalOutputType>
Format prompt with values and pass to LLM
Parameters
- values: any
  keys to pass to prompt template
- OptionalcallbackManager: any
  CallbackManager to use
Returns Promise<EvalOutputType>
Completion from LLM.
Example
```
llm.predict({ adjective: "funny" })
```
Inherited from PairwiseStringEvalChain.predict
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/chains/llm_chain.ts:246

prepOutputs

prepOutputs(
    inputs: Record<string, unknown>,
    outputs: Record<string, unknown>,
    returnOnlyOutputs?: boolean,
): Promise<Record<string, unknown>>
Parameters
- inputs: Record<string, unknown>
- outputs: Record<string, unknown>
- returnOnlyOutputs: boolean = false
Returns Promise<Record<string, unknown>>
Inherited from PairwiseStringEvalChain.prepOutputs
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/chains/base.ts:149

run

run(input: any, config?: any): Promise<string>
Parameters
- input: any
- Optionalconfig: any
Returns Promise<string>
Deprecated
Use .invoke() instead. Will be removed in 0.2.0.
Inherited from PairwiseStringEvalChain.run
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/chains/base.ts:190

serialize

serialize(): SerializedLLMChain
Returns SerializedLLMChain
Deprecated
Inherited from PairwiseStringEvalChain.serialize
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/chains/llm_chain.ts:274

`Static`deserialize

deserialize(data: SerializedLLMChain): Promise<LLMChain<string, any>>
Load a chain from a json-like object describing it.
Parameters
- data: SerializedLLMChain
Returns Promise<LLMChain<string, any>>
Inherited from PairwiseStringEvalChain.deserialize
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/chains/llm_chain.ts:258

`Static`fromLLM

fromLLM(
    llm: BaseLanguageModelInterface,
    criteria?: CriteriaLike,
    chainOptions?: Partial<
        Omit<
            LLMEvalChainInput<EvalOutputType, BaseLanguageModelInterface>,
            "llm",
        >,
    >,
): Promise<PairwiseStringEvalChain>
Create a new instance of the PairwiseStringEvalChain.
Parameters
- llm: BaseLanguageModelInterface
- Optionalcriteria: CriteriaLike
  The criteria to use for evaluation.
- OptionalchainOptions: Partial<
      Omit<
          LLMEvalChainInput<EvalOutputType, BaseLanguageModelInterface>,
          "llm",
      >,
  >
  Options to pass to the chain.
Returns Promise<PairwiseStringEvalChain>
Inherited from PairwiseStringEvalChain.fromLLM
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/evaluation/comparison/pairwise.ts:185

`Static`lc_name

lc_name(): string
Returns string
Overrides PairwiseStringEvalChain.lc_name
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/evaluation/comparison/pairwise.ts:236

`Static`resolvePairwiseCriteria

resolvePairwiseCriteria(criteria?: CriteriaLike): Record<string, string>
Parameters
- Optionalcriteria: CriteriaLike
Returns Record<string, string>
Inherited from PairwiseStringEvalChain.resolvePairwiseCriteria
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/evaluation/comparison/pairwise.ts:116

`Static`resolvePairwisePrompt

resolvePairwisePrompt(prompt?: any): any
Parameters
- Optionalprompt: any
Returns any
Overrides PairwiseStringEvalChain.resolvePairwisePrompt
- Defined in remotes/langchain-ai/langchainjs/v1/libs/langchain/src/evaluation/comparison/pairwise.ts:242

Class LabeledPairwiseStringEvalChain

Hierarchy (View Summary)

Index

Constructors

Properties

Accessors

Methods

Constructors

constructor

Parameters

Returns LabeledPairwiseStringEvalChain

Properties

OptionalcriterionName

OptionalevaluationName

lc_serializable

llm

OptionalllmKwargs

Optionalmemory

outputKey

outputParser

prompt

requiresInput

requiresReference

OptionalskipInputWarning

skipReferenceWarning

Accessors

inputKeys

Returns any

lc_namespace

Returns string[]

outputKeys

Returns string[]

Methods

_chainType

Returns "llm"

_evaluateStringPairs

Parameters

Returns Promise<ChainValues>

Protected_formatValues

Parameters

Returns Promise<any>

_getNumTokens

Parameters

Returns Promise<number>

_prepareOutput

Parameters

Returns ChainValues

apply

Parameters

Returns Promise<ChainValues[]>

Deprecated

call

Parameters

Returns Promise<ChainValues>

checkEvaluationArgs

Parameters

Returns void

Throws

evaluateStringPairs

Parameters

Returns Promise<ChainValues>

invoke

Parameters

Returns Promise<ChainValues>

predict

Parameters

Returns Promise<EvalOutputType>

Example

prepOutputs

Parameters

Returns Promise<Record<string, unknown>>

run

Parameters

Returns Promise<string>

Deprecated

serialize

Returns SerializedLLMChain

Deprecated

Staticdeserialize

Parameters

`Optional`criterionName

`Optional`evaluationName

`Optional`llmKwargs

`Optional`memory

`Optional`skipInputWarning

`Protected`_formatValues

`Static`deserialize

`Static`fromLLM

`Static`lc_name

`Static`resolvePairwiseCriteria

`Static`resolvePairwisePrompt