Class LabeledCriteriaEvalChain

OptionalcriterionName

OptionalevaluationName

lc_serializable

llm

OptionalllmKwargs

Optionalmemory

outputKey

outputParser

prompt

requiresInput

OptionalskipInputWarning

skipReferenceWarning

inputKeys

• get inputKeys(): any

  Returns any

lc_namespace

• get lc_namespace(): string[]

  Returns string[]

outputKeys

• get outputKeys(): string[]

  Returns string[]

_chainType

• _chainType(): "llm"

  Return the string type key uniquely identifying this class of chain.

  Returns "llm"

_evaluateStrings

• _evaluateStrings(args: StringEvaluatorArgs, config?: any): Promise<ChainValues>

  Evaluate Chain or LLM output, based on optional input and label.

  Parameters

  • args: StringEvaluatorArgs
  • Optionalconfig: any

  Returns Promise<ChainValues>

  The evaluation results containing the score or value. It is recommended that the dictionary contain the following keys:

  • score: the score of the evaluation, if applicable.
  • value: the string value of the evaluation, if applicable.
  • reasoning: the reasoning for the evaluation, if applicable.

_getNumTokens

• _getNumTokens(text: string): Promise<number>

  Parameters

  • text: string

  Returns Promise<number>

_prepareOutput

• _prepareOutput(result: ChainValues): ChainValues

  Prepare the output of the evaluation.

  Parameters

  • result: ChainValues

  Returns ChainValues

apply

• apply(inputs: ChainValues[], config?: any[]): Promise<ChainValues[]>

  Parameters

  • inputs: ChainValues[]
  • Optionalconfig: any[]

  Returns Promise<ChainValues[]>

  Deprecated

  Use .batch() instead. Will be removed in 0.2.0.

  Call the chain on all inputs in the list

call

• call(values: any, config?: any): Promise<ChainValues>

  Run the core logic of this chain and add to output if desired.

  Wraps _call and handles memory.

  Parameters

  • values: any
  • Optionalconfig: any

  Returns Promise<ChainValues>

checkEvaluationArgs

• checkEvaluationArgs(reference?: string, input?: string): void

  Check if the evaluation arguments are valid.

  Parameters

  • Optionalreference: string

    The reference label.

  • Optionalinput: string

    The input string.

  Returns void

  Throws

  If the evaluator requires an input string but none is provided, or if the evaluator requires a reference label but none is provided.

evaluateStrings

• evaluateStrings(args: StringEvaluatorArgs, config?: any): Promise<ChainValues>

  Evaluate Chain or LLM output, based on optional input and label.

  Parameters

  • args: StringEvaluatorArgs
  • Optionalconfig: any

  Returns Promise<ChainValues>

  The evaluation results containing the score or value. It is recommended that the dictionary contain the following keys:

  • score: the score of the evaluation, if applicable.
  • value: the string value of the evaluation, if applicable.
  • reasoning: the reasoning for the evaluation, if applicable.

getEvalInput

• getEvalInput(__namedParameters: StringEvaluatorArgs): CriteriaEvalInput

  Parameters

  • __namedParameters: StringEvaluatorArgs

  Returns CriteriaEvalInput

invoke

• invoke(input: ChainValues, options?: any): Promise<ChainValues>

  Invoke the chain with the provided input and returns the output.

  Parameters

  • input: ChainValues

    Input values for the chain run.

  • Optionaloptions: any

  Returns Promise<ChainValues>

  Promise that resolves with the output of the chain run.

predict

• predict(values: any, callbackManager?: any): Promise<EvalOutputType>

  Format prompt with values and pass to LLM

  Parameters

  • values: any

    keys to pass to prompt template

  • OptionalcallbackManager: any

    CallbackManager to use

  Returns Promise<EvalOutputType>

  Completion from LLM.

  Example

  llm.predict({ adjective: "funny" })
  Copy

prepOutputs

• prepOutputs(
      inputs: Record<string, unknown>,
      outputs: Record<string, unknown>,
      returnOnlyOutputs?: boolean,
  ): Promise<Record<string, unknown>>

  Parameters

  • inputs: Record<string, unknown>
  • outputs: Record<string, unknown>
  • returnOnlyOutputs: boolean = false

  Returns Promise<Record<string, unknown>>

run

• run(input: any, config?: any): Promise<string>

  Parameters

  • input: any
  • Optionalconfig: any

  Returns Promise<string>

  Deprecated

  Use .invoke() instead. Will be removed in 0.2.0.

serialize

• serialize(): SerializedLLMChain

  Returns SerializedLLMChain

  Deprecated

Staticdeserialize

• deserialize(data: SerializedLLMChain): Promise<LLMChain<string, any>>

  Load a chain from a json-like object describing it.

  Parameters

  • data: SerializedLLMChain

  Returns Promise<LLMChain<string, any>>

StaticfromLLM

• fromLLM(
      llm: BaseLanguageModelInterface,
      criteria?: CriteriaLike,
      chainOptions?: Partial<
          Omit<
              LLMEvalChainInput<EvalOutputType, BaseLanguageModelInterface>,
              "llm",
          >,
      >,
  ): Promise<CriteriaEvalChain>

  Create a new instance of the CriteriaEvalChain.

  Parameters

  • llm: BaseLanguageModelInterface
  • Optionalcriteria: CriteriaLike
  • OptionalchainOptions: Partial<
        Omit<
            LLMEvalChainInput<EvalOutputType, BaseLanguageModelInterface>,
            "llm",
        >,
    >

    Options to pass to the constructor of the LLMChain.

  Returns Promise<CriteriaEvalChain>

StaticresolveCriteria

• resolveCriteria(criteria?: CriteriaLike): Record<string, string>

  Resolve the criteria to evaluate.

  Parameters

  • Optionalcriteria: CriteriaLike

    The criteria to evaluate the runs against. It can be: - a mapping of a criterion name to its description - a single criterion name present in one of the default criteria - a single ConstitutionalPrinciple instance

  Returns Record<string, string>

  A dictionary mapping criterion names to descriptions.