langchain.js
    Preparing search index...

    Class HuggingFaceInference

    Class implementing the Large Language Model (LLM) interface using the Hugging Face Inference API for text generation.

    const model = new HuggingFaceInference({
      model: "gpt2",
      temperature: 0.7,
      maxTokens: 50,
    });

    const res = await model.invoke(
      "Question: What would be a good company name for a company that makes colorful socks?\nAnswer:"
    );
    console.log({ res });

    Hierarchy (View Summary)

    Implements

    Index

    Constructors

    constructor

    Properties

    apiKey endpointUrl frequencyPenalty includeCredentials lc_serializable maxTokens model stopSequences temperature topK topP

    Accessors

    lc_secrets

    Methods

    _llmType _streamResponseChunks invocationParams

    Constructors

    • Parameters

      • Optionalfields: any

      Returns HuggingFaceInference

    Properties

    apiKey: undefined | string = undefined

    API key to use.

    endpointUrl: undefined | string = undefined

    Custom inference endpoint URL to use

    frequencyPenalty: undefined | number = undefined

    Penalizes repeated tokens according to frequency

    includeCredentials: undefined | string | boolean = undefined

    Credentials to use for the request. If this is a string, it will be passed straight on. If it's a boolean, true will be "include" and false will not send credentials at all.

    lc_serializable: boolean = true
    maxTokens: undefined | number = undefined

    Maximum number of tokens to generate in the completion.

    model: string = "gpt2"

    Model to use

    stopSequences: undefined | string[] = undefined

    The model will stop generating text when one of the strings in the list is generated.

    temperature: undefined | number = undefined

    Sampling temperature to use

    topK: undefined | number = undefined

    Integer to define the top tokens considered within the sample operation to create new text.

    topP: undefined | number = undefined

    Total probability mass of tokens to consider at each step

    Accessors

    • get lc_secrets(): undefined | { [key: string]: string }

      Returns undefined | { [key: string]: string }

    Methods

    _llmType

    • _llmType(): string

      Returns string

    _streamResponseChunks

    • _streamResponseChunks(
          prompt: string,
          options: unknown,
          runManager?: any,
      ): AsyncGenerator<GenerationChunk>

      Parameters

      • prompt: string
      • options: unknown
      • OptionalrunManager: any

      Returns AsyncGenerator<GenerationChunk>

    • Parameters

      • Optionaloptions: unknown

      Returns {
          model: string;
          parameters: {
              max_new_tokens: undefined | number;
              repetition_penalty: undefined | number;
              return_full_text: boolean;
              stop: any;
              temperature: undefined | number;
              top_k: undefined | number;
              top_p: undefined | number;
          };
      }

    Settings

    Member Visibility

    On This Page

    Constructors
    Properties
    Accessors
    Methods