langchain.js
    Preparing search index...

    Class ChatLlamaCpp

    To use this model you need to have the node-llama-cpp module installed. This can be installed using npm install -S node-llama-cpp and the minimum version supported in version 2.0.0. This also requires that have a locally built version of Llama3 installed.

    // Initialize the ChatLlamaCpp model with the path to the model binary file.
    const model = await ChatLlamaCpp.initialize({
      modelPath: "/Replace/with/path/to/your/model/gguf-llama3-Q4_0.bin",
      temperature: 0.5,
    });

    // Call the model with a message and await the response.
    const response = await model.invoke([
      new HumanMessage({ content: "My name is John." }),
    ]);

    // Log the response to the console.
    console.log({ response });

    Hierarchy (View Summary)

    Index

    Constructors

    constructor

    Properties

    _context _model _session lc_serializable maxTokens? temperature? topK? topP? trimWhitespaceSuffix? inputs

    Methods

    _buildPrompt _buildSession _convertMessagesToInteractions _llmType _streamResponseChunks invocationParams initialize lc_name

    Constructors

    • Parameters

      Returns ChatLlamaCpp

    Properties

    _context

    _context: LlamaContext

    _model

    _model: LlamaModel

    _session

    _session: null | LlamaChatSession
    lc_serializable: boolean = true
    maxTokens?: number
    temperature?: number
    topK?: number
    topP?: number
    trimWhitespaceSuffix?: boolean
    inputs: LlamaCppInputs

    Methods

    Protected_buildPrompt

    • _buildPrompt(input: BaseMessage[]): string

      Parameters

      • input: BaseMessage[]

      Returns string

    Protected_buildSession

    • _buildSession(messages: BaseMessage[]): string

      Parameters

      • messages: BaseMessage[]

      Returns string

    Protected_convertMessagesToInteractions

    • _convertMessagesToInteractions(messages: BaseMessage[]): ChatHistoryItem[]

      Parameters

      • messages: BaseMessage[]

      Returns ChatHistoryItem[]

    _llmType

    • _llmType(): string

      Returns string

    _streamResponseChunks

    • _streamResponseChunks(
          input: BaseMessage[],
          _options: unknown,
          runManager?: any,
      ): AsyncGenerator<ChatGenerationChunk>

      Parameters

      • input: BaseMessage[]
      • _options: unknown
      • OptionalrunManager: any

      Returns AsyncGenerator<ChatGenerationChunk>

    • Returns {
          maxTokens: undefined | number;
          temperature: undefined | number;
          topK: undefined | number;
          topP: undefined | number;
          trimWhitespaceSuffix: undefined | boolean;
      }

    • Initializes the llama_cpp model for usage in the chat models wrapper.

      Parameters

      • inputs: LlamaBaseCppInputs

        the inputs passed onto the model.

      Returns Promise<ChatLlamaCpp>

      A Promise that resolves to the ChatLlamaCpp type class.

    • Returns string

    Settings

    Member Visibility

    On This Page

    Constructors
    Properties
    Methods