langchain.js
    Preparing search index...

    Class LlamaCpp

    To use this model you need to have the node-llama-cpp module installed. This can be installed using npm install -S node-llama-cpp and the minimum version supported in version 2.0.0. This also requires that have a locally built version of Llama3 installed.

    Hierarchy (View Summary)

    Index

    Constructors

    constructor

    Properties

    _context _gbnf _jsonSchema _model _session lc_serializable maxTokens? temperature? topK? topP? trimWhitespaceSuffix? inputs

    Methods

    _llmType _streamResponseChunks initialize lc_name

    Constructors

    • Parameters

      Returns LlamaCpp

    Properties

    _context

    _context: LlamaContext

    _gbnf

    _gbnf: undefined | LlamaGrammar

    _jsonSchema

    _jsonSchema: undefined | LlamaJsonSchemaGrammar<GbnfJsonSchema>

    _model

    _model: LlamaModel

    _session

    _session: LlamaChatSession
    lc_serializable: boolean = true
    maxTokens?: number
    temperature?: number
    topK?: number
    topP?: number
    trimWhitespaceSuffix?: boolean
    inputs: LlamaCppInputs

    Methods

    _llmType

    • _llmType(): string

      Returns string

    _streamResponseChunks

    • _streamResponseChunks(
          prompt: string,
          _options: unknown,
          runManager?: any,
      ): AsyncGenerator<GenerationChunk>

      Parameters

      • prompt: string
      • _options: unknown
      • OptionalrunManager: any

      Returns AsyncGenerator<GenerationChunk>

    • Initializes the llama_cpp model for usage.

      Parameters

      Returns Promise<LlamaCpp>

      A Promise that resolves to the LlamaCpp type class.

    • Returns string

    Settings

    Member Visibility

    On This Page

    Constructors
    Properties
    Methods