Inputs for ChatBedrockConverse.
interface ChatBedrockConverseInputBaseChatModelParamsPartial<DefaultProviderInit>Additional inference parameters that the model supports, beyond the
base set of inference parameters that the Converse API supports in the inferenceConfig
field. For more information, see the model parameters link below.
Application Inference Profile ARN to use for the model.
For example, "arn:aws:bedrock:eu-west-1:123456789102:application-inference-profile/fm16bt65tzgx", will override this.model in final /invoke URL call.
Must still provide model as normal modelId to benefit from all the metadata.
See the below link for more details on creating and using application inference profiles.
The BedrockRuntimeClient to use. It gives ability to override the default client with a custom one, allowing you to pass requestHandler {NodeHttpHandler} parameter in case it is not provided here.
Overrideable configuration options for the BedrockRuntimeClient. Allows customization of client configuration such as requestHandler, etc. Will be ignored if 'client' is provided.
AWS Credentials. If no credentials are provided, the default credentials from
@aws-sdk/credential-provider-node will be used.
Whether to disable streaming.
If streaming is bypassed, then stream() will defer to
invoke().
Override the default endpoint hostname.
Configuration information for a guardrail that you want to use in the request.
The maximum number of concurrent calls that can be made.
Defaults to Infinity, which means no limit.
The maximum number of retries that can be made for a single call, with an exponential backoff between each attempt. Defaults to 6.
Max tokens.
Model to use. For example, "anthropic.claude-3-haiku-20240307-v1:0", this is equivalent to the modelId property in the list-foundation-models api. See the below link for a full list of models.
Custom handler to handle failed attempts. Takes the originally thrown error object as input, and should itself throw an error if the input error is not retryable.
Version of AIMessage output format to store in message content.
AIMessage.contentBlocks will lazily parse the contents of content into a
standard format. This flag can be used to additionally store the standard format
as the message content, e.g., for serialization purposes.
.contentBlocks).contentBlocks)You can also set LC_OUTPUT_VERSION as an environment variable to "v1" to
enable this by default.
Model performance configuration. See https://docs.aws.amazon.com/bedrock/latest/userguide/latency-optimized-inference.html
The configuration profile to use.
The AWS region e.g. us-west-2.
Fallback to AWS_DEFAULT_REGION env variable or region specified in ~/.aws/config
in case it is not provided here.
Service tier for model invocation.
Specifies the processing tier type used for serving the request. Supported values are 'priority', 'default', 'flex', and 'reserved'.
If not provided, AWS uses the default tier.
For more information, see: https://docs.aws.amazon.com/bedrock/latest/userguide/service-tiers-inference.html
Whether or not to stream responses
Whether or not to include usage data, like token counts in the streamed response chunks. Passing as a call option will take precedence over the class-level setting.
Which types of tool_choice values the model supports.
Inferred if not specified. Inferred as ['auto', 'any', 'tool'] if a 'claude-3' model is used, ['auto', 'any'] if a 'mistral-large' model is used, empty otherwise.
Temperature.
Timeout for this call in milliseconds.
The percentage of most-likely candidates that the model considers for the next token. For
example, if you choose a value of 0.8 for topP, the model selects from the top 80% of the
probability distribution of tokens that could be next in the sequence.
The default value is the default value for the model that you are using.
For more information, see the inference parameters for foundation models link below.