Trimming options.
An array of trimmed BaseMessages or a Runnable that takes a sequence of BaseMessage-like objects and returns
an array of trimmed BaseMessages.
import { trimMessages, AIMessage, BaseMessage, HumanMessage, SystemMessage } from "@langchain/core/messages";
const messages = [
new SystemMessage("This is a 4 token text. The full message is 10 tokens."),
new HumanMessage({
content: "This is a 4 token text. The full message is 10 tokens.",
id: "first",
}),
new AIMessage({
content: [
{ type: "text", text: "This is the FIRST 4 token block." },
{ type: "text", text: "This is the SECOND 4 token block." },
],
id: "second",
}),
new HumanMessage({
content: "This is a 4 token text. The full message is 10 tokens.",
id: "third",
}),
new AIMessage({
content: "This is a 4 token text. The full message is 10 tokens.",
id: "fourth",
}),
];
function dummyTokenCounter(messages: BaseMessage[]): number {
// treat each message like it adds 3 default tokens at the beginning
// of the message and at the end of the message. 3 + 4 + 3 = 10 tokens
// per message.
const defaultContentLen = 4;
const defaultMsgPrefixLen = 3;
const defaultMsgSuffixLen = 3;
let count = 0;
for (const msg of messages) {
if (typeof msg.content === "string") {
count += defaultMsgPrefixLen + defaultContentLen + defaultMsgSuffixLen;
}
if (Array.isArray(msg.content)) {
count +=
defaultMsgPrefixLen +
msg.content.length * defaultContentLen +
defaultMsgSuffixLen;
}
}
return count;
}
First 30 tokens, not allowing partial messages:
await trimMessages(messages, {
maxTokens: 30,
tokenCounter: dummyTokenCounter,
strategy: "first",
});
Output:
[
new SystemMessage(
"This is a 4 token text. The full message is 10 tokens."
),
new HumanMessage({
content: "This is a 4 token text. The full message is 10 tokens.",
id: "first",
}),
]
First 30 tokens, allowing partial messages:
await trimMessages(messages, {
maxTokens: 30,
tokenCounter: dummyTokenCounter,
strategy: "first",
allowPartial: true,
});
Output:
[
new SystemMessage(
"This is a 4 token text. The full message is 10 tokens."
),
new HumanMessage({
content: "This is a 4 token text. The full message is 10 tokens.",
id: "first",
}),
new AIMessage({
content: [{ type: "text", text: "This is the FIRST 4 token block." }],
id: "second",
}),
]
First 30 tokens, allowing partial messages, have to end on HumanMessage:
await trimMessages(messages, {
maxTokens: 30,
tokenCounter: dummyTokenCounter,
strategy: "first",
allowPartial: true,
endOn: "human",
});
Output:
[
new SystemMessage(
"This is a 4 token text. The full message is 10 tokens."
),
new HumanMessage({
content: "This is a 4 token text. The full message is 10 tokens.",
id: "first",
}),
]
Last 30 tokens, including system message, not allowing partial messages:
await trimMessages(messages, {
maxTokens: 30,
includeSystem: true,
tokenCounter: dummyTokenCounter,
strategy: "last",
});
Output:
[
new SystemMessage(
"This is a 4 token text. The full message is 10 tokens."
),
new HumanMessage({
content: "This is a 4 token text. The full message is 10 tokens.",
id: "third",
}),
new AIMessage({
content: "This is a 4 token text. The full message is 10 tokens.",
id: "fourth",
}),
]
Last 40 tokens, including system message, allowing partial messages:
await trimMessages(messages, {
maxTokens: 40,
tokenCounter: dummyTokenCounter,
strategy: "last",
allowPartial: true,
includeSystem: true,
});
Output:
[
new SystemMessage(
"This is a 4 token text. The full message is 10 tokens."
),
new AIMessage({
content: [{ type: "text", text: "This is the FIRST 4 token block." }],
id: "second",
}),
new HumanMessage({
content: "This is a 4 token text. The full message is 10 tokens.",
id: "third",
}),
new AIMessage({
content: "This is a 4 token text. The full message is 10 tokens.",
id: "fourth",
}),
]
Last 30 tokens, including system message, allowing partial messages, end on HumanMessage:
await trimMessages(messages, {
maxTokens: 30,
tokenCounter: dummyTokenCounter,
strategy: "last",
endOn: "human",
includeSystem: true,
allowPartial: true,
});
Output:
[
new SystemMessage(
"This is a 4 token text. The full message is 10 tokens."
),
new AIMessage({
content: [{ type: "text", text: "This is the FIRST 4 token block." }],
id: "second",
}),
new HumanMessage({
content: "This is a 4 token text. The full message is 10 tokens.",
id: "third",
}),
]
Last 40 tokens, including system message, allowing partial messages, start on HumanMessage:
await trimMessages(messages, {
maxTokens: 40,
tokenCounter: dummyTokenCounter,
strategy: "last",
includeSystem: true,
allowPartial: true,
startOn: "human",
});
Output:
[
new SystemMessage(
"This is a 4 token text. The full message is 10 tokens."
),
new HumanMessage({
content: "This is a 4 token text. The full message is 10 tokens.",
id: "third",
}),
new AIMessage({
content: "This is a 4 token text. The full message is 10 tokens.",
id: "fourth",
}),
]
Trim messages to be below a token count.
Array of BaseMessage instances to trim.
Trimming options.
An array of trimmed BaseMessages or a Runnable that takes a sequence of BaseMessage-like objects and returns
an array of trimmed BaseMessages.
import { trimMessages, AIMessage, BaseMessage, HumanMessage, SystemMessage } from "@langchain/core/messages";
const messages = [
new SystemMessage("This is a 4 token text. The full message is 10 tokens."),
new HumanMessage({
content: "This is a 4 token text. The full message is 10 tokens.",
id: "first",
}),
new AIMessage({
content: [
{ type: "text", text: "This is the FIRST 4 token block." },
{ type: "text", text: "This is the SECOND 4 token block." },
],
id: "second",
}),
new HumanMessage({
content: "This is a 4 token text. The full message is 10 tokens.",
id: "third",
}),
new AIMessage({
content: "This is a 4 token text. The full message is 10 tokens.",
id: "fourth",
}),
];
function dummyTokenCounter(messages: BaseMessage[]): number {
// treat each message like it adds 3 default tokens at the beginning
// of the message and at the end of the message. 3 + 4 + 3 = 10 tokens
// per message.
const defaultContentLen = 4;
const defaultMsgPrefixLen = 3;
const defaultMsgSuffixLen = 3;
let count = 0;
for (const msg of messages) {
if (typeof msg.content === "string") {
count += defaultMsgPrefixLen + defaultContentLen + defaultMsgSuffixLen;
}
if (Array.isArray(msg.content)) {
count +=
defaultMsgPrefixLen +
msg.content.length * defaultContentLen +
defaultMsgSuffixLen;
}
}
return count;
}
First 30 tokens, not allowing partial messages:
await trimMessages(messages, {
maxTokens: 30,
tokenCounter: dummyTokenCounter,
strategy: "first",
});
Output:
[
new SystemMessage(
"This is a 4 token text. The full message is 10 tokens."
),
new HumanMessage({
content: "This is a 4 token text. The full message is 10 tokens.",
id: "first",
}),
]
First 30 tokens, allowing partial messages:
await trimMessages(messages, {
maxTokens: 30,
tokenCounter: dummyTokenCounter,
strategy: "first",
allowPartial: true,
});
Output:
[
new SystemMessage(
"This is a 4 token text. The full message is 10 tokens."
),
new HumanMessage({
content: "This is a 4 token text. The full message is 10 tokens.",
id: "first",
}),
new AIMessage({
content: [{ type: "text", text: "This is the FIRST 4 token block." }],
id: "second",
}),
]
First 30 tokens, allowing partial messages, have to end on HumanMessage:
await trimMessages(messages, {
maxTokens: 30,
tokenCounter: dummyTokenCounter,
strategy: "first",
allowPartial: true,
endOn: "human",
});
Output:
[
new SystemMessage(
"This is a 4 token text. The full message is 10 tokens."
),
new HumanMessage({
content: "This is a 4 token text. The full message is 10 tokens.",
id: "first",
}),
]
Last 30 tokens, including system message, not allowing partial messages:
await trimMessages(messages, {
maxTokens: 30,
includeSystem: true,
tokenCounter: dummyTokenCounter,
strategy: "last",
});
Output:
[
new SystemMessage(
"This is a 4 token text. The full message is 10 tokens."
),
new HumanMessage({
content: "This is a 4 token text. The full message is 10 tokens.",
id: "third",
}),
new AIMessage({
content: "This is a 4 token text. The full message is 10 tokens.",
id: "fourth",
}),
]
Last 40 tokens, including system message, allowing partial messages:
await trimMessages(messages, {
maxTokens: 40,
tokenCounter: dummyTokenCounter,
strategy: "last",
allowPartial: true,
includeSystem: true,
});
Output:
[
new SystemMessage(
"This is a 4 token text. The full message is 10 tokens."
),
new AIMessage({
content: [{ type: "text", text: "This is the FIRST 4 token block." }],
id: "second",
}),
new HumanMessage({
content: "This is a 4 token text. The full message is 10 tokens.",
id: "third",
}),
new AIMessage({
content: "This is a 4 token text. The full message is 10 tokens.",
id: "fourth",
}),
]
Last 30 tokens, including system message, allowing partial messages, end on HumanMessage:
await trimMessages(messages, {
maxTokens: 30,
tokenCounter: dummyTokenCounter,
strategy: "last",
endOn: "human",
includeSystem: true,
allowPartial: true,
});
Output:
[
new SystemMessage(
"This is a 4 token text. The full message is 10 tokens."
),
new AIMessage({
content: [{ type: "text", text: "This is the FIRST 4 token block." }],
id: "second",
}),
new HumanMessage({
content: "This is a 4 token text. The full message is 10 tokens.",
id: "third",
}),
]
Last 40 tokens, including system message, allowing partial messages, start on HumanMessage:
await trimMessages(messages, {
maxTokens: 40,
tokenCounter: dummyTokenCounter,
strategy: "last",
includeSystem: true,
allowPartial: true,
startOn: "human",
});
Output:
[
new SystemMessage(
"This is a 4 token text. The full message is 10 tokens."
),
new HumanMessage({
content: "This is a 4 token text. The full message is 10 tokens.",
id: "third",
}),
new AIMessage({
content: "This is a 4 token text. The full message is 10 tokens.",
id: "fourth",
}),
]
Trim messages to be below a token count.