class ApifyDatasetLoaderA class that extends the BaseDocumentLoader and implements the DocumentLoader interface. It represents a document loader that loads documents from an Apify dataset.
Retrieves the dataset items from the Apify platform and applies the datasetMappingFunction to each item to create an array of Document instances.
Create an ApifyDatasetLoader by calling an Actor on the Apify platform and waiting for its results to be ready.
Create an ApifyDatasetLoader by calling a saved Actor task on the Apify platform and waiting for its results to be ready.
const loader = new ApifyDatasetLoader("your-dataset-id", {
datasetMappingFunction: (item) =>
new Document({
pageContent: item.text || "",
metadata: { source: item.url },
}),
clientOptions: {
token: "your-apify-token",
},
});
const docs = await loader.load();
const chain = new RetrievalQAChain();
const res = await chain.invoke({ query: "What is LangChain?" });
console.log(res.text);
console.log(res.sourceDocuments.map((d) => d.metadata.source));