class CheerioWebBaseLoaderA class that extends the BaseDocumentLoader and implements the DocumentLoader interface. It represents a document loader for loading web-based documents using Cheerio.
The async caller should be used by subclasses to make any async calls, which will thus benefit from the concurrency and retry logic.
Timeout for this call in milliseconds.
A method that loads the text file or blob and returns a promise that
resolves to an array of Document instances. It reads the text from
the file or blob using the readFile function from the
node:fs/promises module or the text() method of the blob. It then
parses the text using the parse() method and creates a Document
instance for each parsed page. The metadata includes the source of the
text (file path or blob) and, if there are multiple pages, the line
number of each page.
Fetches the web document from the webPath and loads it using Cheerio. It returns a CheerioAPI instance.
A static method that imports the readFile function from the
node:fs/promises module. It is used to dynamically import the
function when needed. If the import fails, it throws an error
indicating that the fs/promises module is not available in the
current environment.
Fetches web documents from the given array of URLs and loads them using Cheerio. It returns an array of CheerioAPI instances.
const loader = new CheerioWebBaseLoader("https://exampleurl.com");
const docs = await loader.load();
console.log({ docs });