indexing

Index data from the doc source into the vector store.

Indexing functionality uses a manager to keep track of which documents are in the vector store.

This allows us to keep track of which documents were updated, and which documents were deleted, which documents should be skipped.

For the time being, documents are indexed using their hashes, and users are not able to specify the uid of the document.

HashedDocument is a Document with hashes calculated. Hashes are calculated based on page content and metadata. It is used for indexing.