Base class for retrievers integration tests.
RetrieversIntegrationTests()A BaseRetriever subclass to be tested.
Returns a dictionary of parameters to pass to the retriever constructor.
Returns a str representing the query of an example retriever call.
Returns the name of the parameter for the number of results returned.
Usually something like k or top_k.
Return retriever fixture.
Test the number of results constructor parameter.
Test that the retriever constructor accepts a parameter representing the number of documents to return.
By default, the parameter tested is named k, but it can be overridden by
setting the num_results_arg_name property.
If the retriever doesn't support configuring the number of results returned
via the constructor, this test can be skipped using a pytest xfail on
the test class:
@pytest.mark.xfail(
reason="This retriever doesn't support setting "
"the number of results via the constructor."
)
def test_k_constructor_param(self) -> None:
raise NotImplementedErrorIf this test fails, the retriever constructor does not accept a number
of results parameter, or the retriever does not return the correct number
of documents ( of the one set in num_results_arg_name) when it is
set.
For example, a retriever like...
MyRetriever(k=3).invoke("query")...should return 3 documents when invoked with a query.
Test the number of results parameter in invoke.
Test that the invoke method accepts a parameter representing the number of documents to return.
By default, the parameter is named, but it can be overridden by
setting the num_results_arg_name property.
If the retriever doesn't support configuring the number of results returned
via the invoke method, this test can be skipped using a pytest xfail on
the test class:
@pytest.mark.xfail(
reason="This retriever doesn't support setting "
"the number of results in the invoke method."
)
def test_invoke_with_k_kwarg(self) -> None:
raise NotImplementedErrorIf this test fails, the retriever's invoke method does not accept a number
of results parameter, or the retriever does not return the correct number
of documents (k of the one set in num_results_arg_name) when it is
set.
For example, a retriever like...
MyRetriever().invoke("query", k=3)...should return 3 documents when invoked with a query.
Test invoke returns documents.
If invoked with the example params, the retriever should return a list of Documents.
If this test fails, the retriever's invoke method does not return a list of
Document objects. Please confirm that your
_get_relevant_documents method returns a list of Document objects.
Test ainvoke returns documents.
If ainvoke'd with the example params, the retriever should return a list of
Document objects.
See test_invoke_returns_documents for more information on
troubleshooting.