You cannot select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
langchain/langchain
Zander Chase b0859c9b18
Add New Retriever Interface with Callbacks (#5962)
Handle the new retriever events in a way that (I think) is entirely
backwards compatible? Needs more testing for some of the chain changes
and all.

This creates an entire new run type, however. We could also just treat
this as an event within a chain run presumably (same with memory)

Adds a subclass initializer that upgrades old retriever implementations
to the new schema, along with tests to ensure they work.

First commit doesn't upgrade any of our retriever implementations (to
show that we can pass the tests along with additional ones testing the
upgrade logic).

Second commit upgrades the known universe of retrievers in langchain.

- [X] Add callback handling methods for retriever start/end/error (open
to renaming to 'retrieval' if you want that)
- [X] Update BaseRetriever schema to support callbacks
- [X] Tests for upgrading old "v1" retrievers for backwards
compatibility
- [X] Update existing retriever implementations to implement the new
interface
- [X] Update calls within chains to .{a]get_relevant_documents to pass
the child callback manager
- [X] Update the notebooks/docs to reflect the new interface
- [X] Test notebooks thoroughly


Not handled:
- Memory pass throughs: retrieval memory doesn't have a parent callback
manager passed through the method

---------

Co-authored-by: Nuno Campos <nuno@boringbits.io>
Co-authored-by: William Fu-Hinthorn <13333726+hinthornw@users.noreply.github.com>
1 year ago
..
agents Add `OpenAIMultiFunctionsAgent` to import list in agents directory (#6824) 1 year ago
callbacks Add New Retriever Interface with Callbacks (#5962) 1 year ago
chains Add New Retriever Interface with Callbacks (#5962) 1 year ago
chat_models Tiktoken override (#6697) 1 year ago
client Add Input Mapper in run_on_dataset (#6894) 1 year ago
docstore Fix class promotion (#6187) 1 year ago
document_loaders Add password to PyPDR loader and parser (#6908) 1 year ago
embeddings move octo notebook (#6901) 1 year ago
evaluation Simplify eval arg names (#6944) 1 year ago
experimental Add New Retriever Interface with Callbacks (#5962) 1 year ago
graphs Add KuzuQAChain (#6454) 1 year ago
indexes Create async copy of from_text() inside GraphIndexCreator. (#5214) 1 year ago
llms Bagatur/openllm ensure available (#6960) 1 year ago
load added docstrings where they missed (#6626) 1 year ago
memory Zep Authentication (#6728) 1 year ago
output_parsers rail_parser: Allow creation from pydantic (#6832) 1 year ago
prompts added docstrings where they missed (#6626) 1 year ago
retrievers Add New Retriever Interface with Callbacks (#5962) 1 year ago
tools PowerBI Toolkit additional logs (#6881) 1 year ago
utilities Add `call_actor_task` to the Apify integration (#6862) 1 year ago
vectorstores Add New Retriever Interface with Callbacks (#5962) 1 year ago
__init__.py release v220 (#6962) 1 year ago
base_language.py nc/load (#5733) 1 year ago
cache.py Page per class-style api reference (#6560) 1 year ago
docker-compose.yaml Update docker-compose.yaml (#3582) 1 year ago
document_transformers.py added docstrings where they missed (#6626) 1 year ago
env.py Use LCP Client in Tracer (#5908) 1 year ago
example_generator.py Replace remaining usage of basellm with baselangmodel (#3981) 1 year ago
formatting.py Validate `input_variables` when using `jinja2` templates (#3140) 1 year ago
input.py FileCallbackHandler (#5589) 1 year ago
math_utils.py add get_top_k_cosine_similarity method to get max top k score and index (#5059) 1 year ago
model_laboratory.py Harrison/improve cache (#368) 2 years ago
py.typed Add py.typed marker to package (#121) 2 years ago
python.py Move PythonRepl -> langchain.utilities (#2917) 1 year ago
requests.py fixed aiohttp.client_exceptions.ClientConnectionError: Connection closed (#2718) 1 year ago
schema.py Add New Retriever Interface with Callbacks (#5962) 1 year ago
serpapi.py move serpapi wrapper (#1199) 2 years ago
server.py Fix for ModuleNotFoundError while running langchain-server. Issue #5833 (#6077) 1 year ago
sql_database.py updated sql_database.py for returning sorted table names. (#6692) 1 year ago
text_splitter.py Page per class-style api reference (#6560) 1 year ago
utils.py added docstrings where they missed (#6626) 1 year ago