16 Commits (21a353e9c25017a8a66661edb2ed98e9b152572b)

Author	SHA1	Message	Date
Nuno Campos	81e5b1ad36	Add serialized object to retriever start callback (#7074) ... <!-- Thank you for contributing to LangChain! Replace this comment with: - Description: a description of the change, - Issue: the issue # it fixes (if applicable), - Dependencies: any dependencies required for this change, - Tag maintainer: for a quicker response, tag the relevant maintainer (see below), - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out! If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. Maintainer responsibilities: - General / Misc / if you don't know who to tag: @dev2049 - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev - Models / Prompts: @hwchase17, @dev2049 - Memory: @hwchase17 - Agents / Tools / Toolkits: @vowelparrot - Tracing / Callbacks: @agola11 - Async: @agola11 If no one reviews your PR within a few days, feel free to @-mention the same people again. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md -->	1 year ago
Bagatur	7acd524210	Rm retriever kwargs (#7013) ... Doesn't actually limit the Retriever interface but hopefully in practice it does	1 year ago
Saba Sturua	427551eabf	DocArray as a Retriever (#6031) ... ## DocArray as a Retriever [DocArray](https://github.com/docarray/docarray) is an open-source tool for managing your multi-modal data. It offers flexibility to store and search through your data using various document index backends. This PR introduces `DocArrayRetriever` - which works with any available backend and serves as a retriever for Langchain apps. Also, I added 2 notebooks: DocArray Backends - intro to all 5 currently supported backends, how to initialize, index, and use them as a retriever DocArray Usage - showcasing what additional search parameters you can pass to create versatile retrievers Example: ```python from docarray.index import InMemoryExactNNIndex from docarray import BaseDoc, DocList from docarray.typing import NdArray from langchain.embeddings.openai import OpenAIEmbeddings from langchain.retrievers import DocArrayRetriever # define document schema class MyDoc(BaseDoc): description: str description_embedding: NdArray[1536] embeddings = OpenAIEmbeddings() # create documents descriptions = ["description 1", "description 2"] desc_embeddings = embeddings.embed_documents(texts=descriptions) docs = DocList[MyDoc]( [ MyDoc(description=desc, description_embedding=embedding) for desc, embedding in zip(descriptions, desc_embeddings) ] ) # initialize document index with data db = InMemoryExactNNIndex[MyDoc](docs) # create a retriever retriever = DocArrayRetriever( index=db, embeddings=embeddings, search_field="description_embedding", content_field="description", ) # find the relevant document doc = retriever.get_relevant_documents("action movies") print(doc) ``` #### Who can review? @dev2049 --------- Signed-off-by: jupyterjazz <saba.sturua@jina.ai>	1 year ago
German Martin	736a1819aa	LOTR: Lord of the Retrievers. A retriever that merge several retrievers together applying document_formatters to them. (#5798) ... "One Retriever to merge them all, One Retriever to expose them, One Retriever to bring them all and in and process them with Document formatters." Hi @dev2049! Here bothering people again! I'm using this simple idea to deal with merging the output of several retrievers into one. I'm aware of DocumentCompressorPipeline and ContextualCompressionRetriever but I don't think they allow us to do something like this. Also I was getting in trouble to get the pipeline working too. Please correct me if i'm wrong. This allow to do some sort of "retrieval" preprocessing and then using the retrieval with the curated results anywhere you could use a retriever. My use case is to generate diff indexes with diff embeddings and sources for a more colorful results then filtering them with one or many document formatters. I saw some people looking for something like this, here: https://github.com/hwchase17/langchain/issues/3991 and something similar here: https://github.com/hwchase17/langchain/issues/5555 This is just a proposal I know I'm missing tests , etc. If you think this is a worth it idea I can work on tests and anything you want to change. Let me know! --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	1 year ago
Harrison Chase	ad09367a92	Harrison/pubmed integration (#5664) ... Co-authored-by: younis basher <71520361+younis-ba@users.noreply.github.com> Co-authored-by: Younis Bashir <younis@omicmd.com>	1 year ago
Davis Chase	2b2176a3c1	tfidf retriever (#5114) ... Co-authored-by: vempaliakhil96 <vempaliakhil96@gmail.com>	1 year ago
yujiosaka	2f8eb95a91	Remove unnecessary comment (#4845) ... # Remove unnecessary comment Remove unnecessary comment accidentally included in #4800 ## Before submitting - no test - no document ## Who can review? Community members can review the PR once tests pass. Tag maintainers/contributors who might be interested:	1 year ago
yujiosaka	6561efebb7	Accept uuids kwargs for weaviate (#4800) ... # Accept uuids kwargs for weaviate Fixes #4791	1 year ago
Leonid Ganeline	e17d0319d5	Add `arxiv` retriever (#4538)	1 year ago
Davis Chase	9ec60ad832	Add azure cognitive search retriever (#4467) ... All credit to @UmerHA, made a couple small changes --------- Co-authored-by: UmerHA <40663591+UmerHA@users.noreply.github.com>	1 year ago
Leonid Ganeline	ce15ffae6a	added `Wikipedia` retriever (#4302) ... - added `Wikipedia` retriever. It is effectively a wrapper for `WikipediaAPIWrapper`. It wrapps load() into get_relevant_documents() - sorted `__all__` in the `retrievers/__init__` - added integration tests for the WikipediaRetriever - added an example (as Jupyter notebook) for the WikipediaRetriever	1 year ago
Zander Chase	84cfa76e00	Update Cohere Reranker (#4180) ... The forward ref annotations don't get updated if we only iimport with type checking --------- Co-authored-by: Abhinav Verma <abhinav_win12@yahoo.co.in>	1 year ago
Mike Wang	ce4fea983b	[simple] added test case and improve self class return type annotation (#3773) ... a simple follow up of https://github.com/hwchase17/langchain/pull/3748 - added test case - improve annotation when function return type is class itself.	1 year ago
Harrison Chase	7257f9e015	Harrison/tfidf parameters (#3481) ... Co-authored-by: pao <go5kuramubon@gmail.com> Co-authored-by: KyoHattori <kyo.hattori@abejainc.com>	1 year ago
Harrison Chase	408a0183cd	Harrison/weaviate (#3494) ... Co-authored-by: Nick Rubell <nick@rubell.com>	1 year ago
Davis Chase	46542dc774	Contextual compression retriever (#2915) ... Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	1 year ago