614 Commits (c24c871d8819b0814da0bb0fde93f38685218752)

Author	SHA1	Message	Date
Mohammad Mohtashim	43db4cd20e	core[major]: On Tool End Observation Casting Fix (#18798) ... This PR updates the on_tool_end handlers to return the raw output from the tool instead of casting it to a string. This is technically a breaking change, though it's impact is expected to be somewhat minimal. It will fix behavior in `astream_events` as well. Fixes the following issue #18760 raised by @eyurtsev --------- Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com>	3 months ago
Massimiliano Pronesti	8113d612bb	community[patch]: support modin document loader (#18866) ... Langchain community document loaders support `pyspark`, `polars`, and `pandas` dataframes but not `modin`'s. This PR addresses this point.	3 months ago
Pol Ruiz Farre	a7f63d8cb4	community[patch]: Fix BasePDFLoader suffix for s3 presigned urls (#18844) ... BasePDFLoader doesn't parse the suffix of the file correctly when parsing S3 presigned urls. This fix enables the proper detection and parsing of S3 presigned URLs to prevent errors such as `OSError: [Errno 36] File name too long`. No additional dependencies required.	3 months ago
Joshua Carroll	ddaf9de169	community: Fix bug with StreamlitChatMessageHistory (#18834) ... - **Description:** Fix Streamlit bug which was introduced by https://github.com/langchain-ai/langchain/pull/18250, update integration test - **Issue:** https://github.com/langchain-ai/langchain/issues/18684 - **Dependencies:** None	3 months ago
Tomaz Bratanic	a28be31a96	Switch to md5 for deduplication in neo4j integrations (#18846) ... Deduplicate documents using MD5 of the page_content. Also allows for custom deduplication with graph ingestion method by providing metadata id attribute --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	3 months ago
Leonid Ganeline	476d6dc596	community[patch]: Use getattr for `toolkits` imports (#18825) ... This will preserve the namespace, without actually loading the underlying packages on init.	3 months ago
Luis Antonio Vieira Junior	67c880af74	community[patch]: adding linearization config to AmazonTextractPDFLoader (#17489) ... - **Description:** Adding an optional parameter `linearization_config` to the `AmazonTextractPDFLoader` so the caller can define how the output will be linearized, instead of forcing a predefined set of linearization configs. It will still have a default configuration as this will be an optional parameter. - **Issue:** #17457 - **Dependencies:** The same ones that already exist for `AmazonTextractPDFLoader` - **Twitter handle:** [@lvieirajr19](https://twitter.com/lvieirajr19) --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	3 months ago
Anis ZAKARI	37e89ba5b1	community[patch]: Bedrock add support for mistral models (#18756) ... *Description**: My previous [PR](https://github.com/langchain-ai/langchain/pull/18521) was mistakenly closed, so I am reopening this one. Context: AWS released two Mistral models on Bedrock last Friday (March 1, 2024). This PR includes some code adjustments to ensure their compatibility with the Bedrock class. --------- Co-authored-by: Anis ZAKARI <anis.zakari@hymaia.com> Co-authored-by: Erick Friis <erick@langchain.dev>	3 months ago
Keith Chan	914af69b44	community[patch]: Update azuresearch vectorstore from_texts() method to include fields argument (#17661) ... - **Description:** Update azuresearch vectorstore from_texts() method to include fields argument, necessary for creating an Azure AI Search index with custom fields. - **Issue:** Currently index fields are fixed to default fields if Azure Search index is created using from_texts() method - **Dependencies:** None - **Twitter handle:** None --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	3 months ago
al1p	46f0cea2b9	community[patch][: improved the suffix prompt to avoid loop (#17791) ... Small improvement to the openapi prompt. The agent was not finding the server base URL (looping through all nodes). This small change narrows the search and enables finding the url faster. No dependency Twitter : @al1pra	3 months ago
Théo LEBRUN	cf94091cd0	community[patch]: Skip nested directories when using S3DirectoryLoader (#17829) ... - **Description:** `S3DirectoryLoader` is failing if prefix is a folder (ex: `my_folder/`) because `S3FileLoader` will try to load that folder and will fail. This PR skip nested directories so prefix can be set to folder instead of `my_folder/files_prefix`. - **Issue:** - #11917 - #6535 - #4326 - **Dependencies:** none - **Twitter handle:** @Falydoor - [x] **Add tests and docs**: If you're adding a new integration, please include 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. It lives in `docs/docs/integrations` directory. - [x] **Lint and test**: Run `make format`, `make lint` and `make test` from the root of the package(s) you've modified. See contribution guidelines for more: https://python.langchain.com/docs/contributing/	3 months ago
Venkatesan	7a18b63dbf	community[patch]: Mongo index creation (#17748) ... - [ ] Title: Mongodb: MongoDB connection performance improvement. - [ ] Message: - **Description:** I made collection index_creation as optional. Index Creation is one time process. - **Issue:** MongoDBChatMessageHistory class object is attempting to create an index during connection, causing each request to take longer than usual. This should be optional with a parameter. - **Dependencies:** N/A - **Branch to be checked:** origin/mongo_index_creation --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	3 months ago
wt3639	5b5b37a999	community[patch]: Add embedding instruction to HuggingFaceBgeEmbeddings (#18017) ... - **Description:** Add embedding instruction to HuggingFaceBgeEmbeddings, so that it can be compatible with nomic and other models that need embedding instruction. --------- Co-authored-by: Tao Wu <tao.wu@rwth-aachen.de> Co-authored-by: Bagatur <baskaryan@gmail.com>	3 months ago
Ishani Vyas	2b0cbd65ba	community[patch]: Add Passio Nutrition AI Food Search Tool to Community Package (#18278) ... ## Add Passio Nutrition AI Food Search Tool to Community Package ### Description We propose adding a new tool to the `community` package, enabling integration with Passio Nutrition AI for food search functionality. This tool will provide a simple interface for retrieving nutrition facts through the Passio Nutrition AI API, simplifying user access to nutrition data based on food search queries. ### Implementation Details - **Class Structure:** Implement `NutritionAI`, extending `BaseTool`. It includes an `_run` method that accepts a query string and, optionally, a `CallbackManagerForToolRun`. - **API Integration:** Use `NutritionAIAPI` for the API wrapper, encapsulating all interactions with the Passio Nutrition AI and providing a clean API interface. - **Error Handling:** Implement comprehensive error handling for API request failures. ### Expected Outcome - **User Benefits:** Enable easy querying of nutrition facts from Passio Nutrition AI, enhancing the utility of the `langchain_community` package for nutrition-related projects. - **Functionality:** Provide a straightforward method for integrating nutrition information retrieval into users' applications. ### Dependencies - `langchain_core` for base tooling support - `pydantic` for data validation and settings management - Consider `requests` or another HTTP client library if not covered by `NutritionAIAPI`. ### Tests and Documentation - **Unit Tests:** Include tests that mock network interactions to ensure tool reliability without external API dependency. - **Documentation:** Create an example notebook in `docs/docs/integrations/tools/passio_nutrition_ai.ipynb` showing usage, setup, and example queries. ### Contribution Guidelines Compliance - Adhere to the project's linting and formatting standards (`make format`, `make lint`, `make test`). - Ensure compliance with LangChain's contribution guidelines, particularly around dependency management and package modifications. ### Additional Notes - Aim for the tool to be a lightweight, focused addition, not introducing significant new dependencies or complexity. - Potential future enhancements could include caching for common queries to improve performance. ### Twitter Handle - Here is our Passio AI [twitter handle](https://twitter.com/@passio_ai) where we announce our products. If no one reviews your PR within a few days, please @-mention one of baskaryan, efriis, eyurtsev, hwchase17.	3 months ago
Kushagra	b1f22bf76c	community[minor]: added a feature to filter documents in Mongoloader (#18253) ... "community: added a feature to filter documents in Mongoloader" - **Description:** added a feature to filter documents in Mongoloader - **Feature:** the feature #18251 - **Dependencies:** No - **Twitter handle:** https://twitter.com/im_Kushagra	3 months ago
Eugene Yurtsev	1f50274df7	community[patch]: Add pgvector to docker compose and update settings used in integration test (#18815)	3 months ago
Christophe Bornet	e54a49b697	community[minor]: Add lazy_table_reflection param to SqlDatabase (#18742) ... For some DBs with lots of tables, reflection of all the tables can take very long. So this change will make the tables be reflected lazily when get_table_info() is called and `lazy_table_reflection` is True.	3 months ago
Christophe Bornet	ead2a74806	community: Implement lazy_load() for JSONLoader (#18643) ... Covered by `tests/unit_tests/document_loaders/test_json_loader.py`	3 months ago
Phat Vo	3ecb903d49	community[patch] : Tidy up and update Clarifai SDK functions (#18314) ... Description : * Tidy up, add missing docstring and fix unused params * Enable using session token	3 months ago
Tomaz Bratanic	4bfe888717	comunity[patch]: Fix neo4j sanitizing values (#18750) ... Fixing sanitization for when deeply nested lists appear	3 months ago
Yunmo Koo	fee6f983ef	community[minor]: Integration for `Friendli` LLM and `ChatFriendli` ChatModel. (#17913) ... ## Description - Add [Friendli](https://friendli.ai/) integration for `Friendli` LLM and `ChatFriendli` chat model. - Unit tests and integration tests corresponding to this change are added. - Documentations corresponding to this change are added. ## Dependencies - Optional dependency [`friendli-client`](https://pypi.org/project/friendli-client/) package is added only for those who use `Frienldi` or `ChatFriendli` model. ## Twitter handle - https://twitter.com/friendliai	3 months ago
Smit Parmar	aed46cd6f2	community[patch]: Added support for filter out AWS Kendra search by score confidence (#12920) ... **Description:** It will add support for filter out kendra search by score confidence which will make result more accurate. For example ``` retriever = AmazonKendraRetriever( index_id=kendra_index_id, top_k=5, region_name=region, score_confidence="HIGH" ) ``` Result will not include the records which has score confidence "LOW" or "MEDIUM". Relevant docs https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/kendra/client/query.html https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/kendra/client/retrieve.html **Issue:** the issue # it resolve #11801 **twitter:** [@SmitCode](https://twitter.com/SmitCode)	3 months ago
Ian	390ef6abe3	community[minor]: Add Initial Support for TiDB Vector Store (#15796) ... This pull request introduces initial support for the TiDB vector store. The current version is basic, laying the foundation for the vector store integration. While this implementation provides the essential features, we plan to expand and improve the TiDB vector store support with additional enhancements in future updates. Upcoming Enhancements: * Support for Vector Index Creation: To enhance the efficiency and performance of the vector store. * Support for max marginal relevance search. * Customized Table Structure Support: Recognizing the need for flexibility, we plan for more tailored and efficient data store solutions. Simple use case exmaple ```python from typing import List, Tuple from langchain.docstore.document import Document from langchain_community.vectorstores import TiDBVectorStore from langchain_openai import OpenAIEmbeddings db = TiDBVectorStore.from_texts( embedding=embeddings, texts=['Andrew like eating oranges', 'Alexandra is from England', 'Ketanji Brown Jackson is a judge'], table_name="tidb_vector_langchain", connection_string=tidb_connection_url, distance_strategy="cosine", ) query = "Can you tell me about Alexandra?" docs_with_score: List[Tuple[Document, float]] = db.similarity_search_with_score(query) for doc, score in docs_with_score: print("-" * 80) print("Score: ", score) print(doc.page_content) print("-" * 80) ```	3 months ago
Bagatur	3b1eb1f828	community[patch]: chat hf typing fix (#18693)	3 months ago
Eugene Yurtsev	e188d4ecb0	Add dangerous parameter to requests tool (#18697) ... The tools are already documented as dangerous. Not clear whether adding an opt-in parameter is necessary or not	3 months ago
Erick Friis	1beb84b061	community[patch]: move pdf text tests to integration (#18746)	3 months ago
Christophe Bornet	4a7d73b39d	community: If load() has been overridden, use it in default lazy_load() (#18690)	3 months ago
Christophe Bornet	6cd7607816	community[patch]: Implement lazy_load() for MHTMLLoader (#18648) ... Covered by `tests/unit_tests/document_loaders/test_mhtml.py`	3 months ago
axiangcoding	9745b5894d	community[patch]: Chroma use uuid4 instead of uuid1 to generate random ids (#18723) ... - **Description:** Chroma use uuid4 instead of uuid1 as random ids. Use uuid1 may leak mac address, changing to uuid4 will not cause other effects. - **Issue:** None - **Dependencies:** None - **Twitter handle:** None	3 months ago
Guangdong Liu	ced5e7bae7	community[patch]: Fix sparkllm authentication problem. (#18651) ... - **Description:** fix sparkllm authentication problem.The current timestamp is in RFC1123 format. The time deviation must be controlled within 300s. I changed to re-obtain the url every time I ask a question. https://www.xfyun.cn/doc/spark/general_url_authentication.html#_1-2-%E9%89%B4%E6%9D%83%E5%8F%82%E6%95%B0	3 months ago
Erick Friis	89d32ffbbd	community[patch]: release 0.0.27 (#18708)	3 months ago
Piyush Jain	2b234a4d96	Support for claude v3 models. (#18630) ... Fixes #18513. ## Description This PR attempts to fix the support for Anthropic Claude v3 models in BedrockChat LLM. The changes here has updated the payload to use the `messages` format instead of the formatted text prompt for all models; `messages` API is backwards compatible with all models in Anthropic, so this should not break the experience for any models. ## Notes The PR in the current form does not support the v3 models for the non-chat Bedrock LLM. This means, that with these changes, users won't be able to able to use the v3 models with the Bedrock LLM. I can open a separate PR to tackle this use-case, the intent here was to get this out quickly, so users can start using and test the chat LLM. The Bedrock LLM classes have also grown complex with a lot of conditions to support various providers and models, and is ripe for a refactor to make future changes more palatable. This refactor is likely to take longer, and requires more thorough testing from the community. Credit to PRs [18579](https://github.com/langchain-ai/langchain/pull/18579) and [18548](https://github.com/langchain-ai/langchain/pull/18548) for some of the code here. --------- Co-authored-by: Erick Friis <erick@langchain.dev>	3 months ago
Sam Khano	1b4dcf22f3	community[minor]: Add DocumentDBVectorSearch VectorStore (#17757) ... **Description:** - Added Amazon DocumentDB Vector Search integration (HNSW index) - Added integration tests - Updated AWS documentation with DocumentDB Vector Search instructions - Added notebook for DocumentDB integration with example usage --------- Co-authored-by: EC2 Default User <ec2-user@ip-172-31-95-226.ec2.internal>	3 months ago
Vittorio Rigamonti	51f3902bc4	community[minor]: Adding support for Infinispan as VectorStore (#17861) ... **Description:** This integrates Infinispan as a vectorstore. Infinispan is an open-source key-value data grid, it can work as single node as well as distributed. Vector search is supported since release 15.x For more: [Infinispan Home](https://infinispan.org) Integration tests are provided as well as a demo notebook	3 months ago
Max Jakob	cca0167917	elasticsearch[patch], community[patch]: update references, deprecate community classes (#18506) ... Follow up on https://github.com/langchain-ai/langchain/pull/17467. - Update all references to the Elasticsearch classes to use the partners package. - Deprecate community classes. --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	3 months ago
Djordje	12b4a4d860	community[patch]: Opensearch delete method added - indexing supported (#18522) ... - **Description:** Added delete method for OpenSearchVectorSearch, therefore indexing supported - **Issue:** No - **Dependencies:** No - **Twitter handle:** stkbmf	3 months ago
Christophe Bornet	db8db6faae	community: Implement lazy_load() for PlaywrightURLLoader (#18676) ... Integration tests: `tests/integration_tests/document_loaders/test_url_playwright.py`	3 months ago
Aaron Yi	c092db862e	community[patch]: make metadata and text optional as expected in DocArray (#18678) ... ValidationError: 2 validation errors for DocArrayDoc text Field required [type=missing, input_value={'embedding': [-0.0191128...9, 0.01005221541175212]}, input_type=dict] For further information visit https://errors.pydantic.dev/2.5/v/missing metadata Field required [type=missing, input_value={'embedding': [-0.0191128...9, 0.01005221541175212]}, input_type=dict] For further information visit https://errors.pydantic.dev/2.5/v/missing ``` In the `_get_doc_cls` method, the `DocArrayDoc` class is defined as follows: ```python class DocArrayDoc(BaseDoc): text: Optional[str] embedding: Optional[NdArray] = Field(**embeddings_params) metadata: Optional[dict] ```	3 months ago
Eugene Yurtsev	4c25b49229	community[major]: breaking change in some APIs to force users to opt-in for pickling (#18696) ... This is a PR that adds a dangerous load parameter to force users to opt in to use pickle. This is a PR that's meant to raise user awareness that the pickling module is involved.	3 months ago
Eugene Yurtsev	0e52961562	community[patch]: Patch tdidf retriever (CVE-2024-2057) (#18695) ... This is a patch for `CVE-2024-2057`: https://www.cve.org/CVERecord?id=CVE-2024-2057 This affects users that: * Use the `TFIDFRetriever` * Attempt to de-serialize it from an untrusted source that contains a malicious payload	3 months ago
Christophe Bornet	ea141511d8	core: Move document loader interfaces to core (#17723) ... This is needed to be able to move document loaders to partner packages. --------- Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com>	3 months ago
Christophe Bornet	5985454269	Merge pull request #18539 ... * Implement lazy_load() for GitLoader	3 months ago
Christophe Bornet	9a6f7e213b	Merge pull request #18423 ... * Implement lazy_load() for BSHTMLLoader	3 months ago
Christophe Bornet	b3a0c44838	Merge pull request #18673 ... * Implement lazy_load() for PDFMinerPDFasHTMLLoader and PyMuPDFLoader	3 months ago
Christophe Bornet	68fc0cf909	Merge pull request #18674 ... * Implement lazy_load() for TextLoader	3 months ago
Christophe Bornet	5b92f962f1	Merge pull request #18671 ... * Implement lazy_load() for MastodonTootsLoader	3 months ago
Christophe Bornet	15b1770326	Merge pull request #18421 ... * Implement lazy_load() for AssemblyAIAudioTranscriptLoader	3 months ago
Christophe Bornet	bb284eebe4	Merge pull request #18436 ... * Implement lazy_load() for ConfluenceLoader	3 months ago
Christophe Bornet	691480f491	Merge pull request #18647 ... * Implement lazy_load() for UnstructuredBaseLoader	3 months ago
Christophe Bornet	52ac67c5d8	Merge pull request #18654 ... * Implement lazy_load() for ObsidianLoader	3 months ago