langchain

Commit Graph

Author	SHA1	Message	Date
Isaac Francisco	148088a588	docs: duckduckgosearch options listed (#22568 ) Co-authored-by: Bagatur <baskaryan@gmail.com>	4 months ago
X-HAN	62f13f95e4	community[minor]: add DashScope Rerank (#22403 ) Description: this PR adds DashScope Rerank capability to Langchain, you can find DashScope Rerank API from [here](https://help.aliyun.com/document_detail/2780058.html?spm=a2c4g.2780059.0.0.6d995024FlrJ12) & [here](https://help.aliyun.com/document_detail/2780059.html?spm=a2c4g.2780058.0.0.63f75024cr11N9). [DashScope](https://dashscope.aliyun.com/) is the generative AI service from Alibaba Cloud (Aliyun). You can create DashScope API key from [here](https://bailian.console.aliyun.com/?apiKey=1#/api-key). Dependencies: DashScopeRerank depends on `dashscope` python package. Twitter handle: my twitter/x account is https://x.com/LastMonopoly and I'd like a mention, thanks you! Tests and docs 1. integration test: `test_dashscope_rerank.py` 2. example notebook: `dashscope_rerank.ipynb` Lint and test: I have run `make format`, `make lint` and `make test` from the root of the package I've modified. --------- Co-authored-by: Bagatur <baskaryan@gmail.com> Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com>	4 months ago
Ethan Yang	29064848f9	[Community]add option to delete the prompt from HF output (#22225 ) This will help to solve pattern mismatching issue when parsing the output in Agent. https://github.com/langchain-ai/langchain/issues/21912	4 months ago
Bagatur	584a1e30ac	community[patch]: AzureSearch async functions (#22075 )	4 months ago
ccurme	af129974a3	community: update how OpenAIAssistantV2Runnable creates threads with tool_resources (#22549 ) https://github.com/langchain-ai/langchain/issues/22503	4 months ago
Bagatur	51a0d4574e	community[patch]: Release 0.2.3 (#22562 )	4 months ago
leila-messallem	3280a5b49b	community[patch]: improve test setup to accurately test filtering of labels in neo4j (#22531 ) Description: This PR addresses an issue with an existing test that was not effectively testing the intended functionality. The previous test setup did not adequately validate the filtering of the labels in neo4j, because the nodes and relationship in the test data did not have any properties set. Without properties these labels would not have been returned, regardless of the filtering. --------- Co-authored-by: Oskar Hane <oh@oskarhane.com>	4 months ago
Stefano Lottini	328d0c99f2	community[minor]: Add support for metadata indexing policy in Cassandra vector store (#22548 ) This PR adds a constructor `metadata_indexing` parameter to the Cassandra vector store to allow optional fine-tuning of which fields of the metadata are to be indexed. This is a feature supported by the underlying CassIO library. Indexing mode of "all", "none" or deny- and allow-list based choices are available. The rationale is, in some cases it's advisable to programmatically exclude some portions of the metadata from the index if one knows in advance they won't ever be used at search-time. this keeps the index more lightweight and performant and avoids limitations on the length of _indexed_ strings. I added a integration test of the feature. I also added the possibility of running the integration test with Cassandra on an arbitrary IP address (e.g. Dockerized), via `CASSANDRA_CONTACT_POINTS=10.1.1.5,10.1.1.6 poetry run pytest [...]` or similar. While I was at it, I added a line to the `.gitignore` since the mypy _test_ cache was not ignored yet. My X (Twitter) handle: @rsprrs.	4 months ago
Emilien Chauvet	c3d4126eb1	community[minor]: add user agent for web scraping loaders (#22480 ) Description: This PR adds a `USER_AGENT` env variable that is to be used for web scraping. It creates a util to get that user agent and uses it in the classes used for scraping in [this piece of doc](https://python.langchain.com/v0.1/docs/use_cases/web_scraping/). Identifying your scraper is considered a good politeness practice, this PR aims at easing it. Issue: `None` Dependencies: `None` Twitter handle: `None`	4 months ago
Philippe PRADOS	8250c177de	community[minor]: Add native async support to SQLChatMessageHistory (#22065 ) # package community: Fix SQLChatMessageHistory ## Description Here is a rewrite of `SQLChatMessageHistory` to properly implement the asynchronous approach. The code circumvents [issue 22021](https://github.com/langchain-ai/langchain/issues/22021) by accepting a synchronous call to `def add_messages()` in an asynchronous scenario. This bypasses the bug. For the same reasons as in [PR 22](https://github.com/langchain-ai/langchain-postgres/pull/32) of `langchain-postgres`, we use a lazy strategy for table creation. Indeed, the promise of the constructor cannot be fulfilled without this. It is not possible to invoke a synchronous call in a constructor. We compensate for this by waiting for the next asynchronous method call to create the table. The goal of the `PostgresChatMessageHistory` class (in `langchain-postgres`) is, among other things, to be able to recycle database connections. The implementation of the class is problematic, as we have demonstrated in [issue 22021](https://github.com/langchain-ai/langchain/issues/22021). Our new implementation of `SQLChatMessageHistory` achieves this by using a singleton of type (`Async`)`Engine` for the database connection. The connection pool is managed by this singleton, and the code is then reentrant. We also accept the type `str` (optionally complemented by `async_mode`. I know you don't like this much, but it's the only way to allow an asynchronous connection string). In order to unify the different classes handling database connections, we have renamed `connection_string` to `connection`, and `Session` to `session_maker`. Now, a single transaction is used to add a list of messages. Thus, a crash during this write operation will not leave the database in an unstable state with a partially added message list. This makes the code resilient. We believe that the `PostgresChatMessageHistory` class is no longer necessary and can be replaced by: ``` PostgresChatMessageHistory = SQLChatMessageHistory ``` This also fixes the bug. ## Issue - [issue 22021](https://github.com/langchain-ai/langchain/issues/22021) - Bug in _exit_history() - Bugs in PostgresChatMessageHistory and sync usage - Bugs in PostgresChatMessageHistory and async usage - [issue 36](https://github.com/langchain-ai/langchain-postgres/issues/36) ## Twitter handle: pprados ## Tests - libs/community/tests/unit_tests/chat_message_histories/test_sql.py (add async test) @baskaryan, @eyurtsev or @hwchase17 can you check this PR ? And, I've been waiting a long time for validation from other PRs. Can you take a look? - [PR 32](https://github.com/langchain-ai/langchain-postgres/pull/32) - [PR 15575](https://github.com/langchain-ai/langchain/pull/15575) - [PR 13200](https://github.com/langchain-ai/langchain/pull/13200) --------- Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com>	4 months ago
Vincent Min	59bef31997	community[minor]: Improve InMemoryVectorStore with ability to persist to disk and filter on metadata. (#22186 ) - Description: The InMemoryVectorStore is a nice and simple vector store implementation for quick development and debugging. The current implementation is quite limited in its functionalities. This PR extends the functionalities by adding utility function to persist the vector store to a json file and to load it from a json file. We choose the json file format because it allows inspection of the database contents in a text editor, which is great for debugging. Furthermore, it adds a `filter` keyword that can be used to filter out documents on their `page_content` or `metadata`. - Issue: - - Dependencies: - - Twitter handle: @Vincent_Min	4 months ago
maang-h	89128b7a49	community[patch]: add detailed paragraph and example for BaichuanTextEmbeddings (#22031 ) - Description: add detailed paragraph and example for BaichuanTextEmbeddings - Issue: the issue #21983	4 months ago
Anthony Bernabeu	4e676a63b8	community[minor]: Added filter search for LanceDB (#22461 ) - [ ] community: "vectorstore: added filtering support for LanceDB vector store" - [ ] This PR adds filtering capabilities to LanceDB: - Description: In LanceDB filtering can be applied when searching for data into the vectorstore. It is using the SQL language as mentioned in the LanceDB documentation. - Issue: #18235 - Dependencies: No - [ ] Add tests and docs: If you're adding a new integration, please include 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. It lives in `docs/docs/integrations` directory. - [ ] Lint and test: Run `make format`, `make lint` and `make test` from the root of the package(s) you've modified. See contribution guidelines for more: https://python.langchain.com/docs/contributing/	4 months ago
Erick Friis	58192d617f	community: fix huggingface deprecations (#22522 )	4 months ago
Ofer Mendelevitch	ad502e8d50	community[minor]: Vectara Integration Update - Streaming, FCS, Chat, updates to documentation and example notebooks (#21334 ) Thank you for contributing to LangChain! Description: update to the Vectara / Langchain integration to integrate new Vectara capabilities: - Full RAG implemented as a Runnable with as_rag() - Vectara chat supported with as_chat() - Both support streaming response - Updated documentation and example notebook to reflect all the changes - Updated Vectara templates Twitter handle: ofermend Add tests and docs: no new tests or docs, but updated both existing tests and existing docs	4 months ago
Bagatur	e60f88ccdd	community[patch]: Release 0.2.2 (#22496 )	4 months ago
Joydeep Banik Roy	3796672c67	community, milvus, pinecone, qdrant, mongo: Broadcast operation failure while using simsimd beyond v3.7.7 (#22271 ) - [ ] Packages affected: - community: fix `cosine_similarity` to support simsimd beyond 3.7.7 - partners/milvus: fix `cosine_similarity` to support simsimd beyond 3.7.7 - partners/mongodb: fix `cosine_similarity` to support simsimd beyond 3.7.7 - partners/pinecone: fix `cosine_similarity` to support simsimd beyond 3.7.7 - partners/qdrant: fix `cosine_similarity` to support simsimd beyond 3.7.7 - [ ] Broadcast operation failure while using simsimd beyond v3.7.7: - Description: I was using simsimd 4.3.1 and the unsupported operand type issue popped up. When I checked out the repo and ran the tests, they failed as well (have attached a screenshot for that). Looks like it is a variant of https://github.com/langchain-ai/langchain/issues/18022 . Prior to 3.7.7, simd.cdist returned an ndarray but now it returns simsimd.DistancesTensor which is ineligible for a broadcast operation with numpy. With this change, it also remove the need to explicitly cast `Z` to numpy array - Issue: #19905 - Dependencies: No - Twitter handle: https://x.com/GetzJoydeep <img width="1622" alt="Screenshot 2024-05-29 at 2 50 00 PM" src="https://github.com/langchain-ai/langchain/assets/31132555/fb27b383-a9ae-4a6f-b355-6d503b72db56"> - [ ] Considerations: 1. I started with community but since similar changes were there in Milvus, MongoDB, Pinecone, and QDrant so I modified their files as well. If touching multiple packages in one PR is not the norm, then I can remove them from this PR and raise separate ones 2. I have run and verified that the tests work. Since, only MongoDB had tests, I ran theirs and verified it works as well. Screenshots attached : <img width="1573" alt="Screenshot 2024-05-29 at 2 52 13 PM" src="https://github.com/langchain-ai/langchain/assets/31132555/ce87d1ea-19b6-4900-9384-61fbc1a30de9"> <img width="1614" alt="Screenshot 2024-05-29 at 3 33 51 PM" src="https://github.com/langchain-ai/langchain/assets/31132555/6ce1d679-db4c-4291-8453-01028ab2dca5"> I have added a test for simsimd. I feel it may not go well with the CI/CD setup as installing simsimd is not a dependency requirement. I have just imported simsimd to ensure simsimd cosine similarity is invoked. However, its not a good approach. Suggestions are welcome and I can make the required changes on the PR. Please provide guidance on the same as I am new to the community. --------- Co-authored-by: Bagatur <baskaryan@gmail.com> Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com>	4 months ago
KyrianC	03178ee74f	community[minor]: Add tools calls to `ChatEdenAI` (#22320 ) ### Description Add tools implementation to `ChatEdenAI`: - `bind_tools()` - `with_structured_output()` ### Documentation Updated `docs/docs/integrations/chat/edenai.ipynb` ### Notes We don´t support stream with tools as of yet. If stream is called with tools we directly yield the whole message from `generate` (implemented the same way as Anthropic did).	4 months ago
pranavvuppala	9d4350e69a	docs : Update docstrings for OpenAI base.py (#22221 ) - [x] PR title: Update docstrings for OpenAI base.py -Description: Updated the docstring of few OpenAI functions for a better understanding of the function. - Issue: #21983 --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	4 months ago
Anindyadeep	7a197539aa	communty[patch]: Native RAG Support in Prem AI langchain (#22238 ) This PR adds native RAG support in langchain premai package. The same has been added in the docs too.	4 months ago
Rahul Triptahi	77ad857934	community[minor]: Enable retrieval api calls in PebbloRetrievalQA (#21958 ) Description: Enable app discovery and Prompt/Response apis in PebbloSafeRetrieval Documentation: NA Unit test: N/A --------- Signed-off-by: Rahul Tripathi <rauhl.psit.ec@gmail.com> Co-authored-by: Rahul Tripathi <rauhl.psit.ec@gmail.com>	4 months ago
Bagatur	17c127531a	community[patch]: deprecate all HF classes (#22444 )	4 months ago
Christophe Bornet	9a8fe58ebe	community[minor]: Improve Cassandra VectorStore as_retriever (#22465 ) The Vectorstore's API `as_retriever` doesn't expose explicitly the parameters `search_type` and `search_kwargs` and so these are not well documented. This PR improves `as_retriever` for the Cassandra VectorStore by making these parameters explicit. NB: An alternative would have been to modify `as_retriever` in `Vectorstore`. But there's probably a good reason these were not exposed in the first place ? Is it because implementations may decide to not support them and have fixed values when creating the VectorStoreRetriever ?	4 months ago
Fahreddin Özcan	0061ded002	community[patch]: Upstash Vector Store Namespace Support (#22251 ) This PR introduces namespace support for Upstash Vector Store, which would allow users to partition their data in the vector index. --------- Co-authored-by: Bagatur <baskaryan@gmail.com> Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com>	4 months ago
Dristy Srivastava	ef3df45d9d	community[minor]: Updating payload for pebblo discover API (#22309 ) Description: Updating response for pebblo discover API. Also updating filed name case type Documentation: N/A Unit tests: N/A	4 months ago
ccurme	afe89a1411	community: add standard chat model params to Ollama (#22446 )	4 months ago
Ethan Yang	52da6a160d	community[patch]: Update OpenVINO embedding and reranker to support static input shape (#22171 ) It can help to deploy embedding models on NPU device	4 months ago
maang-h	01352bb55f	community[minor]: Implement MiniMaxChat interface (#22391 ) - Description: Implement MiniMaxChat interface, include: - No longer inherits the LLM class (like other chat model) - Update request parameters (v1 -> v2) - update `base url` - update message role (system, user, assistant) - add `stream` function - no longer use `group id` - Implement the `_stream`, `_agenerate`, and `_astream` interfaces [minimax v2 api document](https://platform.minimaxi.com/document/guides/chat-model/V2?id=65e0736ab2845de20908e2dd)	4 months ago
Brandon Sharp	56e5aa4dd9	community[patch]: Airtable to allow for addtl params (#22092 ) - [X] PR title: "community: added optional params to Airtable table.all()" - [X] PR message: - Description: Add's kwargs to AirtableLoader to allow for kwargs: https://pyairtable.readthedocs.io/en/latest/api.html#pyairtable.Table.all - Issue: N/A - Dependencies: N/A - Twitter handle: parakoopa88 - [X] Add tests and docs: If you're adding a new integration, please include 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. It lives in `docs/docs/integrations` directory. - [X] Lint and test**: Run `make format`, `make lint` and `make test` from the root of the package(s) you've modified. See contribution guidelines for more: https://python.langchain.com/docs/contributing/ If no one reviews your PR within a few days, please @-mention one of baskaryan, efriis, eyurtsev, ccurme, vbarda, hwchase17. --------- Co-authored-by: Bagatur <baskaryan@gmail.com> Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com>	4 months ago
Harichandan Roy	1f751343e2	community[patch]: update embeddings/oracleai.py (#22240 ) Thank you for contributing to LangChain! - [ ] PR title: "package: description" - Where "package" is whichever of langchain, community, core, experimental, etc. is being modified. Use "docs: ..." for purely docs changes, "templates: ..." for template changes, "infra: ..." for CI changes. - Example: "community: add foobar LLM" "community/embeddings: update oracleai.py" - [ ] PR message: *Delete this entire checklist* and replace with - Description: a description of the change - Issue: the issue # it fixes, if applicable - Dependencies: any dependencies required for this change - Twitter handle: if your PR gets announced, and you'd like a mention, we'll gladly shout you out! Adding oracle VECTOR_ARRAY_T support. - [ ] Add tests and docs: If you're adding a new integration, please include 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. It lives in `docs/docs/integrations` directory. Tests are not impacted. - [ ] Lint and test: Run `make format`, `make lint` and `make test` from the root of the package(s) you've modified. See contribution guidelines for more: https://python.langchain.com/docs/contributing/ Done. Additional guidelines: - Make sure optional dependencies are imported within a function. - Please do not add dependencies to pyproject.toml files (even optional ones) unless they are required for unit tests. - Most PRs should not touch more than one package. - Changes should be backwards compatible. - If you are adding something to community, do not re-import it in langchain. If no one reviews your PR within a few days, please @-mention one of baskaryan, efriis, eyurtsev, ccurme, vbarda, hwchase17.	4 months ago
maang-h	13140dc4ff	community[patch]: Update the default api_url and reqeust_body of sparkllm embedding (#22136 ) - Description: When I was running the SparkLLMTextEmbeddings, app_id, api_key and api_secret are all correct, but it cannot run normally using the current URL. ```python # example from langchain_community.embeddings import SparkLLMTextEmbeddings embedding= SparkLLMTextEmbeddings( spark_app_id="my-app-id", spark_api_key="my-api-key", spark_api_secret="my-api-secret" ) embedding= "hello" print(spark.embed_query(text1)) ``` ![sparkembedding](https://github.com/langchain-ai/langchain/assets/55082429/11daa853-4f67-45b2-aae2-c95caa14e38c) So I updated the url and request body parameters according to [Embedding_api](https://www.xfyun.cn/doc/spark/Embedding_api.html), now it is runnable.	4 months ago
Yuwen Hu	ba0dca46d7	community[minor]: Add IPEX-LLM BGE embedding support on both Intel CPU and GPU (#22226 ) Description: [IPEX-LLM](https://github.com/intel-analytics/ipex-llm) is a PyTorch library for running LLM on Intel CPU and GPU (e.g., local PC with iGPU, discrete GPU such as Arc, Flex and Max) with very low latency. This PR adds ipex-llm integrations to langchain for BGE embedding support on both Intel CPU and GPU. Dependencies: `ipex-llm`, `sentence-transformers` Contribution maintainer: @Oscilloscope98 tests and docs: - langchain/docs/docs/integrations/text_embedding/ipex_llm.ipynb - langchain/docs/docs/integrations/text_embedding/ipex_llm_gpu.ipynb - langchain/libs/community/tests/integration_tests/embeddings/test_ipex_llm.py --------- Co-authored-by: Shengsheng Huang <shannie.huang@gmail.com>	4 months ago
Dan	86509161b0	community: fix AzureSearch delete documents (#22315 ) Description Fix AzureSearch delete documents method by using FIELDS_ID variable instead of the hard coded "id" value Issue: This is linked to this issue: https://github.com/langchain-ai/langchain/issues/22314 Co-authored-by: dseban <dan.seban@neoxia.com>	4 months ago
Charles John	2d81a72884	community: fix missing `apify_api_token` field in ApifyWrapper (#22421 ) - Description: The `ApifyWrapper` class expects `apify_api_token` to be passed as a named parameter or set as an environment variable. But the corresponding field was missing in the class definition causing the argument to be ignored when passed as a named param. This patch fixes that.	4 months ago
Joan Fontanals	a7ae16f912	add `embed_image` API to JinaEmbedding (#22416 ) - Description: Add `embed_image` to JinaEmbedding to embed images - Twitter handle: https://x.com/JinaAI_	4 months ago
WU LIFU	86698b02a9	doc: fix wrong documentation on FAISS load_local function (#22310 ) ### Issue: #22299 ### descriptions The documentation appears to be wrong. When the user actually sets this parameter "asynchronous" to be True, it fails because the __init__ function of FAISS class doesn't allow this parameter. In fact, most of the class/instance functions of this class have both the sync/async version, so it looks like what we need is just to remove this parameter from the doc. Thank you for contributing to LangChain! - [x] PR title: "package: description" - Where "package" is whichever of langchain, community, core, experimental, etc. is being modified. Use "docs: ..." for purely docs changes, "templates: ..." for template changes, "infra: ..." for CI changes. - Example: "community: add foobar LLM" - [x] PR message: *Delete this entire checklist* and replace with - Description: a description of the change - Issue: the issue # it fixes, if applicable - Dependencies: any dependencies required for this change - Twitter handle: if your PR gets announced, and you'd like a mention, we'll gladly shout you out! - [ ] Add tests and docs: If you're adding a new integration, please include 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. It lives in `docs/docs/integrations` directory. - [ ] Lint and test: Run `make format`, `make lint` and `make test` from the root of the package(s) you've modified. See contribution guidelines for more: https://python.langchain.com/docs/contributing/ Additional guidelines: - Make sure optional dependencies are imported within a function. - Please do not add dependencies to pyproject.toml files (even optional ones) unless they are required for unit tests. - Most PRs should not touch more than one package. - Changes should be backwards compatible. - If you are adding something to community, do not re-import it in langchain. If no one reviews your PR within a few days, please @-mention one of baskaryan, efriis, eyurtsev, ccurme, vbarda, hwchase17. Co-authored-by: Lifu Wu <lifu@nextbillion.ai>	4 months ago
maang-h	596c062cba	community[patch]: Standardize qianfan model init args name (#22322 ) - Description: - Standardize qianfan chat model intialization arguments name - qianfan_ak (qianfan api key) -> api_key - qianfan_sk (qianfan secret key) -> secret_key - Delete unuse variable - Issue: #20085	4 months ago
Dobiichi-Origami	10b12e1c08	community: adding tool_call_id for every ToolCall (#22323 ) - Description: This PR contains a bugfix which result in malfunction of multi-turn conversation in QianfanChatEndpoint and adaption for ToolCall and ToolMessage	4 months ago
ccurme	f39e1a2288	community, docs: update token usage tracking callback + how-to guides (#22145 )	4 months ago
ChengZi	404d92ded0	milvus: New langchain_milvus package and new milvus features (#21077 ) New features: - New langchain_milvus package in partner - Milvus collection hybrid search retriever - Zilliz cloud pipeline retriever - Milvus Local guid - Rag-milvus template --------- Signed-off-by: ChengZi <chen.zhang@zilliz.com> Signed-off-by: Jael Gu <mengjia.gu@zilliz.com> Co-authored-by: Jael Gu <mengjia.gu@zilliz.com> Co-authored-by: Jackson <jacksonxie612@gmail.com> Co-authored-by: Erick Friis <erick@langchain.dev> Co-authored-by: Erick Friis <erickfriis@gmail.com>	4 months ago
Will Higgins	83d10df78d	community[patch]: Update firecrawl api key name (#22183 ) Change 'FIREWALL' to 'FIRECRAWL' as I believe this may have been in error. Other docs refer to 'FIRECRAWL_API_KEY'. Thank you for contributing to LangChain! - [ ] PR title: "package: description" - Where "package" is whichever of langchain, community, core, experimental, etc. is being modified. Use "docs: ..." for purely docs changes, "templates: ..." for template changes, "infra: ..." for CI changes. - Example: "community: add foobar LLM" - [ ] PR message: *Delete this entire checklist* and replace with - Description: a description of the change - Issue: the issue # it fixes, if applicable - Dependencies: any dependencies required for this change - Twitter handle: if your PR gets announced, and you'd like a mention, we'll gladly shout you out! - [ ] Add tests and docs: If you're adding a new integration, please include 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. It lives in `docs/docs/integrations` directory. - [ ] Lint and test: Run `make format`, `make lint` and `make test` from the root of the package(s) you've modified. See contribution guidelines for more: https://python.langchain.com/docs/contributing/ Additional guidelines: - Make sure optional dependencies are imported within a function. - Please do not add dependencies to pyproject.toml files (even optional ones) unless they are required for unit tests. - Most PRs should not touch more than one package. - Changes should be backwards compatible. - If you are adding something to community, do not re-import it in langchain. If no one reviews your PR within a few days, please @-mention one of baskaryan, efriis, eyurtsev, ccurme, vbarda, hwchase17. --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	4 months ago
Pavlo Paliychuk	342df7cf83	community[minor]: Add Zep Cloud components + docs + examples (#21671 ) Thank you for contributing to LangChain! - [x] PR title: community: Add Zep Cloud components + docs + examples - [x] PR message: We have recently released our new zep-cloud sdks that are compatible with Zep Cloud (not Zep Open Source). We have also maintained our Cloud version of langchain components (ChatMessageHistory, VectorStore) as part of our sdks. This PRs goal is to port these components to langchain community repo, and close the gap with the existing Zep Open Source components already present in community repo (added ZepCloudMemory,ZepCloudVectorStore,ZepCloudRetriever). Also added a ZepCloudChatMessageHistory components together with an expression language example ported from our repo. We have left the original open source components intact on purpose as to not introduce any breaking changes. - Issue: - - Dependencies: Added optional dependency of our new cloud sdk `zep-cloud` - Twitter handle: @paulpaliychuk51 - [x] Add tests and docs - [x] Lint and test: Run `make format`, `make lint` and `make test` from the root of the package(s) you've modified. See contribution guidelines for more: https://python.langchain.com/docs/contributing/ Additional guidelines: - Make sure optional dependencies are imported within a function. - Please do not add dependencies to pyproject.toml files (even optional ones) unless they are required for unit tests. - Most PRs should not touch more than one package. - Changes should be backwards compatible. - If you are adding something to community, do not re-import it in langchain. If no one reviews your PR within a few days, please @-mention one of baskaryan, efriis, eyurtsev, hwchase17.	4 months ago
Jan Soubusta	cccc8fbe2f	community[patch]: DuckDB VS - expose similarity, improve performance of from_texts (#20971 ) 3 fixes of DuckDB vector store: - unify defaults in constructor and from_texts (users no longer have to specify `vector_key`). - include search similarity into output metadata (fixes #20969) - significantly improve performance of `from_documents` Dependencies: added Pandas to speed up `from_documents`. I was thinking about CSV and JSON options, but I expect trouble loading JSON values this way and also CSV and JSON options require storing data to disk. Anyway, the poetry file for langchain-community already contains a dependency on Pandas. --------- Co-authored-by: Bagatur <baskaryan@gmail.com> Co-authored-by: Harrison Chase <hw.chase.17@gmail.com> Co-authored-by: ccurme <chester.curme@gmail.com>	4 months ago
Ameya Shenoy	8ba492ed6a	community[minor]: clickhouse -- ability to use secure connection (#22108 ) - Description: this PR gives clickhouse client the ability to use a secure connection to the clickhosue server - Issue: fixes #22082 - Dependencies: - - Twitter handle: `_codingcoffee_` Signed-off-by: Ameya Shenoy <shenoy.ameya@gmail.com> Co-authored-by: Shresth Rana <shresth@grapevine.in>	4 months ago
Rahul Triptahi	1a485f59b9	community[patch]: Put authorized identities behind a feature flag in SharepointLoader (#22125 ) Description: Put authorised identities behind a feature flag, load_auth. Documentation: N/A Unit tests: N/A --------- Signed-off-by: Rahul Tripathi <rauhl.psit.ec@gmail.com> Co-authored-by: Rahul Tripathi <rauhl.psit.ec@gmail.com>	4 months ago
sasha	1c9ceff503	community: add metadata to chain logging; (#22122 ) Hey, I'm Sasha. The SDK engineer from [Comet](https://comet.com). This PR updates the CometTracer class. Added metadata to CometTracerr. From now on, both chains and spans will send it.	4 months ago
Jirka Lhotka	7c0459faf2	community: Update costs of openai finetuned models (#22124 ) - Description: Update costs of finetuned models and add gpt-3-turbo-0125. Source: https://openai.com/api/pricing/ - Issue: N/A - Dependencies: None	4 months ago
Eugene Yurtsev	d3db83abe3	community[major]: lint for usage of xml library (#22132 ) * Lint for usage of standard xml library * Add forced opt-in for quip client * Actual security issue is with underlying QuipClient not LangChain integration (since the client is doing the parsing), but adding enforcement at the LangChain level.	4 months ago
Christophe Bornet	c838de5027	doc: Add doc for CassandraByteStore (#22126 ) Preview: https://langchain-git-fork-cbornet-doc-cassandrabytestore-langchain.vercel.app/v0.2/docs/integrations/stores/cassandra/	4 months ago
Eugene Yurtsev	2d693c484e	docs: fix some spelling mistakes caught by newest version of code spell (#22090 ) Going to merge this even though it doesn't pass all tests, and open a separate PR for the remaining spelling mistakes.	4 months ago
Pavel Zloi	fe26f937e4	community[minor]: ManticoreSearch engine added to vectorstore (#19117 ) Description: ManticoreSearch engine added to vectorstores Issue: no issue, just a new feature Dependencies: https://pypi.org/project/manticoresearch-dev/ Twitter handle: @EvilFreelancer - Example notebook with test integration: https://github.com/EvilFreelancer/langchain/blob/manticore-search-vectorstore/docs/docs/integrations/vectorstores/manticore_search.ipynb --------- Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com> Co-authored-by: Harrison Chase <hw.chase.17@gmail.com> Co-authored-by: Chester Curme <chester.curme@gmail.com> Co-authored-by: Bagatur <baskaryan@gmail.com>	4 months ago
Bagatur	3d26807b92	community[patch]: Release. 0.2.1 (#22073 )	4 months ago
maang-h	9aba9e3e33	community[patch]: Update the default “API URL” and “MODEL” of sparkllm (#22070 ) - Description: When I was running the sparkllm, I found that the default parameters currently used could no longer run correctly. - original parameters & values: - spark_api_url: "wss://spark-api.xf-yun.com/v3.1/chat" - spark_llm_domain: "generalv3" ```python # example from langchain_community.chat_models import ChatSparkLLM spark = ChatSparkLLM(spark_app_id="my_app_id", spark_api_key="my_api_key", spark_api_secret="my_api_secret") spark.invoke("hello") ``` ![sparkllm](https://github.com/langchain-ai/langchain/assets/55082429/5369bfdf-4305-496a-bcf5-2d3f59d39414) So I updated them to 3.5 (same as sparkllm official website). After the update, they can be used normally. - new parameters & values: - spark_api_url: "wss://spark-api.xf-yun.com/v3.5/chat" - spark_llm_domain: "generalv3.5"	4 months ago
Martin Triska	2df8ac402a	community[minor]: Added propagation of document metadata from O365BaseLoader (#20663 ) Description: - Added propagation of document metadata from O365BaseLoader to FileSystemBlobLoader (O365BaseLoader uses FileSystemBlobLoader under the hood). - This is done by passing dictionary `metadata_dict`: key=filename and value=dictionary containing document's metadata - Modified `FileSystemBlobLoader` to accept the `metadata_dict`, use `mimetype` from it (if available) and pass metadata further into blob loader. Issue: - `O365BaseLoader` under the hood downloads documents to temp folder and then uses `FileSystemBlobLoader` on it. - However metadata about the document in question is lost in this process. In particular: - `mime_type`: `FileSystemBlobLoader` guesses `mime_type` from the file extension, but that does not work 100% of the time. - `web_url`: this is useful to keep around since in RAG LLM we might want to provide link to the source document. In order to work well with document parsers, we pass the `web_url` as `source` (`web_url` is ignored by parsers, `source` is preserved) Dependencies: None Twitter handle: @martintriska1 Please review @baskaryan --------- Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com> Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com>	4 months ago
Eugene Yurtsev	e5541d1da7	community[patch]: Update doc-string in CloudBlobLoader (#22069 ) Update doc-string	4 months ago
Philippe PRADOS	6dd621d636	community[minor]: Add CloudBlobLoader that supports loading data from cloud buckets (#21957 ) Thank you for contributing to LangChain! - [ ] PR title: "Add CloudBlobLoader" - community: Add CloudBlobLoader - [ ] PR message: Add cloud blob loader - Description: Langchain provides several approaches to read different file formats: Specific loaders (`CVSLoader`) or blob-compatible loaders (`FileSystemBlobLoader`). The only implementation proposed for BlobLoader is `FileSystemBlobLoader`. Many projects retrieve files from cloud storage. We propose a new implementation of `BlobLoader` to read files from the three cloud storage systems. The interface is strictly identical to `FileSystemBlobLoader`. The only difference is the constructor, which takes a cloud "url" object such as `s3://my-bucket`, `az://my-bucket`, or `gs://my-bucket`. By streamlining the process, this novel implementation eliminates the requirement to pre-download files from cloud storage to local temporary files (which are seldom removed). The code relies on the [CloudPathLib](https://cloudpathlib.drivendata.org/stable/) library to interpret cloud URLs. This has been added as an optional dependency. ```Python loader = CloudBlobLoader("s3://mybucket/id") for blob in loader.yield_blobs(): print(blob) ``` - [X] Dependencies: CloudPathLib - [X] Twitter handle: pprados - [X] Add tests and docs: Add unit test, but it's easy to convert to integration test, with some files in a cloud storage (see `test_cloud_blob_loader.py`) - [X] Lint and test: Run `make format`, `make lint` and `make test` from the root of the package(s) you've modified. Hello from Paris @hwchase17. Can you review this PR? --------- Co-authored-by: Eugene Yurtsev <eugene@langchain.dev>	4 months ago
Christophe Bornet	74947ec894	community[minor]: Add Cassandra ByteStore (#22064 )	4 months ago
Christophe Bornet	fea6b99b16	community[minor]: Add async methods to CassandraChatMessageHistory (#21975 )	4 months ago
Sky	12d65f17ff	community[patch]: surrealdb provide functions for MMR (Maximal Marginal Relevance) (#21185 ) This PR contains 4 added functions: - max_marginal_relevance_search_by_vector - amax_marginal_relevance_search_by_vector - max_marginal_relevance_search - amax_marginal_relevance_search I'm no langchain expert, but tried do inspect other vectorstore sources like chroma, to build these functions for SurrealDB. If someone has some changes for me, please let me know. Otherwise I would be happy, if these changes are added to the repository, so that I can use the orignal repo and not my local monkey patched version. --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com> Co-authored-by: Bagatur <baskaryan@gmail.com>	4 months ago
Bruno Alvisio	5eabe90494	community[patch]: Adding HEADER to the list of supported locations (#21946 ) Description: adds headers to the list of supported locations when generating the openai function schema	4 months ago
Bagatur	50186da0a1	infra: rm unused # noqa violations (#22049 ) Updating #21137	4 months ago
acho98	45ed5f3f51	community[minor]: Add Clova Embeddings for LangChain Community (#21890 ) - [ ] PR title: "Add Naver ClovaX embedding to LangChain community" - HyperClovaX is a large language model developed by [Naver](https://clova-x.naver.com/welcome). It's a powerful and purpose-trained LLM. - You can visit the embedding service provided by [ClovaX](https://www.ncloud.com/product/aiService/clovaStudio) - You may get CLOVA_EMB_API_KEY, CLOVA_EMB_APIGW_API_KEY, CLOVA_EMB_APP_ID From https://www.ncloud.com/product/aiService/clovaStudio --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	4 months ago
arpitkumar980	444c2a3d9f	community[patch]: sharepoint loader identity enabled (#21176 ) Thank you for contributing to LangChain! - [ ] PR title: "package: description" - Where "package" is whichever of langchain, community, core, experimental, etc. is being modified. Use "docs: ..." for purely docs changes, "templates: ..." for template changes, "infra: ..." for CI changes. - Example: "community: add foobar LLM" - [ ] PR message: *Delete this entire checklist* and replace with - Description: a description of the change - Issue: the issue # it fixes, if applicable - Dependencies: any dependencies required for this change - Twitter handle: if your PR gets announced, and you'd like a mention, we'll gladly shout you out! - [ ] Add tests and docs: If you're adding a new integration, please include 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. It lives in `docs/docs/integrations` directory. - [ ] Lint and test: Run `make format`, `make lint` and `make test` from the root of the package(s) you've modified. See contribution guidelines for more: https://python.langchain.com/docs/contributing/ Additional guidelines:https://github.com/arpitkumar980/langchain.git - Make sure optional dependencies are imported within a function. - Please do not add dependencies to pyproject.toml files (even optional ones) unless they are required for unit tests. - Most PRs should not touch more than one package. - Changes should be backwards compatible. - If you are adding something to community, do not re-import it in langchain. If no one reviews your PR within a few days, please @-mention one of baskaryan, efriis, eyurtsev, hwchase17. --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com> Co-authored-by: Bagatur <baskaryan@gmail.com> Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com>	4 months ago
HuiyuanYan	bf3aefce93	community[patch]: Update tongyi.py to support MultimodalConversation in dashscope. (#21249 ) Add the support of multimodal conversation in dashscope,now we can use multimodal language model "qwen-vl-v1", "qwen-vl-chat-v1", "qwen-audio-turbo" to processing picture an audio. :) - [ ] PR title: "community: add multimodal conversation support in dashscope" - [ ] PR message: *Delete this entire checklist* and replace with - Description: add multimodal conversation support in dashscope - Issue: - Dependencies: dashscope≥1.18.0 - Twitter handle: none :) - [ ] How to use it?: - ```python Tongyi_chat = ChatTongyi( top_p=0.5, dashscope_api_key=api_key, model="qwen-vl-v1" ) response= Tongyi_chat.invoke( input = [ { "role": "user", "content": [ {"image": "https://dashscope.oss-cn-beijing.aliyuncs.com/images/dog_and_girl.jpeg"}, {"text": "这是什么?"} ] } ] ) ``` --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	4 months ago
MSubik	d948783a4c	community[patch]: standardize init args, update for javelin sdk release. (#21980 ) Related to [20085](https://github.com/langchain-ai/langchain/issues/20085) Updated the Javelin chat model to standardize the initialization argument. Also fixed an existing bug, where code was initialized with incorrect call to the JavelinClient defined in the javelin_sdk, resulting in an initialization error. See related [Javelin Documentation](https://docs.getjavelin.io/docs/javelin-python/quickstart).	4 months ago
Mohammad Mohtashim	16617dd239	community[patch]: AzureSearchVectorStoreRetriever Fixed to account for search_kwargs (#21572 ) - Description: Fixed `AzureSearchVectorStoreRetriever` to account for search_kwargs. More explanation is in the mentioned issue. - Issue: #21492 --------- Co-authored-by: MAC <mac@MACs-MacBook-Pro.local> Co-authored-by: Massimiliano Pronesti <massimiliano.pronesti@gmail.com> Co-authored-by: Bagatur <baskaryan@gmail.com>	4 months ago
Jerron Lim	28456c2c33	community[patch]: add args_schema to WikipediaQueryRun (#22019 ) Description: This change adds args_schema (pydantic BaseModel) to WikipediaQueryRun for correct schema formatting on LLM function calls Issue: currently using WikipediaQueryRun with OpenAI function calling returns the following error "TypeError: WikipediaQueryRun._run() got an unexpected keyword argument '__arg1' ". This happens because the schema sent to the LLM is "input: '{"__arg1":"Hunter x Hunter"}'" while the method should be called with the "query" parameter. --------- Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com> Co-authored-by: Bagatur <baskaryan@gmail.com>	4 months ago
Mazen Ramadan	3c1d77dd64	community[minor]: Add Scrapfly Loader community integration (#22036 ) Added [Scrapfly](https://scrapfly.io/) Web Loader integration. Scrapfly is a web scraping API that allows extracting web page data into accessible markdown or text datasets. - __Description__: Added Scrapfly web loader for retrieving web page data as markdown or text. - Dependencies: scrapfly-sdk - Twitter: @thealchemi1st --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	4 months ago
ccurme	b51a1eba4d	langchain, community: move OpenAIAssistantV2Runnable to community (#22044 )	4 months ago
CaroFG	6b98140b38	community[patch]: update for compatibility with Meilisearch v1.8 (#21979 ) Thank you for contributing to LangChain! - [x] PR title: "package: description" - Where "package" is whichever of langchain, community, core, experimental, etc. is being modified. Use "docs: ..." for purely docs changes, "templates: ..." for template changes, "infra: ..." for CI changes. - Example: "community: add foobar LLM" - [x] PR message: *Delete this entire checklist* and replace with - Description: Updates Meilisearch vectorstore for compatibility with v1.8. Adds [”showRankingScore”: true”](https://www.meilisearch.com/docs/reference/api/search#ranking-score) in the search parameters and replaces `_semanticScore` field with ` _rankingScore` - [ ] Add tests and docs: If you're adding a new integration, please include 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. It lives in `docs/docs/integrations` directory. - [x] Lint and test: Run `make format`, `make lint` and `make test` from the root of the package(s) you've modified. See contribution guidelines for more: https://python.langchain.com/docs/contributing/ Additional guidelines: - Make sure optional dependencies are imported within a function. - Please do not add dependencies to pyproject.toml files (even optional ones) unless they are required for unit tests. - Most PRs should not touch more than one package. - Changes should be backwards compatible. - If you are adding something to community, do not re-import it in langchain. If no one reviews your PR within a few days, please @-mention one of baskaryan, efriis, eyurtsev, ccurme, vbarda, hwchase17.	4 months ago
Oleksii Pokotylo	98c0b093bb	community[patch]: Extend AzureSearch with `maximal_marginal_relevance`, `from_embeddings` (#21065 ) Description: - Extend AzureSearch with `maximal_marginal_relevance` (for vector and hybrid search) - Add construction `from_embeddings` - if the user has already embedded the texts - Add `add_embeddings` - Refactor common parts (`_simple_search`, `_results_to_documents`, `_reorder_results_with_maximal_marginal_relevance`) - Add `vector_search_dimensions` as a parameter to the constructor to avoid extra calls to `embed_query` (most of the time the user applies the same model and knows the dimension) Issue: none Dependencies: none - [x] Add tests and docs: The docstrings have been added to the new functions, and unified for the existing ones. The example notebook is great in illustrating the main usage of AzureSearch, adding the new methods would only dilute the main content. - [x] Lint and test --------- Co-authored-by: Oleksii Pokotylo <oleksii.pokotylo@pwc.com> Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com> Co-authored-by: Bagatur <baskaryan@gmail.com>	4 months ago
SaschaStoll	709664a079	community[patch]: Performant filter columns option for Hanavector (#21971 ) Description: Backwards compatible extension of the initialisation interface of HanaDB to allow the user to specify specific_metadata_columns that are used for metadata storage of selected keys which yields increased filter performance. Any not-mentioned metadata remains in the general metadata column as part of a JSON string. Furthermore switched to executemany for batch inserts into HanaDB. Issue: N/A Dependencies: no new dependencies added Twitter handle: @sapopensource --------- Co-authored-by: Martin Kolb <martin.kolb@sap.com> Co-authored-by: Bagatur <baskaryan@gmail.com>	4 months ago
Eric Zhang	e7e41eaabe	langchain: add RankLLM Reranker (#21171 ) Integrate RankLLM reranker (https://github.com/castorini/rank_llm) into LangChain An example notebook is given in `docs/docs/integrations/retrievers/rankllm-reranker.ipynb` --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	4 months ago
maang-h	fc93bed8c4	community: Fix CSVLoader columns is None (#20701 ) - Bug code: In langchain_community/document_loaders/csv_loader.py:100 - Description: currently, when 'CSVLoader' reads the column as None in the 'csv' file, it will report an error because the 'CSVLoader' does not verify whether the column is of str type and does not consider how to handle the corresponding 'row_data' when the column is' None 'in the csv. This pr provides a solution. - Issue: Fix #20699 - thinking: 1. Refer to the processing method for 'langchain_community/document_loaders/csv_loader.py:100' when 'v' equals'None', and apply the same method to 'k'. (Reference`csv.DictReader` ,'k' will only be None when ` len(columns) < len(number_row_data)` is established) 2. ‘k’ equals None only holds when it is the last column, and its corresponding 'v' type is a list. Therefore, I referred to the data format in 'Document' and used ',' to concatenated the elements in the list.(But I'm not sure if you accept this form, if you have any other ideas, communicate) --------- Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com>	4 months ago
Sihan Chen	1f81277b9b	community[minor]: allow enabling proxy in aiohttp session in AsyncHTML (#19499 ) Allow enabling proxy in aiohttp session async html	4 months ago
Eugene Yurtsev	36813d2f00	community[patch]: Fix remaining __inits__ in community (#22037 ) Fixes the __init__ files in community to use __all__ which is statically defined.	4 months ago
Eugene Yurtsev	58360a1e53	community[patch]: Add unit test to verify that init is correctly defined (#22030 ) Fix some __init__ files and add a unit test	4 months ago
Matthew Hoffman	4f2e3bd7fd	community[patch]: fix public interface for embeddings module (#21650 ) ## Description The existing public interface for `langchain_community.emeddings` is broken. In this file, `__all__` is statically defined, but is subsequently overwritten with a dynamic expression, which type checkers like pyright do not support. pyright actually gives the following diagnostic on the line I am requesting we remove: [reportUnsupportedDunderAll](https://github.com/microsoft/pyright/blob/main/docs/configuration.md#reportUnsupportedDunderAll): ``` Operation on "__all__" is not supported, so exported symbol list may be incorrect ``` Currently, I get the following errors when attempting to use publicablly exported classes in `langchain_community.emeddings`: ```python import langchain_community.embeddings langchain_community.embeddings.HuggingFaceEmbeddings(...) # error: "HuggingFaceEmbeddings" is not exported from module "langchain_community.embeddings" (reportPrivateImportUsage) ``` This is solved easily by removing the dynamic expression.	4 months ago
Eugene Yurtsev	8d82160a8a	community[patch]: Clean up logic in import checking unit test (#22026 ) Clean up unit test	4 months ago
Tomaz Bratanic	d8a1f1114d	community[patch]: Handle exceptions where node props aren't consistent in neo4j schema (#22027 )	4 months ago
WeichenXu	b0ef5e778a	community[patch]: Fix ChatDatabricsk in case that streaming response doesn't have role field in delta chunk (#21897 ) Thank you for contributing to LangChain! - [X] PR title: "package: description" - Where "package" is whichever of langchain, community, core, experimental, etc. is being modified. Use "docs: ..." for purely docs changes, "templates: ..." for template changes, "infra: ..." for CI changes. - Example: "community: add foobar LLM" Description: Fix ChatDatabricsk in case that streaming response doesn't have role field in delta chunk - [ ] Add tests and docs: If you're adding a new integration, please include 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. It lives in `docs/docs/integrations` directory. - [X] Lint and test: Run `make format`, `make lint` and `make test` from the root of the package(s) you've modified. See contribution guidelines for more: https://python.langchain.com/docs/contributing/ Additional guidelines: - Make sure optional dependencies are imported within a function. - Please do not add dependencies to pyproject.toml files (even optional ones) unless they are required for unit tests. - Most PRs should not touch more than one package. - Changes should be backwards compatible. - If you are adding something to community, do not re-import it in langchain. If no one reviews your PR within a few days, please @-mention one of baskaryan, efriis, eyurtsev, hwchase17. --------- Signed-off-by: Weichen Xu <weichen.xu@databricks.com>	4 months ago
Eugene Yurtsev	aed64daabb	community[patch]: Add unit test to catch bad __all__ definitions (#21996 ) This will catch all dynamic __all__ definitions.	4 months ago
Kefan You	24b5c27bb1	community[patch]: raise_for_status logic missing in async _fetch of WebBaseLoader (#21948 ) ## 'raise_for_status' parameter of WebBaseLoader works in sync load but not in async load. In webBaseLoader: Sync load is calling `_scrape` and has `raise_for_status` properly handled. ``` def _scrape( self, url: str, parser: Union[str, None] = None, bs_kwargs: Optional[dict] = None, ) -> Any: from bs4 import BeautifulSoup if parser is None: if url.endswith(".xml"): parser = "xml" else: parser = self.default_parser self._check_parser(parser) html_doc = self.session.get(url, self.requests_kwargs) if self.raise_for_status: html_doc.raise_for_status() if self.encoding is not None: html_doc.encoding = self.encoding elif self.autoset_encoding: html_doc.encoding = html_doc.apparent_encoding return BeautifulSoup(html_doc.text, parser, (bs_kwargs or {})) ``` Async load is calling `_fetch` but missing `raise_for_status` logic. ``` async def _fetch( self, url: str, retries: int = 3, cooldown: int = 2, backoff: float = 1.5 ) -> str: async with aiohttp.ClientSession() as session: for i in range(retries): try: async with session.get( url, headers=self.session.headers, ssl=None if self.session.verify else False, cookies=self.session.cookies.get_dict(), ) as response: return await response.text() ``` Co-authored-by: kefan.you <darkfss@sina.com>	4 months ago
Pengcheng Liu	4cf523949a	community[patch]: Update model client to support vision model in Tong… (#21474 ) - Description: Tongyi uses different client for chat model and vision model. This PR chooses proper client based on model name to support both chat model and vision model. Reference [tongyi document](https://help.aliyun.com/zh/dashscope/developer-reference/tongyi-qianwen-vl-plus-api?spm=a2c4g.11186623.0.0.27404c9a7upm11) for details. ``` from langchain_core.messages import HumanMessage from langchain_community.chat_models import ChatTongyi llm = ChatTongyi(model_name='qwen-vl-max') image_message = { "image": "https://lilianweng.github.io/posts/2023-06-23-agent/agent-overview.png" } text_message = { "text": "summarize this picture", } message = HumanMessage(content=[text_message, image_message]) llm.invoke([message]) ``` - Issue: None - Dependencies: None - Twitter handle: None	4 months ago
Sevin F. Varoglu	1bc0ea5496	community[patch]: update OctoAIEmbeddings to subclass OpenAIEmbeddings (#21805 )	4 months ago
Bagatur	72d4a8eeed	community[patch]: AzureSearch dont overwrite default async (#21989 )	4 months ago
Yulong Wang	8e1aeb8ad5	community[patch]: Fix typo in arxiv tool's doc (#21970 ) Fix typo in arxiv tool's doc	4 months ago
Robert Caulk	54adcd9e82	community[minor]: add AskNews retriever and AskNews tool (#21581 ) We add a tool and retriever for the [AskNews](https://asknews.app) platform with example notebooks. The retriever can be invoked with: ```py from langchain_community.retrievers import AskNewsRetriever retriever = AskNewsRetriever(k=3) retriever.invoke("impact of fed policy on the tech sector") ``` To retrieve 3 documents in then news related to fed policy impacts on the tech sector. The included notebook also includes deeper details about controlling filters such as category and time, as well as including the retriever in a chain. The tool is quite interesting, as it allows the agent to decide how to obtain the news by forming a query and deciding how far back in time to look for the news: ```py from langchain_community.tools.asknews import AskNewsSearch from langchain import hub from langchain.agents import AgentExecutor, create_openai_functions_agent from langchain_openai import ChatOpenAI tool = AskNewsSearch() instructions = """You are an assistant.""" base_prompt = hub.pull("langchain-ai/openai-functions-template") prompt = base_prompt.partial(instructions=instructions) llm = ChatOpenAI(temperature=0) asknews_tool = AskNewsSearch() tools = [asknews_tool] agent = create_openai_functions_agent(llm, tools, prompt) agent_executor = AgentExecutor( agent=agent, tools=tools, verbose=True, ) agent_executor.invoke({"input": "How is the tech sector being affected by fed policy?"}) ``` --------- Co-authored-by: Emre <e@emre.pm>	4 months ago
Jesse S	fc79b372cb	community[minor]: add aerospike vectorstore integration (#21735 ) Please let me know if you see any possible areas of improvement. I would very much appreciate your constructive criticism if time allows. Description: - Added a aerospike vector store integration that utilizes [Aerospike-Vector-Search](https://aerospike.com/products/vector-database-search-llm/) add-on. - Added both unit tests and integration tests - Added a docker compose file for spinning up a test environment - Added a notebook Dependencies: any dependencies required for this change - aerospike-vector-search Twitter handle: - No twitter, you can use my GitHub handle or LinkedIn if you'd like Thanks! --------- Co-authored-by: Jesse Schumacher <jschumacher@aerospike.com> Co-authored-by: Bagatur <baskaryan@gmail.com>	4 months ago
Prince Canuma	3587c60396	community[patch]: Fix MLX LLM Stream (#20575 ) Closes #20561 This PR fixes MLX LLM stream `AttributeError`. Recently, `mlx-lm` changed the token decoding logic, which affected the LC+MLX integration. Additionally, I made minor fixes such as: docs example broken link and enforcing pipeline arguments (max_tokens, temp and etc) for invoke. - Issue: #20561 - Twitter handle: @Prince_Canuma	4 months ago
Rahul Triptahi	96bd0b0844	community[patch]: Remove redundant pebblo cloud api call (#21589 ) Description: removed redundant pebblo cloud api call. Changed classified `doc` key to `ai_apps_data`. Documentation: N/A Unit tests: N/A	4 months ago
Param Singh	d07885f8b7	community[patch]: standardized sparkllm init args (#21633 ) Related to #20085 @baskaryan Thank you for contributing to LangChain! community:sparkllm[patch]: standardized init args updated `spark_api_key` so that aliased to `api_key`. Added integration test for `sparkllm` to test that it continues to set the same underlying attribute. updated temperature with Pydantic Field, added to the integration test. Ran `make format`,`make test`, `make lint`, `make spell_check`	4 months ago
Dhruv Chawla	d4359d3de6	community[patch]: Update UpTrain Callback Handler to support the new UpTrain evaluation schema (#21656 ) UpTrain has a new dashboard now that makes it easier to view projects and evaluations. Using this requires specifying both project_name and evaluation_name when performing evaluations. I have updated the code to support it.	4 months ago
Alex Riina	c0e3c3a350	openai[patch], community[patch]: add pricing and max context window for GPT-4o (#21673 ) # Add pricing and max context window for GPT-4o - community: add cost per 1k tokens and max context window - partners: add max context window Description: adds static information about GPT-4o based on https://openai.com/api/pricing/ and https://platform.openai.com/docs/models/gpt-4o so that GPT-4o reporting is accurate. --------- Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com> Co-authored-by: Bagatur <baskaryan@gmail.com>	4 months ago
缨缨	bd39b2ccdf	community: enable SupabaseVectorStore to support extended table fields (#21762 ) Thank you for contributing to LangChain! - [x] PR title: "community: enable SupabaseVectorStore to support extended table fields" - [x] PR message: - Added extension fields to the function _add_vectors so that users can add other custom fields when insert a record into the database. eg: ![image](https://github.com/langchain-ai/langchain/assets/10885578/e1d5ca20-936e-4cab-ba69-8fdd23b8ce8f) --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	4 months ago
Jens	b0b302ec6b	community[patch]: fixed aleph alpha default emedding request (#21826 ) - Description: In the aleph alpha client the paramater `normalize` is not optional. Setting this to `None` gives an error. - Dependencies: None Co-authored-by: Jens Lücke <jens.luecke@tngtech.com> Co-authored-by: Jens <jens.luecke@hu-berlin.de> Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com>	4 months ago
Jorge Piedrahita Ortiz	e6207ad4f3	community[patch]: Sambanova integration api update (#21848 ) - Description:: SambaStudio generic endpoint compatibility added Improved error description, and handling streaming examples added	4 months ago
Liuww	332ffed393	community[patch]: Adopting the lighter-weight xinference_client (#21900 ) While integrating the xinference_embedding, we observed that the downloaded dependency package is quite substantial in size. With a focus on resource optimization and efficiency, if the project requirements are limited to its vector processing capabilities, we recommend migrating to the xinference_client package. This package is more streamlined, significantly reducing the storage space requirements of the project and maintaining a feature focus, making it particularly suitable for scenarios that demand lightweight integration. Such an approach not only boosts deployment efficiency but also enhances the application's maintainability, rendering it an optimal choice for our current context. --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	4 months ago
Jiří Spilka	6499897c87	community[patch]: update apify integration to attribute API activity to langchain (#21909 ) Description: Add `Origin/langchain` to Apify's client's user-agent to attribute API activity to LangChain (at Apify, we aim to monitor our integrations to evaluate whether we should invest more in the LangChain integration regarding functionality and content) Issue: None Dependencies: None Twitter handle: None	4 months ago
Tomaz Bratanic	d85e46321a	community[patch]: Better error message for neo4j vector when text is null (#21861 )	4 months ago
WilliamEspegren	30bca57aae	doc list not empty (#21208 ) Make sure the doc list is not empty, and set Metadata: true in param, to enable the user to disable metadata for slightly faster crawls.	4 months ago
TJ	8cd6ed3e1e	community[patch]: Update documentation string in databricks chat model (#21915 ) Update typos in documentation string in databricks chat model	4 months ago
Erick Friis	169f525cfb	community: release 0.2.0 (#21834 )	4 months ago
Eugene Yurtsev	e3f30b4cde	docs: clean up link to bing search (#21825 ) Documentation should be inlined, not linking to medium article.	4 months ago
Sen Lin	eb7f07ae36	community[patch]: fix typo in ValueError message in load_local function (#21818 ) Description: Corrected an error in the `allow_dangerous_deserialization` message within the `load_local` functions	4 months ago
Jorge Piedrahita Ortiz	700b1c7212	community: sambaverse api update (#21816 ) - Description: fix sambaverse integration to make it compatible with sambaverse API update / minor changes in docs	4 months ago
maang-h	9f8d18c028	community[patch]: Fix unintended newline in print statement in exception for BaichuanTextEmbeddings (#21820 ) - Code: langchain_community/embeddings/baichuan.py:82 - Description: When I make an error using 'baichuan embeddings', the printed error message is wrapped (there is actually no need to wrap) ```python # example from langchain_community.embeddings import BaichuanTextEmbeddings # error key BAICHUAN_API_KEY = "sk-xxxxxxxxxxxxx" embeddings = BaichuanTextEmbeddings(baichuan_api_key=BAICHUAN_API_KEY) text_1 = "今天天气不错" query_result = embeddings.embed_query(text_1) ``` ![unintended newline](https://github.com/langchain-ai/langchain/assets/55082429/e1178ce8-62bb-405d-a4af-e3b28eabc158)	4 months ago
Eugene Yurtsev	8607735b80	langchain[patch],community[patch]: Move unit tests that depend on community to community (#21685 )	4 months ago
Marco Lamina	d0fae6cd54	community: Add token cost for GPT-4o model (#21771 ) Adding [token cost for the new GPT-4o model](https://openai.com/api/pricing/): * Input cost US$5.00 / 1M tokens * Output cost US$15.00 / 1M tokens	4 months ago
Massimiliano Pronesti	0c0db7c5db	feat(community): support semantic hybrid score threshold in Azure AI Search (#21527 ) Support semantic hybrid search with a score threshold -- similar to what we do for similarity search and for hybrid search (#20907).	4 months ago
Stefano Lottini	040597e832	community: init signature revision for Cassandra LLM cache classes + small maintenance (#17765 ) This PR improves on the `CassandraCache` and `CassandraSemanticCache` classes, mainly in the constructor signature, and also introduces several minor improvements around these classes. ### Init signature A (sigh) breaking change is tentatively introduced to the constructor. To me, the advantages outweigh the possible discomfort: the new syntax places the DB-connection objects `session` and `keyspace` later in the param list, so that they can be given a default value. This is what enables the pattern of _not_ specifying them, provided one has previously initialized the Cassandra connection through the versatile utility method `cassio.init(...)`. In this way, a much less unwieldy instantiation can be done, such as `CassandraCache()` and `CassandraSemanticCache(embedding=xyz)`, everything else falling back to defaults. A downside is that, compared to the earlier signature, this might turn out to be breaking for those doing positional instantiation. As a way to mitigate this problem, this PR typechecks its first argument trying to detect the legacy usage. (And to make this point less tricky in the future, most arguments are left to be keyword-only). If this is considered too harsh, I'd like guidance on how to further smoothen this transition. Our plan is to make the pattern of optional session/keyspace a standard across all Cassandra classes, so that a repeatable strategy would be ideal. A possibility would be to keep positional arguments for legacy reasons but issue a deprecation warning if any of them is actually used, to later remove them with 0.2 - please advise on this point. ### Other changes - class docstrings: enriched, completely moved to class level, added note on `cassio.init(...)` pattern, added tiny sample usage code. - semantic cache: revised terminology to never mention "distance" (it is in fact a similarity!). Kept the legacy constructor param with a deprecation warning if used. - `llm_caching` notebook: uniform flow with the Cassandra and Astra DB separate cases; better and Cassandra-first description; all imports made explicit and from community where appropriate. - cache integration tests moved to community (incl. the imported tools), env var bugfix for `CASSANDRA_CONTACT_POINTS`. --------- Co-authored-by: Erick Friis <erick@langchain.dev>	4 months ago
Kyle Cassidy	eca8c4bcc6	Standardized openai init params (#21739 ) ## Patch Summary community:openai[patch]: standardize init args ## Details I made changes to the OpenAI Chat API wrapper test in the Langchain open-source repository - File: `libs/community/tests/unit_tests/chat_models/test_openai.py` - Changes: - Updated `max_retries` with Pydantic Field - Updated the corresponding unit test - Related Issues: #20085 - Updated max_retries with Pydantic Field, updated the unit test. --------- Co-authored-by: JuHyung Son <sonju0427@gmail.com>	4 months ago
Ethan Yang	e44b448ec3	community: update openvino doc with streaming support (#21519 ) Co-authored-by: Chester Curme <chester.curme@gmail.com>	4 months ago
ccurme	19e6bf814b	community: fix CI (#21766 )	4 months ago
Mish Ushakov	d77e60a7f4	community: updated Browserbase loader (#21757 ) Thank you for contributing to LangChain! - [x] PR title: "community: updated Browserbase loader" - [x] PR message: Updates the Browserbase loader with more options and improved docs. - [x] Lint and test: Run `make format`, `make lint` and `make test` from the root of the package(s) you've modified. See contribution guidelines for more: https://python.langchain.com/docs/contributing/	4 months ago
Cheese	0ead09f84d	community: Implement `bind_tools` for ChatTongyi (#20725 ) ## Description Implement `bind_tools` in ChatTongyi. Usage example: ```py from langchain_core.tools import tool from langchain_community.chat_models.tongyi import ChatTongyi @tool def multiply(first_int: int, second_int: int) -> int: """Multiply two integers together.""" return first_int * second_int llm = ChatTongyi(model="qwen-turbo") llm_with_tools = llm.bind_tools([multiply]) msg = llm_with_tools.invoke("What's 5 times forty two") print(msg) ``` Streaming is also supported. ## Dependencies No Dependency is required for this change. --------- Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com> Co-authored-by: Chester Curme <chester.curme@gmail.com>	4 months ago
Harrison Chase	15be439719	Harrison/move flashrank rerank (#21448 ) third party integration, should be in community	4 months ago
Rajendra Kadam	54e003268e	langchain[minor]: Add PebbloRetrievalQA chain with Identity & Semantic Enforcement support (#20641 ) - Description: PebbloRetrievalQA chain introduces identity enforcement using vector-db metadata filtering - Dependencies: None - Issue: None - Documentation: Adding documentation for PebbloRetrievalQA chain in a separate PR(https://github.com/langchain-ai/langchain/pull/20746) - Unit tests: New unit-tests added --------- Co-authored-by: Eugene Yurtsev <eugene@langchain.dev>	4 months ago
Erick Friis	c77d2f2b06	multiple: core 0.2 nonbreaking dep, check_diff community->langchain dep (#21646 ) 0.2 is not a breaking release for core (but it is for langchain and community) To keep the core+langchain+community packages in sync at 0.2, we will relax deps throughout the ecosystem to tolerate `langchain-core` 0.2	4 months ago
Anush	edd68e4ad4	qdrant: init package (#21146 ) ## Description This PR introduces the new `langchain-qdrant` partner package, intending to deprecate the community package. ## Changes - Moved the Qdrant vector store implementation `/libs/partners/qdrant` with integration tests. - The conditional imports of the client library are now regular with minor implementation improvements. - Added a deprecation warning to `langchain_community.vectorstores.qdrant.Qdrant`. - Replaced references/imports from `langchain_community` with either `langchain_core` or by moving the definitions to the `langchain_qdrant` package itself. - Updated the Qdrant vector store documentation to reflect the changes. ## Testing - `QDRANT_URL` and [`QDRANT_API_KEY`](`583e36bf6b`) env values need to be set to [run integration tests](`d608c93d1f`) in the [cloud](https://cloud.qdrant.tech). - If a Qdrant instance is running at `http://localhost:6333`, the integration tests will use it too. - By default, tests use an [`in-memory`](https://github.com/qdrant/qdrant-client?tab=readme-ov-file#local-mode) instance(Not comprehensive). --------- Co-authored-by: Erick Friis <erick@langchain.dev> Co-authored-by: Erick Friis <erickfriis@gmail.com>	4 months ago
Prashanth Rao	63c3a0e56c	[community][graph]: Update KuzuQAChain and docs (#21218 ) This PR makes some small updates for `KuzuQAChain` for graph QA. - Updated Cypher generation prompt (we now support `WHERE EXISTS`) and generalize it more - Support different LLMs for Cypher generation and QA - Update docs and examples	4 months ago
Jofthomas	afd85b60fc	huggingface: init package (#21097 ) First Pr for the langchain_huggingface partner Package - Moved some of the hugging face related class from `community` to the new `partner package` Still needed : - Documentation - Tests - Support for the new apply_chat_template in `ChatHuggingFace` - Confirm choice of class to support for embeddings witht he sentence-transformer team. cc : @efriis --------- Co-authored-by: Cyril Kondratenko <kkn1993@gmail.com> Co-authored-by: Erick Friis <erick@langchain.dev>	4 months ago
Tomaz Bratanic	9fce03e7db	community[patch]: Fix neo4j enhanced schema (#21582 )	4 months ago
Christophe Bornet	66a4da8ad0	community[patch]: Improve Cassandra VectorStore docsctrings (#21620 )	4 months ago
Eugene Yurtsev	25fbe356b4	community[patch]: upgrade to recent version of mypy (#21616 ) This PR upgrades community to a recent version of mypy. It inserts type: ignore on all existing failures.	4 months ago
Jorge Piedrahita Ortiz	4378fbbef0	community[patch]: Fix typos in Sambanova integration doc-strings (#21617 ) - Description: Sambanova integration docstrings updated, bad formated --------- Co-authored-by: Eugene Yurtsev <eugene@langchain.dev>	4 months ago
Christophe Bornet	bcf53f93e1	[community]: Add missing docstring param to CassandraLoader (#21611 )	4 months ago
Christophe Bornet	e6fa4547b1	community[minor]: Add alazy_load to AsyncHtmlLoader (#21536 ) Also fixes a bug that `_scrape` was called and was doing a second HTTP request synchronously. Twitter handle: cbornet_	4 months ago
Leonid Ganeline	500569da48	community[patch]: `vectorstores` import update (#21169 ) Issue: we have several helper functions to import third-party libraries like lancedb.import_lancedb in [community.vectorstores](https://api.python.langchain.com/en/latest/vectorstores/langchain_community.vectorstores.lancedb.import_lancedb.html#langchain_community.vectorstores.lancedb.import_lancedb). And we have core.utils.utils.guard_import that works exactly for this purpose. The import_<package> functions work inconsistently and rather be private functions. Change: replaced these functions with the guard_import function. Related to #21133	4 months ago
ccurme	3003363605	langchain, community: remove cap on sqlalchemy and bump duckdb (#21509 )	4 months ago
Erick Friis	3db85cbb5b	community: deps (#21508 )	5 months ago
ccurme	375f447e58	community: fix builds with min dependencies (#21495 )	5 months ago
ccurme	3bb9bec314	bedrock: add unit test for retriever (#21485 ) This was implemented in https://github.com/langchain-ai/langchain/pull/21349 but dropped before merge.	5 months ago
Renu Rozera	4035a1d234	Add source metadata to bedrock retriever response (#21349 ) Thank you for contributing to LangChain! - [X] PR title: "community: Add source metadata to bedrock retriever response" - [X] PR message: - Description: Bedrock retrieve API returns extra metadata in the response which is currently not returned in the retriever response - Issue: The change adds the metadata from bedrock retrieve API response to the bedrock retriever in a backward compatible way. Renamed metadata to sourceMetadata as metadata term is being used in the Document already. This is in sync with what we are doing in llama-index as well. - Dependencies: No - [X] Add tests and docs: 1. Added unit tests 2. Notebook already exists and does not need any change 3. Response from end to end testing, just to ensure backward compatibility: `[Document(page_content='Exoplanets.', metadata={'location': {'s3Location': {'uri': 's3://bucket/file_name.txt'}, 'type': 'S3'}, 'score': 0.46886647, 'source_metadata': {'x-amz-bedrock-kb-source-uri': 's3://bucket/file_name.txt', 'tag': 'space', 'team': 'Nasa', 'year': 1946.0}})]` - [X] Lint and test: Run `make format`, `make lint` and `make test` from the root of the package(s) you've modified. See contribution guidelines for more: https://python.langchain.com/docs/contributing/ Additional guidelines: - Make sure optional dependencies are imported within a function. - Please do not add dependencies to pyproject.toml files (even optional ones) unless they are required for unit tests. - Most PRs should not touch more than one package. - Changes should be backwards compatible. - If you are adding something to community, do not re-import it in langchain. If no one reviews your PR within a few days, please @-mention one of baskaryan, efriis, eyurtsev, hwchase17. --------- Co-authored-by: Piyush Jain <piyushjain@duck.com>	5 months ago
Erick Friis	f178c67ad0	community: release 0.2.0rc1, bump deps (#21470 )	5 months ago
roiperlman	9992beaff9	community: Add arguments to whisper parser (#20378 ) Description: Added a few additional arguments to the whisper parser, which can be consumed by the underlying API. The prompt is especially important to fine-tune transcriptions. --------- Co-authored-by: Roi Perlman <roi@fivesigmalabs.com> Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com> Co-authored-by: Bagatur <baskaryan@gmail.com>	5 months ago
Yash	cb31c3611f	Ndb enterprise (#21233 ) Description: Adds NeuralDBClientVectorStore to the langchain, which is our enterprise client. --------- Co-authored-by: kartikTAI <129414343+kartikTAI@users.noreply.github.com> Co-authored-by: Kartik Sarangmath <kartik@thirdai.com>	5 months ago
Oguz Vuruskaner	5b35f077f9	[community][fix](DeepInfraEmbeddings): Implement chunking for large batches (#21189 ) Description: This PR introduces chunking logic to the `DeepInfraEmbeddings` class to handle large batch sizes without exceeding maximum batch size of the backend. This enhancement ensures that embedding generation processes large batches by breaking them down into smaller, manageable chunks, each conforming to the maximum batch size limit. Issue: Fixes #21189 Dependencies: No new dependencies introduced.	5 months ago
Sokolov Fedor	f4ddf64faa	community: Add MarkdownifyTransformer to langchain_community.document_transformers (#21247 ) - Added new document_transformer: MarkdonifyTransformer, that uses `markdonify` package with customizable options to convert HTML to Markdown. It's similar to Html2TextTransformer, but has more flexible options and also I've noticed that sometimes MarkdownifyTransformer performs better than html2text one, so that's why I use markdownify on my project. - Added docs and tests - Usage: ```python from langchain_community.document_transformers import MarkdownifyTransformer markdownify = MarkdownifyTransformer() docs_transform = markdownify.transform_documents(docs) ``` - Example of better performance on simple task, that I've noticed: ``` <html> <head><title>Reports on product movement</title></head> <body> <p data-block-key="2wst7">The reports on product movement will be useful for forming supplier orders and controlling outcomes.</p> </body> ``` Html2TextTransformer: ```python [Document(page_content='The reports on product movement will be useful for forming supplier orders and\ncontrolling outcomes.\n\n')] # Here we can see 'and\ncontrolling', which has extra '\n' in it ``` MarkdownifyTranformer: ```python [Document(page_content='Reports on product movement\n\nThe reports on product movement will be useful for forming supplier orders and controlling outcomes.')] ``` --------- Co-authored-by: Sokolov Fedor <f.sokolov@sokolov-macbook.bbrouter> Co-authored-by: Harrison Chase <hw.chase.17@gmail.com> Co-authored-by: Sokolov Fedor <f.sokolov@sokolov-macbook.local> Co-authored-by: Sokolov Fedor <f.sokolov@192.168.1.6>	5 months ago
Alex JW	d3ce6aad2e	community: Instantiate GPT4AllEmbeddings with parameters (#21238 ) ### GPT4AllEmbeddings parameters --- Description: As of right now the Embed4All class inside _GPT4AllEmbeddings_ is instantiated as it's default which leaves no room to customize the chosen model and it's behavior. Thus: - GPT4AllEmbeddings can now be instantiated with custom parameters like a different model that shall be used. --------- Co-authored-by: AlexJauchWalser <alexander.jauch-walser@knime.com>	5 months ago
Philippe PRADOS	7be68228da	community[patch]: Make sql record manager fully compatible with async (#20735 ) The `_amake_session()` method does not allow modifying the `self.session_factory` with anything other than `async_sessionmaker`. This prohibits advanced uses of `index()`. In a RAG architecture, it is necessary to import document chunks. To keep track of the links between chunks and documents, we can use the `index()` API. This API proposes to use an SQL-type record manager. In a classic use case, using `SQLRecordManager` and a vector database, it is impossible to guarantee the consistency of the import. Indeed, if a crash occurs during the import (problem with the network, ...) there is an inconsistency between the SQL database and the vector database. With the [PR](https://github.com/langchain-ai/langchain-postgres/pull/32) we are proposing for `langchain-postgres`, it is now possible to guarantee the consistency of the import of chunks into a vector database. It's possible only if the outer session is built with the connection. ```python def main(): db_url = "postgresql+psycopg://postgres:password_postgres@localhost:5432/" engine = create_engine(db_url, echo=True) embeddings = FakeEmbeddings() pgvector:VectorStore = PGVector( embeddings=embeddings, connection=engine, ) record_manager = SQLRecordManager( namespace="namespace", engine=engine, ) record_manager.create_schema() with engine.connect() as connection: session_maker = scoped_session(sessionmaker(bind=connection)) # NOTE: Update session_factories record_manager.session_factory = session_maker pgvector.session_maker = session_maker with connection.begin(): loader = CSVLoader( "data/faq/faq.csv", source_column="source", autodetect_encoding=True, ) result = index( source_id_key="source", docs_source=loader.load()[:1], cleanup="incremental", vector_store=pgvector, record_manager=record_manager, ) print(result) ``` The same thing is possible asynchronously, but a bug in `sql_record_manager.py` in `_amake_session()` must first be fixed. ```python async def _amake_session(self) -> AsyncGenerator[AsyncSession, None]: """Create a session and close it after use.""" # FIXME: REMOVE if not isinstance(self.session_factory, async_sessionmaker):~~ if not isinstance(self.engine, AsyncEngine): raise AssertionError("This method is not supported for sync engines.") async with self.session_factory() as session: yield session ``` Then, it is possible to do the same thing asynchronously: ```python async def main(): db_url = "postgresql+psycopg://postgres:password_postgres@localhost:5432/" engine = create_async_engine(db_url, echo=True) embeddings = FakeEmbeddings() pgvector:VectorStore = PGVector( embeddings=embeddings, connection=engine, ) record_manager = SQLRecordManager( namespace="namespace", engine=engine, async_mode=True, ) await record_manager.acreate_schema() async with engine.connect() as connection: session_maker = async_scoped_session( async_sessionmaker(bind=connection), scopefunc=current_task) record_manager.session_factory = session_maker pgvector.session_maker = session_maker async with connection.begin(): loader = CSVLoader( "data/faq/faq.csv", source_column="source", autodetect_encoding=True, ) result = await aindex( source_id_key="source", docs_source=loader.load()[:1], cleanup="incremental", vector_store=pgvector, record_manager=record_manager, ) print(result) asyncio.run(main()) ``` --------- Signed-off-by: Rahul Tripathi <rauhl.psit.ec@gmail.com> Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com> Co-authored-by: Sean <sean@upstage.ai> Co-authored-by: JuHyung-Son <sonju0427@gmail.com> Co-authored-by: Erick Friis <erick@langchain.dev> Co-authored-by: YISH <mokeyish@hotmail.com> Co-authored-by: Bagatur <baskaryan@gmail.com> Co-authored-by: Jason_Chen <820542443@qq.com> Co-authored-by: Joan Fontanals <joan.fontanals.martinez@jina.ai> Co-authored-by: Pavlo Paliychuk <pavlo.paliychuk.ca@gmail.com> Co-authored-by: fzowl <160063452+fzowl@users.noreply.github.com> Co-authored-by: samanhappy <samanhappy@gmail.com> Co-authored-by: Lei Zhang <zhanglei@apache.org> Co-authored-by: Tomaz Bratanic <bratanic.tomaz@gmail.com> Co-authored-by: merdan <48309329+merdan-9@users.noreply.github.com> Co-authored-by: ccurme <chester.curme@gmail.com> Co-authored-by: Andres Algaba <andresalgaba@gmail.com> Co-authored-by: davidefantiniIntel <115252273+davidefantiniIntel@users.noreply.github.com> Co-authored-by: Jingpan Xiong <71321890+klaus-xiong@users.noreply.github.com> Co-authored-by: kaka <kaka@zbyte-inc.cloud> Co-authored-by: jingsi <jingsi@leadincloud.com> Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com> Co-authored-by: Rahul Triptahi <rahul.psit.ec@gmail.com> Co-authored-by: Rahul Tripathi <rauhl.psit.ec@gmail.com> Co-authored-by: Shengsheng Huang <shannie.huang@gmail.com> Co-authored-by: Michael Schock <mjschock@users.noreply.github.com> Co-authored-by: Anish Chakraborty <anish749@users.noreply.github.com> Co-authored-by: am-kinetica <85610855+am-kinetica@users.noreply.github.com> Co-authored-by: Dristy Srivastava <58721149+dristysrivastava@users.noreply.github.com> Co-authored-by: Matt <matthew.gotteiner@microsoft.com> Co-authored-by: William FH <13333726+hinthornw@users.noreply.github.com>	5 months ago
Andreas Motl	17e42bbd18	community[patch]: pgvector: Slight refactoring to make code a bit more reusable (#16243 ) - Description: Improve [pgvector vector store adapter](https://github.com/langchain-ai/langchain/blob/v0.1.1/libs/community/langchain_community/vectorstores/pgvector.py) to make it reusable by adapters deriving from that. - Issue: NA - Dependencies: NA - References: https://github.com/crate-workbench/langchain/pull/1 - Addressed to: @eyurtsev, @cbornet Hi from the CrateDB team, first of all, thanks a stack for conceiving and maintaining LangChain. We are currently [preparing a patch](https://github.com/crate-workbench/langchain/pull/1) for adding [CrateDB](https://github.com/crate/crate) to the list of community adapters. Because CrateDB aims to be compatible with PostgreSQL to some degree, the vector store subsystem in LangChain derives functionality from the corresponding implementation for pgvector. Therefore, in order to make the implementation more reusable, we needed to rename the private methods `__from` and `__query_collection` to the less private counterparts `_from` and `_query_collection`, so they can be overwritten, in order to unlock other adapters deriving from [pgvector](https://github.com/langchain-ai/langchain/blob/v0.1.1/libs/community/langchain_community/vectorstores/pgvector.py). With kind regards, Andreas.	5 months ago
Mehrdad Shokri	f103927b88	bugfix(community): fix Playwright import paths. (#21395 ) - Description: Fix import class name exporeted from 'playwright.async_api' and 'playwright.sync_api' to match the correct name in playwright tool. Change import from inline guard_import to helper function that calls guard_import to make code more readable in gmail tool. Upgrade playwright version to 1.43.0 - Issue: #21354 - Dependencies: upgrade playwright version(this is not required for the bugfix itself, just trying to keep dependencies fresh. I can remove the playwright version upgrade if you want.)	5 months ago
Shailendra Mishra	aa966b6161	Replaced bind variable in SQL with formatted string for compatibility with sql syntax. (#21439 ) Thank you for contributing to LangChain! - [ ] PR title: "package: description" - Where "package" is whichever of langchain, community, core, experimental, etc. is being modified. Use "docs: ..." for purely docs changes, "templates: ..." for template changes, "infra: ..." for CI changes. - Example: "community: add foobar LLM" - [ ] PR message: *Delete this entire checklist* and replace with - Description: a description of the change - Issue: the issue # it fixes, if applicable - Dependencies: any dependencies required for this change - Twitter handle: if your PR gets announced, and you'd like a mention, we'll gladly shout you out! - [ ] Add tests and docs: If you're adding a new integration, please include 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. It lives in `docs/docs/integrations` directory. - [ ] Lint and test: Run `make format`, `make lint` and `make test` from the root of the package(s) you've modified. See contribution guidelines for more: https://python.langchain.com/docs/contributing/ Additional guidelines: - Make sure optional dependencies are imported within a function. - Please do not add dependencies to pyproject.toml files (even optional ones) unless they are required for unit tests. - Most PRs should not touch more than one package. - Changes should be backwards compatible. - If you are adding something to community, do not re-import it in langchain. If no one reviews your PR within a few days, please @-mention one of baskaryan, efriis, eyurtsev, hwchase17.	5 months ago
Eugene Yurtsev	f92006de3c	multiple: langchain 0.2 in master (#21191 ) 0.2rc migrations - [x] Move memory - [x] Move remaining retrievers - [x] graph_qa chains - [x] some dependency from evaluation code potentially on math utils - [x] Move openapi chain from `langchain.chains.api.openapi` to `langchain_community.chains.openapi` - [x] Migrate `langchain.chains.ernie_functions` to `langchain_community.chains.ernie_functions` - [x] migrate `langchain/chains/llm_requests.py` to `langchain_community.chains.llm_requests` - [x] Moving `langchain_community.cross_enoders.base:BaseCrossEncoder` -> `langchain_community.retrievers.document_compressors.cross_encoder:BaseCrossEncoder` (namespace not ideal, but it needs to be moved to `langchain` to avoid circular deps) - [x] unit tests langchain -- add pytest.mark.community to some unit tests that will stay in langchain - [x] unit tests community -- move unit tests that depend on community to community - [x] mv integration tests that depend on community to community - [x] mypy checks Other todo - [x] Make deprecation warnings not noisy (need to use warn deprecated and check that things are implemented properly) - [x] Update deprecation messages with timeline for code removal (likely we actually won't be removing things until 0.4 release) -- will give people more time to transition their code. - [ ] Add information to deprecation warning to show users how to migrate their code base using langchain-cli - [ ] Remove any unnecessary requirements in langchain (e.g., is SQLALchemy required?) --------- Co-authored-by: Erick Friis <erick@langchain.dev>	5 months ago
Dobiichi-Origami	5b00885b49	community: add `bind_tools` and `with_structured_output` support to `QianfanChatEndpoint` (#21412 ) …Endpoint` Thank you for contributing to LangChain! - [x] PR title: "package: description" - Where "package" is whichever of langchain, community, core, experimental, etc. is being modified. Use "docs: ..." for purely docs changes, "templates: ..." for template changes, "infra: ..." for CI changes. - Example: "community: add foobar LLM" - [x] PR message: *Delete this entire checklist* and replace with - Description: add `bind_tools` and `with_structured_output` support to `QianfanChatEndpoint` - [x] Lint and test: Run `make format`, `make lint` and `make test` from the root of the package(s) you've modified. See contribution guidelines for more: https://python.langchain.com/docs/contributing/	5 months ago
Leonid Ganeline	791d59a2c8	community: `callbacks` guard_imports (#21173 ) Issue: we have several helper functions to import third-party libraries like import_uptrain in [community.callbacks](https://api.python.langchain.com/en/latest/callbacks/langchain_community.callbacks.uptrain_callback.import_uptrain.html#langchain_community.callbacks.uptrain_callback.import_uptrain). And we have core.utils.utils.guard_import that works exactly for this purpose. The import_<package> functions work inconsistently and rather be private functions. Change: replaced these functions with the guard_import function. Related to #21133	5 months ago
Rahul Triptahi	7994cba18d	[Community][Minor]: Fetch loader_source of GoogleDriveLoader in PebbloSafeLoader. (#21314 ) Description: This PR includes fix for loader_source to be fetched from metadata in case of GdriveLoaders. Documentation: NA Unit Test: NA Signed-off-by: Rahul Tripathi <rauhl.psit.ec@gmail.com> Co-authored-by: Rahul Tripathi <rauhl.psit.ec@gmail.com>	5 months ago
Eugene Yurtsev	6a1d61dbf1	community[patch]: Fix in memory vectorstore to take into account ids when adding docs (#21384 ) Should respect `ids` if passed	5 months ago
Miroslav	04e2611fea	Added additional headers for HuggingFaceInferenceAPIEmbeddings endpoint. (#21282 ) Thank you for contributing to LangChain! - [ ] HuggingFaceInferenceAPIEmbeddings: "Additional Headers" - Where: langchain, community, embeddings. huggingface.py. - Community: add additional headers when needed by custom HuggingFace TEI embedding endpoints. HuggingFaceInferenceAPIEmbeddings" - [ ] PR message: *Delete this entire checklist* and replace with - Description: Adding the `additional_headers` to be passed to requests library if needed - Dependencies: none - [ ] Add tests and docs: If you're adding a new integration, please include 1. Tested with locally available TEI endpoints with and without `additional_headers` 2. Example Usage ```python embeddings=HuggingFaceInferenceAPIEmbeddings( api_key=MY_CUSTOM_API_KEY, api_url=MY_CUSTOM_TEI_URL, additional_headers={ "Content-Type": "application/json" } ) ``` Additional guidelines: - Make sure optional dependencies are imported within a function. - Please do not add dependencies to pyproject.toml files (even optional ones) unless they are required for unit tests. - Most PRs should not touch more than one package. - Changes should be backwards compatible. - If you are adding something to community, do not re-import it in langchain. If no one reviews your PR within a few days, please @-mention one of baskaryan, efriis, eyurtsev, hwchase17. --------- Co-authored-by: Massimiliano Pronesti <massimiliano.pronesti@gmail.com> Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	5 months ago
Guangdong Liu	1fe66f5d39	community(patch) fix MoonshotChat moonshot_api_key is invaild for api key (#21361 ) Description: close https://github.com/langchain-ai/langchain/issues/21237 @baskaryan, @eyurtsev	5 months ago
Wu Enze	32c61b3ece	community[patch]: chat message history mypy fixes #17048 (#20114 ) Relates [#17048] Description : Applied fix to redis and neo4j file. Error was : `Cannot override writeable attribute with read-only property` fix with the same solution of [[langchain/libs/community/langchain_community/chat_message_histories/elasticsearch.py](`d5c412b0a9/libs/community/langchain_community/chat_message_histories/elasticsearch.py (L170-L175)`)] --------- Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com> Co-authored-by: Chester Curme <chester.curme@gmail.com>	5 months ago
nrpd25	95cc8e3fc3	premai[patch]:Standardized model init args (#21308 ) [Standardized model init args #20085](https://github.com/langchain-ai/langchain/issues/20085) - Enable premai chat model to be initialized with `model_name` as an alias for `model`, `api_key` as an alias for `premai_api_key`. - Add initialization test `test_premai_initialization` --------- Co-authored-by: Chester Curme <chester.curme@gmail.com>	5 months ago
Tomaz Bratanic	ac14f171ac	Add indexed properties to neo4j enhanced schema (#21335 )	5 months ago
scaserini	a6cdf6572f	community: add Kendra DocumentRelevanceOverrideConfigurations request parameter (#20695 ) - Description: add DocumentRelevanceOverrideConfigurations request parameter to Kendra retriever Co-authored-by: Simone Caserini <simone.caserini@klarna.com> Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com>	5 months ago
Jorge Piedrahita Ortiz	e65652c3e8	community: add SambaNova embeddings integration (#21227 ) - Description: SambaNova hosted embeddings integration	5 months ago
Jorge Piedrahita Ortiz	df1c10260c	community: minor changes sambanova integration (#21231 ) - Description: fix: variable names in root validator not allowing pass credentials as named parameters in llm instancing, also added sambanova's sambaverse and sambastudio llms to __init__.py for module import	5 months ago
Jan Soubusta	d9a61c0fa9	fix: respect table_name argument when calling from_texts (#21252 ) valid for from_documents() as well fixes #21251	5 months ago
Pedro Lima	bebf46c4a2	community: added args_schema to YahooFinanceNewsTool (#21232 ) Description: this change adds args_schema (pydantic BaseModel) to YahooFinanceNewsTool for correct schema formatting on LLM function calls Issue: currently using YahooFinanceNewsTool with OpenAI function calling returns the following error "TypeError("YahooFinanceNewsTool._run() got an unexpected keyword argument '__arg1'")". This happens because the schema sent to the LLM is "input: "{'__arg1': 'MSFT'}"" while the method should be called with the "query" parameter. Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com>	5 months ago
Mark Cusack	060987d755	community[minor]: Add indexing via locality sensitive hashing to the Yellowbrick vector store (#20856 ) - Description: Add LSH-based indexing to the Yellowbrick vector store module - Twitter handle: @markcusack --------- Co-authored-by: markcusack <markcusack@markcusacksmac.lan> Co-authored-by: markcusack <markcusack@Mark-Cusack-sMac.local> Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com>	5 months ago
Rashmi Pawar	a2fdabdad2	mark NemoEmbeddings as deprecated (#21239 ) The NemoEmbeddings is deprecated, instead use langchain-nvidia-ai-endpoints NVIDIAEmbeddings interface. cc: @mattf --------- Co-authored-by: Daniel Glogowski <167348611+dglogo@users.noreply.github.com> Co-authored-by: andyjessen <62343929+andyjessen@users.noreply.github.com> Co-authored-by: Chris Germann <88305668+TAAGECH9@users.noreply.github.com> Co-authored-by: gere <gere@kapo.zh.ch> Co-authored-by: Chester Curme <chester.curme@gmail.com>	5 months ago
Erick Friis	5c000f8d79	community: release 0.0.37 (#21332 )	5 months ago
Erick Friis	7ecf9996f1	community: Revert "community: langkit dependency" (#21333 ) Reverts langchain-ai/langchain#21174 Hey team - going to revert this because it doesn't seem necessary for testing. We should only be adding optional + extended_testing dependencies for deps that have extended tests. otherwise it just increases probability of dependency conflicts in the community lockfile.	5 months ago
Param Singh	fee91d43b7	baichuan[patch]:standardize chat init args (#21298 ) Thank you for contributing to LangChain! community:baichuan[patch]: standardize init args updated `baichuan_api_key` so that aliased to `api_key`. Added test that it continues to set the same underlying attribute. Test checks for `SecretStr` updated `temperature` with Pydantic Field, added unit test. Related to https://github.com/langchain-ai/langchain/issues/20085	5 months ago
Christophe Bornet	484a009012	community[minor]: Relax constraints on Cassandra VectorStore constructors (#21209 ) If Session and/or keyspace are not provided, they are resolved from cassio's context. So they are not required. This change is fully backward compatible.	5 months ago
Leonid Ganeline	6feddfae88	community: langkit dependency (#21174 ) Issue: the `langkit` package is not presented in the `pyproject.toml` but it is a requirement for the `WhyLabsCallbackHandler` Change: added `langkit` --------- Co-authored-by: Chester Curme <chester.curme@gmail.com>	5 months ago
Rohan Aggarwal	8021d2a2ab	community[minor]: Oraclevs integration (#21123 ) Thank you for contributing to LangChain! - Oracle AI Vector Search Oracle AI Vector Search is designed for Artificial Intelligence (AI) workloads that allows you to query data based on semantics, rather than keywords. One of the biggest benefit of Oracle AI Vector Search is that semantic search on unstructured data can be combined with relational search on business data in one single system. This is not only powerful but also significantly more effective because you don't need to add a specialized vector database, eliminating the pain of data fragmentation between multiple systems. - Oracle AI Vector Search is designed for Artificial Intelligence (AI) workloads that allows you to query data based on semantics, rather than keywords. One of the biggest benefit of Oracle AI Vector Search is that semantic search on unstructured data can be combined with relational search on business data in one single system. This is not only powerful but also significantly more effective because you don't need to add a specialized vector database, eliminating the pain of data fragmentation between multiple systems. This Pull Requests Adds the following functionalities Oracle AI Vector Search : Vector Store Oracle AI Vector Search : Document Loader Oracle AI Vector Search : Document Splitter Oracle AI Vector Search : Summary Oracle AI Vector Search : Oracle Embeddings - We have added unit tests and have our own local unit test suite which verifies all the code is correct. We have made sure to add guides for each of the components and one end to end guide that shows how the entire thing runs. - We have made sure that make format and make lint run clean. Additional guidelines: - Make sure optional dependencies are imported within a function. - Please do not add dependencies to pyproject.toml files (even optional ones) unless they are required for unit tests. - Most PRs should not touch more than one package. - Changes should be backwards compatible. - If you are adding something to community, do not re-import it in langchain. If no one reviews your PR within a few days, please @-mention one of baskaryan, efriis, eyurtsev, hwchase17. --------- Co-authored-by: skmishraoracle <shailendra.mishra@oracle.com> Co-authored-by: hroyofc <harichandan.roy@oracle.com> Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com> Co-authored-by: Bagatur <baskaryan@gmail.com>	5 months ago
Leonid Ganeline	9639457222	community[patch]: `tools` imports (#21156 ) Issue: we have several helper functions to import third-party libraries like tools.gmail.utils.import_google in [community.tools](https://api.python.langchain.com/en/latest/community_api_reference.html#id37). And we have core.utils.utils.guard_import that works exactly for this purpose. The import_<package> functions work inconsistently and rather be private functions. Change: replaced these functions with the guard_import function. Related to #21133	5 months ago
ccurme	6da3d92b42	(all): update removal in deprecation warnings from 0.2 to 0.3 (#21265 ) We are pushing out the removal of these to 0.3. `find . -type f -name "*.py" -exec sed -i '' 's/removal="0\.2/removal="0.3/g' {} +`	5 months ago
Eugene Yurtsev	0989c48028	langchain[minor]: Re-add deleted ainetwork tool (#21254 ) * Adding __init__.py to turn it into a package in community * Adding proxy imports that assume that langchain_community is optional	5 months ago
Christophe Bornet	2fbe82f5e6	community[minor]: Relax constraints on CassandraChatMessageHistory constructor (#21241 )	5 months ago
Christophe Bornet	683fb45c6b	community[patch]: Refactor CassandraDatabase wrapper (#21075 ) * Introduce individual `fetch_` methods for easier typing. * Rework some docstrings to google style * Move some logic to the tool * Merge the 2 cassandra utility files	5 months ago
Raghav Dixit	7d451d0041	community[patch]: Update lancedb.py (#21192 ) very minor update in LanceDB integration, 'metric' argument was missing.	5 months ago
Eugene Yurtsev	3cd7fced5f	langchain[patch],community[minor]: Migrate memory implementations to community (#20845 ) Migrates memory implementations to community	5 months ago
Eugene Yurtsev	c9119b0e75	langchain[patch],community[minor]: Move some unit tests from langchain to community, use core for fake models (#21190 )	5 months ago
Tomaz Bratanic	9e53fa7d2e	Some more fixes to neo4j enhanced schema (#21139 )	5 months ago
Eugene Yurtsev	44602bdc20	langchain[patch],community[minor]: Move load_tools to community (#21158 ) Move load tools to community	5 months ago
Eugene Yurtsev	bec3eee3fa	langchain[patch]: Migrate retrievers to use optional langchain community imports (#21155 )	5 months ago
Eugene Yurtsev	0e5bf16d00	langchain[patch]: Migrate document loaders to use optional langchain community imports (#21095 )	5 months ago
Harrison Chase	4d1c21d97d	community[patch]: Fix alternative name in deprecation notice for sql_database (#21144 )	5 months ago
East Agile	2a6f78a53f	community[minor]: Rememberizer retriever (#20052 ) Description: This pull request introduces a new feature for LangChain: the integration with the Rememberizer API through a custom retriever. This enables LangChain applications to allow users to load and sync their data from Dropbox, Google Drive, Slack, their hard drive into a vector database that LangChain can query. Queries involve sending text chunks generated within LangChain and retrieving a collection of semantically relevant user data for inclusion in LLM prompts. User knowledge dramatically improved AI applications. The Rememberizer integration will also allow users to access general purpose vectorized data such as Reddit channel discussions and US patents. Issue: N/A Dependencies: N/A Twitter handle: https://twitter.com/Rememberizer	5 months ago
Eugene Yurtsev	1ce1a10f2b	langchain[patch],community[minor]: Move graph index creator (#20795 ) Move graph index creator to community	5 months ago
Noah	45ddf4d26f	community[patch]: Update comments for lazy_load method (#21063 ) - [ ] PR message: - Description: Refactored the lazy_load method to use asynchronous execution for improved performance. The method now initiates scraping of all URLs simultaneously using asyncio.gather, enhancing data fetching efficiency. Each Document object is yielded immediately once its content becomes available, streamlining the entire process. - Issue: N/A - Dependencies: Requires the asyncio library for handling asynchronous tasks, which should already be part of standard Python libraries in Python 3.7 and above. - Email: [r73327118@gmail.com](mailto:r73327118@gmail.com) --------- Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com> Co-authored-by: Bagatur <baskaryan@gmail.com>	5 months ago
Ismail Hossain Polas	1fdf63fa6c	community[patch]: update package name to bagelML (#19948 ) Description This pull request updates the Bagel Network package name from "betabageldb" to "bagelML" to align with the latest changes made by the Bagel Network team. The following modifications have been made: - Updated all references to the old package name ("betabageldb") with the new package name ("bagelML") throughout the codebase. - Modified the documentation, and any relevant scripts to reflect the package name change. - Tested the changes to ensure that the functionality remains intact and no breaking changes were introduced. By merging this pull request, our project will stay up to date with the latest Bagel Network package naming convention, ensuring compatibility and smooth integration with their updated library. Please review the changes and provide any feedback or suggestions. Thank you!	5 months ago
tianzedavid	5a8909440b	docs: remove repetitive words (#21058 ) remove repetitive words Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com>	5 months ago
Tomaz Bratanic	c9e96bb5e2	community[patch]: Fix neo4j enhanced schema bugs (#21072 )	5 months ago
MacanPN	0f7f448603	community[patch]: add delete() method to AzureSearch vector store (#21127 ) Issue: Currently `AzureSearch` vector store does not implement `delete` method. This PR implements it. This also makes it compatible with LangChain indexer. Dependencies: None Twitter handle: @martintriska1 --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	5 months ago
Erick Friis	8a62fb0570	community: release 0.0.36 (#21118 )	5 months ago
Jamsheed Mistri	3e749369ef	community[minor]: bump version of LayerupSecurity, add support for untrusted_input parameter (#19985 ) Description: update version of LayerupSecurity package for the Layerup Security integration. Add untrusted_input parameter.	5 months ago
fubuki8087	f1c3687aa5	community[patch]: Using the right encoding to parse the web page in RecursiveUrlLoader (#20632 ) As shown in #13749 , `RecursiveUrlLoader` has encoding issue. This PR is to solve this. --------- Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com> Co-authored-by: Bagatur <baskaryan@gmail.com>	5 months ago
Jakub Pawłowski	b0b1a67771	community[patch]: Skip unexpected 404 HTTP Error in Arxiv download (#21042 ) ### Description: When attempting to download PDF files from arXiv, an unexpected 404 error frequently occurs. This error halts the operation, regardless of whether there are additional documents to process. As a solution, I suggest implementing a mechanism to ignore and communicate this error and continue processing the next document from the list. Proposed Solution: To address the issue of unexpected 404 errors during PDF downloads from arXiv, I propose implementing the following solution: - Error Handling: Implement error handling mechanisms to catch and handle 404 errors gracefully. - Communication: Inform the user or logging system about the occurrence of the 404 error. - Continued Processing: After encountering a 404 error, continue processing the remaining documents from the list without interruption. This solution ensures that the application can handle unexpected errors without terminating the entire operation. It promotes resilience and robustness in the face of intermittent issues encountered during PDF downloads from arXiv. ### Issue: #20909 ### Dependencies: none --------- Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com>	5 months ago
Erick Friis	b9c53e95b7	community: release 0.0.35 (#21104 )	5 months ago
Eugene Yurtsev	3c064a757f	core[minor],langchain[patch],community[patch]: Move storage interfaces to core (#20750 ) * Move storage interface to core * Move in memory and file system implementation to core	5 months ago
Charlie Marsh	8f38b7a725	multiple: Remove unnecessary Ruff suppression comments (#21050 ) ## Summary I ran `ruff check --extend-select RUF100 -n` to identify `# noqa` comments that weren't having any effect in Ruff, and then `ruff check --extend-select RUF100 -n --fix` on select files to remove all of the unnecessary `# noqa: F401` violations. It's possible that these were needed at some point in the past, but they're not necessary in Ruff v0.1.15 (used by LangChain) or in the latest release. Co-authored-by: Erick Friis <erick@langchain.dev>	5 months ago
Christophe Bornet	5c77f45b06	community[minor]: Add async methods to CassandraCache and CassandraSemanticCache (#20654 )	5 months ago
Kuro Denjiro	fa4124b821	community[minor]: add mintbase loader to langchain (#20089 ) - [x] Add Near NFT loader: "community: Load NFT near block chain using mintbase graph API" - [x] PR message: - Description: a description of the change - Twitter handle:Kurodenjiro --------- Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com> Co-authored-by: Bagatur <baskaryan@gmail.com>	5 months ago
Alexander Dicke	d7e12750df	community[patch]: allows using `text-generation-inference` /generate route with `HuggingFaceEndpoint` (#20100 ) - Description: allows to use the /generate route of `text-generation-inference` with the `HuggingFaceEndpoint`	5 months ago
davidkgp	28b0b0d863	community[patch]: Fix for github issue #17690 (#20117 ) …/17690 Thank you for contributing to LangChain! - [x] Fix Google Lens knowledge graph issue: "langchain: community" - Fix for [No "knowledge_graph" property in Google Lens API call from SerpAPI](https://github.com/langchain-ai/langchain/issues/17690) - [x] PR message: *Delete this entire checklist* and replace with - Description: handled the existence of keys in the json response of Google Lens - Issue: [No "knowledge_graph" property in Google Lens API call from SerpAPI](https://github.com/langchain-ai/langchain/issues/17690) - [x] Lint and test: Run `make format`, `make lint` and `make test` from the root of the package(s) you've modified. See contribution guidelines for more: https://python.langchain.com/docs/contributing/ If no one reviews your PR within a few days, please @-mention one of baskaryan, efriis, eyurtsev, hwchase17. Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com>	5 months ago
高远	a7a4630bf4	community[patch]: Modify the text field type and add new exception handling (#20116 ) Co-authored-by: gaoyuan <gaoyuan.20001218@bytedance.com>	5 months ago
Rahul Triptahi	c172611647	community[patch]: Add classifier_url argument in PebbloSafeLoader and documentation update. (#21030 ) Description: Add classifier_url argument in PebbloSafeLoader. Documentation: Updated PebbloSafeLoader documentation with above change and new links for pebblo github pages. --------- Signed-off-by: Rahul Tripathi <rauhl.psit.ec@gmail.com> Co-authored-by: Rahul Tripathi <rauhl.psit.ec@gmail.com>	5 months ago
Leonid Ganeline	85094cbb3a	docs: community docstring updates (#21040 ) Added missed docstrings. Updated docstrings to consistent format.	5 months ago
Rodrigo Nogueira	90f19028e5	community[patch]: Add maritalk streaming (sync and async) (#19203 ) Co-authored-by: RosevalJr <rdmalajr@gmail.com> Co-authored-by: Roseval Donisete Malaquias Junior <roseval@maritaca.ai>	5 months ago
Cahid Arda Öz	cc6191cb90	community[minor]: Add support for Upstash Vector (#20824 ) ## Description Adding `UpstashVectorStore` to utilize [Upstash Vector](https://upstash.com/docs/vector/overall/getstarted)! #17012 was opened to add Upstash Vector to langchain but was closed to wait for filtering. Now filtering is added to Upstash vector and we open a new PR. Additionally, [embedding feature](https://upstash.com/docs/vector/features/embeddingmodels) was added and we add this to our vectorstore aswell. ## Dependencies [upstash-vector](https://pypi.org/project/upstash-vector/) should be installed to use `UpstashVectorStore`. Didn't update dependencies because of [this comment in the previous PR](https://github.com/langchain-ai/langchain/pull/17012#pullrequestreview-1876522450). ## Tests Tests are added and they pass. Tests are naturally network bound since Upstash Vector is offered through an API. There was [a discussion in the previous PR about mocking the unittests](https://github.com/langchain-ai/langchain/pull/17012#pullrequestreview-1891820567). We didn't make changes to this end yet. We can update the tests if you can explain how the tests should be mocked. --------- Co-authored-by: ytkimirti <yusuftaha9@gmail.com> Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com> Co-authored-by: Bagatur <baskaryan@gmail.com>	5 months ago
chyroc	3e241956d3	community[minor]: add coze chat model (#20770 ) add coze chat model, to call coze.com apis	5 months ago
Massimiliano Pronesti	ce89b34fc0	community[patch]: support hybrid search with threshold in Azure AI Search Retriever (#20907 ) Support hybrid search with a score threshold -- similar to what we do for similarity search.	5 months ago
Andrei Panferov	b3efa38cc0	community[patch]: GigaChat model selection fix (#20988 ) Fixed the error that the model name is never actually put into GigaChat request payload, always defaulting to `GigaChat-Lite`. With this fix, model selection through ```python import os from langchain.chat_models.gigachat import GigaChat chat = GigaChat( name="GigaChat-Pro", # <- HERE!!!!! ... ) ``` should actually work, as intended in [here](`804390ba4b/libs/community/langchain_community/llms/gigachat.py (L36)`). --------- Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com>	5 months ago
Patrick McFadin	3331865f6b	community[minor]: add Cassandra Database Toolkit (#20246 ) Description: ToolKit and Tools for accessing data in a Cassandra Database primarily for Agent integration. Initially, this includes the following tools: - `cassandra_db_schema` Gathers all schema information for the connected database or a specific schema. Critical for the agent when determining actions. - `cassandra_db_select_table_data` Selects data from a specific keyspace and table. The agent can pass paramaters for a predicate and limits on the number of returned records. - `cassandra_db_query` Expiriemental alternative to `cassandra_db_select_table_data` which takes a query string completely formed by the agent instead of parameters. May be removed in future versions. Includes unit test and two notebooks to demonstrate usage. Dependencies: cassio Twitter handle: @PatrickMcFadin --------- Co-authored-by: Phil Miesle <phil.miesle@datastax.com> Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com> Co-authored-by: Bagatur <baskaryan@gmail.com>	5 months ago
Igor Brai	b3e74f2b98	community[minor]: add mojeek search util (#20922 ) Description: This pull request introduces a new feature to community tools, enhancing its search capabilities by integrating the Mojeek search engine Dependencies: None --------- Co-authored-by: Igor Brai <igor@mojeek.com> Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com> Co-authored-by: ccurme <chester.curme@gmail.com>	5 months ago
Tomaz Bratanic	67428c4052	community[patch]: Neo4j enhanced schema (#20983 ) Scan the database for example values and provide them to an LLM for better inference of Text2cypher	5 months ago
Pengcheng Liu	1fad39be1c	community[minor]: Add LarkSuite wiki document loader. (#21016 ) Description: Add LarkSuite wiki document loader. Refer to [LarkSuite api document ](https://open.feishu.cn/document/server-docs/docs/wiki-v2/space-node/list)for details. Issue: None Dependencies: None Twitter handle: None	5 months ago
Leonid Ganeline	dc7c06bc07	community[minor]: import fix (#20995 ) Issue: When the third-party package is not installed, whenever we need to `pip install <package>` the ImportError is raised. But sometimes, the `ValueError` or `ModuleNotFoundError` is raised. It is bad for consistency. Change: replaced the `ValueError` or `ModuleNotFoundError` with `ImportError` when we raise an error with the `pip install <package>` message. Note: Ideally, we replace all `try: import... except... raise ... `with helper functions like `import_aim` or just use the existing [langchain_core.utils.utils.guard_import](https://api.python.langchain.com/en/latest/utils/langchain_core.utils.utils.guard_import.html#langchain_core.utils.utils.guard_import) But it would be much bigger refactoring. @baskaryan Please, advice on this.	5 months ago
WilliamEspegren	804390ba4b	community: Spider integration (#20937 ) Added the [Spider.cloud](https://spider.cloud) document loader. [Spider](https://github.com/spider-rs/spider) is the [fastest](https://github.com/spider-rs/spider/blob/main/benches/BENCHMARKS.md) and cheapest crawler that returns LLM-ready data. ``` - Description: Adds Spider data loader - Dependencies: spider-client - Twitter handle: @WilliamEspegren ``` --------- Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com> Co-authored-by: Bagatur <baskaryan@gmail.com> Co-authored-by: = <=> Co-authored-by: Chester Curme <chester.curme@gmail.com>	5 months ago
Chip Davis	e818c75f8a	infra: test directory loader multithreaded (#20281 ) This is a unit test for #20230 which was a fix for using multithreaded mode with directory loader @eyurtsev	5 months ago
Guilherme Zanotelli	f931a9ce60	community[patch]: Pass kwargs to SPARQLStore from RdfGraph (#20385 ) This introduces `store_kwargs` which behaves similarly to `graph_kwargs` on the `RdfGraph` object, which will enable users to pass `headers` and other arguments to the underlying `SPARQLStore` object. I have also made a [PR in `rdflib` to support passing `default_graph`](https://github.com/RDFLib/rdflib/pull/2761). Example usage: ```python from langchain_community.graphs import RdfGraph graph = RdfGraph( query_endpoint="http://localhost/sparql", standard="rdf", store_kwargs=dict( default_graph="http://example.com/mygraph" ) ) ``` <!--If no one reviews your PR within a few days, please @-mention one of baskaryan, efriis, eyurtsev, hwchase17.--> --------- Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com> Co-authored-by: Bagatur <baskaryan@gmail.com>	5 months ago
Jorge Piedrahita Ortiz	40b2e2916b	community[minor]: Sambanova llm integration (#20955 ) - Description: Added [Sambanova systems](https://sambanova.ai/) integration, including sambaverse and sambastudio LLMs - Dependencies: sseclient-py (optional) --------- Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com> Co-authored-by: Bagatur <baskaryan@gmail.com>	5 months ago
Rahul Triptahi	955cf186d2	community[patch]: Ingest source, owner and full_path if present in Document's metadata. (#20949 ) Description: The PebbloSafeLoader should first check for owner, full_path and size in metadata before implementing its own logic. Dependencies: None Documentation: NA. Signed-off-by: Rahul Tripathi <rauhl.psit.ec@gmail.com> Co-authored-by: Rahul Tripathi <rauhl.psit.ec@gmail.com>	5 months ago
Amine Djeghri	790ea75cf7	community[minor]: add exllamav2 library for GPTQ & EXL2 models (#17817 ) Added 3 files : - Library : ExLlamaV2 - Test integration - Notebook --------- Co-authored-by: Bagatur <baskaryan@gmail.com> Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com>	5 months ago
Naveen Tatikonda	8bbdb4f6a0	community[patch]: Add OpenSearch as semantic cache (#20254 ) ### Description Use OpenSearch vector store as Semantic Cache. ### Twitter Handle @OpenSearchProj --------- Signed-off-by: Naveen Tatikonda <navtat@amazon.com> Co-authored-by: Harish Tatikonda <harishtatikonda@Harishs-MacBook-Air.local> Co-authored-by: EC2 Default User <ec2-user@ip-172-31-31-155.ec2.internal> Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com> Co-authored-by: Bagatur <baskaryan@gmail.com>	5 months ago
Mayank Solanki	8c085fc697	community[patch]: Added a function `from_existing_collection` in `Qdrant` vector database. (#20779 ) Issue: #20514 The current implementation of `construct_instance` expects a `texts: List[str]` that will call the embedding function. This might not be needed when we already have a client with collection and `path, you don't want to add any text. This PR adds a class method that returns a qdrant instance with an existing client. Here everytime `cb6e5e56c2/libs/community/langchain_community/vectorstores/qdrant.py (L1592)` `construct_instance` is called, this line sends some text for embedding generation. --------- Co-authored-by: Anush <anushshetty90@gmail.com>	5 months ago
Leonid Kuligin	893a924b90	core[minor], community[patch], langchain[patch]: move BaseChatLoader to core (#19607 ) Thank you for contributing to LangChain! - [ ] PR title: "core: move BaseChatLoader and BaseToolkit from community" - [ ] PR message: move BaseChatLoader and BaseToolkit --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	5 months ago
Lei Zhang	9281841cfe	community[patch]: fix integrated test case test_recursive_url_loader.py assertions (issue-20919) (#20920 ) Description: Fix integrated test case test_recursive_url_loader.py Local testing successful ```shell (venv) lei@LeideMacBook-Pro community % poetry run pytest tests/integration_tests/document_loaders/test_recursive_url_loader.py ================================================================================ test session starts ================================================================================ platform darwin -- Python 3.11.4, pytest-7.4.4, pluggy-1.4.0 -- /Users/zhanglei/Work/github/langchain/venv/bin/python cachedir: .pytest_cache rootdir: /Users/zhanglei/Work/github/langchain/libs/community configfile: pyproject.toml plugins: syrupy-4.6.1, asyncio-0.20.3, cov-4.1.0, vcr-1.0.2, mock-3.12.0, anyio-3.7.1, dotenv-0.5.2, requests-mock-1.11.0, socket-0.6.0 asyncio: mode=Mode.AUTO collected 6 items tests/integration_tests/document_loaders/test_recursive_url_loader.py::test_async_recursive_url_loader PASSED [ 16%] tests/integration_tests/document_loaders/test_recursive_url_loader.py::test_async_recursive_url_loader_deterministic PASSED [ 33%] tests/integration_tests/document_loaders/test_recursive_url_loader.py::test_sync_recursive_url_loader FAILED [ 50%] tests/integration_tests/document_loaders/test_recursive_url_loader.py::test_sync_async_equivalent PASSED [ 66%] tests/integration_tests/document_loaders/test_recursive_url_loader.py::test_loading_invalid_url PASSED [ 83%] tests/integration_tests/document_loaders/test_recursive_url_loader.py::test_sync_async_metadata_necessary_properties PASSED [100%] ===================================================================================== FAILURES ====================================================================================== __________________________________________________________________________ test_sync_recursive_url_loader ___________________________________________________________________________ def test_sync_recursive_url_loader() -> None: url = "https://docs.python.org/3.9/" loader = RecursiveUrlLoader( url, extractor=lambda _: "placeholder", use_async=False, max_depth=2 ) docs = loader.load() > assert len(docs) == 23 E AssertionError: assert 24 == 23 E + where 24 = len([Document(page_content='placeholder', metadata={'source': 'https://docs.python.org/3.9/', 'content_type': 'text/html', 'title': '3.9.18 Documentation', 'language': None}), Document(page_content='placeholder', metadata={'source': 'https://docs.python.org/3.9/py-modindex.html', 'content_type': 'text/html', 'title': 'Python Module Index — Python 3.9.18 documentation', 'language': None}), Document(page_content='placeholder', metadata={'source': 'https://docs.python.org/3.9/download.html', 'content_type': 'text/html', 'title': 'Download — Python 3.9.18 documentation', 'language': None}), Document(page_content='placeholder', metadata={'source': 'https://docs.python.org/3.9/howto/index.html', 'content_type': 'text/html', 'title': 'Python HOWTOs — Python 3.9.18 documentation', 'language': None}), Document(page_content='placeholder', metadata={'source': 'https://docs.python.org/3.9/whatsnew/index.html', 'content_type': 'text/html', 'title': 'Whatâ\x80\x99s New in Python — Python 3.9.18 documentation', 'language': None}), Document(page_content='placeholder', metadata={'source': 'https://docs.python.org/3.9/c-api/index.html', 'content_type': 'text/html', 'title': 'Python/C API Reference Manual — Python 3.9.18 documentation', 'language': None}), ...]) tests/integration_tests/document_loaders/test_recursive_url_loader.py:38: AssertionError ================================================================================= warnings summary ================================================================================== tests/integration_tests/document_loaders/test_recursive_url_loader.py::test_async_recursive_url_loader tests/integration_tests/document_loaders/test_recursive_url_loader.py::test_async_recursive_url_loader_deterministic tests/integration_tests/document_loaders/test_recursive_url_loader.py::test_sync_recursive_url_loader tests/integration_tests/document_loaders/test_recursive_url_loader.py::test_sync_async_equivalent tests/integration_tests/document_loaders/test_recursive_url_loader.py::test_sync_async_metadata_necessary_properties /Users/zhanglei/.pyenv/versions/3.11.4/lib/python3.11/html/parser.py:170: XMLParsedAsHTMLWarning: It looks like you're parsing an XML document using an HTML parser. If this really is an HTML document (maybe it's XHTML?), you can ignore or filter this warning. If it's XML, you should know that using an XML parser will be more reliable. To parse this document as XML, make sure you have the lxml package installed, and pass the keyword argument `features="xml"` into the BeautifulSoup constructor. k = self.parse_starttag(i) -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html ================================================================================ slowest 5 durations ================================================================================ 56.75s call tests/integration_tests/document_loaders/test_recursive_url_loader.py::test_async_recursive_url_loader_deterministic 38.99s call tests/integration_tests/document_loaders/test_recursive_url_loader.py::test_async_recursive_url_loader 31.20s call tests/integration_tests/document_loaders/test_recursive_url_loader.py::test_sync_async_metadata_necessary_properties 30.37s call tests/integration_tests/document_loaders/test_recursive_url_loader.py::test_sync_async_equivalent 15.44s call tests/integration_tests/document_loaders/test_recursive_url_loader.py::test_sync_recursive_url_loader ============================================================================== short test summary info ============================================================================== FAILED tests/integration_tests/document_loaders/test_recursive_url_loader.py::test_sync_recursive_url_loader - AssertionError: assert 24 == 23 ================================================================ 1 failed, 5 passed, 5 warnings in 172.97s (0:02:52) ================================================================ (venv) zhanglei@LeideMacBook-Pro community % poetry run pytest tests/integration_tests/document_loaders/test_recursive_url_loader.py ================================================================================ test session starts ================================================================================ platform darwin -- Python 3.11.4, pytest-7.4.4, pluggy-1.4.0 -- /Users/zhanglei/Work/github/langchain/venv/bin/python cachedir: .pytest_cache rootdir: /Users/zhanglei/Work/github/langchain/libs/community configfile: pyproject.toml plugins: syrupy-4.6.1, asyncio-0.20.3, cov-4.1.0, vcr-1.0.2, mock-3.12.0, anyio-3.7.1, dotenv-0.5.2, requests-mock-1.11.0, socket-0.6.0 asyncio: mode=Mode.AUTO collected 6 items tests/integration_tests/document_loaders/test_recursive_url_loader.py::test_async_recursive_url_loader PASSED [ 16%] tests/integration_tests/document_loaders/test_recursive_url_loader.py::test_async_recursive_url_loader_deterministic PASSED [ 33%] tests/integration_tests/document_loaders/test_recursive_url_loader.py::test_sync_recursive_url_loader PASSED [ 50%] tests/integration_tests/document_loaders/test_recursive_url_loader.py::test_sync_async_equivalent PASSED [ 66%] tests/integration_tests/document_loaders/test_recursive_url_loader.py::test_loading_invalid_url PASSED [ 83%] tests/integration_tests/document_loaders/test_recursive_url_loader.py::test_sync_async_metadata_necessary_properties PASSED [100%] ================================================================================= warnings summary ================================================================================== tests/integration_tests/document_loaders/test_recursive_url_loader.py::test_async_recursive_url_loader tests/integration_tests/document_loaders/test_recursive_url_loader.py::test_async_recursive_url_loader_deterministic tests/integration_tests/document_loaders/test_recursive_url_loader.py::test_sync_recursive_url_loader tests/integration_tests/document_loaders/test_recursive_url_loader.py::test_sync_async_equivalent tests/integration_tests/document_loaders/test_recursive_url_loader.py::test_sync_async_metadata_necessary_properties /Users/zhanglei/.pyenv/versions/3.11.4/lib/python3.11/html/parser.py:170: XMLParsedAsHTMLWarning: It looks like you're parsing an XML document using an HTML parser. If this really is an HTML document (maybe it's XHTML?), you can ignore or filter this warning. If it's XML, you should know that using an XML parser will be more reliable. To parse this document as XML, make sure you have the lxml package installed, and pass the keyword argument `features="xml"` into the BeautifulSoup constructor. k = self.parse_starttag(i) -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html ================================================================================ slowest 5 durations ================================================================================ 46.99s call tests/integration_tests/document_loaders/test_recursive_url_loader.py::test_async_recursive_url_loader_deterministic 32.43s call tests/integration_tests/document_loaders/test_recursive_url_loader.py::test_async_recursive_url_loader 31.23s call tests/integration_tests/document_loaders/test_recursive_url_loader.py::test_sync_async_equivalent 30.75s call tests/integration_tests/document_loaders/test_recursive_url_loader.py::test_sync_async_metadata_necessary_properties 15.89s call tests/integration_tests/document_loaders/test_recursive_url_loader.py::test_sync_recursive_url_loader ===================================================================== 6 passed, 5 warnings in 157.42s (0:02:37) ===================================================================== (venv) lei@LeideMacBook-Pro community % ``` Issue: https://github.com/langchain-ai/langchain/issues/20919 Twitter handle: @coolbeevip	5 months ago
Matt	28df4750ef	community[patch]: Add initial tests for AzureSearch vector store (#17663 ) Description: AzureSearch vector store has no tests. This PR adds initial tests to validate the code can be imported and used. Issue: N/A Dependencies: azure-search-documents and azure-identity are added as optional dependencies for testing --------- Co-authored-by: Matt Gotteiner <[email protected]> Co-authored-by: Bagatur <baskaryan@gmail.com>	5 months ago
Dristy Srivastava	5f1d1666e3	community[patch]: Add support for pebblo server and client version (#20269 ) Description: _PebbloSafeLoader_: Add support for pebblo server and client version Documentation: NA Unit test: NA Issue: NA Dependencies: None --------- Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com>	5 months ago
am-kinetica	b54b19ba1c	community[minor]: Implemented Kinetica Document Loader and added notebooks (#20002 ) - [ ] Kinetica Document Loader: "community: a class to load Documents from Kinetica" - [ ] Kinetica Document Loader: - Description: implemented KineticaLoader in `kinetica_loader.py` - Dependencies: install the Kinetica API using `pip install gpudb==7.2.0.1 `	5 months ago
Shengsheng Huang	fd1061e7bf	community[patch]: add more data types support to ipex-llm llm integration (#20833 ) - Description: - add support for more data types: by default `IpexLLM` will load the model in int4 format. This PR adds more data types support such as `sym_in5`, `sym_int8`, etc. Data formats like NF3, NF4, FP4 and FP8 are only supported on GPU and will be added in future PR. - Fix a small issue in saving/loading, update api docs - Dependencies: `ipex-llm` library - Document: In `docs/docs/integrations/llms/ipex_llm.ipynb`, added instructions for saving/loading low-bit model. - Tests: added new test cases to `libs/community/tests/integration_tests/llms/test_ipex_llm.py`, added config params. - Contribution maintainer: @shane-huang	5 months ago
Rahul Triptahi	dc921f0823	community[patch]: Add semantic info to metadata, classified by pebblo-server. (#20468 ) Description: Add support for Semantic topics and entities. Classification done by pebblo-server is not used to enhance metadata of Documents loaded by document loaders. Dependencies: None Documentation: Updated. Signed-off-by: Rahul Tripathi <rauhl.psit.ec@gmail.com> Co-authored-by: Rahul Tripathi <rauhl.psit.ec@gmail.com>	5 months ago
Jingpan Xiong	1202017c56	community[minor]: Add relyt vector database (#20316 ) Co-authored-by: kaka <kaka@zbyte-inc.cloud> Co-authored-by: Bagatur <baskaryan@gmail.com> Co-authored-by: jingsi <jingsi@leadincloud.com> Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com>	5 months ago
davidefantiniIntel	f386f71bb3	community: fix tqdm import (#20263 ) Description: Fix tqdm import in QuantizedBiEncoderEmbeddings	5 months ago
Andres Algaba	05ae8ca7d4	community[patch]: deprecate persist method in Chroma (#20855 ) Thank you for contributing to LangChain! - [x] PR title - [x] PR message: - Description: Deprecate persist method in Chroma no longer exists in Chroma 0.4.x - Issue: #20851 - Dependencies: None - Twitter handle: AndresAlgaba1 - [x] Add tests and docs: If you're adding a new integration, please include 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. It lives in `docs/docs/integrations` directory. - [x] Lint and test: Run `make format`, `make lint` and `make test` from the root of the package(s) you've modified. See contribution guidelines for more: https://python.langchain.com/docs/contributing/ --------- Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com>	5 months ago
ccurme	b8db73233c	core, community: deprecate tool.__call__ (#20900 ) Does not update docs.	5 months ago
Tomaz Bratanic	520972fd0f	community[patch]: Support passing graph object to Neo4j integrations (#20876 ) For driver connection reusage, we introduce passing the graph object to neo4j integrations	5 months ago
Lei Zhang	748a6ae609	community[patch]: add HTTP response headers Content-Type to metadata of RecursiveUrlLoader document (#20875 ) Description: The RecursiveUrlLoader loader offers a link_regex parameter that can filter out URLs. However, this filtering capability is limited, and if the internal links of the website change, unexpected resources may be loaded. These resources, such as font files, can cause problems in subsequent embedding processing. > https://blog.langchain.dev/assets/fonts/source-sans-pro-v21-latin-ext_latin-regular.woff2?v=0312715cbf We can add the Content-Type in the HTTP response headers to the document metadata so developers can choose which resources to use. This allows developers to make their own choices. For example, the following may be a good choice for text knowledge. - text/plain - simple text file - text/html - HTML web page - text/xml - XML format file - text/json - JSON format data - application/pdf - PDF file - application/msword - Word document and ignore the following - text/css - CSS stylesheet - text/javascript - JavaScript script - application/octet-stream - binary data - image/jpeg - JPEG image - image/png - PNG image - image/gif - GIF image - image/svg+xml - SVG image - audio/mpeg - MPEG audio files - video/mp4 - MP4 video file - application/font-woff - WOFF font file - application/font-ttf - TTF font file - application/zip - ZIP compressed file - application/octet-stream - binary data Twitter handle: @coolbeevip --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	5 months ago
Joan Fontanals	baefbfb14e	community[mionr]: add Jina Reranker in retrievers module (#19406 ) - Description: Adapt JinaEmbeddings to run with the new Jina AI Rerank API - Twitter handle: https://twitter.com/JinaAI_ - [ ] Add tests and docs: If you're adding a new integration, please include 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. It lives in `docs/docs/integrations` directory. - [ ] Lint and test: Run `make format`, `make lint` and `make test` from the root of the package(s) you've modified. See contribution guidelines for more: https://python.langchain.com/docs/contributing/ --------- Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com> Co-authored-by: Bagatur <baskaryan@gmail.com>	5 months ago
Jason_Chen	53bb7dbd29	community[patch]: add BeautifulSoupTransformer remove_unwanted_classnames method (#20467 ) Add the remove_unwanted_classnames method to the BeautifulSoupTransformer class, which can filter more effectively. --------- Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com> Co-authored-by: Bagatur <baskaryan@gmail.com>	5 months ago
Bagatur	5b83130855	core[minor], langchain[patch], community[patch]: mv StructuredQuery (#20849 ) mv StructuredQuery to core	5 months ago
Mish Ushakov	6ccecf2363	community[minor]: added Browserbase loader (#20478 )	5 months ago
ccurme	481d3855dc	patch: remove usage of llm, chat model __call__ (#20788 ) - `llm(prompt)` -> `llm.invoke(prompt)` - `llm(prompt=prompt` -> `llm.invoke(prompt)` (same with `messages=`) - `llm(prompt, callbacks=callbacks)` -> `llm.invoke(prompt, config={"callbacks": callbacks})` - `llm(prompt, kwargs)` -> `llm.invoke(prompt, kwargs)`	5 months ago
Raghav Dixit	9b7fb381a4	community[patch]: LanceDB integration patch update (#20686 ) Description : - added functionalities - delete, index creation, using existing connection object etc. - updated usage - Added LaceDB cloud OSS support make lint_diff , make test checks done	5 months ago
volodymyr-memsql	493afe4d8d	community[patch]: add hybrid search to singlestoredb vectorstore (#20793 ) Implemented the ability to enable full-text search within the SingleStore vector store, offering users a versatile range of search strategies. This enhancement allows users to seamlessly combine full-text search with vector search, enabling the following search strategies: * Search solely by vector similarity. * Conduct searches exclusively based on text similarity, utilizing Lucene internally. * Filter search results by text similarity score, with the option to specify a threshold, followed by a search based on vector similarity. * Filter results by vector similarity score before conducting a search based on text similarity. * Perform searches using a weighted sum of vector and text similarity scores. Additionally, integration tests have been added to comprehensively cover all scenarios. Updated notebook with examples. CC: @baskaryan, @hwchase17 --------- Co-authored-by: Volodymyr Tkachuk <vtkachuk-ua@singlestore.com> Co-authored-by: Bagatur <baskaryan@gmail.com>	5 months ago
Tomaz Bratanic	9efab3ed66	community[patch]: Add driver config param for neo4j graph (#20772 ) Co-authored-by: Bagatur <baskaryan@gmail.com>	5 months ago
Leonid Ganeline	13751c3297	community: `tigergraph` fixes (#20034 ) - added guard on the `pyTigerGraph` import - added a missed example page in the `docs/integrations/graphs/` - formatted the `docs/integrations/providers/` page to the consistent format. Added links.	5 months ago
Martin Kolb	0186e4e633	community[patch]: Advanced filtering for HANA Cloud Vector Engine (#20821 ) - Description: This PR adds support for advanced filtering to the integration of HANA Vector Engine. The newly supported filtering operators are: $eq, $ne, $gt, $gte, $lt, $lte, $between, $in, $nin, $like, $and, $or - Issue: N/A - Dependencies: no new dependencies added Added integration tests to: `libs/community/tests/integration_tests/vectorstores/test_hanavector.py` Description of the new capabilities in notebook: `docs/docs/integrations/vectorstores/hanavector.ipynb`	5 months ago
Alex Sherstinsky	12e5ec6de3	community: Support both Predibase SDK-v1 and SDK-v2 in Predibase-LangChain integration (#20859 )	5 months ago
JeffKatzy	5ab3f9a995	community[patch]: standardize chat init args (#20844 ) Thank you for contributing to LangChain! community:perplexity[patch]: standardize init args updated pplx_api_key and request_timeout so that aliased to api_key, and timeout respectively. Added test that both continue to set the same underlying attributes. Related to [20085](https://github.com/langchain-ai/langchain/issues/20085) --------- Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com>	5 months ago
Massimiliano Pronesti	8d1167b32f	community[patch]: add support for similarity_score_threshold search in… (#20852 ) See https://github.com/langchain-ai/langchain/issues/20600#issuecomment-2075569338 for details. @chrislrobert	5 months ago
Eugene Yurtsev	30e48c9878	core[patch],community[patch]: Move file chat history back to community (#20834 ) Marking as patch since we haven't had releases in between. This just reverting part of a PR from yesterday.	5 months ago
Nestor Qin	9111d3a636	community[patch]: Fix message formatting for Anthropic models on Amazon Bedrock (#20801 ) Description: This PR fixes an issue in message formatting function for Anthropic models on Amazon Bedrock. Currently, LangChain BedrockChat model will crash if it uses Anthropic models and the model return a message in the following type: - `AIMessageChunk` Moreover, when use BedrockChat with for building Agent, the following message types will trigger the same issue too: - `HumanMessageChunk` - `FunctionMessage` Issue: https://github.com/langchain-ai/langchain/issues/18831 Dependencies: No. Testing: Manually tested. The following code was failing before the patch and works after. ``` @tool def square_root(x: str): "Useful when you need to calculate the square root of a number" return math.sqrt(int(x)) llm = ChatBedrock( model_id="anthropic.claude-3-sonnet-20240229-v1:0", model_kwargs={ "temperature": 0.0 }, ) prompt = ChatPromptTemplate.from_messages( [ ("system", FUNCTION_CALL_PROMPT), ("human", "Question: {user_input}"), MessagesPlaceholder(variable_name="agent_scratchpad"), ] ) tools = [square_root] tools_string = format_tool_to_anthropic_function(square_root) agent = ( RunnablePassthrough.assign( user_input=lambda x: x['user_input'], agent_scratchpad=lambda x: format_to_openai_function_messages( x["intermediate_steps"] ) ) \| prompt \| llm \| AnthropicFunctionsAgentOutputParser() ) agent_executor = AgentExecutor(agent=agent, tools=tools, verbose=True, return_intermediate_steps=True) output = agent_executor.invoke({ "user_input": "What is the square root of 2?", "tools_string": tools_string, }) ``` List of messages returned from Bedrock: ``` <SystemMessage> content='You are a helpful assistant.' <HumanMessage> content='Question: What is the square root of 2?' <AIMessageChunk> content="Okay, let's calculate the square root of 2.<scratchpad>\nTo calculate the square root of a number, I can use the square_root tool:\n\n<function_calls>\n <invoke>\n <tool_name>square_root</tool_name>\n <parameters>\n <__arg1>2</__arg1>\n </parameters>\n </invoke>\n</function_calls>\n</scratchpad>\n\n<function_results>\n<search_result>\nThe square root of 2 is approximately 1.414213562373095\n</search_result>\n</function_results>\n\n<answer>\nThe square root of 2 is approximately 1.414213562373095\n</answer>" id='run-92363df7-eff6-4849-bbba-fa16a1b2988c'" <FunctionMessage> content='1.4142135623730951' name='square_root' ```	5 months ago
Aliaksandr Kuzmik	5560cc448c	community[patch]: fix CometTracer bug (#20796 ) Hi! My name is Alex, I'm an SDK engineer from [Comet](https://www.comet.com/site/) This PR updates the `CometTracer` class. Fixed an issue when `CometTracer` failed while logging the data to Comet because this data is not JSON-encodable. The problem was in some of the `Run` attributes that could contain non-default types inside, now these attributes are taken not from the run instance, but from the `run.dict()` return value.	5 months ago
Eugene Yurtsev	645b1e142e	core[minor],langchain[patch],community[patch]: Move InMemory and File implementations of Chat History to core (#20752 ) This PR moves the implementations for chat history to core. So it's easier to determine which dependencies need to be broken / add deprecation warnings	5 months ago
ccurme	7a922f3e48	core, openai: support custom token encoders (#20762 )	5 months ago
Christophe Bornet	0ae5027d98	community[patch]: Remove usage of deprecated StoredBlobHistory in CassandraChatMessageHistory (#20666 )	5 months ago
Eugene Yurtsev	38adbfdf34	community[patch],core[minor]: Move BaseToolKit to core.tools (#20669 )	5 months ago
Mark Needham	ce23f8293a	Community patch clickhouse make it possible to not specify index (#20460 ) Vector indexes in ClickHouse are experimental at the moment and can sometimes break/change behaviour. So this PR makes it possible to say that you don't want to specify an index type. Any queries against the embedding column will be brute force/linear scan, but that gives reasonable performance for small-medium dataset sizes. --------- Co-authored-by: Erick Friis <erick@langchain.dev> Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com>	5 months ago
ccurme	c010ec8b71	patch: deprecate (a)get_relevant_documents (#20477 ) - `.get_relevant_documents(query)` -> `.invoke(query)` - `.get_relevant_documents(query=query)` -> `.invoke(query)` - `.get_relevant_documents(query, callbacks=callbacks)` -> `.invoke(query, config={"callbacks": callbacks})` - `.get_relevant_documents(query, kwargs)` -> `.invoke(query, kwargs)` --------- Co-authored-by: Erick Friis <erick@langchain.dev>	5 months ago
Matheus Henrique Raymundo	bb69819267	community: Fix the stop sequence key name for Mistral in Bedrock (#20709 ) Fixing the wrong stop sequence key name that causes an error on AWS Bedrock. You can check the MistralAI bedrock parameters [here](https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-mistral.html) This change fixes this [issue](https://github.com/langchain-ai/langchain/issues/20095)	5 months ago
Bagatur	1c7b3c75a7	community[patch], experimental[patch]: support tool-calling sql and p… (#20639 ) d agents	5 months ago
shumway743	cb6e5e56c2	community[minor]: add graph store implementation for apache age (#20582 ) Description: implemented GraphStore class for Apache Age graph db Dependencies: depends on psycopg2 Unit and integration tests included. Formatting and linting have been run. --------- Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com> Co-authored-by: Bagatur <baskaryan@gmail.com>	5 months ago
Christophe Bornet	c909ae0152	community[minor]: Add async methods to CassandraVectorStore (#20602 ) Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com>	5 months ago
Dmitry Tyumentsev	f111efeb6e	community[patch]: YandexGPT API add ability to disable request logging (#20670 ) Closes (#20622) Added the ability to [disable logging of requests to YandexGPT](https://yandex.cloud/en/docs/foundation-models/operations/yandexgpt/disable-logging).	5 months ago
Erick Friis	73809817ff	community: release 0.0.34 (#20672 )	5 months ago
Tomaz Bratanic	8c08cf4619	community: Add support for relationship indexes in neo4j vector (#20657 ) Neo4j has added relationship vector indexes. We can't populate them, but we can use existing indexes for retrieval	5 months ago
Charlie Holtz	1cbab0ebda	community: update Replicate to work with official models (#20633 ) Description: you don't need to pass a version for Replicate official models. That was broken on LangChain until now! You can now run: ``` llm = Replicate( model="meta/meta-llama-3-8b-instruct", model_kwargs={"temperature": 0.75, "max_length": 500, "top_p": 1}, ) prompt = """ User: Answer the following yes/no question by reasoning step by step. Can a dog drive a car? Assistant: """ llm(prompt) ``` I've updated the replicate.ipynb to reflect that. twitter: @charliebholtz --------- Co-authored-by: Erick Friis <erick@langchain.dev>	5 months ago
Congyu	dd5139e304	community[patch]: truncate zhipuai `temperature` and `top_p` parameters to [0.01, 0.99] (#20261 ) ZhipuAI API only accepts `temperature` parameter between `(0, 1)` open interval, and if `0` is passed, it responds with status code `400`. However, 0 and 1 is often accepted by other APIs, for example, OpenAI allows `[0, 2]` for temperature closed range. This PR truncates temperature parameter passed to `[0.01, 0.99]` to improve the compatibility between langchain's ecosystem's and ZhipuAI (e.g., ragas `evaluate` often generates temperature 0, which results in a lot of 400 invalid responses). The PR also truncates `top_p` parameter since it has the same restriction. Reference: [glm-4 doc](https://open.bigmodel.cn/dev/api#glm-4) (which unfortunately is in Chinese though). --------- Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com>	5 months ago
Lance Martin	d5c22b80a5	community[patch]: Fix Ollama for LLaMA3 (#20624 ) We see verbose generations w/ LLaMA3 and Ollama - https://smith.langchain.com/public/88c4cd21-3d57-4229-96fe-53443398ca99/r --- Fix here implies that when stop was being set to an empty list, the stream had no conditions under which to stop, which could lead to excessive or unintended output. Test LLaMA2 - https://smith.langchain.com/public/57dfc64a-591b-46fa-a1cd-8783acaefea2/r Test LLaMA3 - https://smith.langchain.com/public/76ff5f47-ac89-4772-a7d2-5caa907d3fd6/r https://smith.langchain.com/public/a31d2fad-9094-4c93-949a-964b27630ccb/r Test Mistral - https://smith.langchain.com/public/a4fe7114-c308-4317-b9fd-6c86d31f1c5b/r --------- Co-authored-by: Erick Friis <erick@langchain.dev>	5 months ago
hulitaitai	7d0a008744	community[minor]: Add audio-parser "faster-whisper" in audio.py (#20012 ) faster-whisper is a reimplementation of OpenAI's Whisper model using CTranslate2, which is up to 4 times faster than enai/whisper for the same accuracy while using less memory. The efficiency can be further improved with 8-bit quantization on both CPU and GPU. It can automatically detect the following 14 languages and transcribe the text into their respective languages: en, zh, fr, de, ja, ko, ru, es, th, it, pt, vi, ar, tr. The gitbub repository for faster-whisper is : https://github.com/SYSTRAN/faster-whisper --------- Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com>	5 months ago
Guangdong Liu	e3c2431c5b	comminuty[patch]:Fix Error in apache doris insert (#19989 ) - Issue: #19886	5 months ago
Tomaz Bratanic	27370b679e	community[patch]: Ignore null and invalid embedding values for neo4j metadata filtering (#20558 )	5 months ago
Massimiliano Pronesti	2542a09abc	community[patch]: AzureSearch incorrectly converted to retriever (#20601 ) Closes #20600. Please see the issue for more details.	5 months ago
Christophe Bornet	8f0b5687a3	community[minor]: Add hybrid search to Cassandra VectorStore (#20286 ) Only supported by Astra DB at the moment. Twitter handle: cbornet_	5 months ago
Christophe Bornet	d2d01370bc	community[minor]: Add async methods to CassandraLoader (#20609 ) Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com>	5 months ago
balloonio	e786da7774	community[patch]: Invoke callback prior to yielding token fix [HuggingFaceTextGenInference] (#20426 ) …gFaceTextGenInference) - [x] PR title: community[patch]: Invoke callback prior to yielding token fix for [HuggingFaceTextGenInference] - [x] PR message: - Description: Invoke callback prior to yielding token in stream method in [HuggingFaceTextGenInference] - Issue: https://github.com/langchain-ai/langchain/issues/16913 - Dependencies: None - Twitter handle: @bolun_zhang If no one reviews your PR within a few days, please @-mention one of baskaryan, efriis, eyurtsev, hwchase17. --------- Co-authored-by: Chester Curme <chester.curme@gmail.com>	5 months ago
Ethan Yang	2d6d796040	community: Add save_model function for openvino reranker and embedding (#19896 )	5 months ago
zR	9c1d7f2405	update zhipuai notebook (#20595 ) fix timeout issue fix zhipuai usecase notebookbook Thank you for contributing to LangChain! - [ ] PR title: "package: description" - Where "package" is whichever of langchain, community, core, experimental, etc. is being modified. Use "docs: ..." for purely docs changes, "templates: ..." for template changes, "infra: ..." for CI changes. - Example: "community: add foobar LLM" - [ ] PR message: *Delete this entire checklist* and replace with - Description: a description of the change - Issue: the issue # it fixes, if applicable - Dependencies: any dependencies required for this change - Twitter handle: if your PR gets announced, and you'd like a mention, we'll gladly shout you out! - [ ] Add tests and docs: If you're adding a new integration, please include 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. It lives in `docs/docs/integrations` directory. - [ ] Lint and test: Run `make format`, `make lint` and `make test` from the root of the package(s) you've modified. See contribution guidelines for more: https://python.langchain.com/docs/contributing/ Additional guidelines: - Make sure optional dependencies are imported within a function. - Please do not add dependencies to pyproject.toml files (even optional ones) unless they are required for unit tests. - Most PRs should not touch more than one package. - Changes should be backwards compatible. - If you are adding something to community, do not re-import it in langchain. If no one reviews your PR within a few days, please @-mention one of baskaryan, efriis, eyurtsev, hwchase17.	5 months ago
ccurme	c897264b9b	community: (milvus) check for num_shards (#20603 ) @rgupta2508 I believe this change is necessary following https://github.com/langchain-ai/langchain/pull/20318 because of how Milvus handles defaults: `59bf5e811a/pymilvus/client/prepare.py (L82-L85)` ```python num_shards = kwargs[next(iter(same_key))] if not isinstance(num_shards, int): msg = f"invalid num_shards type, got {type(num_shards)}, expected int" raise ParamError(message=msg) req.shards_num = num_shards ``` this way lets Milvus control the default value (instead of maintaining a separate default in Langchain). Let me know if I've got this wrong or you feel it's unnecessary. Thanks.	5 months ago
Rohit Gupta	25c4c24e89	Support to create shards_num in milvus vectorstores (#20318 ) To support number of the shards for the collection to create in milvus vvectorstores. Thank you for contributing to LangChain! - [ ] PR title: "package: description" - Where "package" is whichever of langchain, community, core, experimental, etc. is being modified. Use "docs: ..." for purely docs changes, "templates: ..." for template changes, "infra: ..." for CI changes. - Example: "community: add foobar LLM" - [ ] PR message: *Delete this entire checklist* and replace with - Description: a description of the change - Issue: the issue # it fixes, if applicable - Dependencies: any dependencies required for this change - Twitter handle: if your PR gets announced, and you'd like a mention, we'll gladly shout you out! - [ ] Add tests and docs: If you're adding a new integration, please include 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. It lives in `docs/docs/integrations` directory. - [ ] Lint and test: Run `make format`, `make lint` and `make test` from the root of the package(s) you've modified. See contribution guidelines for more: https://python.langchain.com/docs/contributing/ Additional guidelines: - Make sure optional dependencies are imported within a function. - Please do not add dependencies to pyproject.toml files (even optional ones) unless they are required for unit tests. - Most PRs should not touch more than one package. - Changes should be backwards compatible. - If you are adding something to community, do not re-import it in langchain. If no one reviews your PR within a few days, please @-mention one of baskaryan, efriis, eyurtsev, hwchase17.	5 months ago
Erick Friis	e395115807	docs: aws docs updates (#20571 )	5 months ago
Erick Friis	f09bd0b75b	upstage: init package (#20574 ) Co-authored-by: Sean Cho <sean@upstage.ai> Co-authored-by: JuHyung-Son <sonju0427@gmail.com>	5 months ago
Marco Perini	11c9ed3362	community[patch]: exposing headless flag parameter to AsyncChromiumLoader class (#20424 ) - Description: added the headless parameter as optional argument to the langchain_community.document_loaders AsyncChromiumLoader class - Dependencies: None - Twitter handle: @perinim_98 If no one reviews your PR within a few days, please @-mention one of baskaryan, efriis, eyurtsev, hwchase17. --------- Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com> Co-authored-by: Bagatur <baskaryan@gmail.com>	5 months ago
Christophe Bornet	a22da4315b	community[patch]: Replace function in CassandraVectorStore with simpler lambda (#20323 )	5 months ago
Christophe Bornet	75733c5cc1	community[minor]: Improve CassandraVectorStore from_texts (#20284 )	5 months ago
Tomer Cagan	463160c3f6	community: fix `DirectoryLoader` progress bar (#19821 ) Description: currently, the `DirectoryLoader` progress-bar maximum value is based on an incorrect number of files to process In langchain_community/document_loaders/directory.py:127: ```python paths = p.rglob(self.glob) if self.recursive else p.glob(self.glob) items = [ path for path in paths if not (self.exclude and any(path.match(glob) for glob in self.exclude)) ] ``` `paths` returns both files and directories. `items` is later used to determine the maximum value of the progress-bar which gives an incorrect progress indication.	5 months ago
Pengcheng Liu	ecd19a9e58	community[patch]: Add function call support in Tongyi chat model. (#20119 ) - [ ] PR message: - Description: This pr adds function calling support in Tongyi chat model. - Issue: None - Dependencies: None - Twitter handle: None Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com>	5 months ago
kaijietti	80679ab906	zep[patch]: implement add_messages and aadd_messages (#20099 ) This PR implement `add_messages` and `aadd_messages` to avoid unnecessary round-trips.	5 months ago
Sevin F. Varoglu	3f156e0ece	community[minor]: add ChatOctoAI (#20059 ) This PR adds ChatOctoAI, a chat model integration for OctoAI.	5 months ago
Eun Hye Kim	b34f1086fe	community[patch]: Add streaming logic in ChatHuggingFace (#18784 ) - Add functions (_stream, _astream) - Connect to _generate and _agenerate Thank you for contributing to LangChain! - [x] PR title: "community: Add streaming logic in ChatHuggingFace" - [x] PR message: *Delete this entire checklist* and replace with - Description: Addition functions (_stream, _astream) and connection to _generate and _agenerate - Issue: #18782 - Dependencies: none - Twitter handle: @lunara_x	5 months ago
pjb157	479be3cc91	community[minor]: Unify Titan Takeoff Integrations and Adding Embedding Support (#18775 ) Community: Unify Titan Takeoff Integrations and Adding Embedding Support Description: Titan Takeoff no longer reflects this either of the integrations in the community folder. The two integrations (TitanTakeoffPro and TitanTakeoff) where causing confusion with clients, so have moved code into one place and created an alias for backwards compatibility. Added Takeoff Client python package to do the bulk of the work with the requests, this is because this package is actively updated with new versions of Takeoff. So this integration will be far more robust and will not degrade as badly over time. Issue: Fixes bugs in the old Titan integrations and unified the code with added unit test converge to avoid future problems. Dependencies: Added optional dependency takeoff-client, all imports still work without dependency including the Titan Takeoff classes but just will fail on initialisation if not pip installed takeoff-client Twitter @MeryemArik9 Thanks all :) --------- Co-authored-by: Bagatur <baskaryan@gmail.com> Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com>	5 months ago
Rahul Triptahi	2cbfc94bcb	community[patch]: Add support for authorized identities in PebbloSafeLoader. (#20055 ) Description: Add support for authorized identities in PebbloSafeLoader. Now with this change, PebbloSafeLoader will extract authorized_identities from metadata and send it to pebblo server Dependencies: None Documentation: None Signed-off-by: Rahul Tripathi <rauhl.psit.ec@gmail.com> Co-authored-by: Rahul Tripathi <rauhl.psit.ec@gmail.com>	5 months ago
Guangdong Liu	b78ede2f96	community[patch]: standardize init args (#20166 ) Related to https://github.com/langchain-ai/langchain/issues/20085 @baskaryan	5 months ago
Guangdong Liu	3729bec1a2	community[patch]: standardize init args (#20210 ) Related to https://github.com/langchain-ai/langchain/issues/20085 @baskaryan	5 months ago
sdan	a7c5e41443	community[minor]: Added VLite as VectorStore (#20245 ) Support [VLite](https://github.com/sdan/vlite) as a new VectorStore type. Description: vlite is a simple and blazing fast vector database(vdb) made with numpy. It abstracts a lot of the functionality around using a vdb in the retrieval augmented generation(RAG) pipeline such as embeddings generation, chunking, and file processing while still giving developers the functionality to change how they're made/stored. Before submitting: Added tests [here](`c09c2ebd5c/libs/community/tests/integration_tests/vectorstores/test_vlite.py`) Added ipython notebook [here](`c09c2ebd5c/docs/docs/integrations/vectorstores/vlite.ipynb`) Added simple docs on how to use [here](`c09c2ebd5c/docs/docs/integrations/providers/vlite.mdx`) Profiles Maintainers: @sdan Twitter handles: [@sdand](https://x.com/sdand) --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	5 months ago
Hyeongchan Kim	7824291252	community[patch]: Fix not to cast to str type when `file_path` is None (#20057 ) From `langchain_community 0.0.30`, there's a bug that cannot send a file-like object via `file` parameter instead of `file path` due to casting the `file_path` to str type even if `file_path` is None. which means that when I call the `partition_via_api()`, exactly one of `filename` and `file` must be specified by the following error message. however, from `langchain_community 0.0.30`, `file_path` is casted into `str` type even `file_path` is None in `get_elements_from_api()` and got an error at `exactly_one(filename=filename, file=file)`. here's an error message ``` ---> 51 exactly_one(filename=filename, file=file) 53 if metadata_filename and file_filename: 54 raise ValueError( 55 "Only one of metadata_filename and file_filename is specified. " 56 "metadata_filename is preferred. file_filename is marked for deprecation.", 57 ) File /opt/homebrew/lib/python3.11/site-packages/unstructured/partition/common.py:441, in exactly_one(**kwargs) 439 else: 440 message = f"{names[0]} must be specified." --> 441 raise ValueError(message) ValueError: Exactly one of filename and file must be specified. ``` So, I simply made a change that casting to str type when `file_path` is not None. I use `UnstructuredAPIFileLoader` like below. ``` from langchain_community.document_loaders.unstructured import UnstructuredAPIFileLoader documents: list = UnstructuredAPIFileLoader( file_path=None, file=file, # file-like object, io.BytesIO type mode='elements', url='http://127.0.0.1:8000/general/v0/general', content_type='application/pdf', metadata_filename='asdf.pdf', ).load_and_split() ```	5 months ago
MacanPN	bce69ae43d	community[patch]: Changes to base_o365 and sharepoint document loaders (#20373 ) ## Description: The PR introduces 3 changes: 1. added `recursive` property to `O365BaseLoader`. (To keep the behavior unchanged, by default is set to `False`). When `recursive=True`, `_load_from_folder()` also recursively loads all nested folders. 2. added `folder_id` to SharePointLoader.(similar to (this PR)[https://github.com/langchain-ai/langchain/pull/10780] ) This provides an alternative to `folder_path` that doesn't seem to reliably work. 3. when none of `document_ids`, `folder_id`, `folder_path` is provided, the loader fetches documets from root folder. Combined with `recursive=True` this provides an easy way of loading all compatible documents from SharePoint. The PR contains the same logic as [this stale PR](https://github.com/langchain-ai/langchain/pull/10780) by @WaleedAlfaris. I'd like to ask his blessing for moving forward with this one. ## Issue: - As described in https://github.com/langchain-ai/langchain/issues/19938 and https://github.com/langchain-ai/langchain/pull/10780 the sharepoint loader often does not seem to work with folder_path. - Recursive loading of subfolders is a missing functionality ## Dependecies: None Twitter handle: @martintriska1 @WRhetoric This is my first PR here, please be gentle :-) Please review @baskaryan --------- Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com>	5 months ago
Sevin F. Varoglu	54d388d898	community[patch]: update OctoAI endpoint to subclass BaseOpenAI (#19757 ) This PR updates OctoAIEndpoint LLM to subclass BaseOpenAI as OctoAI is an OpenAI-compatible service. The documentation and tests have also been updated.	5 months ago
Benito Geordie	57b226532d	community[minor]: Added integrations for ThirdAI's NeuralDB as a Retriever (#17334 ) Description: Adds ThirdAI NeuralDB retriever integration. NeuralDB is a CPU-friendly and fine-tunable text retrieval engine. We previously added a vector store integration but we think that it will be easier for our customers if they can also find us under under langchain-community/retrievers. --------- Co-authored-by: kartikTAI <129414343+kartikTAI@users.noreply.github.com> Co-authored-by: Kartik Sarangmath <kartik@thirdai.com>	5 months ago
WeichenXu	e9fc87aab1	community[patch]: Make ChatDatabricks model supports streaming response (#19912 ) Description: Make ChatDatabricks model supports stream Issue: N/A Dependencies: MLflow nightly build version (we will release next MLflow version soon) Twitter handle: N/A Manually test: (Before testing, please install `pip install git+https://github.com/mlflow/mlflow.git`) ```python # Test Databricks Foundation LLM model from langchain.chat_models import ChatDatabricks chat_model = ChatDatabricks( endpoint="databricks-llama-2-70b-chat", max_tokens=500 ) from langchain_core.messages import AIMessageChunk for chunk in chat_model.stream("What is mlflow?"): print(chunk.content, end="\|") ``` - [x] Add tests and docs: If you're adding a new integration, please include 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. It lives in `docs/docs/integrations` directory. - [x] Lint and test: Run `make format`, `make lint` and `make test` from the root of the package(s) you've modified. See contribution guidelines for more: https://python.langchain.com/docs/contributing/ Additional guidelines: - Make sure optional dependencies are imported within a function. - Please do not add dependencies to pyproject.toml files (even optional ones) unless they are required for unit tests. - Most PRs should not touch more than one package. - Changes should be backwards compatible. - If you are adding something to community, do not re-import it in langchain. If no one reviews your PR within a few days, please @-mention one of baskaryan, efriis, eyurtsev, hwchase17. --------- Signed-off-by: Weichen Xu <weichen.xu@databricks.com> Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com> Co-authored-by: Bagatur <baskaryan@gmail.com>	5 months ago
Dhruv Chawla	d6d559d50d	community[minor]: add UpTrainCallbackHandler (#19956 ) - Description: This PR adds a callback handler for UpTrain. It performs evaluations in the RAG pipeline to check the quality of retrieved documents, generated queries and responses. - Dependencies: - The UpTrainCallbackHandler requires the uptrain package --------- Co-authored-by: Eugene Yurtsev <eugene@langchain.dev>	5 months ago
Ravindu Somawansa	5acc7ba622	community[minor]: Add glue catalog loader (#20220 ) Add Glue Catalog loader	5 months ago
Martín Gotelli Ferenaz	b48add4353	community[patch]: Fix pgvector deprecated filter clause usage with OR and AND conditions (#20446 ) Description: Support filter by OR and AND for deprecated PGVector version Issue: #20445 Dependencies: N/A Twitter handle: @martinferenaz	5 months ago
Eugene Yurtsev	c50099161b	community[patch]: Use uuid4 not uuid1 (#20487 ) Using UUID1 is incorrect since it's time dependent, which makes it easy to generate the exact same uuid	5 months ago
Erick Friis	86cf1d3ee1	community: release 0.0.33 (#20490 )	5 months ago
Leonid Kuligin	676c68d318	community[patch]: deprecating remaining google_community integrations (#20471 ) Deprecating remaining google community integrations	5 months ago
balloonio	b66a4f48fa	community[patch]: Invoke callback prior to yielding token fix [DeepInfra] (#20427 ) - [x] PR title: community[patch]: Invoke callback prior to yielding token fix for [DeepInfra] - [x] PR message: - Description: Invoke callback prior to yielding token in stream method in [DeepInfra] - Issue: https://github.com/langchain-ai/langchain/issues/16913 - Dependencies: None - Twitter handle: @bolun_zhang If no one reviews your PR within a few days, please @-mention one of baskaryan, efriis, eyurtsev, hwchase17.	5 months ago
Juan Carlos José Camacho	450c458f8f	community[minor]: Add Datahareld tool (#19680 ) Description: Integrate [dataherald](https://www.dataherald.com) tool, It is a natural language-to-SQL tool. Dependencies: Install dataherald sdk to use it, ``` pip install dataherald ``` --------- Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com> Co-authored-by: Bagatur <baskaryan@gmail.com> Co-authored-by: Christophe Bornet <cbornet@hotmail.com>	5 months ago
Egor Krasheninnikov	c8391d4ff1	community[patch]: Fix YandexGPT embeddings (#19720 ) Fix of YandexGPT embeddings. The current version uses a single `model_name` for queries and documents, essentially making the `embed_documents` and `embed_query` methods the same. Yandex has a different endpoint (`model_uri`) for encoding documents, see [this](https://yandex.cloud/en/docs/yandexgpt/concepts/embeddings). The bug may impact retrievers built with `YandexGPTEmbeddings` (for instance FAISS database as retriever) since they use both `embed_documents` and `embed_query`. A simple snippet to test the behaviour: ```python from langchain_community.embeddings.yandex import YandexGPTEmbeddings embeddings = YandexGPTEmbeddings() q_emb = embeddings.embed_query('hello world') doc_emb = embeddings.embed_documents(['hello world', 'hello world']) q_emb == doc_emb[0] ``` The response is `True` with the current version and `False` with the changes I made. Twitter: @egor_krash --------- Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com> Co-authored-by: Bagatur <baskaryan@gmail.com>	5 months ago
Guangdong Liu	4be7ca7b4c	community[patch]:sparkllm standardize init args (#20194 ) Related to https://github.com/langchain-ai/langchain/issues/20085 @baskaryan	5 months ago
Yuki Oshima	0758da8940	community[patch]: Set default value for _ListSQLDatabaseToolInput tool_input (#20409 ) Description: `_ListSQLDatabaseToolInput` raise error if model returns `{}`. For example, gpt-4-turbo returns `{}` with SQL Agent initialized by `create_sql_agent`. So, I set default value `""` for `_ListSQLDatabaseToolInput` tool_input. This is actually a gpt-4-turbo issue, not a LangChain issue, but I thought it would be helpful to set a default value `""`. This problem is discussed in detail in the following Issue. Issue: https://github.com/langchain-ai/langchain/issues/20405 Dependencies: none Sorry, I did not add or change the test code, as tests for this components was not exist . However, I have tested the following code based on the [SQL Agent Document](https://python.langchain.com/docs/use_cases/sql/agents/), to make sure it works. ``` from langchain_community.agent_toolkits.sql.base import create_sql_agent from langchain_community.utilities.sql_database import SQLDatabase from langchain_openai import ChatOpenAI db = SQLDatabase.from_uri("sqlite:///Chinook.db") llm = ChatOpenAI(model="gpt-4-turbo", temperature=0) agent_executor = create_sql_agent(llm, db=db, agent_type="openai-tools", verbose=True) result = agent_executor.invoke("List the total sales per country. Which country's customers spent the most?") print(result["output"]) ```	5 months ago
ccurme	38faa74c23	community[patch]: update use of deprecated llm methods (#20393 ) .predict and .predict_messages for BaseLanguageModel and BaseChatModel	5 months ago
Corey Zumar	3a068b26f3	community[patch]: Databricks - fix scope of dangerous deserialization error in Databricks LLM connector (#20368 ) fix scope of dangerous deserialization error in Databricks LLM connector --------- Signed-off-by: dbczumar <corey.zumar@databricks.com>	5 months ago
balloonio	e7b1a44c5b	community[patch]: Invoke callback prior to yielding token fix for Llamafile (#20365 ) - [x] PR title: community[patch]: Invoke callback prior to yielding token fix for Llamafile - [x] PR message: - Description: Invoke callback prior to yielding token in stream method in community llamafile.py - Issue: https://github.com/langchain-ai/langchain/issues/16913 - Dependencies: None - Twitter handle: @bolun_zhang If no one reviews your PR within a few days, please @-mention one of baskaryan, efriis, eyurtsev, hwchase17.	5 months ago
balloonio	93caa568f9	community[patch]: Invoke callback prior to yielding token fix for HuggingFaceEndpoint (#20366 ) - [x] PR title: community[patch]: Invoke callback prior to yielding token fix for HuggingFaceEndpoint - [x] PR message: - Description: Invoke callback prior to yielding token in stream method in community HuggingFaceEndpoint - Issue: https://github.com/langchain-ai/langchain/issues/16913 - Dependencies: None - Twitter handle: @bolun_zhang If no one reviews your PR within a few days, please @-mention one of baskaryan, efriis, eyurtsev, hwchase17. --------- Co-authored-by: Chester Curme <chester.curme@gmail.com>	5 months ago
Nicolas	ad04585e30	community[minor]: Firecrawl.dev integration (#20364 ) Added the [FireCrawl](https://firecrawl.dev) document loader. Firecrawl crawls and convert any website into LLM-ready data. It crawls all accessible subpages and give you clean markdown for each. - Description: Adds FireCrawl data loader - Dependencies: firecrawl-py - Twitter handle: @mendableai ccing contributors: (@ericciarla @nickscamara) --------- Co-authored-by: Bagatur <baskaryan@gmail.com> Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com>	5 months ago
P. Taylor Goetz	9317df7f16	community[patch]: Add "model" attribute to the payload sent to Ollama in `ChatOllama` (#20354 ) Example Ollama API calls: Request without "model": ``` curl --location 'http://localhost:11434/api/chat' \ --header 'Content-Type: application/json' \ --data '{ "messages": [ { "role": "user", "content": "What is the capitol of PA?" } ], "stream": false }' ``` Response: ``` {"error":"model is required"} ``` Request with "model": ``` curl --location 'http://localhost:11434/api/chat' \ --header 'Content-Type: application/json' \ --data '{ "model": "openchat", "messages": [ { "role": "user", "content": "What is the capitol of PA?" } ], "stream": false }' ``` Response: ``` { "eval_duration" : 733248000, "created_at" : "2024-04-11T23:04:08.735766843Z", "model" : "openchat", "message" : { "content" : " The capital city of Pennsylvania is Harrisburg.", "role" : "assistant" }, "total_duration" : 3138731168, "prompt_eval_count" : 25, "load_duration" : 466562959, "done" : true, "prompt_eval_duration" : 1938495000, "eval_count" : 10 } ```	5 months ago
Alex Sherstinsky	fad0962643	community: for Predibase -- enable both Predibase-hosted and HuggingFace-hosted fine-tuned adapter repositories (#20370 )	5 months ago
Isak Nyberg	bac9fb9a7c	community: add gpt-4 pricing in callback (#20292 ) Added the pricing for `gpt-4-turbo` and `gpt-4-turbo-2024-04-09` in the callback method. related to issue #17173 https://openai.com/pricing#language-models	6 months ago
Leonid Ganeline	7cf2d2759d	community[patch]: docstrings update (#20301 ) Added missed docstrings. Format docstings to the consistent form.	6 months ago
Eugene Yurtsev	22fd844e8a	community[patch]: Add deprecation warnings to postgres implementation (#20222 ) Add deprecation warnings to postgres implementation that are in langchain-postgres.	6 months ago
Leonid Ganeline	4cb5f4c353	community[patch]: import flattening fix (#20110 ) This PR should make it easier for linters to do type checking and for IDEs to jump to definition of code. See #20050 as a template for this PR. - As a byproduct: Added 3 missed `test_imports`. - Added missed `SolarChat` in to __init___.py Added it into test_import ut. - Added `# type: ignore` to fix linting. It is not clear, why linting errors appear after ^ changes. --------- Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com>	6 months ago
Bagatur	e39fdfddf1	community[patch]: Release 0.0.32 (#20236 )	6 months ago
Chip Davis	806d4ae48f	community[patch]: fixed multithreading returning List[List[Documents]] instead of List[Documents] (#20230 ) Description: When multithreading is set to True and using the DirectoryLoader, there was a bug that caused the return type to be a double nested list. This resulted in other places upstream not being able to utilize the from_documents method as it was no longer a `List[Documents]` it was a `List[List[Documents]]`. The change made was to just loop through the `future.result()` and yield every item. Issue: #20093 Dependencies: N/A Twitter handle: N/A	6 months ago
seray	add31f46d0	community[patch]: OpenLLM Async Client Fixes and Timeout Parameter (#20007 ) Same changes as this merged [PR](https://github.com/langchain-ai/langchain/pull/17478) (https://github.com/langchain-ai/langchain/pull/17478), but for the async client, as the same issues persist. - Replaced 'responses' attribute of OpenLLM's GenerationOutput schema to 'outputs'. reference: `66de54eae7/openllm-core/src/openllm_core/_schemas.py (L135)` - Added timeout parameter for the async client. --------- Co-authored-by: Seray Arslan <seray.arslan@knime.com>	6 months ago
Erick Friis	37a9e23c05	community: switch to falkordb python client (#20229 )	6 months ago
David Lee	0394c6e126	community[minor]: add allow_dangerous_requests for OpenAPI toolkits (#19493 ) OpenAPI allow_dangerous_requests: community: add allow_dangerous_requests for OpenAPI toolkits Description: a description of the change Due to BaseRequestsTool changes, we need to pass allow_dangerous_requests manually. `b617085af0/libs/community/langchain_community/tools/requests/tool.py (L26-L46)` While OpenAPI toolkits didn't pass it in the arguments. `b617085af0/libs/community/langchain_community/agent_toolkits/openapi/planner.py (L262-L269)` Issue: the issue # it fixes, if applicable https://github.com/langchain-ai/langchain/issues/19440 If not passing allow_dangerous_requests, it won't be able to do requests. Dependencies: any dependencies required for this change Not much --------- Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com> Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com>	6 months ago
Timothy	0c848a25ad	community[patch]: GCSDirectoryLoader bugfix (#20005 ) - Description: Bug fix. Removed extra line in `GCSDirectoryLoader` to allow catching Exceptions. Now also logs the file path if Exception is raised for easier debugging. - Issue: #20198 Bug since langchain-community==0.0.31 - Dependencies: No change - Twitter handle: timothywong731 --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	6 months ago
jeff kit	ac42e96e4c	community[patch], langchain[minor]: Enhance Tencent Cloud VectorDB, langchain: make Tencent Cloud VectorDB self query retrieve compatible (#19651 ) - make Tencent Cloud VectorDB support metadata filtering. - implement delete function for Tencent Cloud VectorDB. - support both Langchain Embedding model and Tencent Cloud VDB embedding model. - Tencent Cloud VectorDB support filter search keyword, compatible with langchain filtering syntax. - add Tencent Cloud VectorDB TranslationVisitor, now work with self query retriever. - more documentations. --------- Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com>	6 months ago
Bagatur	1a34c65e01	community[patch]: pass through sql agent kwargs (#19962 ) Fix #19961	6 months ago
Guangdong Liu	97d91ec17c	community[patch]: standardize baichuan init args (#20209 ) Related to https://github.com/langchain-ai/langchain/issues/20085 @baskaryan	6 months ago
Piyush Jain	cd7abc495a	community[minor]: add neptune analytics graph (#20047 ) Replacement for PR [#19772](https://github.com/langchain-ai/langchain/pull/19772). --------- Co-authored-by: Dave Bechberger <dbechbe@amazon.com> Co-authored-by: bechbd <bechbd@users.noreply.github.com>	6 months ago
Shuqian	ad9750403b	community[minor]: add bedrock anthropic callback for token usage counting (#19864 ) Description: add bedrock anthropic callback for token usage counting, consulted openai callback. --------- Co-authored-by: Massimiliano Pronesti <massimiliano.pronesti@gmail.com>	6 months ago
Prince Canuma	1f9f4d8742	community[minor]: Add support for MLX models (chat & llm) (#18152 ) Description: This PR adds support for MLX models both chat (i.e., instruct) and llm (i.e., pretrained) types/ Dependencies: mlx, mlx_lm, transformers Twitter handle: @Prince_Canuma --------- Co-authored-by: Bagatur <baskaryan@gmail.com> Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com>	6 months ago
Leonid Ganeline	2f8dd1a161	community[patch]: `cross_encoders` flatten namespaces (#20183 ) Issue `langchain_community.cross_encoders` didn't have flattening namespace code in the __init__.py file. Changes: - added code to flattening namespaces (used #20050 as a template) - added ut for a change - added missed `test_imports` for `chat_loaders` and `chat_message_histories` modules	6 months ago
Alex Sherstinsky	5f563e040a	community: extend Predibase integration to support fine-tuned LLM adapters (#19979 ) - [x] PR title: "package: description" - Where "package" is whichever of langchain, community, core, experimental, etc. is being modified. Use "docs: ..." for purely docs changes, "templates: ..." for template changes, "infra: ..." for CI changes. - Example: "community: add foobar LLM" - [x] PR message: *Delete this entire checklist* and replace with - Description: Langchain-Predibase integration was failing, because it was not current with the Predibase SDK; in addition, Predibase integration tests were instantiating the Langchain Community `Predibase` class with one required argument (`model`) missing. This change updates the Predibase SDK usage and fixes the integration tests. - Twitter handle: `@alexsherstinsky` - [x] Lint and test: Run `make format`, `make lint` and `make test` from the root of the package(s) you've modified. See contribution guidelines for more: https://python.langchain.com/docs/contributing/ Additional guidelines: - Make sure optional dependencies are imported within a function. - Please do not add dependencies to pyproject.toml files (even optional ones) unless they are required for unit tests. - Most PRs should not touch more than one package. - Changes should be backwards compatible. - If you are adding something to community, do not re-import it in langchain. If no one reviews your PR within a few days, please @-mention one of baskaryan, efriis, eyurtsev, hwchase17. --------- Co-authored-by: Erick Friis <erick@langchain.dev>	6 months ago
Bagatur	5ae0e687b3	docs: use standard openai params (#20160 ) Part of #20085	6 months ago
david02871	e1a24d09c5	community: Add PHP language parser to document_loaders (#19850 ) Description: Added a PHP language parser to document_loaders Issue: N/A Dependencies: N/A Twitter handle: N/A --------- Co-authored-by: Chester Curme <chester.curme@gmail.com>	6 months ago
Marlene	2f03bc397e	Community: Updating Azure Retriever and Docs to be Azure AI Search instead of Azure Cognitive Search (#19925 ) Last year Microsoft [changed the name](https://learn.microsoft.com/en-us/azure/search/search-what-is-azure-search) of Azure Cognitive Search to Azure AI Search. This PR updates the Langchain Azure Retriever API and it's associated docs to reflect this change. It may be confusing for users to see the name Cognitive here and AI in the Microsoft documentation which is why this is needed. I've also added a more detailed example to the Azure retriever doc page. There are more places that need a similar update but I'm breaking it up so the PRs are not too big 😄 Fixing my errors from the previous PR. Twitter: @marlene_zw Two new tests added to test backward compatibility in `libs/community/tests/integration_tests/retrievers/test_azure_cognitive_search.py` --------- Co-authored-by: Chester Curme <chester.curme@gmail.com>	6 months ago
Rahul Triptahi	820b713086	community[minor]: Add support for Pebblo cloud_api_key in PebbloSafeLoader (#19855 ) Description: _PebbloSafeLoader_: Add support for pebblo's cloud api-key in PebbloSafeLoader - This Pull request enables PebbloSafeLoader to accept pebblo's cloud api-key and send the semantic classification data to pebblo cloud. Documentation: Updated Unit test: Added Issue: NA Dependencies: - None Twitter handle: @rahul_tripathi2 Signed-off-by: Rahul Tripathi <rauhl.psit.ec@gmail.com> Co-authored-by: Rahul Tripathi <rauhl.psit.ec@gmail.com>	6 months ago
Eugene Yurtsev	520ff50adc	community[patch]: Improve import callbacks to make it IDE friendly (#20050 ) * declares __all__ as a list of strings (instead of dynamically computing it) * import type definitions when TYPE_CHECKING is true	6 months ago
Leonid Ganeline	3aacd11846	community[minor]: added missed class to __all__ (#19888 ) Added missed `UnstructuredCHMLoader` class to the document_loader.\_\_init\_\_.py \_\_all\_\_	6 months ago
Tomaz Bratanic	df25829f33	community[minor]: Add metadata filtering support for neo4j vector (#20001 )	6 months ago
Ben Mitchell	b52b78478f	community[minor]: Implement Async OpenSearch `afrom_texts` & `afrom_embeddings` (#20009 ) - Description: Adds async variants of afrom_texts and afrom_embeddings into `OpenSearchVectorSearch`, which allows for `afrom_documents` to be called. - Issue: I implemented this because my use case involves an async scraper generating documents as and when they're ready to be ingested by Embedding/OpenSearch - Dependencies: None that I'm aware Co-authored-by: Ben Mitchell <b.mitchell@reply.com>	6 months ago
Tomaz Bratanic	09a0ecd000	langchain[minor]: Tests update metadata filtering examples of documents (#19963 ) Removing metadata properties that are dicts as some databases don't support that, and those properties aren't used in tests anyhow..	6 months ago
happy-go-lucky	c6432abdbe	community[patch]: Implement delete method and all async methods in opensearch_vector_search (#17321 ) - Description: In order to use index and aindex in libs/langchain/langchain/indexes/_api.py, I implemented delete method and all async methods in opensearch_vector_search - Dependencies: No changes	6 months ago
Cheng, Penghui	cc407e8a1b	community[minor]: weight only quantization with intel-extension-for-transformers. (#14504 ) Support weight only quantization with intel-extension-for-transformers. [Intel® Extension for Transformers](https://github.com/intel/intel-extension-for-transformers) is an innovative toolkit to accelerate Transformer-based models on Intel platforms, in particular effective on 4th Intel Xeon Scalable processor [Sapphire Rapids](https://www.intel.com/content/www/us/en/products/docs/processors/xeon-accelerated/4th-gen-xeon-scalable-processors.html) (codenamed Sapphire Rapids). The toolkit provides the below key features: * Seamless user experience of model compressions on Transformer-based models by extending [Hugging Face transformers](https://github.com/huggingface/transformers) APIs and leveraging [Intel® Neural Compressor](https://github.com/intel/neural-compressor) * Advanced software optimizations and unique compression-aware runtime. * Optimized Transformer-based model packages. * [NeuralChat](https://github.com/intel/intel-extension-for-transformers/blob/main/intel_extension_for_transformers/neural_chat), a customizable chatbot framework to create your own chatbot within minutes by leveraging a rich set of plugins and SOTA optimizations. * [Inference](https://github.com/intel/intel-extension-for-transformers/blob/main/intel_extension_for_transformers/llm/runtime/graph) of Large Language Model (LLM) in pure C/C++ with weight-only quantization kernels. This PR is an integration of weight only quantization feature with intel-extension-for-transformers. Unit test is in lib/langchain/tests/integration_tests/llm/test_weight_only_quantization.py The notebook is in docs/docs/integrations/llms/weight_only_quantization.ipynb. The document is in docs/docs/integrations/providers/weight_only_quantization.mdx. --------- Signed-off-by: Cheng, Penghui <penghui.cheng@intel.com> Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com> Co-authored-by: Bagatur <baskaryan@gmail.com>	6 months ago
Eugene Yurtsev	d293431e10	core[minor]: Add aload to document loader (#19936 ) Add aload to document loader	6 months ago
Leonid Kuligin	eb0521064e	deprecating integrations moved to langchain_google_community (#19841 ) Thank you for contributing to LangChain! - [ ] PR title: "community: deprecating integrations moved to langchain_google_community" - [ ] PR message: deprecating integrations moved to langchain_google_community --------- Co-authored-by: ccurme <chester.curme@gmail.com>	6 months ago
Erick Friis	f0d5b59962	core[patch]: remove requests (#19891 ) Removes required usage of `requests` from `langchain-core`, all of which has been deprecated. - removes Tracer V1 implementations - removes old `try_load_from_hub` github-based hub implementations Removal done in a way where imports will still succeed, and usage will fail with a `RuntimeError`.	6 months ago
Peter Vandenabeele	e830a4e731	community[patch]: Add remove_comments option (default True): do not extract html comments (#13259 ) - Description: add `remove_comments` option (default: True): do not extract html _comments_, - Issue: None, - Dependencies: None, - Tag maintainer: @nfcampos , - Twitter handle: peter_v I ran `make format`, `make lint` and `make test`. Discussion: I my use case, I prefer to not have the comments in the extracted text: * e.g. from a Google tag that is added in the html as comment * e.g. content that the authors have temporarily hidden to make it non visible to the regular reader Removing the comments makes the extracted text more alike the intended text to be seen by the reader. Choice to make: do we prefer to make the default for this `remove_comments` option to be True or False? I have changed it to True in a second commit, since that is how I would prefer to use it by default. Have the cleaned text (without technical Google tags etc.) and also closer to the actually visible and intended content. I am not sure what is best aligned with the conventions of langchain in general ... INITIAL VERSION (new version above): ~Choice to make: do we prefer to make the default for this `ignore_comments` option to be True or False? I have set it to False now to be backwards compatible. On the other hand, I would use it mostly with True. I am not sure what is best aligned with the conventions of langchain in general ...~ --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	6 months ago
Jamsheed Mistri	4f70bc119d	community[minor]: add Layerup Security integration (#19787 ) Description: adds integration with [Layerup Security](https://uselayerup.com). Docs can be found [here](https://docs.uselayerup.com). Integrates directly with our Python SDK. Dependencies: [LayerupSecurity](https://pypi.org/project/LayerupSecurity/) Note: all methods for our product require a paid API key, so I only included 1 test which checks for an invalid API key response. I have tested extensively locally. Twitter handle: [@layerup_](https://twitter.com/layerup_) --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	6 months ago
Anıl Berk Altuner	4384fa8e49	community[minor]: Add Dria retriever (#17098 ) [Dria](https://dria.co/) is a hub of public RAG models for developers to both contribute and utilize a shared embedding lake. This PR adds a retriever that can retrieve documents from Dria.	6 months ago
Ethan Yang	48f84e253e	community[minor]: Add OpenVINO rerank model support (#19791 ) @eaidova @AlexKoff88 Could you help to review, thanks --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	6 months ago
Chenhui Zhang	a1f3e9f537	community[minor]: Update ChatZhipuAI to support GLM-4 model (#16695 ) Description: Update `ChatZhipuAI` to support the latest `glm-4` model. Issue: N/A Dependencies: httpx, httpx-sse, PyJWT The previous `ChatZhipuAI` implementation requires the `zhipuai` package, and cannot call the latest GLM model. This is because - The old version `zhipuai==1.` doesn't support the latest model. - `zhipuai==2.` requires `pydantic V2`, which is incompatible with 'langchain-community'. This re-implementation invokes the GLM model by sending HTTP requests to [open.bigmodel.cn](https://open.bigmodel.cn/dev/api) via the `httpx` package, and uses the `httpx-sse` package to handle stream events. --------- Co-authored-by: zR <2448370773@qq.com>	6 months ago

... 5 6 7 8 9 ...

1411 Commits (35ebd2620c56d8a109321d7eb3f7676e20175d69)