121 Commits (413a56b8f1a71613457cf14b027a51b95a0fd21f)

Author	SHA1	Message	Date
Bob Lin	e93be14c11	Improvement: Allow passing parameters to the underlying es_client. Closes: #14403 (#14435) ... ### Description In https://github.com/langchain-ai/langchain/issues/14403, the user mentioned that he hopes not to verify ssl and needs to pass more parameters I found that the `Elasticsearch` class [has very many parameters](98f2af2134/elasticsearch/_sync/client/__init__.py (L131-L191) ): <img width="1097" alt="Screenshot 2023-12-08 at 4 24 39 PM" src="https://github.com/langchain-ai/langchain/assets/10000925/f2201554-b41a-4388-a8e8-c14a2d0466d4"> In order to adapt to more situations, I want to add the kwargs parameter so that users can enter more `Elasticsearch` parameters. Like [redis](https://github.com/langchain-ai/langchain/blob/master/libs/langchain/langchain/vectorstores/redis/base.py#L253), [tair](https://github.com/langchain-ai/langchain/blob/master/libs/langchain/langchain/vectorstores/tair.py#L32), [myscale](https://github.com/langchain-ai/langchain/blob/master/libs/langchain/langchain/vectorstores/myscale.py#L112) and so on.	9 months ago
codehound42	8aa921d3a4	Support `score_threshold` in SupabaseVectorStore similarity search (#14439) ... Description: Add support for setting the `score_threshold` for similarity search in SupabaseVectoreStore. This pull request addresses issue #14438 Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	9 months ago
YISH	eecfa81918	Add the collection_description parameter to Milvus (#14524) ... Because Milvus' collection_name doesn't support UFT8 characters in other languages, I want the `collection_descriotion`. <!-- Thank you for contributing to LangChain! Replace this entire comment with: - **Description:** a description of the change, - **Issue:** the issue # it fixes (if applicable), - **Dependencies:** any dependencies required for this change, - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below), - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out! Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. It lives in `docs/extras` directory. If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17. -->	9 months ago
Evgenii Molov	b4ec340fb3	Fix failing serpapi response processing for Google Maps API (#14817) ... **Description:** Fix for processing for serpapi response for Google Maps API **Issue:** Due to the fact corresponding [api](https://serpapi.com/google-maps-api) returns 'local_results' as list, and old version requested `res["local_results"].keys()` of the list. As the result we got exception: ```AttributeError: 'list' object has no attribute 'keys'```. Way to reproduce wrong behaviour: ``` params = { "engine": "google_maps", "type": "search", "google_domain": "google.de", "ll": "@51.1917,10.525,14z", "hl": "de", "gl": "de", } search = SerpAPIWrapper(params=params) results = search.run("cafe") ``` --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com> Co-authored-by: Ran <rccalman@gmail.com>	9 months ago
YISH	da0f750a0b	Milvus allows to store metadata as json field (#14636) ... Because Milvus doesn't support nullable fields, but document metadata is very rich, so it makes more sense to store it as json. https://github.com/milvus-io/pymilvus/issues/1705#issuecomment-1731112372 <!-- Thank you for contributing to LangChain! Replace this entire comment with: - **Description:** a description of the change, - **Issue:** the issue # it fixes (if applicable), - **Dependencies:** any dependencies required for this change, - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below), - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out! Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. It lives in `docs/extras` directory. If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17. --> --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	9 months ago
Ashley Xu	0ce7858529	feat: add Google BigQueryVectorSearch in vectorstore (#14829) ... BigQuery vector search lets you use GoogleSQL to do semantic search, using vector indexes for fast but approximate results, or using brute force for exact results. This PR integrates LangChain vectorstore with BigQuery Vector Search. <!-- Thank you for contributing to LangChain! Replace this entire comment with: - **Description:** a description of the change, - **Issue:** the issue # it fixes (if applicable), - **Dependencies:** any dependencies required for this change, - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below), - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out! Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally. See contribution guidelines for more information on how to write/run tests, lint, etc: https://python.langchain.com/docs/contributing/ If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. It lives in `docs/extras` directory. If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17. --> --------- Co-authored-by: Vlad Kolesnikov <vladkol@google.com>	9 months ago
JaguarDB	02f59c2035	Use args option in jaguar so it takes more options in similarity search (#15080) ... - **Description:** replace score_threshold with args - **Issue:** needs a way to pass more options to similarity search - **Dependencies:** None - **Twitter handle:** @workbot --------- Co-authored-by: JY <jyjy@jaguardb>	9 months ago
chyroc	37ad6ec248	Refactor: use SecretStr for tongyi chat-model (#15102)	9 months ago
Shaurya Rohatgi	e1c2cd7a28	community: Semanticscholar tool to search 200M+ scientific articles (#15151) ... - **Description:** Tool now supports querying over 200 million scientific articles, vastly expanding its reach beyond the 2 million articles accessible through Arxiv. This update significantly broadens access to the entire scope of scientific literature. - **Dependencies:** semantischolar https://github.com/danielnsilva/semanticscholar - **Twitter handle:** @shauryr --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	9 months ago
dudub12	7e6b0056b8	SQLDatabase drop the column names in the result. (#15361) ... Fix for the following bug: https://github.com/langchain-ai/langchain/issues/15360 --------- Co-authored-by: dudu butbul <100126964+dudu-upstream@users.noreply.github.com>	9 months ago
chyroc	07d294b5ec	Fix: fix Bing Search empty result exception, fix #15384 (#15387) ... fix https://github.com/langchain-ai/langchain/issues/15384	9 months ago
Bagatur	1678d6ca17	langchain[patch], experimental[patch], docs: update tools imports (#15433)	9 months ago
Leonid Ganeline	b8c6ebf647	refactor `utils` (#15432) ... The `langchain` [still holds several artifacts](https://api.python.langchain.com/en/latest/langchain_api_reference.html#module-langchain.utils) that belongs to `community`. If they moved then `langchain.utils` namespace would be removed completely. - moved `ernie_functions` artifacts to `community`	9 months ago
Bagatur	fa5d49f2c1	docs, experimental[patch], langchain[patch], community[patch]: update storage imports (#15429) ... ran ```bash g grep -l "langchain.vectorstores" | xargs -L 1 sed -i '' "s/langchain\.vectorstores/langchain_community.vectorstores/g" g grep -l "langchain.document_loaders" | xargs -L 1 sed -i '' "s/langchain\.document_loaders/langchain_community.document_loaders/g" g grep -l "langchain.chat_loaders" | xargs -L 1 sed -i '' "s/langchain\.chat_loaders/langchain_community.chat_loaders/g" g grep -l "langchain.document_transformers" | xargs -L 1 sed -i '' "s/langchain\.document_transformers/langchain_community.document_transformers/g" g grep -l "langchain\.graphs" | xargs -L 1 sed -i '' "s/langchain\.graphs/langchain_community.graphs/g" g grep -l "langchain\.memory\.chat_message_histories" | xargs -L 1 sed -i '' "s/langchain\.memory\.chat_message_histories/langchain_community.chat_message_histories/g" gco master libs/langchain/tests/unit_tests/*/test_imports.py gco master libs/langchain/tests/unit_tests/**/test_public_api.py ```	9 months ago
Bagatur	480626dc99	docs, community[patch], experimental[patch], langchain[patch], cli[pa… (#15412) ... …tch]: import models from community ran ```bash git grep -l 'from langchain\.chat_models' | xargs -L 1 sed -i '' "s/from\ langchain\.chat_models/from\ langchain_community.chat_models/g" git grep -l 'from langchain\.llms' | xargs -L 1 sed -i '' "s/from\ langchain\.llms/from\ langchain_community.llms/g" git grep -l 'from langchain\.embeddings' | xargs -L 1 sed -i '' "s/from\ langchain\.embeddings/from\ langchain_community.embeddings/g" git checkout master libs/langchain/tests/unit_tests/llms git checkout master libs/langchain/tests/unit_tests/chat_models git checkout master libs/langchain/tests/unit_tests/embeddings/test_imports.py make format cd libs/langchain; make format cd ../experimental; make format cd ../core; make format ```	9 months ago
Mohammad Mohtashim	b6c57d38fa	Langchain_community: Small Fix when loading facebook messages (#15358) ... - **Description:** SingleFileFacebookMessengerChatLoader did not handle the case for when messages had stickers and/or photos so fixed that. - **Issue:** #15356 --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	9 months ago
Mateusz Szewczyk	cbfaccc424	WatsonxLLM updates/enhancements (#14598) ... - **Description:** updates/enhancements to IBM [watsonx.ai](https://www.ibm.com/products/watsonx-ai) LLM provider (prompt tuned models and prompt templates deployments support) - **Dependencies:** [ibm-watsonx-ai](https://pypi.org/project/ibm-watsonx-ai/), - **Tag maintainer:** : @hwchase17 , @eyurtsev , @baskaryan - **Twitter handle:** details in comment below. Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally. ✅ --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	9 months ago
Manjunath Janardhan	7a0feba9f7	GITLAB_URL should take default https://gitlab.com instead of error (#14638) ... The fix #14221 has broken default gitlab url which is forcing the users to specify GITLAB_URL for default one. With this fix if GITLAB_URL is not set, the default gitlab url will be taken. - **Description:** Add the GITHUB URL instead of None - **Issue:** the issue #14221 has broken the default github URL - **Dependencies:** None - **Tag maintainer:** @hwchase17 - **Twitter handle:** manjunath_shiva	9 months ago
David	dcf047c48f	add api_base to _client_params (community version of #14393) (#14644) ... - **Description:** This PR adds `api_base` to `_client_params` in the `chat_model` of LiteLLM to ensure it's included in API calls. Previously, `api_base` was set on the client but was not included in the parameters passed to the completion function. This change ensures that `api_base` is correctly passed to all API calls. - **Issue:** #14338 - **Tag maintainer:** @hwchase17 @agola11 - **Twitter handle:** @LMS_David_RS	9 months ago
xuxiang	dd1d818a82	Fixing the Issue with DashScopeEmbeddings Handling More than 25 Rows of Data (#14662) ... <!-- Thank you for contributing to LangChain! Replace this entire comment with: - **Description:** a description of the change, - **Issue:** the issue # it fixes (if applicable), - **Dependencies:** any dependencies required for this change, - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below), - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out! Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. It lives in `docs/extras` directory. If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17. --> This change addresses the issue where DashScopeEmbeddingAPI limits requests to 25 lines of data, and DashScopeEmbeddings did not handle cases with more than 25 lines, leading to errors. I have implemented a fix to manage data exceeding this limit efficiently. --------- Co-authored-by: xuxiang <xuxiang@aliyun.com>	9 months ago
Christophe Bornet	e2a8962ba6	Add AstraDB document loader (#14747) ... - **Description:** this adds the AstraDB document loader and an integration test - **Twitter handle:** cbornet_	9 months ago
Igor Dvorkin	76923e5743	Restore self message sent before OSX 12 Monterey (#14818) ... <!-- Thank you for contributing to LangChain! Replace this entire comment with: - **Description:** a description of the change, - **Issue:** the issue # it fixes (if applicable), - **Dependencies:** any dependencies required for this change, - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below), - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out! Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. It lives in `docs/extras` directory. If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17. -->	9 months ago
savoiepe	d006be60ec	Added more filtering options to pgvector vectorstore (#14852) ... - **Description:** Using PGVector vector store, it was only possible to filter for values equals, in or not in metadata. Extended this feature to work with the following keywords : IN, NIN, BETWEEN, GT, LT, NE, EQ, LIKE, CONTAINS, OR, AND --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	9 months ago
chyroc	32e96a471c	Refactor: use SecretStr for llm_rails embeddings (#15090)	9 months ago
chyroc	b440f92d81	Refactor: use SecretStr for embaas embeddings (#15091)	9 months ago
chyroc	ea6cf0f1b1	Refactor: use SecretStr for edenai embeddings (#15092)	9 months ago
chyroc	32e6e9de13	Refactor: use SecretStr for palm chat-model (#15100)	9 months ago
chyroc	b6952d41e5	Refactor: use SecretStr for GPTRouter chat-model (#15101)	9 months ago
Nan LI	f506b4cfd2	community: Integration of New Chat Model Based on ChatGLM3 via ZhipuAI API (#15105) ... - **Description:** - This PR introduces a significant enhancement to the LangChain project by integrating a new chat model powered by the third-generation base large model, ChatGLM3, via the zhipuai API. - This advanced model supports functionalities like function calls, code interpretation, and intelligent Agent capabilities. - The additions include the chat model itself, comprehensive documentation in the form of Python notebook docs, and thorough testing with both unit and integrated tests. - **Dependencies:** This update relies on the ZhipuAI package as a key dependency. - **Twitter handle:** If this PR receives spotlight attention, we would be honored to receive a mention for our integration of the advanced ChatGLM3 model via the ZhipuAI API. Kindly tag us at @kaiwu. To ensure quality and standards, we have performed extensive linting and testing. Commands such as make format, make lint, and make test have been run from the root of the modified package to ensure compliance with LangChain's coding standards. TO DO: Continue refining and enhancing both the unit tests and integrated tests. --------- Co-authored-by: jing <jingguo92@gmail.com> Co-authored-by: hyy1987 <779003812@qq.com> Co-authored-by: jianchuanqi <qijianchuan@hotmail.com> Co-authored-by: lirq <whuclarence@gmail.com> Co-authored-by: whucalrence <81530213+whucalrence@users.noreply.github.com> Co-authored-by: Jing Guo <48378126+JaneCrystall@users.noreply.github.com>	9 months ago
Hin	2cf1e73d12	Feat add volcano embedding (#14693) ... Description: Volcano Ark is an enterprise-grade large-model service platform for developers, providing a full range of functions and services such as model training, inference, evaluation, fine-tuning. You can visit its homepage at https://www.volcengine.com/docs/82379/1099455 for details. This change could help developers use the platform for embedding. Issue: None Dependencies: volcengine Tag maintainer: @baskaryan Twitter handle: @hinnnnnnnnnnnns --------- Co-authored-by: lujingxuansc <lujingxuansc@bytedance.com>	9 months ago
David Křístek	a010f29013	fix: call correct stream method in ollama (#15104) ... Co-authored-by: David Kristek <david@David--MacBook-Pro.local>	9 months ago
Christian Janiake	be578f32be	community:Lazy load wikipedia dump file (#15111) ... **Description:** the MWDumpLoader implementation currently does not support the lazy_load method, and the files are usually very large. We are proposing refactoring the load function, extracting two private functions with the functionality of loading the dump file and parsing a single page, to reuse the code in the lazy_load implementation.	9 months ago
chyroc	a4ae4bc361	feat: mask api_key for konko (#14010) ... for https://github.com/langchain-ai/langchain/issues/12165	9 months ago
joel-teratis	62d32bd214	fix(minor): added missing **kwargs parameter to chroma query function (#14919) ... **Description:** This PR adds the `**kwargs` parameter to six calls in the `chroma.py` package. All functions already were able to receive `kwargs` but they were discarded before. **Issue:** When passing `kwargs` to functions in the `chroma.py` package they are being ignored. For example: ``` chroma_instance.similarity_search_with_score( query, k=100, include=["metadatas", "documents", "distances", "embeddings"], # this parameter gets ignored ) ``` The `include` parameter does not get passed on to the next function and does not have any effect. **Dependencies:** None	9 months ago
NuODaniel	7773943a51	community:qianfan endpoint support init params & remove useless params definietion (#15381) ... - **Description:** - support custom kwargs in object initialization. For instantance, QPS differs from multiple object(chat/completion/embedding with diverse models), for which global env is not a good choice for configuration. - **Issue:** no - **Dependencies:** no - **Twitter handle:** no @baskaryan PTAL	9 months ago
Nuno Campos	99000c612e	Propagate context vars in all classes/methods (#15329) ... - Any direct usage of ThreadPoolExecutor or asyncio.run_in_executor needs manual handling of context vars <!-- Thank you for contributing to LangChain! Please title your PR "<package>: <description>", where <package> is whichever of langchain, community, core, experimental, etc. is being modified. Replace this entire comment with: - **Description:** a description of the change, - **Issue:** the issue # it fixes if applicable, - **Dependencies:** any dependencies required for this change, - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out! Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` from the root of the package you've modified to check this locally. See contribution guidelines for more information on how to write/run tests, lint, etc: https://python.langchain.com/docs/contributing/ If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. It lives in `docs/docs/integrations` directory. If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17. -->	9 months ago
Ankush Gola	7eec8f2487	Delete V1 tracer and refactor tracer tests to core (#15326)	9 months ago
chyroc	7ce338201c	Patch: improve check openai version (#15301)	9 months ago
Nuno Campos	eb5e250188	Propagate context vars in all classes/methods ... - Any direct usage of ThreadPoolExecutor or asyncio.run_in_executor needs manual handling of context vars	9 months ago
Shuai Liu	4b53440e70	Upgrades the Tongyi LLM and ChatTongyi Model (#14793) ... - **Description:** fixes and upgrades for the Tongyi LLM and ChatTongyi Model - Fixed typos; it should be `Tongyi`, not `OpenAI`. - Fixed a bug in `stream_generate_with_retry`; it's a real stream generator now. - Fixed a bug in `validate_environment`; the `dashscope_api_key` should be properly handled when set by environment variables or initialization parameters. - Changed the `dashscope` response to incremental output by setting the parameter `incremental_output`, which eliminates the need for the prefix-removal trick. - Removed some unused parameters, like `n`, `prefix_messages`. - Added `_stream` method. - Added async methods support, such as `_astream`, `_agenerate`, `_abatch`. - **Dependencies:** No new dependencies. - **Tag maintainer:** @hwchase17 > PS: Some may be confused about the terms `dashscope`, `tongyi`, and `Qwen`: > - `dashscope`: A platform to deploy LLMs and provide APIs to invoke the LLM. > - `tongyi`: A brand name or overall term about Alibaba Cloud's LLM/AI. > - `Qwen`: An LLM that is open-sourced and deployed in `dashscope`. > > We use the `dashscope` SDK to interact with the `tongyi`-`Qwen` LLM. --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	9 months ago
Diego Rani Mazine	ec72225265	refactor: enable connection pool usage in PGVector (#11514) ... - **Description:** `PGVector` refactored to use connection pool. - **Issue:** #11433, - **Tag maintainer:** @hwchase17 @eyurtsev, --------- Co-authored-by: Diego Rani Mazine <diego.mazine@mercadolivre.com> Co-authored-by: Nuno Campos <nuno@langchain.dev>	9 months ago
joshy-deshaw	bf5385592e	core, community: propagate context between threads (#15171) ... While using `chain.batch`, the default implementation uses a `ThreadPoolExecutor` and run the chains in separate threads. An issue with this approach is that that [the token counting callback](https://python.langchain.com/docs/modules/callbacks/token_counting) fails to work as a consequence of the context not being propagated between threads. This PR adds context propagation to the new threads and adds some thread synchronization in the OpenAI callback. With this change, the token counting callback works as intended. Having the context propagation change would be highly beneficial for those implementing custom callbacks for similar functionalities as well. --------- Co-authored-by: Nuno Campos <nuno@langchain.dev>	9 months ago
shroominic	694bbb14cd	community: fix typo in async ollama chat (#15276) ... Made a stupid typo in the last PR which got already merged😅	9 months ago
triThirty	fea4888e72	community: Enhance Github error prompt (#15248) ... - **Description:** The Github error prompt is confused because of JWT enctrypt to somebody not familiar with Github connection method. This PR is to add some useful error prompt to help users troubleshooting. - **Issue:** https://github.com/langchain-ai/langchain/issues/14550#issuecomment-1867445049 - **Dependencies:** None, - **Twitter handle:** None	9 months ago
Bob Lin	a464eb4394	community: Make doctran synchronous (#15264) ... ### Description I found that the methods in [the doctran library](https://github.com/psychic-api/doctran) have been restructured into [synchronized versions](14944a59f7), And [the example ipynb](https://github.com/psychic-api/doctran/blob/main/examples.ipynb) also shows that the code is synchronized, but the README has not been updated yet. so we need to modify the code and update the documentation. ### Issue https://github.com/langchain-ai/langchain/issues/14645	9 months ago
chyroc	6fb3cc6f27	Fix: Use `Union` instead of `|` to improve compatibility, fix #15244 (#15245)	9 months ago
chyroc	1abcf441ae	Refactor: use SecretStr for Predibase llms (#15119)	9 months ago
chyroc	0a9a73a9c9	Refactor: use SecretStr for PipelineAI llms (#15120)	9 months ago
chyroc	d63ceb65b3	Refactor: use SecretStr for StochasticAI llms (#15118)	9 months ago
chyroc	674fde87d2	Refactor: use SecretStr for VolcEngineMaas llms (#15117)	9 months ago