124 Commits (e57e50b2132f882cf334c27b518cc4f609be9081)

Author	SHA1	Message	Date
Leonid Ganeline	b8c6ebf647	refactor `utils` (#15432) ... The `langchain` [still holds several artifacts](https://api.python.langchain.com/en/latest/langchain_api_reference.html#module-langchain.utils) that belongs to `community`. If they moved then `langchain.utils` namespace would be removed completely. - moved `ernie_functions` artifacts to `community`	9 months ago
Bagatur	fa5d49f2c1	docs, experimental[patch], langchain[patch], community[patch]: update storage imports (#15429) ... ran ```bash g grep -l "langchain.vectorstores" | xargs -L 1 sed -i '' "s/langchain\.vectorstores/langchain_community.vectorstores/g" g grep -l "langchain.document_loaders" | xargs -L 1 sed -i '' "s/langchain\.document_loaders/langchain_community.document_loaders/g" g grep -l "langchain.chat_loaders" | xargs -L 1 sed -i '' "s/langchain\.chat_loaders/langchain_community.chat_loaders/g" g grep -l "langchain.document_transformers" | xargs -L 1 sed -i '' "s/langchain\.document_transformers/langchain_community.document_transformers/g" g grep -l "langchain\.graphs" | xargs -L 1 sed -i '' "s/langchain\.graphs/langchain_community.graphs/g" g grep -l "langchain\.memory\.chat_message_histories" | xargs -L 1 sed -i '' "s/langchain\.memory\.chat_message_histories/langchain_community.chat_message_histories/g" gco master libs/langchain/tests/unit_tests/*/test_imports.py gco master libs/langchain/tests/unit_tests/**/test_public_api.py ```	9 months ago
Bagatur	480626dc99	docs, community[patch], experimental[patch], langchain[patch], cli[pa… (#15412) ... …tch]: import models from community ran ```bash git grep -l 'from langchain\.chat_models' | xargs -L 1 sed -i '' "s/from\ langchain\.chat_models/from\ langchain_community.chat_models/g" git grep -l 'from langchain\.llms' | xargs -L 1 sed -i '' "s/from\ langchain\.llms/from\ langchain_community.llms/g" git grep -l 'from langchain\.embeddings' | xargs -L 1 sed -i '' "s/from\ langchain\.embeddings/from\ langchain_community.embeddings/g" git checkout master libs/langchain/tests/unit_tests/llms git checkout master libs/langchain/tests/unit_tests/chat_models git checkout master libs/langchain/tests/unit_tests/embeddings/test_imports.py make format cd libs/langchain; make format cd ../experimental; make format cd ../core; make format ```	9 months ago
Mohammad Mohtashim	b6c57d38fa	Langchain_community: Small Fix when loading facebook messages (#15358) ... - **Description:** SingleFileFacebookMessengerChatLoader did not handle the case for when messages had stickers and/or photos so fixed that. - **Issue:** #15356 --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	9 months ago
Mateusz Szewczyk	cbfaccc424	WatsonxLLM updates/enhancements (#14598) ... - **Description:** updates/enhancements to IBM [watsonx.ai](https://www.ibm.com/products/watsonx-ai) LLM provider (prompt tuned models and prompt templates deployments support) - **Dependencies:** [ibm-watsonx-ai](https://pypi.org/project/ibm-watsonx-ai/), - **Tag maintainer:** : @hwchase17 , @eyurtsev , @baskaryan - **Twitter handle:** details in comment below. Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally. ✅ --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	9 months ago
Manjunath Janardhan	7a0feba9f7	GITLAB_URL should take default https://gitlab.com instead of error (#14638) ... The fix #14221 has broken default gitlab url which is forcing the users to specify GITLAB_URL for default one. With this fix if GITLAB_URL is not set, the default gitlab url will be taken. - **Description:** Add the GITHUB URL instead of None - **Issue:** the issue #14221 has broken the default github URL - **Dependencies:** None - **Tag maintainer:** @hwchase17 - **Twitter handle:** manjunath_shiva	9 months ago
David	dcf047c48f	add api_base to _client_params (community version of #14393) (#14644) ... - **Description:** This PR adds `api_base` to `_client_params` in the `chat_model` of LiteLLM to ensure it's included in API calls. Previously, `api_base` was set on the client but was not included in the parameters passed to the completion function. This change ensures that `api_base` is correctly passed to all API calls. - **Issue:** #14338 - **Tag maintainer:** @hwchase17 @agola11 - **Twitter handle:** @LMS_David_RS	9 months ago
xuxiang	dd1d818a82	Fixing the Issue with DashScopeEmbeddings Handling More than 25 Rows of Data (#14662) ... <!-- Thank you for contributing to LangChain! Replace this entire comment with: - **Description:** a description of the change, - **Issue:** the issue # it fixes (if applicable), - **Dependencies:** any dependencies required for this change, - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below), - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out! Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. It lives in `docs/extras` directory. If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17. --> This change addresses the issue where DashScopeEmbeddingAPI limits requests to 25 lines of data, and DashScopeEmbeddings did not handle cases with more than 25 lines, leading to errors. I have implemented a fix to manage data exceeding this limit efficiently. --------- Co-authored-by: xuxiang <xuxiang@aliyun.com>	9 months ago
Christophe Bornet	e2a8962ba6	Add AstraDB document loader (#14747) ... - **Description:** this adds the AstraDB document loader and an integration test - **Twitter handle:** cbornet_	9 months ago
Igor Dvorkin	76923e5743	Restore self message sent before OSX 12 Monterey (#14818) ... <!-- Thank you for contributing to LangChain! Replace this entire comment with: - **Description:** a description of the change, - **Issue:** the issue # it fixes (if applicable), - **Dependencies:** any dependencies required for this change, - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below), - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out! Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. It lives in `docs/extras` directory. If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17. -->	9 months ago
savoiepe	d006be60ec	Added more filtering options to pgvector vectorstore (#14852) ... - **Description:** Using PGVector vector store, it was only possible to filter for values equals, in or not in metadata. Extended this feature to work with the following keywords : IN, NIN, BETWEEN, GT, LT, NE, EQ, LIKE, CONTAINS, OR, AND --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	9 months ago
chyroc	32e96a471c	Refactor: use SecretStr for llm_rails embeddings (#15090)	9 months ago
chyroc	b440f92d81	Refactor: use SecretStr for embaas embeddings (#15091)	9 months ago
chyroc	ea6cf0f1b1	Refactor: use SecretStr for edenai embeddings (#15092)	9 months ago
chyroc	32e6e9de13	Refactor: use SecretStr for palm chat-model (#15100)	9 months ago
chyroc	b6952d41e5	Refactor: use SecretStr for GPTRouter chat-model (#15101)	9 months ago
Nan LI	f506b4cfd2	community: Integration of New Chat Model Based on ChatGLM3 via ZhipuAI API (#15105) ... - **Description:** - This PR introduces a significant enhancement to the LangChain project by integrating a new chat model powered by the third-generation base large model, ChatGLM3, via the zhipuai API. - This advanced model supports functionalities like function calls, code interpretation, and intelligent Agent capabilities. - The additions include the chat model itself, comprehensive documentation in the form of Python notebook docs, and thorough testing with both unit and integrated tests. - **Dependencies:** This update relies on the ZhipuAI package as a key dependency. - **Twitter handle:** If this PR receives spotlight attention, we would be honored to receive a mention for our integration of the advanced ChatGLM3 model via the ZhipuAI API. Kindly tag us at @kaiwu. To ensure quality and standards, we have performed extensive linting and testing. Commands such as make format, make lint, and make test have been run from the root of the modified package to ensure compliance with LangChain's coding standards. TO DO: Continue refining and enhancing both the unit tests and integrated tests. --------- Co-authored-by: jing <jingguo92@gmail.com> Co-authored-by: hyy1987 <779003812@qq.com> Co-authored-by: jianchuanqi <qijianchuan@hotmail.com> Co-authored-by: lirq <whuclarence@gmail.com> Co-authored-by: whucalrence <81530213+whucalrence@users.noreply.github.com> Co-authored-by: Jing Guo <48378126+JaneCrystall@users.noreply.github.com>	9 months ago
Hin	2cf1e73d12	Feat add volcano embedding (#14693) ... Description: Volcano Ark is an enterprise-grade large-model service platform for developers, providing a full range of functions and services such as model training, inference, evaluation, fine-tuning. You can visit its homepage at https://www.volcengine.com/docs/82379/1099455 for details. This change could help developers use the platform for embedding. Issue: None Dependencies: volcengine Tag maintainer: @baskaryan Twitter handle: @hinnnnnnnnnnnns --------- Co-authored-by: lujingxuansc <lujingxuansc@bytedance.com>	9 months ago
David Křístek	a010f29013	fix: call correct stream method in ollama (#15104) ... Co-authored-by: David Kristek <david@David--MacBook-Pro.local>	9 months ago
Christian Janiake	be578f32be	community:Lazy load wikipedia dump file (#15111) ... **Description:** the MWDumpLoader implementation currently does not support the lazy_load method, and the files are usually very large. We are proposing refactoring the load function, extracting two private functions with the functionality of loading the dump file and parsing a single page, to reuse the code in the lazy_load implementation.	9 months ago
chyroc	a4ae4bc361	feat: mask api_key for konko (#14010) ... for https://github.com/langchain-ai/langchain/issues/12165	9 months ago
joel-teratis	62d32bd214	fix(minor): added missing **kwargs parameter to chroma query function (#14919) ... **Description:** This PR adds the `**kwargs` parameter to six calls in the `chroma.py` package. All functions already were able to receive `kwargs` but they were discarded before. **Issue:** When passing `kwargs` to functions in the `chroma.py` package they are being ignored. For example: ``` chroma_instance.similarity_search_with_score( query, k=100, include=["metadatas", "documents", "distances", "embeddings"], # this parameter gets ignored ) ``` The `include` parameter does not get passed on to the next function and does not have any effect. **Dependencies:** None	9 months ago
NuODaniel	7773943a51	community:qianfan endpoint support init params & remove useless params definietion (#15381) ... - **Description:** - support custom kwargs in object initialization. For instantance, QPS differs from multiple object(chat/completion/embedding with diverse models), for which global env is not a good choice for configuration. - **Issue:** no - **Dependencies:** no - **Twitter handle:** no @baskaryan PTAL	9 months ago
Nuno Campos	99000c612e	Propagate context vars in all classes/methods (#15329) ... - Any direct usage of ThreadPoolExecutor or asyncio.run_in_executor needs manual handling of context vars <!-- Thank you for contributing to LangChain! Please title your PR "<package>: <description>", where <package> is whichever of langchain, community, core, experimental, etc. is being modified. Replace this entire comment with: - **Description:** a description of the change, - **Issue:** the issue # it fixes if applicable, - **Dependencies:** any dependencies required for this change, - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out! Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` from the root of the package you've modified to check this locally. See contribution guidelines for more information on how to write/run tests, lint, etc: https://python.langchain.com/docs/contributing/ If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. It lives in `docs/docs/integrations` directory. If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17. -->	9 months ago
Ankush Gola	7eec8f2487	Delete V1 tracer and refactor tracer tests to core (#15326)	9 months ago
chyroc	7ce338201c	Patch: improve check openai version (#15301)	9 months ago
Nuno Campos	eb5e250188	Propagate context vars in all classes/methods ... - Any direct usage of ThreadPoolExecutor or asyncio.run_in_executor needs manual handling of context vars	9 months ago
Shuai Liu	4b53440e70	Upgrades the Tongyi LLM and ChatTongyi Model (#14793) ... - **Description:** fixes and upgrades for the Tongyi LLM and ChatTongyi Model - Fixed typos; it should be `Tongyi`, not `OpenAI`. - Fixed a bug in `stream_generate_with_retry`; it's a real stream generator now. - Fixed a bug in `validate_environment`; the `dashscope_api_key` should be properly handled when set by environment variables or initialization parameters. - Changed the `dashscope` response to incremental output by setting the parameter `incremental_output`, which eliminates the need for the prefix-removal trick. - Removed some unused parameters, like `n`, `prefix_messages`. - Added `_stream` method. - Added async methods support, such as `_astream`, `_agenerate`, `_abatch`. - **Dependencies:** No new dependencies. - **Tag maintainer:** @hwchase17 > PS: Some may be confused about the terms `dashscope`, `tongyi`, and `Qwen`: > - `dashscope`: A platform to deploy LLMs and provide APIs to invoke the LLM. > - `tongyi`: A brand name or overall term about Alibaba Cloud's LLM/AI. > - `Qwen`: An LLM that is open-sourced and deployed in `dashscope`. > > We use the `dashscope` SDK to interact with the `tongyi`-`Qwen` LLM. --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	9 months ago
Bagatur	8bfac1a319	community[patch]: Release 0.0.7 (#15320)	9 months ago
Diego Rani Mazine	ec72225265	refactor: enable connection pool usage in PGVector (#11514) ... - **Description:** `PGVector` refactored to use connection pool. - **Issue:** #11433, - **Tag maintainer:** @hwchase17 @eyurtsev, --------- Co-authored-by: Diego Rani Mazine <diego.mazine@mercadolivre.com> Co-authored-by: Nuno Campos <nuno@langchain.dev>	9 months ago
joshy-deshaw	bf5385592e	core, community: propagate context between threads (#15171) ... While using `chain.batch`, the default implementation uses a `ThreadPoolExecutor` and run the chains in separate threads. An issue with this approach is that that [the token counting callback](https://python.langchain.com/docs/modules/callbacks/token_counting) fails to work as a consequence of the context not being propagated between threads. This PR adds context propagation to the new threads and adds some thread synchronization in the OpenAI callback. With this change, the token counting callback works as intended. Having the context propagation change would be highly beneficial for those implementing custom callbacks for similar functionalities as well. --------- Co-authored-by: Nuno Campos <nuno@langchain.dev>	9 months ago
shroominic	694bbb14cd	community: fix typo in async ollama chat (#15276) ... Made a stupid typo in the last PR which got already merged😅	9 months ago
triThirty	fea4888e72	community: Enhance Github error prompt (#15248) ... - **Description:** The Github error prompt is confused because of JWT enctrypt to somebody not familiar with Github connection method. This PR is to add some useful error prompt to help users troubleshooting. - **Issue:** https://github.com/langchain-ai/langchain/issues/14550#issuecomment-1867445049 - **Dependencies:** None, - **Twitter handle:** None	9 months ago
Bob Lin	a464eb4394	community: Make doctran synchronous (#15264) ... ### Description I found that the methods in [the doctran library](https://github.com/psychic-api/doctran) have been restructured into [synchronized versions](14944a59f7), And [the example ipynb](https://github.com/psychic-api/doctran/blob/main/examples.ipynb) also shows that the code is synchronized, but the README has not been updated yet. so we need to modify the code and update the documentation. ### Issue https://github.com/langchain-ai/langchain/issues/14645	9 months ago
chyroc	6fb3cc6f27	Fix: Use `Union` instead of `|` to improve compatibility, fix #15244 (#15245)	9 months ago
chyroc	1abcf441ae	Refactor: use SecretStr for Predibase llms (#15119)	9 months ago
chyroc	0a9a73a9c9	Refactor: use SecretStr for PipelineAI llms (#15120)	9 months ago
chyroc	d63ceb65b3	Refactor: use SecretStr for StochasticAI llms (#15118)	9 months ago
chyroc	674fde87d2	Refactor: use SecretStr for VolcEngineMaas llms (#15117)	9 months ago
chyroc	3cc1da2b38	Refactor: use SecretStr for Petals llms (#15121)	9 months ago
shroominic	e6f0cee896	community: Async Ollama + ChatOllama (#15169) ... **Description:** Adding async methods to booth OllamaLLM and ChatOllama to enable async streaming and async .on_llm_new_token callbacks. **Issue:** ChatOllama is not working in combination with an AsyncCallbackManager because the .on_llm_new_token method is not awaited.	9 months ago
Phill Zarfos	35896faab7	community: correct spelling mistakes of "Suffle" and "reporoducibility" (#15172) ... - **Description:** Correct spelling mistakes of "Suffle" and "reporoducibility" in `DirectoryLoader` class - **Issue:** N/A - **Dependencies:** N/A - **Twitter handle:** N/A	9 months ago
chyroc	3a3f880e5a	Patch: improve ollama 404 api error message, fix #15147 (#15156) ... Make this issue more clearly exposed to developers	9 months ago
Ivan	59d4b80a92	[community]: Elasticsearch chat history encoding (#15055) ... - Added ensure_ascii property to ElasticsearchChatMessageHistory <!-- Thank you for contributing to LangChain! Please title your PR "<package>: <description>", where <package> is whichever of langchain, community, core, experimental, etc. is being modified. Replace this entire comment with: - **Description:** a description of the change, - **Issue:** the issue # it fixes if applicable, - **Dependencies:** any dependencies required for this change, - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out! Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` from the root of the package you've modified to check this locally. See contribution guidelines for more information on how to write/run tests, lint, etc: https://python.langchain.com/docs/contributing/ If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. It lives in `docs/docs/integrations` directory. If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17. --> --------- Co-authored-by: Ivan Chetverikov <ivan.chetverikov@raftds.com> Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	9 months ago
Corey Brown	9e492620d4	Don't reassign chunk_type (#14923) ... **Description**: The parameter chunk_type was being hard coded to "extractive_answers", so that when "snippet" was being passed, it was being ignored. This change simply doesn't do that.	9 months ago
Takuya Igei	6da2246215	Add support Vertex AI Gemini uses a public image URL (#14949) ... ## What Since `langchain_google_genai.ChatGoogleGenerativeAI` supported A public image URL, we add to support it in `langchain.chat_models.ChatVertexAI` as well. ### Example ```py from langchain.chat_models.vertexai import ChatVertexAI from langchain_core.messages import HumanMessage llm = ChatVertexAI(model_name="gemini-pro-vision") image_message = { "type": "image_url", "image_url": { "url": "https://python.langchain.com/assets/images/cell-18-output-1-0c7fb8b94ff032d51bfe1880d8370104.png", }, } text_message = { "type": "text", "text": "What is shown in this image?", } message = HumanMessage(content=[text_message, image_message]) output = llm([message]) print(output.content) ``` ## Refs - https://python.langchain.com/docs/integrations/llms/google_vertex_ai_palm - https://python.langchain.com/docs/integrations/chat/google_generative_ai	9 months ago
Archan Ghosh	affa3e755a	Update arxiv.py with get_summaries_as_docs inside of Arxivloader (#14953) ... Added the call function get_summaries_as_docs inside of Arxivloader - **Description:** Added a function that returns the documents from get_summaries_as_docs, as the call signature is present in the parent file but never used from Arxivloader, this can be used from Arxivloader itself just like .load() as both the signatures are same. - **Issue:** Reduces time to load papers as no pdf is processed only metadata is pulled from Arxiv allowing users for faster load times on bulk loads. Users can then choose one or more paper and use ID directly with .load() to load pdf thereby loading all the contents of the paper.	9 months ago
ccurme	f2782f4c86	community: add args_schema to GmailSendMessage (#14973) ... - **Description:** `tools.gmail.send_message` implements a `SendMessageSchema` that is not used anywhere. `GmailSendMessage` also does not have an `args_schema` attribute (this led to issues when invoking the tool with an OpenAI functions agent, at least for me). Here we add the missing attribute and a minimal test for the tool. - **Issue:** N/A - **Dependencies:** N/A - **Twitter handle:** N/A --------- Co-authored-by: Chester Curme <chestercurme@microsoft.com>	9 months ago
Philip Kiely - Baseten	6342da333a	community: refactor Baseten integration with new API endpoints & docs (#15017) ... - **Description:** In response to user feedback, this PR refactors the Baseten integration with updated model endpoints, as well as updates relevant documentation. This PR has been tested by end users in production and works as expected. - **Issue:** N/A - **Dependencies:** This PR actually removes the dependency on the `baseten` package! - **Twitter handle:** https://twitter.com/basetenco	9 months ago
Blane Honeycutt	3fc1b3553b	Community: Adds ability to pass a Config to the boto3 client used by Bedrock (#15029) ... # Description This PR adds the ability to pass a `botocore.config.Config` instance to the boto3 client instantiated by the Bedrock LLM. Currently, the Bedrock LLM doesn't support a way to pass a Config, which means that some settings (e.g., timeouts and retry configuration) require instantiating a new boto3 client with a Config and then replacing the LLM's client: ```python llm = Bedrock( region_name='us-west-2', model_id="anthropic.claude-v2", model_kwargs={'max_tokens_to_sample': 4096, 'temperature': 0}, ) llm.client = boto_client('bedrock-runtime', region_name='us-west-2', config=Config({'read_timeout': 300})) ``` # Issue N/A # Dependencies N/A	9 months ago