1915 Commits (fc40bd4cdb53d5fca19ef2a27d615e422d71bb57)

Author	SHA1	Message	Date
pedro-inf-custodio	0fb5f857f9	IMPROVEMENT WebResearchRetriever error handling in urls with connection error (#13401) ... - **Description:** Added a method `fetch_valid_documents` to `WebResearchRetriever` class that will test the connection for every url in `new_urls` and remove those that raise a `ConnectionError`. - **Issue:** [Previous PR](https://github.com/langchain-ai/langchain/pull/13353), - **Dependencies:** None, - **Tag maintainer:** @efriis Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. It lives in `docs/extras` directory. If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.	10 months ago
Piyush Jain	d2335d0114	IMPROVEMENT Neptune graph updates (#13491) ... ## Description This PR adds an option to allow unsigned requests to the Neptune database when using the `NeptuneGraph` class. ```python graph = NeptuneGraph( host='<my-cluster>', port=8182, sign=False ) ``` Also, added is an option in the `NeptuneOpenCypherQAChain` to provide additional domain instructions to the graph query generation prompt. This will be injected in the prompt as-is, so you should include any provider specific tags, for example `<instructions>` or `<INSTR>`. ```python chain = NeptuneOpenCypherQAChain.from_llm( llm=llm, graph=graph, extra_instructions=""" Follow these instructions to build the query: 1. Countries contain airports, not the other way around 2. Use the airport code for identifying airports """ ) ```	10 months ago
William FH	5a28dc3210	Override Keys Option (#13537) ... Should be able to override the global key if you want to evaluate different outputs in a single run	10 months ago
Bagatur	e584b28c54	bump 337 (#13534)	10 months ago
Bagatur	2e2114d2d0	FEATURE: Runnable with message history (#13418) ... Add RunnableWithMessageHistory class that can wrap certain runnables and manages chat history for them.	10 months ago
Bagatur	0fc3af8932	IMPROVEMENT: update assistants output and doc (#13480)	10 months ago
Hugues Chocart	35e04f204b	[LLMonitorCallbackHandler] Various improvements (#13151) ... Small improvements for the llmonitor callback handler, like better support for non-openai models. --------- Co-authored-by: vincelwt <vince@lyser.io>	10 months ago
Noah Stapp	c1b041c188	Add Wrapping Library Metadata to MongoDB vector store (#13084) ... **Description** MongoDB drivers are used in various flavors and languages. Making sure we exercise our due diligence in identifying the "origin" of the library calls makes it best to understand how our Atlas servers get accessed.	10 months ago
Guy Korland	7f8fd70ac4	Add optional arguments to FalkorDBGraph constructor (#13459) ... **Description:** Add optional arguments to FalkorDBGraph constructor **Tag maintainer:** baskaryan **Twitter handle:** @g_korland	10 months ago
chris stucchio	d7f014cd89	Bug: OpenAIFunctionsAgentOutputParser doesn't handle functions with no args (#13467) ... **Description/Issue:** When OpenAI calls a function with no args, the args are `""` rather than `"{}"`. Then `json.loads("")` blows up. This PR handles it correctly. **Dependencies:** None	10 months ago
Yujie Qian	41a433fa33	IMPROVEMENT: add input_type to VoyageEmbeddings (#13488) ... - **Description:** add input_type to VoyageEmbeddings	10 months ago
David Duong	ea6e017b85	Add serialisation arguments to Bedrock and ChatBedrock (#13465)	10 months ago
Erick Friis	427331d621	IMPROVEMENT Lock pydantic v1 in app template, cli 0.0.18 (#13485)	10 months ago
Erick Friis	75363f048f	BUG Fix app_name in cli app new (#13482)	10 months ago
ifduyue	324ab382ad	Use List instead of list (#13443) ... Unify List usages in libs/langchain/langchain/text_splitter.py, only one place it's `list`, all other ocurrences are `List`	10 months ago
Stefano Lottini	b029d9f4e6	Astra DB: minor improvements to docstrings and demo notebook (#13449) ... This PR brings a few minor improvements to the docs, namely class/method docstrings and the demo notebook. - A note on how to control concurrency levels to tune performance in bulk inserts, both in the class docstring and the demo notebook; - Slightly increased concurrency defaults after careful experimentation (still on the conservative side even for clients running on less-than-typical network/hardware specs) - renamed the DB token variable to the standardized `ASTRA_DB_APPLICATION_TOKEN` name (used elsewhere, e.g. in the Astra DB docs) - added a note and a reference (add_text docstring, demo notebook) on allowed metadata field names. Thank you!	10 months ago
Eugene Yurtsev	1e43fd6afe	Add ahandle_event to _all_ (#13469) ... Add ahandle_event for backwards compatibility as it is used by langserve	10 months ago
Harrison Chase	f90249305a	callback refactor (#13372) ... Co-authored-by: Nuno Campos <nuno@boringbits.io>	10 months ago
Bagatur	a9b2c943e6	bump 336, exp 44 (#13420)	10 months ago
Bagatur	1372296dc8	FIX: Infer runnable agent single or multi action (#13412)	10 months ago
Eugene Yurtsev	accadccf8e	Use secretstr for api keys for javelin-ai-gateway (#13417) ... - Make javelin_ai_gateway_api_key a SecretStr --------- Co-authored-by: Hiroshi Tashiro <hiroshitash@gmail.com>	10 months ago
William FH	ba501b27a0	Fix Runnable Lambda Afunc Repr (#13413) ... Otherwise, you get an error when using async functions. h/t to Chris Ruppelt	10 months ago
Sumukh Sridhara	1726d5dcdd	Merge pull request #13232 ... * PGVector needs to close its connection if its garbage collected	10 months ago
Nuno Campos	85a77d2c27	IMPROVEMENT Passthrough kwargs in runnable lambda (#13405) ... <!-- Thank you for contributing to LangChain! Replace this entire comment with: - **Description:** a description of the change, - **Issue:** the issue # it fixes (if applicable), - **Dependencies:** any dependencies required for this change, - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below), - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out! Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. It lives in `docs/extras` directory. If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17. -->	10 months ago
Bagatur	76c317ed78	DOCS: update rag use case (#13319)	10 months ago
Clay Elmore	8823e3831f	FEAT Bedrock cohere embedding support (#13366) ... - **Description:** adding cohere embedding support to bedrock embedding class - **Issue:** N/A - **Dependencies:** None - **Tag maintainer:** @3coins - **Twitter handle:** celmore25 --------- Co-authored-by: Erick Friis <erick@langchain.dev>	10 months ago
Nuno Campos	d5aeff706a	Make it easier to subclass RunnableEach (#13346) ... <!-- Thank you for contributing to LangChain! Replace this entire comment with: - **Description:** a description of the change, - **Issue:** the issue # it fixes (if applicable), - **Dependencies:** any dependencies required for this change, - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below), - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out! Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. It lives in `docs/extras` directory. If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17. -->	10 months ago
竹内謙太	3b5e8bacfa	FEAT Add some properties to NotionDBLoader (#13358) ... <!-- Thank you for contributing to LangChain! Replace this entire comment with: - **Description:** a description of the change, - **Issue:** the issue # it fixes (if applicable), - **Dependencies:** any dependencies required for this change, - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below), - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out! Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. It lives in `docs/extras` directory. If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17. --> fix #13356 Add supports following properties for metadata to NotionDBLoader. - `checkbox` - `email` - `number` - `select` There are no relevant tests for this code to be updated.	10 months ago
Fielding Johnston	37eb44c591	BUG Add limit_to_domains to APIChain based tools (#13367) ... - **Description:** Adds `limit_to_domains` param to the APIChain based tools (open_meteo, TMDB, podcast_docs, and news_api) - **Issue:** I didn't open an issue, but after upgrading to 0.0.328 using these tools would throw an error. - **Dependencies:** N/A - **Tag maintainer:** @baskaryan **Note**: I included the trailing / simply because the docs here did fc886cc303/docs/docs/use_cases/apis.ipynb (L246) , but I checked the code and it is using `urlparse`. SoI followed the docs since it comes down to stylee.	10 months ago
Bagatur	38180ad25f	bump openai support (#13262)	10 months ago
Erick Friis	7c3066f9ec	more cli interactivity, bugfix (#13360)	10 months ago
Predrag Gruevski	d63d4994c0	Bump all libraries to the latest `ruff` version. (#13350) ... This version of `ruff` is the one we'll be using to lint the docs and cookbooks (#12677), so I'm making it used everywhere else too.	10 months ago
Massimiliano Pronesti	344cab0739	IMPROVEMENT: support Openai API v1 for Azure OpenAI completions (#13231) ... Hi, this PR adds support for OpenAI API v1 for Azure OpenAI completion API. @baskaryan @hwchase17 --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	10 months ago
dependabot[bot]	fc886cc303	Bump pyarrow from 13.0.0 to 14.0.1 in /libs/langchain (#13363) ... Bumps [pyarrow](https://github.com/apache/arrow) from 13.0.0 to 14.0.1. <details> <summary>Commits</summary> <ul> <li><a href="ba53748361"><code>ba53748</code></a> MINOR: [Release] Update versions for 14.0.1</li> <li><a href="529f3768fa"><code>529f376</code></a> MINOR: [Release] Update .deb/.rpm changelogs for 14.0.1</li> <li><a href="b84bbcac64"><code>b84bbca</code></a> MINOR: [Release] Update CHANGELOG.md for 14.0.1</li> <li><a href="f141709763"><code>f141709</code></a> <a href="https://redirect.github.com/apache/arrow/issues/38607">GH-38607</a>: [Python] Disable PyExtensionType autoload (<a href="https://redirect.github.com/apache/arrow/issues/38608">#38608</a>)</li> <li><a href="5a37e74198"><code>5a37e74</code></a> <a href="https://redirect.github.com/apache/arrow/issues/38431">GH-38431</a>: [Python][CI] Update fs.type_name checks for s3fs tests (<a href="https://redirect.github.com/apache/arrow/issues/38455">#38455</a>)</li> <li><a href="2dcee3f82c"><code>2dcee3f</code></a> MINOR: [Release] Update versions for 14.0.0</li> <li><a href="297428cbf2"><code>297428c</code></a> MINOR: [Release] Update .deb/.rpm changelogs for 14.0.0</li> <li><a href="3e9734f883"><code>3e9734f</code></a> MINOR: [Release] Update CHANGELOG.md for 14.0.0</li> <li><a href="9f90995c8c"><code>9f90995</code></a> <a href="https://redirect.github.com/apache/arrow/issues/38332">GH-38332</a>: [CI][Release] Resolve symlinks in RAT lint (<a href="https://redirect.github.com/apache/arrow/issues/38337">#38337</a>)</li> <li><a href="bd61239a32"><code>bd61239</code></a> <a href="https://redirect.github.com/apache/arrow/issues/35531">GH-35531</a>: [Python] C Data Interface PyCapsule Protocol (<a href="https://redirect.github.com/apache/arrow/issues/37797">#37797</a>)</li> <li>Additional commits viewable in <a href="https://github.com/apache/arrow/compare/go/v13.0.0...go/v14.0.1">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/langchain-ai/langchain/network/alerts). </details> --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Predrag Gruevski <2348618+obi1kenobi@users.noreply.github.com>	10 months ago
Erick Friis	c0e6045c0b	cli 0.0.17 (#13359)	10 months ago
Erick Friis	927824b7cb	CLI interactivity (#13148) ... Will implement more later	10 months ago
billytrend-cohere	2f6fe6ddf3	Fix latest message index (#13355) ... There is a bug which caused the earliest message rather than the latest message being sent	10 months ago
Harrison Chase	be854225c7	add more reasonable arxiv retriever (#13327)	10 months ago
Krish Dholakia	5a920e14c0	fix litellm openai imports (#13307)	10 months ago
Bagatur	1c67db4c18	Move OAI assistants to langchain and add callbacks (#13236)	10 months ago
Erick Friis	280ecfd8eb	IMPROVEMENT redirect root to docs in langserve app template (#13303)	10 months ago
mertkayhan	9b4974871d	IMPROVEMENT Increase flexibility of ElasticVectorSearch (#6863) ... Hey @rlancemartin, @eyurtsev , I did some minimal changes to the `ElasticVectorSearch` client so that it plays better with existing ES indices. Main changes are as follows: 1. You can pass the dense vector field name into `_default_script_query` 2. You can pass a custom script query implementation and the respective parameters to `similarity_search_with_score` 3. You can pass functions for building page content and metadata for the resulting `Document` <!-- Thank you for contributing to LangChain! Replace this comment with: - Description: a description of the change, - Issue: the issue # it fixes (if applicable), - Dependencies: any dependencies required for this change, - Tag maintainer: for a quicker response, tag the relevant maintainer (see below), - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out! If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 4. an example notebook showing its use. Maintainer responsibilities: - General / Misc / if you don't know who to tag: @dev2049 - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev - Models / Prompts: @hwchase17, @dev2049 - Memory: @hwchase17 - Agents / Tools / Toolkits: @vowelparrot - Tracing / Callbacks: @agola11 - Async: @agola11 If no one reviews your PR within a few days, feel free to @-mention the same people again. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md -->	10 months ago
Erick Friis	50a5c919f0	IMPROVEMENT self-query template (#13305) ... - [ ] https://github.com/langchain-ai/langchain/pull/12694#discussion_r1391334719 -> keep date - [x] https://github.com/langchain-ai/langchain/pull/12694#discussion_r1391336586	10 months ago
Yasin	b46f88d364	IMPROVEMENT add license file to subproject (#8403) ... <!-- Thank you for contributing to LangChain! Replace this comment with: - Description: a description of the change, - Issue: the issue # it fixes (if applicable), - Dependencies: any dependencies required for this change, - Tag maintainer: for a quicker response, tag the relevant maintainer (see below), - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out! Please make sure you're PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally. If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. Maintainer responsibilities: - General / Misc / if you don't know who to tag: @baskaryan - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev - Models / Prompts: @hwchase17, @baskaryan - Memory: @hwchase17 - Agents / Tools / Toolkits: @hinthornw - Tracing / Callbacks: @agola11 - Async: @agola11 If no one reviews your PR within a few days, feel free to @-mention the same people again. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md --> hi! This is pretty straight-forward: The sdist package does not contain the license file (which is needed by e.g. conda) because the package is built from the subdir and can't see the license. I _copied_ the license but since I'm unfamiliar with the projects direction, I'm not sure that's correct. thanks! --------- Co-authored-by: Erick Friis <erick@langchain.dev>	10 months ago
Rui Ramos	ff19a62afc	Fix Pinecone cosine relevance score (#8920) ... Fixes: #8207 Description: Pinecone returns scores (not distances) with cosine similarity. The values according to the docs are [-1, 1], although I could never reproduce negative values. This PR ensures that the score returned from Pinecone is preserved, rather than inverted, so the most relevant documents can be filtered (eg when using similarity thresholds) I'll leave this as a draft PR as I couldn't run the tests (my pinecone account might not be enough - some errors were being thrown around namespaces) so hopefully someone who _can_ will pick this up. Maintainers: @rlancemartin, @eyurtsev --------- Co-authored-by: Erick Friis <erick@langchain.dev>	10 months ago
Bagatur	2e42ed5de6	Self-query template (#12694) ... Co-authored-by: Erick Friis <erick@langchain.dev>	10 months ago
Konstantin Spieß	1e43025bf5	Fix serialization issue in Matching Engine Vector Store (#13266) ... - **Description:** Fixed a serialization issue in the add_texts method of the Matching Engine Vector Store caused by a typo, leading to an attempt to serialize the json module itself. - **Issue:** #12154 - **Dependencies:** ./. - **Tag maintainer:**	10 months ago
William FH	9169d77cf6	Update error message in evaluation runner (#13296)	10 months ago
takatost	f22f273f93	FIX: 'from_texts' method in Weaviate with non-existent kwargs param (#11604) ... Due to the possibility of external inputs including UUIDs, there may be additional values in **kwargs, while Weaviate's `__init__` method does not support passing extra **kwarg parameters. --------- Co-authored-by: Erick Friis <erick@langchain.dev>	10 months ago
Frank995	971d2b2e34	Add missing filter to max_marginal_relevance_search inner call to max_marginal_relevance_search_by_vector (#13260) ... When calling max_marginal_relevance_search from PGVector the filter param is not carried over to max_marginal_relevance_search_by_vector --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	10 months ago
chevalmuscle	3ad78e48e2	Use endpoint_url if provided with boto3 session for dynamodb (#11622) ... - **Description:** Uses `endpoint_url` if provided with a boto3 session. When running dynamodb locally, credentials are required even if invalid. With this change, it will be possible to pass a boto3 session with credentials and specify an endpoint_url --------- Co-authored-by: Bagatur <baskaryan@gmail.com> Co-authored-by: Erick Friis <erick@langchain.dev>	10 months ago
Erick Friis	18acc22f29	Ollama pass kwargs as options instead of top (#13280) ... Noticed params are really in `options` instead while reviewing #12895	10 months ago
刘 方瑞	46af56dc4f	Add MyScaleWithoutJSON which allows user to wrap columns into Document's Metadata (#13164) ... <!-- Thank you for contributing to LangChain! Replace this entire comment with: - **Description:** a description of the change, - **Issue:** the issue # it fixes (if applicable), - **Dependencies:** any dependencies required for this change, - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below), - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out! Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. It lives in `docs/extras` directory. If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17. --> Replace this entire comment with: - **Description:** Add MyScaleWithoutJSON which allows user to wrap columns into Document's Metadata - **Tag maintainer:** @baskaryan	10 months ago
Michael Landis	2aa13f1e10	chore: bump momento dependency version and refactor search hit usage (#13111) ... **Description** Bumps the Momento dependency to the latest version and refactors the usage of `SearchHit` in the Momento Vector Index (MVI) vector store integration. This change is a one liner where we use the preferred attribute `score` to read the query-document similarity instead of `distance`. The latest versions of Momento clients will use this attribute going forward. **Dependencies** Updated the Momento dependency to latest version. **Tests** 💚 I re-ran the existing MVI integration tests (`tests/integration_tests/vectorstores/test_momento_vector_index.py`) and they pass. **Review** cc @baskaryan @eyurtsev	10 months ago
kYLe	cc55d2fcee	Add OpenAI API v1 support for ChatAnyscale and fixed a bug with openai_api_key (#13237) ... 1. Add OpenAI API v1 support 2. Fixed a bug to call `get_secret_value` on a str value (values["openai_api_key"])	10 months ago
Govind.S.B	9024593468	added system prompt and template fields to ollama (#13022) ... **Description** the ollama api now supports passing system prompt and template directly instead of modifying the model file , but the ollama integration in langchain did not have this change updated . The update just adds these two parameters to it ( there are 2 more parameters that are pending to be updated, I was not sure about their utility wrt to langchain ) Refer : 8713ac23a8 **Issue** : None Applicable **Dependencies** : None Changed **Twitter handle** : https://twitter.com/violetto96 --------- Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com>	10 months ago
langchain-infra	f55f67055f	Add dockerfile template (#13240)	10 months ago
Guillem Orellana Trullols	0f31cd8b49	Remove `_get_kwarg_value` function (#13184) ... `_get_kwarg_value` function is useless, one can rely on python builtin functionalities to do the exact same thing. - **Description:** Removed `_get_kwarg_value`. Helps with code readability. - **Issue:** the issue # it fixes (if applicable), - **Twitter handle:** @Guillem_96	10 months ago
SuperDa Fu	e1c020dfe1	dalle add model parameter (#13201) ... - **Description:** dalle_image_generator adding a new model parameter, - **Issue:** N/A, - **Dependencies:** - **Tag maintainer: @hwchase17 - **Twitter handle:** --------- Co-authored-by: dafu <xiangbingze@wenru.wang> Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com> Co-authored-by: Erick Friis <erickfriis@gmail.com>	10 months ago
Dennis de Greef	64e11592bb	Improve CSV reader which can't call .strip() on NoneType (#13079) ... Improve CSV reader which can't call .strip() on NoneType if there are less cells in the row compared to the header <!-- Thank you for contributing to LangChain! Replace this entire comment with: - **Description:** I have a CSV file as followed ``` headerA,headerB,headerC v1A,v1B,v1C, v2A,v2B v3A,v3B,v3C ``` In this case, row 2 is missing a value, which results in reading a None type. The strip() method can not be called on None, hence raising. In this PR I am making the change to only call strip if the value if not None. - **Issue:** the issue # it fixes (if applicable), - **Dependencies:** any dependencies required for this change, - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below), - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out! Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. It lives in `docs/extras` directory. If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17. -->	10 months ago
glad4enkonm	339973db47	Update ollama.py (#12895) ... duplicate option removed **Description:** An issue fix, http stop option duplicate removed. **Issue:** the issue #12892 fix **Dependencies:** no **Tag maintainer:** @eyurtsev --------- Co-authored-by: Erick Friis <erick@langchain.dev>	10 months ago
Isak Nyberg	8f81703d76	Add new models to openai callback (#13244) ... **Description:** Adding the new models to the openai callback function, info taken from [model announcement](https://platform.openai.com/docs/models) and [pricing](https://openai.com/pricing) A short description for a short PR :)	10 months ago
Bagatur	ea6dd3a550	bump 335 (#13261)	10 months ago
William FH	a837b03e55	Update langsmith version 0.63 (#13208)	10 months ago
Harrison Chase	7f1d26160d	update tools (#13243)	10 months ago
Nuno Campos	8d6faf5665	Make it easier to subclass runnable binding with custom init args (#13189)	10 months ago
Peter Vandenabeele	7f1964b264	Fix BeautifulSoupTransformer: no more duplicates and correct order of tags + tests (#12596)	10 months ago
Erick Friis	9c7afa8adb	Upgrade cohere embedding model to v3 (#13219) ... Just updates API docs, doesn't change default param from 2.0 (could be breaking change)	10 months ago
Erick Friis	8fdf15c023	Fix Document Loader Unit Test - Docusaurus (#13228)	10 months ago
Lee	72ad448daa	feat: Docusaurus Loader (#9138) ... Added a Docusaurus Loader Issue: #6353 I had to implement this for working with the Ionic documentation, and wanted to open this up as a draft to get some guidance on building this out further. I wasn't sure if having it be a light extension of the SitemapLoader was in the spirit of a proper feature for the library -- but I'm grateful for the opportunities Langchain has given me and I'd love to build this out properly for the sake of the community. Any feedback welcome!	10 months ago
Tomaz Bratanic	0dc4ab0be1	Neo4j chat message history (#13008)	10 months ago
fyasla	d266b3ea4a	issue #12165 mask API key in chat_models/azureml_endpoint module (#12836) ... - **Description:** `AzureMLChatOnlineEndpoint` object from langchain/chat_models/azureml_endpoint.py safe to print without having any secrets included in raw format in the string representation. - **Issue:** #12165, - **Tag maintainer:** @eyurtsev --------- Co-authored-by: Faysal Bougamale <faysal.bougamale@horiba.com> Co-authored-by: Bagatur <baskaryan@gmail.com>	10 months ago
Anush	52f34de9b7	feat: FastEmbed embedding provider (#13109) ... ## Description: This PR intends to add [Qdrant/FastEmbed](https://qdrant.github.io/fastembed/) as a local embeddings provider, associated tests and documentation. **Documentation preview:** https://langchain-git-fork-anush008-master-langchain.vercel.app/docs/integrations/text_embedding/fastembed --------- Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com>	10 months ago
Eugene Yurtsev	b0e8cbe0b3	Add RunnableSequence documentation (#13094) ... Add RunnableSequence documentation	10 months ago
Eugene Yurtsev	869df62736	Document RunnableWithFallbacks (#13088) ... Add documentation to RunnableWithFallbacks	10 months ago
Eugene Yurtsev	8313c218da	Add more runnable documentation (#13083) ... - Adding documentation to the runnable. - Documentation is not organized in the best way for the runnable; i.e., in terms of LCEL vs. other standard methods, will follow up with more edits.	10 months ago
Bagatur	24386e0860	bump 334, exp 40 (#13211)	10 months ago
Lance Martin	d2e50b3108	Add Chroma multimodal cookbook (#12952) ... Pending: * https://github.com/chroma-core/chroma/pull/1294 * https://github.com/chroma-core/chroma/pull/1293 --------- Co-authored-by: Erick Friis <erick@langchain.dev> Co-authored-by: Bagatur <baskaryan@gmail.com>	10 months ago
The1Bill	55912868da	Update toolkit.py to remove single quotes around table names (#12445) ... **Description:** Removing the single quote wrapper around the table names in the SQL agent toolkit.py file as it misleads the LLM into querying against tables with single quotes around their names. **Issue:** #7457 **Dependencies:** None **Tag maintainer:** @hwchase17 **Twitter handle:** None	10 months ago
Nuno Campos	362a446999	Changes to root listener (#12174) ... - Implement config_specs to include session_id - Remove Runnable method and update notebook - Add more details to notebook, eg. show input schema and config schema before and after adding message history --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	10 months ago
Nuno Campos	b2b94424db	Update return type for Runnable.__or__ (#12880) ... <!-- Thank you for contributing to LangChain! Replace this entire comment with: - **Description:** a description of the change, - **Issue:** the issue # it fixes (if applicable), - **Dependencies:** any dependencies required for this change, - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below), - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out! Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. It lives in `docs/extras` directory. If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17. -->	10 months ago
Harrison Chase	0a2b1c7471	improve duck duck go tool (#13165)	10 months ago
Shinya Maeda	28cc60b347	Fix langchain.llms OpenAI completion doesn't work due to v1 client update (#13099) ... This commit fixes the issue that langchain.llms OpenAI completion stopped working since the V1 openai client update. Replace this entire comment with: - **Description:** This PR fixes the issue [AttributeError: module 'openai' has no attribute 'Completion'](https://github.com/langchain-ai/langchain/issues/12967) similar to 8e0cb2eb84 and https://github.com/langchain-ai/langchain/pull/12969, - **Issue:** https://github.com/langchain-ai/langchain/issues/12967, - **Dependencies:** `openai` v1.x.x client, - **Tag maintainer:** @baskaryan, - **Twitter handle:** @dosuken123 Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. It lives in `docs/extras` directory. If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17. --------- Co-authored-by: Erick Friis <erick@langchain.dev> Co-authored-by: Bagatur <baskaryan@gmail.com>	10 months ago
Bagatur	ff43cd6701	OpenAI remove httpx typing (#13154) ... Addresses #13124	10 months ago
Bagatur	8b2a82b5ce	Bagatur/docs smith context (#13139)	10 months ago
Bagatur	f04cc4b7e1	bump 333 (#13131)	10 months ago
billytrend-cohere	b346d4a455	Add message to documents (#12552) ... This adds the response message as a document to the rag retriever so users can choose to use this. Also drops document limit. --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	10 months ago
Harrison Chase	5f38770161	Support oai tool call (#13110) ... Co-authored-by: Bagatur <baskaryan@gmail.com> Co-authored-by: Nuno Campos <nuno@boringbits.io>	10 months ago
Holt Skinner	0fc8fd12bd	feat: Vertex AI Search - Add Snippet Retrieval for Non-Advanced Website Data Stores (#13020) ... https://cloud.google.com/generative-ai-app-builder/docs/snippets#snippets --------- Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com>	10 months ago
Jacob Lee	76283e9625	Adds embeddings filter option to return scores in state (#12489) ... CC @baskaryan @assafelovic	10 months ago
jakerachleff	18601bd4c8	Get project from langchain sdk (#13100) ... ## Description We need to centralize the API we use to get the project name for our tracers. This PR makes it so we always get this from a shared function in the langsmith sdk. ## Dependencies Upgraded langsmith from 0.52 to 0.62 to include the new API `get_tracer_project`	10 months ago
Bagatur	72e12f6bcf	update more azure docs (#13093)	10 months ago
Bagatur	1703f132c6	update azure embedding docs (#13091)	10 months ago
Bagatur	9fdfac22c2	bump 332 (#13089)	10 months ago
Bagatur	1f85ec34d5	bump 331rc3 exp 39 (#13086)	10 months ago
Anton Troynikov	9f077270c8	Don't pass EF to chroma (#13085) ... - **Description:** Recently Chroma rolled out a breaking change on the way we handle embedding functions, in order to support multi-modal collections. This broke the way LangChain's `Chroma` objects get created, because we were passing the EF down into the Chroma collection: https://docs.trychroma.com/migration#migration-to-0416---november-7-2023 However, internally, we are never actually using embeddings on the chroma collection - LangChain's `Chroma` object calls it instead. Thus we just don't pass an `embedding_function` to Chroma itself, which fixes the issue.	10 months ago
Erick Friis	f15f8e01cf	Azure OpenAI Embeddings (#13039) ... Co-authored-by: Bagatur <baskaryan@gmail.com>	10 months ago
David Peterson	37561d8986	Add Proper Import Error (#13042) ... - **Description:** The issue was not listing the proper import error for amazon textract loader. - **Issue:** Time wasted trying to figure out what to install... (langchain docs don't list the dependency either) - **Dependencies:** N/A - **Tag maintainer:** @sbusso - **Twitter handle:** @h9ste --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	10 months ago
Eugene Yurtsev	06c503f672	Add RunnableRetry Documentation (#13074)	10 months ago
Bagatur	55aeff6777	oai assistant multiple actions (#13068)	10 months ago
Erick Friis	a9b70baef9	cli updates, 0.0.16 (#13034) ... - confirm flags, serve detection - 0.0.16 - always gen code - pip bool	10 months ago
Erick Friis	506f81563f	Update Deps in Experimental (#13029)	10 months ago
Stefano Lottini	4f4b020582	Add "Astra DB" vector store integration (#12966) ... # Astra DB Vector store integration - **Description:** This PR adds a `VectorStore` implementation for DataStax Astra DB using its HTTP API - **Issue:** (no related issue) - **Dependencies:** A new required dependency is `astrapy` (`>=0.5.3`) which was added to pyptoject.toml, optional, as per guidelines - **Tag maintainer:** I recently mentioned to @baskaryan this integration was coming - **Twitter handle:** `@rsprrs` if you want to mention me This PR introduces the `AstraDB` vector store class, extensive integration test coverage, a reworking of the documentation which conflates Cassandra and Astra DB on a single "provider" page and a new, completely reworked vector-store example notebook (common to the Cassandra store, since parts of the flow is shared by the two APIs). I also took care in ensuring docs (and redirects therein) are behaving correctly. All style, linting, typechecks and tests pass as far as the `AstraDB` integration is concerned. I could build the documentation and check it all right (but ran into trouble with the `api_docs_build` makefile target which I could not verify: `Error: Unable to import module 'plan_and_execute.agent_executor' with error: No module named 'langchain_experimental'` was the first of many similar errors) Thank you for a review! Stefano --------- Co-authored-by: Erick Friis <erick@langchain.dev>	10 months ago
Yang, Bo	600caff03c	Add `Memorize` tool (#11722) ... - **Description:** Add `Memorize` tool - **Tag maintainer:** @hwchase17 This PR added a new tool `Memorize` so that an agent can use it to fine-tune itself. This tool requires `TrainableLLM` introduced in #11721 DEMO: 6a9003d5db 	10 months ago
Bagatur	cf481c9418	bump exp 38 (#13016)	10 months ago
Bagatur	57e19989f6	Bagatur/oai assistant (#13010)	10 months ago
Erick Friis	74134dd7e1	cli pyproject updating (#12945) ... `langchain app add` and `langchain app remove` will now keep the dependencies list updated. --------- Co-authored-by: Nuno Campos <nuno@boringbits.io>	10 months ago
Bagatur	6175dc30aa	bump 331rc2 (#13006)	10 months ago
Erick Friis	0c81cd923e	oai v1 embeddings (#12969) ... Initial PR to get OpenAIEmbeddings working with the new sdk fyi @rlancemartin Fixes #12943 --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	10 months ago
Bagatur	fdbb45d79e	bump 331rc1 (#12965)	10 months ago
Bagatur	3bb8030a6e	fix max_tokens (#12964)	10 months ago
Bagatur	a9002a82b8	bump 331rc0 (#12963)	10 months ago
Harrison Chase	c27400efeb	Support multimodal messages (#11320) ... Co-authored-by: Bagatur <baskaryan@gmail.com>	10 months ago
Bagatur	4f7dff9d66	Record system fingerprint chat openai (#12960)	10 months ago
Bagatur	8e0cb2eb84	ChatOpenAI and AzureChatOpenAI openai>=1 compatible (#12948)	10 months ago
Kacper Łukawski	52d0055a91	Add support of Cohere Embed v3 (#12940) ... Cohere released the new embedding API (Embed v3: https://txt.cohere.com/introducing-embed-v3/) that treats document and query embeddings differently. This PR updated the `CohereEmbeddings` to use them appropriately. It also works with the old models.	10 months ago
Praveen Venkateswaran	8e0dcb37d2	Add SecretStr for Symbl.ai Nebula API (#12896) ... Description: This PR masks API key secrets for the Nebula model from Symbl.ai Issue: #12165 Maintainer: @eyurtsev --------- Co-authored-by: Praveen Venkateswaran <praveen.venkateswaran@ibm.com>	10 months ago
Vinzenz Klass	59d0bd2150	feat: acquire advisory lock before creating extension in pgvector (#12935) ... - **Description:** Acquire advisory lock before attempting to create extension on postgres server, preventing errors in concurrent executions. - **Issue:** #12933 - **Dependencies:** None --------- Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com>	10 months ago
Eugene Yurtsev	b376854b26	Fix for anyscale chat model api key (#12938) ... * ChatAnyscale was missing coercion to SecretStr for anyscale api key * The model inherits from ChatOpenAI so it should not force the openai api key to be secret str until openai model has the same changes https://github.com/langchain-ai/langchain/issues/12841	10 months ago
hmasdev	622bf12c2e	fix regex pattern of structured output parser (#12929) ... - **Description:** fix the regex pattern of [StructuredChatOutputParser](https://github.com/langchain-ai/langchain/blob/master/libs/langchain/langchain/agents/structured_chat/output_parser.py#L18) and add unit tests for the code change. - **Issue:** #12158 #12922 - **Dependencies:** None - **Tag maintainer:** - **Twitter handle:** @hmdev3 - **NOTE:** This PR conflicts #7495 . After #7495 is merged, I am going to update PR.	10 months ago
wemysschen	8d7144e6a6	fix baiducloud directory loader import file loader (#12924) ... **Issue:** fix baiducloud BOS directory loader imports its file loader --------- Co-authored-by: wemysschen <root@icoding-cwx.bcc-szzj.baidu.com>	10 months ago
Kacper Łukawski	621419f71e	Fix normalizing the cosine distance in Qdrant (#12934) ... Qdrant was incorrectly calculating the cosine similarity and returning `0.0` for the best match, instead of `1.0`. Internally Qdrant returns a cosine score from `-1.0` (worst match) to `1.0` (best match), and the current formula reflects it.	10 months ago
Hech	8fe6bcc662	Fix return metadata when searching for DingoDB (#12937)	10 months ago
Jakub Novák	ada3d2cbd1	Add possibility to pass on_artifacts for a specific conversation (#12687) ... Possibility to pass on_artifacts to a conversation. It can be then achieved by adding this way: ```python result = agent.run( input=message.text, metadata={ "on_artifact": CALLBACK_FUNCTION }, ) ```	10 months ago
Bagatur	53f453f01a	bump 331 (#12932)	10 months ago
Erick Friis	5000c7308e	cli template gitignores (#12914) ... - ap gitignore - package	10 months ago
Harrison Chase	aba407f774	use keys not items (#12918)	10 months ago
wemysschen	e14aa37d59	fix bes vector store search (#12828) ... **Issue:** fix search body in baidu cloud vectorsearch --------- Co-authored-by: wemysschen <root@icoding-cwx.bcc-szzj.baidu.com>	11 months ago
Lance Martin	ea1ab391d4	Open Clip multimodal embeddings (#12754)	11 months ago
Bagatur	ebee616822	bump 330 (#12853)	11 months ago
Erick Friis	6c237716c4	Update readmes with new cli install (#12847) ... Old command still works. Just simplifying. Merge after releasing CLI 0.0.15	11 months ago
Erick Friis	7db49d3842	Confirm sys.path includes current dir for app serve (#12851) ... - Make sure sys.path is set properly for langchain app serve - bump	11 months ago
Erick Friis	1bc35f61cb	CLI 0.0.14, Uvicorn update and no more [serve] (#12845) ... Calls uvicorn directly from cli: Reload works if you define app by import string instead of object. (was doing subprocess in order to get reloading) Version bump to 0.0.14 Remove the need for [serve] for simplicity. Readmes are updated in #12847 to avoid cluttering this PR	11 months ago
William FH	18005c6384	Disable trace_on_chain_group auto-tracing (#12807) ... Previously we treated trace_on_chain_group as a command to always start tracing. This is unintuitive (makes the function do 2 things), and makes it harder to toggle tracing	11 months ago
Erick Friis	0da75b9ebd	Autopopulate module name in cli init (#12814)	11 months ago
William FH	98aff29fbd	Add Dataset Page to printout (#12816)	11 months ago
Manuel Rech	2e2b9c76d9	Keep also original query - multi_query.py (#12696) ... When you use a MultiQuery it might be useful to use the original query as well as the newly generated ones to maximise the changes to retriever the correct document. I haven't created an issue, it seems a very small and easy thing. --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	11 months ago
Bagatur	658a3a8607	FEAT: Merge TileDB vecstore (#12811)	11 months ago
Akio Nishimura	c04647bb4e	Correct number of elements in config list in `batch()` and `abatch()` of `BaseLLM` (#12713) ... - **Description:** Correct number of elements in config list in `batch()` and `abatch()` of `BaseLLM` in case `max_concurrency` is not None. - **Issue:** #12643 - **Twitter handle:** @akionux --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	11 months ago
James Braza	88b506b321	Adds missing `urllib.parse` for IDE warning of `PubMedAPIWrapper` (#12808) ... Resolves an IDE (PyCharm 2023.2.3 PE) warning around `urllib.parse.quote`, also enabling CTRL-click	11 months ago
Bagatur	a2bb0dd445	TileDB update import unit tests	11 months ago
Nikos Papailiou	2fdaa1e5fd	Add TileDB vectorstore implementation (#12624) ... - **Description:** Add [TileDB](https://tiledb.com) vectorstore implementation. TileDB offers ANN search capabilities using the [TileDB-Vector-Search](https://github.com/TileDB-Inc/TileDB-Vector-Search) module. It provides serverless execution of ANN queries and storage of vector indexes both on local disk and cloud object stores (i.e. AWS S3). More details in: - [Why TileDB as a Vector Database](https://tiledb.com/blog/why-tiledb-as-a-vector-database) - [TileDB 101: Vector Search](https://tiledb.com/blog/tiledb-101-vector-search) - **Twitter handle:** @tiledb	11 months ago
盐粒 Yanli	1b233798a0	feat: Supprt pgvecto.rs as a VectorStore (#12718) ... Supprt [pgvecto.rs](https://github.com/tensorchord/pgvecto.rs) as a new VectorStore type. This introduces a new dependency [pgvecto_rs](https://pypi.org/project/pgvecto_rs/) and upgrade SQLAlchemy to ^2. Relate to https://github.com/tensorchord/pgvecto.rs/issues/11 --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	11 months ago
Daniel Chalef	0cbdba6a9b	zep: VectorStore: Use Native MMR (#12690) ... - refactor to use Zep's native MMR; update example - @baskaryan @eyurtsev	11 months ago
Daniel Chalef	cc3d3920e3	Zep: Summary Search and Example (#12686) ... Zep now has the ability to search over chat history summaries. This PR adds support for doing so. More here: https://blog.getzep.com/zep-v0-17/ @baskaryan @eyurtsev	11 months ago
Bagatur	526313002c	add import tests to all modules (#12806)	11 months ago
Harrison Chase	6609a6033f	fix vectorstore imports (#12804) ... Co-authored-by: Bagatur <baskaryan@gmail.com>	11 months ago
Nuno Campos	f66a9d2adf	Automatically add configurable key to config_schema if config_specs i… (#12798) ... …s present <!-- Thank you for contributing to LangChain! Replace this entire comment with: - **Description:** a description of the change, - **Issue:** the issue # it fixes (if applicable), - **Dependencies:** any dependencies required for this change, - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below), - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out! Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. It lives in `docs/extras` directory. If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17. -->	11 months ago
Praveen Venkateswaran	21eeba075c	enable the device_map parameter in huggingface pipeline (#12731) ... ### Enabling `device_map` in HuggingFacePipeline For multi-gpu settings with large models, the [accelerate](https://huggingface.co/docs/accelerate/usage_guides/big_modeling#using--accelerate) library provides the `device_map` parameter to automatically distribute the model across GPUs / disk. The [Transformers pipeline](3520e37e86/src/transformers/pipelines/__init__.py (L543)) enables users to specify `device` (or) `device_map`, and handles cases (with warnings) when both are specified. However, Langchain's HuggingFacePipeline only supports specifying `device` when calling transformers which limits large models and multi-gpu use-cases. Additionally, the [default value](8bd3ce59cd/libs/langchain/langchain/llms/huggingface_pipeline.py (L72)) of `device` is initialized to `-1` , which is incompatible with the transformers pipeline when `device_map` is specified. This PR addresses the addition of `device_map` as a parameter , and solves the incompatibility of `device = -1` when `device_map` is also specified. An additional test has been added for this feature. Additionally, some existing tests no longer work since 1. `max_new_tokens` has to be specified under `pipeline_kwargs` and not `model_kwargs` 2. The GPT2 tokenizer raises a `ValueError: Pipeline with tokenizer without pad_token cannot do batching`, since the `tokenizer.pad_token` is `None` ([related issue](https://github.com/huggingface/transformers/issues/19853) on the transformers repo). This PR handles fixing these tests as well. Co-authored-by: Praveen Venkateswaran <praveen.venkateswaran@ibm.com>	11 months ago
Mark Bell	3276aa3e17	__getattr__ should rase AttributeError not ImportError on missing attributes (#12801) ... [The python spec](https://docs.python.org/3/reference/datamodel.html#object.__getattr__) requires that `__getattr__` throw `AttributeError` for missing attributes but there are several places throwing `ImportError` in the current code base. This causes a specific problem with `hasattr` since it calls `__getattr__` then looks only for `AttributeError` exceptions. At present, calling `hasattr` on any of these modules will raise an unexpected exception that most code will not handle as `hasattr` throwing exceptions is not expected. In our case this is triggered by an exception tracker (Airbrake) that attempts to collect the version of all installed modules with code that looks like: `if hasattr(mod, "__version__"):`. With `HEAD` this is causing our exception tracker to fail on all exceptions. I only changed instances of unknown attributes raising `ImportError` and left instances of known attributes raising `ImportError`. It feels a little weird but doesn't seem to break anything.	11 months ago