langchain

Commit Graph

Author	SHA1	Message	Date
Bagatur	0ca8d4449c	add ls guide redirect (#11623 )	11 months ago
Bagatur	eedfddac2d	Restructure docs (#11620 )	11 months ago
Bagatur	7232e082de	bump 312 (#11621 )	11 months ago
Eugene Yurtsev	58220cda72	Remove LLM Bash and related bash utilities (#11619 ) Deprecate LLMBash and related bash utilities	11 months ago
ElliotKetchup	683f4a93b9	Update azureml_chat_endpoint code exemple (#11602 ) - Description: azureml_chat_endpoint code exemple now takes endpoint_url and endpoint_api_key parameter into consideration, - Issue: None), - Dependencies: None, - Tag maintainer: None, - Twitter handle: @ElliotAlladaye	11 months ago
Yong woo Song	fca34eb122	Fix: invalid link to chat model in openai platform docs (#11609 ) There is some invalid link in open ai platform [docs](https://python.langchain.com/docs/integrations/platforms/openai). So i fixed it to valid links. - `/docs/integrations/chat_models/openai` -> `/docs/integrations/chat/openai` - `/docs/integrations/chat_models/azure_openai` -> `/docs/integrations/chat/azure_chat_openai` Thanks! ☺️	11 months ago
Shubham Kushwaha	49de862076	Arcee.ai LLM & Retriever integration (#11579 ) - Description: This PR introduces a new LLM and Retriever API to https://arcee.ai for the python client - Issue: implements the integrations as requested in #11578 , - Dependencies: no dependencies are required, - Tag maintainer: @hwchase17 - Twitter handle: shwooobham ✅ `make format`, `make lint` and `make test` runs locally. ```shell =========== 1245 passed, 277 skipped, 20 warnings in 16.26s =========== ./scripts/check_pydantic.sh . ./scripts/check_imports.sh poetry run ruff . [ "." = "" ] \|\| poetry run black . --check All done! ✨ 🍰 ✨ 1818 files would be left unchanged. [ "." = "" ] \|\| poetry run mypy . Success: no issues found in 1815 source files [ "." = "" ] \|\| poetry run black . All done! ✨ 🍰 ✨ 1818 files left unchanged. [ "." = "" ] \|\| poetry run ruff --select I --fix . poetry run codespell --toml pyproject.toml poetry run codespell --toml pyproject.toml -w ``` Contributions 1. Arcee (langchain/llms), ArceeRetriever (langchain/retrievers), ArceeWrapper (langchain/utilities) 2. docs for Arcee (llms/arcee.py) and ArceeRetriever(retrievers/arcee.py) 3. cc: @jacobsolawetz @ben-epstein --------- Co-authored-by: Shubham <shubham@sORo.local>	11 months ago
Eugene Yurtsev	b6a2507794	Docs to use LLMSymbolicMath and LLMBash + utilities from experimental (#11614 ) Update docs in lieu of: https://github.com/langchain-ai/langchain/discussions/11352	11 months ago
Eugene Yurtsev	b56ca0c2a4	Deprecate LLMSymbolicMath from langchain core (#11615 ) Deprecate LLMSymbolicMath from langchain core package.	11 months ago
Leonid Ganeline	59adeaddb3	docs: update `dependents` (#11502 ) A regular update of dependents.	11 months ago
Eugene Yurtsev	c9bce5bbfb	Add version to langchain_experimental (#11613 ) Add version to langchain experimental	11 months ago
Predrag Gruevski	22abeb9f6c	Disable loading jinja2 `PromptTemplate` from file. (#10252 ) jinja2 templates are not sandboxed and are at risk for arbitrary code execution. To mitigate this risk: - We no longer support loading jinja2-formatted prompt template files. - `PromptTemplate` with jinja2 may still be constructed manually, but the class carries a security warning reminding the user to not pass untrusted input into it. Resolves #4394.	11 months ago
Bagatur	b642d00f9f	rm slack from community.md (#11610 )	11 months ago
Nuno Campos	c7c03d4709	Fix mutation bugs in callback manager configure (#11603 ) <!-- Thank you for contributing to LangChain! Replace this entire comment with: - Description: a description of the change, - Issue: the issue # it fixes (if applicable), - Dependencies: any dependencies required for this change, - Tag maintainer: for a quicker response, tag the relevant maintainer (see below), - Twitter handle: we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out! Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. It lives in `docs/extras` directory. If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17. -->	11 months ago
cccs-eric	e2a9072b80	Fix CohereRerank configuration (#11583 ) Description: CohereRerank is missing `cohere_api_key` as a field and since extras are forbidden, it is not possible to pass-in the key. The only way is to use an env variable named `COHERE_API_KEY`. For example, if trying to create a compressor like this: ```python cohere_api_key = "......Cohere api key......" compressor = CohereRerank(cohere_api_key=cohere_api_key) ``` you will get the following error: ``` File "/langchain/.venv/lib/python3.10/site-packages/pydantic/v1/main.py", line 341, in __init__ raise validation_error pydantic.v1.error_wrappers.ValidationError: 1 validation error for CohereRerank cohere_api_key extra fields not permitted (type=value_error.extra) ```	11 months ago
Anar	55fef4b64b	implemented add files method in LLMRails (#11518 ) This PR provides add files method with LLMRails. Implemented here are: docs/extras/integrations/vectorstores/llm-rails.ipynb --------- Co-authored-by: Anar Aliyev <aaliyev@mgmt.cloudnet.services>	11 months ago
unifyh	fd7f129f10	Docs: Fix broken line breaks in snippets (#11523 ) Description: This PR fix some code snippets that have raw `\n`'s instead of actual line breaks. Issue: Currently some snippets look like this: ![image](https://github.com/langchain-ai/langchain/assets/18213435/355b4911-38e9-4ba4-8570-f928557b6c13) Affected pages: - https://python.langchain.com/docs/integrations/providers/predictionguard#example-usage - https://python.langchain.com/docs/modules/agents/how_to/custom_llm_agent#set-up-environment - https://python.langchain.com/docs/modules/chains/foundational/llm_chain#get-started - https://python.langchain.com/docs/integrations/providers/shaleprotocol#how-to Tag maintainer: @hwchase17	11 months ago
Stephen Hankinson	316dddc7cd	fix wording of query_sql_database_tool_description (#11530 ) - Description: Fixes minor typo for the query_sql_database_tool_description in the db toolkit - Issue: N/A - Dependencies: N/A - Tag maintainer: @nfcampos - Twitter handle: N/A	11 months ago
Ash Vardanian	1acfe86353	Accelerating Math Utils with SimSIMD (#11566 ) LangChain relies on NumPy to compute cosine distances, which becomes a bottleneck with the growing dimensionality and number of embeddings. To avoid this bottleneck, in our libraries at [Unum](https://github.com/unum-cloud), we have created a specialized package - [SimSIMD](https://github.com/ashvardanian/simsimd), that knows how to use newer hardware capabilities. Compared to SciPy and NumPy, it reaches 3x-200x performance for various data types. Since publication, several LangChain users have asked me if I can integrate it into LangChain to accelerate their workflows, so here I am 🤗 ## Benchmarking To conduct benchmarks locally, run this in your Jupyter: ```py import numpy as np import scipy as sp import simsimd as simd import timeit as tt def cosine_similarity_np(X: np.ndarray, Y: np.ndarray) -> np.ndarray: X_norm = np.linalg.norm(X, axis=1) Y_norm = np.linalg.norm(Y, axis=1) with np.errstate(divide="ignore", invalid="ignore"): similarity = np.dot(X, Y.T) / np.outer(X_norm, Y_norm) similarity[np.isnan(similarity) \| np.isinf(similarity)] = 0.0 return similarity def cosine_similarity_sp(X: np.ndarray, Y: np.ndarray) -> np.ndarray: return 1 - sp.spatial.distance.cdist(X, Y, metric='cosine') def cosine_similarity_simd(X: np.ndarray, Y: np.ndarray) -> np.ndarray: return 1 - simd.cdist(X, Y, metric='cosine') X = np.random.randn(1, 1536).astype(np.float32) Y = np.random.randn(1, 1536).astype(np.float32) repeat = 1000 print("NumPy: {:,.0f} ops/s, SciPy: {:,.0f} ops/s, SimSIMD: {:,.0f} ops/s".format( repeat / tt.timeit(lambda: cosine_similarity_np(X, Y), number=repeat), repeat / tt.timeit(lambda: cosine_similarity_sp(X, Y), number=repeat), repeat / tt.timeit(lambda: cosine_similarity_simd(X, Y), number=repeat), )) ``` ## Results I ran this on an M2 Pro Macbook for various data types and different number of rows in `X` and reformatted the results as a table for readability: \| Data Type \| NumPy \| SciPy \| SimSIMD \| \| :--- \| ---: \| ---: \| ---: \| \| `f32, 1` \| 59,114 ops/s \| 80,330 ops/s \| 475,351 ops/s \| \| `f16, 1` \| 32,880 ops/s \| 82,420 ops/s \| 650,177 ops/s \| \| `i8, 1` \| 47,916 ops/s \| 115,084 ops/s \| 866,958 ops/s \| \| `f32, 10` \| 40,135 ops/s \| 24,305 ops/s \| 185,373 ops/s \| \| `f16, 10` \| 7,041 ops/s \| 17,596 ops/s \| 192,058 ops/s \| \| `f16, 10` \| 21,989 ops/s \| 25,064 ops/s \| 619,131 ops/s \| \| `f32, 100` \| 3,536 ops/s \| 3,094 ops/s \| 24,206 ops/s \| \| `f16, 100` \| 900 ops/s \| 2,014 ops/s \| 23,364 ops/s \| \| `i8, 100` \| 5,510 ops/s \| 3,214 ops/s \| 143,922 ops/s \| It's important to note that SimSIMD will underperform if both matrices are huge. That, however, seems to be an uncommon usage pattern for LangChain users. You can find a much more detailed performance report for different hardware models here: - [Apple M2 Pro](https://ashvardanian.com/posts/simsimd-faster-scipy/#appendix-1-performance-on-apple-m2-pro). - [4th Gen Intel Xeon Platinum](https://ashvardanian.com/posts/simsimd-faster-scipy/#appendix-2-performance-on-4th-gen-intel-xeon-platinum-8480). - [AWS Graviton 3](https://ashvardanian.com/posts/simsimd-faster-scipy/#appendix-3-performance-on-aws-graviton-3). ## Additional Notes 1. Previous version used `X = np.array(X)`, to repackage lists of lists. It's an anti-pattern, as it will use double-precision floating-point numbers, which are slow on both CPUs and GPUs. I have replaced it with `X = np.array(X, dtype=np.float32)`, but a more selective approach should be discussed. 2. In numerical computations, it's recommended to explicitly define tolerance levels, which were previously avoided in `np.allclose(expected, actual)` calls. For now, I've set absolute tolerance to distance computation errors as 0.01: `np.allclose(expected, actual, atol=1e-2)`. --- - Dependencies: adds `simsimd` dependency - Tag maintainer: @hwchase17 - Twitter handle: @ashvardanian --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	11 months ago
benchello	5de64e6d60	Add option to specify metadata columns in CSV loader (#11576 ) #### Description This PR adds the option to specify additional metadata columns in the CSVLoader beyond just `Source`. The current CSV loader includes all columns in `page_content` and if we want to have columns specified for `page_content` and `metadata` we have to do something like the below.: ``` csv = pd.read_csv( "path_to_csv" ).to_dict("records") documents = [ Document( page_content=doc["content"], metadata={ "last_modified_by": doc["last_modified_by"], "point_of_contact": doc["point_of_contact"], } ) for doc in csv ] ``` #### Usage Example Usage: ``` csv_test = CSVLoader( file_path="path_to_csv", metadata_columns=["last_modified_by", "point_of_contact"] ) ``` Example CSV: ``` content, last_modified_by, point_of_contact "hello world", "Person A", "Person B" ``` Example Result: ``` Document { page_content: "hello world" metadata: { row: '0', source: 'path_to_csv', last_modified_by: 'Person A', point_of_contact: 'Person B', } ``` --------- Co-authored-by: Ben Chello <bchello@dropbox.com> Co-authored-by: Bagatur <baskaryan@gmail.com>	11 months ago
Stephen Hankinson	447a523662	fix comments in output format (#11536 ) - Description: Fixes the comments in the ConvoOutputParser. Because the \\\\ is escaping a single \\, they render something like: `"action_input": string \ The input to the action` in the prompt. Changing this to \\\\\\\\ lets it escape two slashes so that it renders a proper comment: `"action_input": string \\ The input to the action` - Issue: N/A - Dependencies: - Tag maintainer: @hwchase17 - Twitter handle:	11 months ago
Michael Landis	8e45f720a8	feat: add momento vector index as a vector store provider (#11567 ) Description: - Added Momento Vector Index (MVI) as a vector store provider. This includes an implementation with docstrings, integration tests, a notebook, and documentation on the docs pages. - Updated the Momento dependency in pyproject.toml and the lock file to enable access to MVI. - Refactored the Momento cache and chat history session store to prefer using "MOMENTO_API_KEY" over "MOMENTO_AUTH_TOKEN" for consistency with MVI. This change is backwards compatible with the previous "auth_token" variable usage. Updated the code and tests accordingly. Dependencies: - Updated Momento dependency in pyproject.toml. Testing: - Run the integration tests with a Momento API key. Get one at the [Momento Console](https://console.gomomento.com) for free. MVI is available in AWS us-west-2 with a superuser key. - `MOMENTO_API_KEY=<your key> poetry run pytest tests/integration_tests/vectorstores/test_momento_vector_index.py` Tag maintainer: @eyurtsev Twitter handle: Please mention @momentohq for this addition to langchain. With the integration of Momento Vector Index, Momento caching, and session store, Momento provides serverless support for the core langchain data needs. Also mention @mlonml for the integration.	11 months ago
Eugene Yurtsev	ca2eed36b7	LangChain cli fix a few bugs (#11573 ) Code was assuming that `git` and `poetry` exist. In addition, it was not ignoring pycache files that get generated during run time	11 months ago
MSFTeegarden	923e9f9596	Add Azure Redis example (#11570 ) Description This PR adds an additional Example to the Redis integration documentation. [The example](https://learn.microsoft.com/azure/azure-cache-for-redis/cache-tutorial-vector-similarity) is a step-by-step walkthrough of using Azure Cache for Redis and Azure OpenAI for vector similarity search, using LangChain extensively throughout. Issue Nothing specific, just adding an additional example. Dependencies None. Tag Maintainer Tagging @hwchase17 :)	11 months ago
Hugues Chocart	258ae1ba5f	[LLMonitor Callback Handler]: Add error handling (#11563 ) Wraps every callback handler method in error handlers to avoid breaking users' programs when an error occurs inside the handler. Thanks @valdo99 for the suggestion 🙂	11 months ago
Eugene Yurtsev	2aabfafe1e	Module documentation for langchain runnables (#11550 ) Add in code documentation for langchain runnables module.	11 months ago
Eugene Yurtsev	d8fa94e6fa	RunnablePassthrough: In code documentation (#11552 ) Add in code documentation for a runnable passthrough	11 months ago
Eugene Yurtsev	b42f218cfc	RunnableLambda: Add in code docs (#11521 ) Add in code docs for Runnable Lambda	11 months ago
maks-operlejn-ds	f64522fbaf	Reset deanonymizer mapping (#11559 ) @hwchase17 @baskaryan	11 months ago
maks-operlejn-ds	b14b65d62a	Support all presidio entities (#11558 ) https://microsoft.github.io/presidio/supported_entities/ @baskaryan @hwchase17	11 months ago
maks-operlejn-ds	4d62def9ff	Better deanonymizer matching strategy (#11557 ) @baskaryan, @hwchase17	11 months ago
Ash Vardanian	a992b9670d	Fix: Missing DuckDuckGo package version (#11535 ) [The `duckduckgo-search` v3.9.2 was removed from PyPi](https://pypi.org/project/duckduckgo-search/#history). That breaks the build. - Description: refreshes the Poetry dependency to v3.9.3 - Tag maintainer: @baskaryan - Twitter handle: @ashvardanian	11 months ago
Bagatur	0a754fa286	redirect langsmith guides (#11562 )	11 months ago
Nuno Campos	2f2a5fd582	Update Dockerfile.base (#11556 )	11 months ago
Bagatur	8932ed3f07	bump 311 (#11555 )	11 months ago
Bagatur	e7a0def1bc	QoL improvements to query constructor (#11504 ) updating query constructor and self query retriever to - make it easier to pass in examples - validate attributes used in query - remove invalid parts of query - make it easier to get + edit prompt - make query constructor a runnable - make self query retriever use as runnable	11 months ago
Taikono-Himazin	eec53fa294	Added autodetect_encoding option to csvLoader (#11327 )	11 months ago
Holt Skinner	09c66fe04f	feat: Update Google Document AI Parser (#11413 ) - Description: Code Refactoring, Documentation Improvements for Google Document AI PDF Parser - Adds Online (synchronous) processing option. - Adds default field mask to limit payload size. - Skips Human review by default. - Issue: Fixes #10589 --------- Co-authored-by: Erick Friis <erick@langchain.dev>	11 months ago
Nuno Campos	628cc4cce8	Rename RunnableMap to RunnableParallel (#11487 ) - keep alias for RunnableMap - update docs to use RunnableParallel and RunnablePassthrough.assign <!-- Thank you for contributing to LangChain! Replace this entire comment with: - Description: a description of the change, - Issue: the issue # it fixes (if applicable), - Dependencies: any dependencies required for this change, - Tag maintainer: for a quicker response, tag the relevant maintainer (see below), - Twitter handle: we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out! Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. It lives in `docs/extras` directory. If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17. -->	11 months ago
Eugene Yurtsev	6a10e8ef31	Add documentation to Runnable (#11516 )	11 months ago
William FH	eb572f41a6	Add LangSmith Run Chat Loader (#11458 )	11 months ago
David Duong	484947c492	Fetch up-to-date attributes for env-pulled kwargs during serialisation of OpenAI classes (#11499 )	11 months ago
Leonid Ganeline	c3d2b01adf	docs: `integrations/retrievers` cleanup (#11388 ) fixed several notebooks: - headers - formats --------- Co-authored-by: Erick Friis <erick@langchain.dev>	11 months ago
Bagatur	5470e730d2	raise openapi import error (#11495 )	11 months ago
Erick Friis	29f5f70415	Rename some last hwchase17/langchain links (#11494 )	11 months ago
Fabrice Pont	872836c541	feat: add markdown list parser (#11411 ) Description: add `MarkdownListOutputParser` as a new `ListOutputParser` Issue: #11410	11 months ago
Erick Friis	8f50b616c5	Remove optional from vectara source (#11493 ) fyi @ofermend --------- Co-authored-by: Ofer Mendelevitch <ofer@vectara.com> Co-authored-by: Ofer Mendelevitch <ofermend@gmail.com>	11 months ago
Maciej Dzieżyc	bcd308c368	Fix Open in Colab link for ClearML docs 2 (#11491 ) Description: Fixed the Open in Colab link for ClearML docs Issue: https://github.com/allegroai/clearml/issues/1125 Twitter handle: DziezycMaciej	11 months ago
Bagatur	88ab69c288	mv docs extras (#11399 )	11 months ago
Bagatur	53887242a1	bump 310 (#11486 )	11 months ago

1 2 3 4 5 ...

5111 Commits (0ca8d4449cb80cdc52e3f526a383af297a645488) All Branches Search

5111 Commits (0ca8d4449cb80cdc52e3f526a383af297a645488)

All Branches