langchain

Commit Graph

Author	SHA1	Message	Date
Erick Friis	2ffb2144a6	experimental[patch]: release 0.0.54 (#19000 )	6 months ago
Erick Friis	873d06c009	langchain[patch]: release 0.1.12 (#18999 )	6 months ago
Leonid Ganeline	9c8523b529	community[patch]: flattening imports 3 (#18939 ) @eyurtsev	6 months ago
Erick Friis	af50f21765	community[patch]: release 0.0.28 (#18993 )	6 months ago
Erick Friis	4881bb669c	core[patch]: release 0.1.31 (#18989 )	6 months ago
Erick Friis	a29e8d8594	elasticsearch[patch]: fix integration tests for release (#18980 )	6 months ago
Erick Friis	0d1f6c417c	elasticsearch[patch]: release 0.1.1 (#18978 )	6 months ago
Max Jakob	911ccf9aa6	docs: elasticsearch retriever (#18965 ) Add documentation notebook for `ElasticsearchRetriever`. ## Dependencies - [ ] Release new `langchain-elasticsearch` version 0.2.0 that includes `ElasticsearchRetriever`	6 months ago
Dobiichi-Origami	471f2ed40a	community[patch]: re-arrange the addtional_kwargs of returned qianfan structure to avoid _merge_dict issue (#18889 ) fix issue: https://github.com/langchain-ai/langchain/issues/18441 PTAL, thanks @baskaryan, @efriis, @eyurtsev, @hwchase17. --------- Co-authored-by: Bagatur <baskaryan@gmail.com> Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com>	6 months ago
Naman Jain	75122646b5	core[patch]: fixed circular dependency with json schema (#18657 ) Description: Circular dependencies when parsing references leading to `RecursionError: maximum recursion depth exceeded` issue. This PR address the issue by handling previously seen refs as in any typical DFS to avoid infinite depths. Issue: https://github.com/langchain-ai/langchain/issues/12163 Twitter handle: https://twitter.com/theBhulawat - [x] Add tests and docs: If you're adding a new integration, please include 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. It lives in `docs/docs/integrations` directory. - [x] Lint and test: Run `make format`, `make lint` and `make test` from the root of the package(s) you've modified. See contribution guidelines for more: https://python.langchain.com/docs/contributing/ --------- Co-authored-by: Bagatur <baskaryan@gmail.com> Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com>	6 months ago
Tymofii	0bec1f6877	commnity[patch]: refactor code for faiss vectorstore, update faiss vectorstore documentation (#18092 ) Description: Refactor code of FAISS vectorcstore and update the related documentation. Details: - replace `.format()` with f-strings for strings formatting; - refactor definition of a filtering function to make code more readable and more flexible; - slightly improve efficiency of `max_marginal_relevance_search_with_score_by_vector` method by removing unnecessary looping over the same elements; - slightly improve efficiency of `delete` method by using set data structure for checking if the element was already deleted; Issue: fix small inconsistency in the documentation (the old example was incorrect and unappliable to faiss vectorstore) Dependencies: basic langchain-community dependencies and `faiss` (for CPU or for GPU) Twitter handle: antonenkodev	6 months ago
Roshan Santhosh	acf1ecc081	langchain[patch]: update llm_router.py (#18865 ) Issue : _call method of LLMRouterChain uses predict_and_parse, which is slated for deprecation. Description : Instead of using predict_and_parse, this replaces it with individual predict and parse functions.	6 months ago
Bagatur	18de77cc8c	core[minor]: add streaming support to OAI tool parsers (#18940 ) Co-authored-by: Erick Friis <erick@langchain.dev>	6 months ago
Bagatur	e0e688a277	core[minor]: generation info on msg (#18592 ) related to #16403 #17188	6 months ago
Tomaz Bratanic	cda43c5a11	experimental[patch]: Fix LLM graph transformer default prompt (#18856 ) Some LLMs do not allow multiple user messages in sequence.	6 months ago
Bagatur	19721246f5	core[patch]: support labeled json schema as tools (#18935 )	6 months ago
Erick Friis	0d888a65cb	core[patch]: move some attr/methods to BaseLanguageModel (#18936 ) Cleans up some shared code between `BaseLLM` and `BaseChatModel`. One functional difference to make it more consistent (see comment)	6 months ago
aditya thomas	5c2f7e6b2b	partners[openai]: update the docstring of OpenAI, OpenAIEmbeddings and ChatOpenAI classes (#18908 ) Description: Update the docstring of OpenAI, OpenAIEmbeddings and ChatOpenAI classes Issue: Update import module paths to the current LangChain API Dependencies: None Lint and test: `make format` and `make lint` were run This incorporates the review comments from langchain-ai/langchain#18637 which I closed due to an issue I had in updating that pr branch --------- Co-authored-by: Erick Friis <erick@langchain.dev>	6 months ago
Leonid Ganeline	11195cfa42	community[patch]: speed up import times in the community package (#18928 ) This PR speeds up import times in the community package	6 months ago
aditya thomas	8544f748f2	community[patch]: update AnthropicLLM deprecation message (#18869 ) Description: Update AnthropicLLM deprecation message import path for ChatAnthropic Issue: Incorrect import path in deprecation message Dependencies: None Lint and test: `make format`, `make lint` and `make test` were run	6 months ago
Virat Singh	cafffe8a21	community: Add PolygonAggregates tool (#18882 ) Description: In this PR, I am adding a `PolygonAggregates` tool, which can be used to get historical stock price data (called aggregates by Polygon) for a given ticker. Polygon [docs](https://polygon.io/docs/stocks/get_v2_aggs_ticker__stocksticker__range__multiplier___timespan___from___to) for this endpoint. Twitter: [@virattt](https://twitter.com/virattt)	6 months ago
Erick Friis	93ef8ead0b	mongodb[patch]: fix core dep (#18926 )	6 months ago
Mohammad Mohtashim	43db4cd20e	core[major]: On Tool End Observation Casting Fix (#18798 ) This PR updates the on_tool_end handlers to return the raw output from the tool instead of casting it to a string. This is technically a breaking change, though it's impact is expected to be somewhat minimal. It will fix behavior in `astream_events` as well. Fixes the following issue #18760 raised by @eyurtsev --------- Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com>	6 months ago
Massimiliano Pronesti	8113d612bb	community[patch]: support modin document loader (#18866 ) Langchain community document loaders support `pyspark`, `polars`, and `pandas` dataframes but not `modin`'s. This PR addresses this point.	6 months ago
Pol Ruiz Farre	a7f63d8cb4	community[patch]: Fix BasePDFLoader suffix for s3 presigned urls (#18844 ) BasePDFLoader doesn't parse the suffix of the file correctly when parsing S3 presigned urls. This fix enables the proper detection and parsing of S3 presigned URLs to prevent errors such as `OSError: [Errno 36] File name too long`. No additional dependencies required.	6 months ago
Joshua Carroll	ddaf9de169	community: Fix bug with StreamlitChatMessageHistory (#18834 ) - Description: Fix Streamlit bug which was introduced by https://github.com/langchain-ai/langchain/pull/18250, update integration test - Issue: https://github.com/langchain-ai/langchain/issues/18684 - Dependencies: None	6 months ago
Tomaz Bratanic	a28be31a96	Switch to md5 for deduplication in neo4j integrations (#18846 ) Deduplicate documents using MD5 of the page_content. Also allows for custom deduplication with graph ingestion method by providing metadata id attribute --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	6 months ago
Tomaz Bratanic	246724faab	LLM graph transformer prompt engineering (#18843 ) A bit of prompt engineering to improve results	6 months ago
Erick Friis	b48865bf94	langchain[patch]: attach hub metadata (#18830 )	6 months ago
Ammar	34b31a8cc7	core: add in-code docs for RunnableAssign class (#18826 ) Description: Improves the docstring for `RunnableAssign` by providing a concise description and a self-contained code example. Issue: #18803	6 months ago
Leonid Ganeline	476d6dc596	community[patch]: Use getattr for `toolkits` imports (#18825 ) This will preserve the namespace, without actually loading the underlying packages on init.	6 months ago
Erick Friis	bbb609ac9d	core[patch]: fix arbitrary config keys (#18827 )	6 months ago
Luis Antonio Vieira Junior	67c880af74	community[patch]: adding linearization config to AmazonTextractPDFLoader (#17489 ) - Description: Adding an optional parameter `linearization_config` to the `AmazonTextractPDFLoader` so the caller can define how the output will be linearized, instead of forcing a predefined set of linearization configs. It will still have a default configuration as this will be an optional parameter. - Issue: #17457 - Dependencies: The same ones that already exist for `AmazonTextractPDFLoader` - Twitter handle: [@lvieirajr19](https://twitter.com/lvieirajr19) --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	6 months ago
Anis ZAKARI	37e89ba5b1	community[patch]: Bedrock add support for mistral models (#18756 ) Description*: My previous [PR](https://github.com/langchain-ai/langchain/pull/18521) was mistakenly closed, so I am reopening this one. Context: AWS released two Mistral models on Bedrock last Friday (March 1, 2024). This PR includes some code adjustments to ensure their compatibility with the Bedrock class. --------- Co-authored-by: Anis ZAKARI <anis.zakari@hymaia.com> Co-authored-by: Erick Friis <erick@langchain.dev>	6 months ago
Alexander Dicke	66576948e0	experimental[minor]: adds mixtral wrapper (#17423 ) Description: Adds a chat wrapper for Mixtral models using the [prompt template](https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1#instruction-format). --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	6 months ago
Keith Chan	914af69b44	community[patch]: Update azuresearch vectorstore from_texts() method to include fields argument (#17661 ) - Description: Update azuresearch vectorstore from_texts() method to include fields argument, necessary for creating an Azure AI Search index with custom fields. - Issue: Currently index fields are fixed to default fields if Azure Search index is created using from_texts() method - Dependencies: None - Twitter handle: None --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	6 months ago
al1p	46f0cea2b9	community[patch][: improved the suffix prompt to avoid loop (#17791 ) Small improvement to the openapi prompt. The agent was not finding the server base URL (looping through all nodes). This small change narrows the search and enables finding the url faster. No dependency Twitter : @al1pra	6 months ago
Dmitry Kankalovich	f5117e907d	openai[patch]: Proper example for AzureOpenAI usage in error message (#17798 ) # Proper example for AzureOpenAI usage in error message The original error message is wrong in part of a usage example it gives. Corrected to the right one. Co-authored-by: Dzmitry Kankalovich <dzmitry_kankalovich@epam.com> Co-authored-by: Bagatur <baskaryan@gmail.com>	6 months ago
Théo LEBRUN	cf94091cd0	community[patch]: Skip nested directories when using S3DirectoryLoader (#17829 ) - Description: `S3DirectoryLoader` is failing if prefix is a folder (ex: `my_folder/`) because `S3FileLoader` will try to load that folder and will fail. This PR skip nested directories so prefix can be set to folder instead of `my_folder/files_prefix`. - Issue: - #11917 - #6535 - #4326 - Dependencies: none - Twitter handle: @Falydoor - [x] Add tests and docs: If you're adding a new integration, please include 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. It lives in `docs/docs/integrations` directory. - [x] Lint and test: Run `make format`, `make lint` and `make test` from the root of the package(s) you've modified. See contribution guidelines for more: https://python.langchain.com/docs/contributing/	6 months ago
Venkatesan	7a18b63dbf	community[patch]: Mongo index creation (#17748 ) - [ ] Title: Mongodb: MongoDB connection performance improvement. - [ ] Message: - Description: I made collection index_creation as optional. Index Creation is one time process. - Issue: MongoDBChatMessageHistory class object is attempting to create an index during connection, causing each request to take longer than usual. This should be optional with a parameter. - Dependencies: N/A - Branch to be checked: origin/mongo_index_creation --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	6 months ago
wt3639	5b5b37a999	community[patch]: Add embedding instruction to HuggingFaceBgeEmbeddings (#18017 ) - Description: Add embedding instruction to HuggingFaceBgeEmbeddings, so that it can be compatible with nomic and other models that need embedding instruction. --------- Co-authored-by: Tao Wu <tao.wu@rwth-aachen.de> Co-authored-by: Bagatur <baskaryan@gmail.com>	6 months ago
Erick Friis	a8de6d1533	anthropic[patch]: integration test update (#18823 )	6 months ago
wewebber-merlin	d1f5bc4906	anthropic[patch]: add kwargs to format_output base (#18715 ) _generate() and _agenerate() both accept kwargs, then pass them on to _format_output; but _format_output doesn't accept kwargs. Attempting to pass, e.g., timeout=50 to _generate (or invoke()) results in a TypeError. Thank you for contributing to LangChain! - [ ] PR title: "package: description" - Where "package" is whichever of langchain, community, core, experimental, etc. is being modified. Use "docs: ..." for purely docs changes, "templates: ..." for template changes, "infra: ..." for CI changes. - Example: "community: add foobar LLM" - [ ] PR message: *Delete this entire checklist* and replace with - Description: a description of the change - Issue: the issue # it fixes, if applicable - Dependencies: any dependencies required for this change - Twitter handle: if your PR gets announced, and you'd like a mention, we'll gladly shout you out! - [ ] Add tests and docs: If you're adding a new integration, please include 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. It lives in `docs/docs/integrations` directory. - [ ] Lint and test: Run `make format`, `make lint` and `make test` from the root of the package(s) you've modified. See contribution guidelines for more: https://python.langchain.com/docs/contributing/ Additional guidelines: - Make sure optional dependencies are imported within a function. - Please do not add dependencies to pyproject.toml files (even optional ones) unless they are required for unit tests. - Most PRs should not touch more than one package. - Changes should be backwards compatible. - If you are adding something to community, do not re-import it in langchain. If no one reviews your PR within a few days, please @-mention one of baskaryan, efriis, eyurtsev, hwchase17. --------- Co-authored-by: Erick Friis <erick@langchain.dev>	6 months ago
Erick Friis	aa7bce6b13	anthropic[patch]: release 0.1.4 (#18822 )	6 months ago
Erick Friis	a5bcddc738	anthropic[patch]: streaming param (#18819 )	6 months ago
Erick Friis	8c0b215c02	anthropic[patch]: fix format output args (#18816 )	6 months ago
Ishani Vyas	2b0cbd65ba	community[patch]: Add Passio Nutrition AI Food Search Tool to Community Package (#18278 ) ## Add Passio Nutrition AI Food Search Tool to Community Package ### Description We propose adding a new tool to the `community` package, enabling integration with Passio Nutrition AI for food search functionality. This tool will provide a simple interface for retrieving nutrition facts through the Passio Nutrition AI API, simplifying user access to nutrition data based on food search queries. ### Implementation Details - Class Structure: Implement `NutritionAI`, extending `BaseTool`. It includes an `_run` method that accepts a query string and, optionally, a `CallbackManagerForToolRun`. - API Integration: Use `NutritionAIAPI` for the API wrapper, encapsulating all interactions with the Passio Nutrition AI and providing a clean API interface. - Error Handling: Implement comprehensive error handling for API request failures. ### Expected Outcome - User Benefits: Enable easy querying of nutrition facts from Passio Nutrition AI, enhancing the utility of the `langchain_community` package for nutrition-related projects. - Functionality: Provide a straightforward method for integrating nutrition information retrieval into users' applications. ### Dependencies - `langchain_core` for base tooling support - `pydantic` for data validation and settings management - Consider `requests` or another HTTP client library if not covered by `NutritionAIAPI`. ### Tests and Documentation - Unit Tests: Include tests that mock network interactions to ensure tool reliability without external API dependency. - Documentation: Create an example notebook in `docs/docs/integrations/tools/passio_nutrition_ai.ipynb` showing usage, setup, and example queries. ### Contribution Guidelines Compliance - Adhere to the project's linting and formatting standards (`make format`, `make lint`, `make test`). - Ensure compliance with LangChain's contribution guidelines, particularly around dependency management and package modifications. ### Additional Notes - Aim for the tool to be a lightweight, focused addition, not introducing significant new dependencies or complexity. - Potential future enhancements could include caching for common queries to improve performance. ### Twitter Handle - Here is our Passio AI [twitter handle](https://twitter.com/@passio_ai) where we announce our products. If no one reviews your PR within a few days, please @-mention one of baskaryan, efriis, eyurtsev, hwchase17.	6 months ago
Kushagra	b1f22bf76c	community[minor]: added a feature to filter documents in Mongoloader (#18253 ) "community: added a feature to filter documents in Mongoloader" - Description: added a feature to filter documents in Mongoloader - Feature: the feature #18251 - Dependencies: No - Twitter handle: https://twitter.com/im_Kushagra	6 months ago
Eugene Yurtsev	1f50274df7	community[patch]: Add pgvector to docker compose and update settings used in integration test (#18815 )	6 months ago
Erick Friis	ad29806255	nvidia-trt, nvidia-ai-endpoints: move to repo (#18814 ) NVIDIA maintained in https://github.com/langchain-ai/langchain-nvidia	6 months ago
Christophe Bornet	e54a49b697	community[minor]: Add lazy_table_reflection param to SqlDatabase (#18742 ) For some DBs with lots of tables, reflection of all the tables can take very long. So this change will make the tables be reflected lazily when get_table_info() is called and `lazy_table_reflection` is True.	6 months ago
Christophe Bornet	ead2a74806	community: Implement lazy_load() for JSONLoader (#18643 ) Covered by `tests/unit_tests/document_loaders/test_json_loader.py`	6 months ago
Erick Friis	a88f62ec3c	langchain[patch]: getattr import from langchain.chains (#18160 )	6 months ago
Eugene Yurtsev	cdfb5b4ca1	core[minor]: Chat Models to fallback astream to fallback on sync stream if available (#18748 ) Allows all chat models that implement _stream, but not _astream to still have async streaming to work. Amongst other things this should resolve issues with streaming community model implementations through langserve since langserve is exclusively async.	6 months ago
aditya thomas	e00c1ff2b0	infra: ChatOpenAI unit tests for invoke() and ainvoke() (#18792 ) Description: Replacing the deprecated predict() and apredict() methods in the unit tests Issue: Not applicable Dependencies: None Lint and test: `make format`, `make lint` and `make test` have been run	6 months ago
Bagatur	3e29c04213	core[minor]: add BaseMessage.response_metadata (#18699 )	6 months ago
Bagatur	bc6249c889	langchain[patch]: runnable agent streaming param (#18761 ) Usage: ```python agent = RunnableAgent(runnable=runnable, .., stream_runnable=False) ``` or for convenience ```python agent_executor = AgentExecutor(agent=agent, ..., stream_runnable=False) ```	6 months ago
Tomaz Bratanic	c8c592d3f1	experimental[minor]: Add LLM graph transformer (#18733 ) Add a class that constructs knowledge graphs based on text using an LLM.	6 months ago
Phat Vo	3ecb903d49	community[patch] : Tidy up and update Clarifai SDK functions (#18314 ) Description : * Tidy up, add missing docstring and fix unused params * Enable using session token	6 months ago
Max Jakob	61a2eba081	elasticsearch[patch]: add top-level import, remove obsolete dependency (#18644 ) Make `ElasticsearchRetriever` available as top-level import. The `langchain` package depends on `langchain-community` so we do not need to depend on it explicitly.	6 months ago
Tomaz Bratanic	010a234f1e	docs: Fix diffbot graph transformer description (#18736 ) The previous docstring was invalid	6 months ago
Jan Nissen	b8922480ed	core[patch]: improve PydanticOutputParser typing (#18740 ) This PR adds generic typing to `PydanticOutputParser` so we get a typed output from `.parse` instead of `Any`. It should provide a better DX by way of Intellisense and for anyone strictly typing. Pre-change: ![Screenshot 2024-03-07 at 10 22 31 AM](https://github.com/langchain-ai/langchain/assets/22690160/fd22dde0-9fdc-4283-b283-4c98f0bc46e5) Post-change: ![Screenshot 2024-03-07 at 10 26 31 AM](https://github.com/langchain-ai/langchain/assets/22690160/7e23d2b7-8f8c-494f-80b3-187530a173ee) I haven't dug too deep, but I think a similar change could probably be added to `JsonOutputParser` so we don't have to pull up `.parse`. Co-authored-by: Jan Nissen <jan23@gmail.com>	6 months ago
Massimiliano Pronesti	3b975c6ebe	experimental[minor]: add support for modin in pandas agent (#18749 ) Added support for Intel's [modin](https://github.com/modin-project/modin) in `create_pandas_dataframe_agent`.	6 months ago
Tomaz Bratanic	4bfe888717	comunity[patch]: Fix neo4j sanitizing values (#18750 ) Fixing sanitization for when deeply nested lists appear	6 months ago
Eugene Yurtsev	6caceb5473	core[patch]: Automatic upgrade to AddableDict in transform and atransform (#18743 ) Automatic upgrade to transform and atransform Closes: https://github.com/langchain-ai/langchain/issues/18741 https://github.com/langchain-ai/langgraph/issues/136 https://github.com/langchain-ai/langserve/issues/504	6 months ago
Yunmo Koo	fee6f983ef	community[minor]: Integration for `Friendli` LLM and `ChatFriendli` ChatModel. (#17913 ) ## Description - Add [Friendli](https://friendli.ai/) integration for `Friendli` LLM and `ChatFriendli` chat model. - Unit tests and integration tests corresponding to this change are added. - Documentations corresponding to this change are added. ## Dependencies - Optional dependency [`friendli-client`](https://pypi.org/project/friendli-client/) package is added only for those who use `Frienldi` or `ChatFriendli` model. ## Twitter handle - https://twitter.com/friendliai	6 months ago
Smit Parmar	aed46cd6f2	community[patch]: Added support for filter out AWS Kendra search by score confidence (#12920 ) Description: It will add support for filter out kendra search by score confidence which will make result more accurate. For example ``` retriever = AmazonKendraRetriever( index_id=kendra_index_id, top_k=5, region_name=region, score_confidence="HIGH" ) ``` Result will not include the records which has score confidence "LOW" or "MEDIUM". Relevant docs https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/kendra/client/query.html https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/kendra/client/retrieve.html Issue: the issue # it resolve #11801 twitter: [@SmitCode](https://twitter.com/SmitCode)	6 months ago
Ian	390ef6abe3	community[minor]: Add Initial Support for TiDB Vector Store (#15796 ) This pull request introduces initial support for the TiDB vector store. The current version is basic, laying the foundation for the vector store integration. While this implementation provides the essential features, we plan to expand and improve the TiDB vector store support with additional enhancements in future updates. Upcoming Enhancements: * Support for Vector Index Creation: To enhance the efficiency and performance of the vector store. * Support for max marginal relevance search. * Customized Table Structure Support: Recognizing the need for flexibility, we plan for more tailored and efficient data store solutions. Simple use case exmaple ```python from typing import List, Tuple from langchain.docstore.document import Document from langchain_community.vectorstores import TiDBVectorStore from langchain_openai import OpenAIEmbeddings db = TiDBVectorStore.from_texts( embedding=embeddings, texts=['Andrew like eating oranges', 'Alexandra is from England', 'Ketanji Brown Jackson is a judge'], table_name="tidb_vector_langchain", connection_string=tidb_connection_url, distance_strategy="cosine", ) query = "Can you tell me about Alexandra?" docs_with_score: List[Tuple[Document, float]] = db.similarity_search_with_score(query) for doc, score in docs_with_score: print("-" * 80) print("Score: ", score) print(doc.page_content) print("-" * 80) ```	6 months ago
Bagatur	3b1eb1f828	community[patch]: chat hf typing fix (#18693 )	6 months ago
Jib	d60e93b6ae	langchain-mongodb: Standardize mongodb collection/index names in tests (#18755 ) ## Description: MongoDB integration tests link to a provided Atlas Cluster. We have very stringent permissions set against the cluster provided. In order to make it easier to track and isolate the collections each test gets run against, we've updated the collection names to map the test file name. i.e. `langchain_{filename}` => `langchain_test_vectorstores` Fixes integration test results ![image](https://github.com/langchain-ai/langchain/assets/2887713/41f911b9-55f7-4fe4-9134-5514b82009f9) ## Dependencies: Provided MONGODB_ATLAS_URI - [x] Lint and test: Run `make format`, `make lint` and `make test` from the root of the package(s) you've modified. See contribution guidelines for more: https://python.langchain.com/docs/contributing/ cc: @shaneharvey, @blink1073 , @NoahStapp , @caseyclements	6 months ago
Eugene Yurtsev	ca299a8e08	Docs: Add custom parsing documentation and extending langchain (#18331 ) * Added extending langchain.mdx -- we'll need to add links as we add more custom documentation * Added partial documentation about parsers	6 months ago
Eugene Yurtsev	8c71f92cb2	core: upgrade mypy to recent mypy (#18753 ) Testing this works per package on CI	6 months ago
Eugene Yurtsev	e188d4ecb0	Add dangerous parameter to requests tool (#18697 ) The tools are already documented as dangerous. Not clear whether adding an opt-in parameter is necessary or not	6 months ago
Erick Friis	1beb84b061	community[patch]: move pdf text tests to integration (#18746 )	6 months ago
Christophe Bornet	4a7d73b39d	community: If load() has been overridden, use it in default lazy_load() (#18690 )	6 months ago
Christophe Bornet	6cd7607816	community[patch]: Implement lazy_load() for MHTMLLoader (#18648 ) Covered by `tests/unit_tests/document_loaders/test_mhtml.py`	6 months ago
axiangcoding	9745b5894d	community[patch]: Chroma use uuid4 instead of uuid1 to generate random ids (#18723 ) - Description: Chroma use uuid4 instead of uuid1 as random ids. Use uuid1 may leak mac address, changing to uuid4 will not cause other effects. - Issue: None - Dependencies: None - Twitter handle: None	6 months ago
Guangdong Liu	ced5e7bae7	community[patch]: Fix sparkllm authentication problem. (#18651 ) - Description: fix sparkllm authentication problem.The current timestamp is in RFC1123 format. The time deviation must be controlled within 300s. I changed to re-obtain the url every time I ask a question. https://www.xfyun.cn/doc/spark/general_url_authentication.html#_1-2-%E9%89%B4%E6%9D%83%E5%8F%82%E6%95%B0	6 months ago
Erick Friis	89d32ffbbd	community[patch]: release 0.0.27 (#18708 )	6 months ago
Erick Friis	c09b520ce4	core[patch]: release 0.1.30 (#18706 )	6 months ago
Piyush Jain	2b234a4d96	Support for claude v3 models. (#18630 ) Fixes #18513. ## Description This PR attempts to fix the support for Anthropic Claude v3 models in BedrockChat LLM. The changes here has updated the payload to use the `messages` format instead of the formatted text prompt for all models; `messages` API is backwards compatible with all models in Anthropic, so this should not break the experience for any models. ## Notes The PR in the current form does not support the v3 models for the non-chat Bedrock LLM. This means, that with these changes, users won't be able to able to use the v3 models with the Bedrock LLM. I can open a separate PR to tackle this use-case, the intent here was to get this out quickly, so users can start using and test the chat LLM. The Bedrock LLM classes have also grown complex with a lot of conditions to support various providers and models, and is ripe for a refactor to make future changes more palatable. This refactor is likely to take longer, and requires more thorough testing from the community. Credit to PRs [18579](https://github.com/langchain-ai/langchain/pull/18579) and [18548](https://github.com/langchain-ai/langchain/pull/18548) for some of the code here. --------- Co-authored-by: Erick Friis <erick@langchain.dev>	6 months ago
Sam Khano	1b4dcf22f3	community[minor]: Add DocumentDBVectorSearch VectorStore (#17757 ) Description: - Added Amazon DocumentDB Vector Search integration (HNSW index) - Added integration tests - Updated AWS documentation with DocumentDB Vector Search instructions - Added notebook for DocumentDB integration with example usage --------- Co-authored-by: EC2 Default User <ec2-user@ip-172-31-95-226.ec2.internal>	6 months ago
Vittorio Rigamonti	51f3902bc4	community[minor]: Adding support for Infinispan as VectorStore (#17861 ) Description: This integrates Infinispan as a vectorstore. Infinispan is an open-source key-value data grid, it can work as single node as well as distributed. Vector search is supported since release 15.x For more: [Infinispan Home](https://infinispan.org) Integration tests are provided as well as a demo notebook	6 months ago
Max Jakob	cca0167917	elasticsearch[patch], community[patch]: update references, deprecate community classes (#18506 ) Follow up on https://github.com/langchain-ai/langchain/pull/17467. - Update all references to the Elasticsearch classes to use the partners package. - Deprecate community classes. --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	6 months ago
Djordje	12b4a4d860	community[patch]: Opensearch delete method added - indexing supported (#18522 ) - Description: Added delete method for OpenSearchVectorSearch, therefore indexing supported - Issue: No - Dependencies: No - Twitter handle: stkbmf	6 months ago
Erick Friis	687d27567d	openai[patch]: unit test azure init (#18703 )	6 months ago
Christophe Bornet	db8db6faae	community: Implement lazy_load() for PlaywrightURLLoader (#18676 ) Integration tests: `tests/integration_tests/document_loaders/test_url_playwright.py`	6 months ago
Aaron Yi	c092db862e	community[patch]: make metadata and text optional as expected in DocArray (#18678 ) ValidationError: 2 validation errors for DocArrayDoc text Field required [type=missing, input_value={'embedding': [-0.0191128...9, 0.01005221541175212]}, input_type=dict] For further information visit https://errors.pydantic.dev/2.5/v/missing metadata Field required [type=missing, input_value={'embedding': [-0.0191128...9, 0.01005221541175212]}, input_type=dict] For further information visit https://errors.pydantic.dev/2.5/v/missing ``` In the `_get_doc_cls` method, the `DocArrayDoc` class is defined as follows: ```python class DocArrayDoc(BaseDoc): text: Optional[str] embedding: Optional[NdArray] = Field(**embeddings_params) metadata: Optional[dict] ```	6 months ago
Eugene Yurtsev	4c25b49229	community[major]: breaking change in some APIs to force users to opt-in for pickling (#18696 ) This is a PR that adds a dangerous load parameter to force users to opt in to use pickle. This is a PR that's meant to raise user awareness that the pickling module is involved.	6 months ago
Eugene Yurtsev	0e52961562	community[patch]: Patch tdidf retriever (CVE-2024-2057) (#18695 ) This is a patch for `CVE-2024-2057`: https://www.cve.org/CVERecord?id=CVE-2024-2057 This affects users that: * Use the `TFIDFRetriever` * Attempt to de-serialize it from an untrusted source that contains a malicious payload	6 months ago
Erick Friis	2619420df1	mongodb[patch]: release 0.1.1 (#18692 )	6 months ago
Christophe Bornet	ea141511d8	core: Move document loader interfaces to core (#17723 ) This is needed to be able to move document loaders to partner packages. --------- Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com>	6 months ago
Christophe Bornet	5985454269	Merge pull request #18539 * Implement lazy_load() for GitLoader	6 months ago
Christophe Bornet	9a6f7e213b	Merge pull request #18423 * Implement lazy_load() for BSHTMLLoader	6 months ago
Christophe Bornet	b3a0c44838	Merge pull request #18673 * Implement lazy_load() for PDFMinerPDFasHTMLLoader and PyMuPDFLoader	6 months ago
Christophe Bornet	68fc0cf909	Merge pull request #18674 * Implement lazy_load() for TextLoader	6 months ago
Christophe Bornet	5b92f962f1	Merge pull request #18671 * Implement lazy_load() for MastodonTootsLoader	6 months ago
Christophe Bornet	15b1770326	Merge pull request #18421 * Implement lazy_load() for AssemblyAIAudioTranscriptLoader	6 months ago
Christophe Bornet	bb284eebe4	Merge pull request #18436 * Implement lazy_load() for ConfluenceLoader	6 months ago
Christophe Bornet	691480f491	Merge pull request #18647 * Implement lazy_load() for UnstructuredBaseLoader	6 months ago
Christophe Bornet	52ac67c5d8	Merge pull request #18654 * Implement lazy_load() for ObsidianLoader	6 months ago
Christophe Bornet	b9c0cf9025	Merge pull request #18656 * Implement lazy_load() for PsychicLoader	6 months ago
Christophe Bornet	aa7ac57b67	community: Implement lazy_load() for TrelloLoader (#18658 ) Covered by `tests/unit_tests/document_loaders/test_trello.py`	6 months ago
Christophe Bornet	302985fea1	community: Implement lazy_load() for SlackDirectoryLoader (#18675 ) Integration tests: `tests/integration_tests/document_loaders/test_slack.py`	6 months ago
Christophe Bornet	ed36f9f604	community: Implement lazy_load() for WhatsAppChatLoader (#18677 ) Integration test: `tests/integration_tests/document_loaders/test_whatsapp_chat.py`	6 months ago
Christophe Bornet	f414f5cdb9	community[minor]: Implement lazy_load() for WikipediaLoader (#18680 ) Integration test: `tests/integration_tests/document_loaders/test_wikipedia.py`	6 months ago
Bagatur	4cbfeeb1c2	community[patch]: Release 0.0.26 (#18683 )	6 months ago
Christophe Bornet	1100f8de7a	community[minor]: Implement lazy_load() for ArxivLoader (#18664 ) Integration tests: `tests/integration_tests/utilities/test_arxiv.py` and `tests/integration_tests/document_loaders/test_arxiv.py`	6 months ago
Christophe Bornet	2d96803ddd	community[minor]: Implement lazy_load() for OutlookMessageLoader (#18668 ) Integration test: `tests/integration_tests/document_loaders/test_email.py`	6 months ago
Christophe Bornet	ae167fb5b2	community[minor]: Implement lazy_load() for SitemapLoader (#18667 ) Integration tests: `test_sitemap.py` and `test_docusaurus.py`	6 months ago
Christophe Bornet	623dfcc55c	community[minor]: Implement lazy_load() for FacebookChatLoader (#18669 ) Integration test: `tests/integration_tests/document_loaders/test_facebook_chat.py`	6 months ago
Christophe Bornet	20794bb889	community[minor]: Implement lazy_load() for GitbookLoader (#18670 ) Integration test: `tests/integration_tests/document_loaders/test_gitbook.py`	6 months ago
Liang Zhang	81985b31e6	community[patch]: Databricks SerDe uses cloudpickle instead of pickle (#18607 ) - Description: Databricks SerDe uses cloudpickle instead of pickle when serializing a user-defined function transform_input_fn since pickle does not support functions defined in `__main__`, and cloudpickle supports this. - Dependencies: cloudpickle>=2.0.0 Added a unit test.	6 months ago
Christophe Bornet	7d6de96186	community[patch]: Implement lazy_load() for CubeSemanticLoader (#18535 ) Covered by `test_cube_semantic.py`	6 months ago
Christophe Bornet	a6b5d45e31	community[patch]: Implement lazy_load() for EverNoteLoader (#18538 ) Covered by `test_evernote_loader.py`	6 months ago
Max Jakob	ee7a7954b9	elasticsearch: add `ElasticsearchRetriever` (#18587 ) Implement [Retriever](https://python.langchain.com/docs/modules/data_connection/retrievers/) interface for Elasticsearch. I opted to only expose the `body`, which gives you full flexibility, and none the other 68 arguments of the [search method](https://elasticsearch-py.readthedocs.io/en/v8.12.1/api/elasticsearch.html#elasticsearch.Elasticsearch.search). Added a user agent header for usage tracking in Elastic Cloud. --------- Co-authored-by: Erick Friis <erick@langchain.dev>	6 months ago
Jib	8bc347c5fc	mongodb[patch]: include LLM caches in toplevel library import (#18601 )	6 months ago
Sunchao Wang	dc81dba6cf	community[patch]: Improve amadeus tool and doc (#18509 ) Description: This pull request addresses two key improvements to the langchain repository: Fix for Crash in Flight Search Interface: Previously, the code would crash when encountering a failure scenario in the flight ticket search interface. This PR resolves this issue by implementing a fix to handle such scenarios gracefully. Now, the code handles failures in the flight search interface without crashing, ensuring smoother operation. Documentation Update for Amadeus Toolkit: Prior to this update, examples provided in the documentation for the Amadeus Toolkit were unable to run correctly due to outdated information. This PR includes an update to the documentation, ensuring that all examples can now be executed successfully. With this update, users can effectively utilize the Amadeus Toolkit with accurate and functioning examples. These changes aim to enhance the reliability and usability of the langchain repository by addressing issues related to error handling and ensuring that documentation remains up-to-date and actionable. Issue: https://github.com/langchain-ai/langchain/issues/17375 Twitter Handle: SingletonYxx	6 months ago
Christophe Bornet	f77f7dc3ec	community[patch]: Fix VectorStoreQATool (#18529 ) Fix #18460	6 months ago
Dounx	ad48f55357	community[minor]: add Yuque document loader (#17924 ) This pull request support loading documents from Yuque with Langchain. Yuque is a professional cloud-based knowledge base for team collaboration in documentation. Website: https://www.yuque.com OpenAPI: https://www.yuque.com/yuque/developer/openapi	6 months ago
Kazuki Maeda	60c5d964a8	community[minor]: use jq schema for content_key in json_loader (#18003 ) ### Description Changed the value specified for `content_key` in JSONLoader from a single key to a value based on jq schema. I created [similar PR](https://github.com/langchain-ai/langchain/pull/11255) before, but it has several conflicts because of the architectural change associated stable version release, so I re-create this PR to fit new architecture. ### Why For json data like the following, specify `.data[].attributes.message` for page_content and `.data[].attributes.id` or `.data[].attributes.attributes. tags`, etc., the `content_key` must also parse the json structure. <details> <summary>sample json data</summary> ```json { "data": [ { "attributes": { "message": "message1", "tags": [ "tag1" ] }, "id": "1" }, { "attributes": { "message": "message2", "tags": [ "tag2" ] }, "id": "2" } ] } ``` </details> <details> <summary>sample code</summary> ```python def metadata_func(record: dict, metadata: dict) -> dict: metadata["source"] = None metadata["id"] = record.get("id") metadata["tags"] = record["attributes"].get("tags") return metadata sample_file = "sample1.json" loader = JSONLoader( file_path=sample_file, jq_schema=".data[]", content_key=".attributes.message", ## content_key is parsable into jq schema is_content_key_jq_parsable=True, ## this is added parameter metadata_func=metadata_func ) data = loader.load() data ``` </details> ### Dependencies none ### Twitter handle [kzk_maeda](https://twitter.com/kzk_maeda)	6 months ago
Max Jakob	81e9ab6e3a	docs: Update elasticsearch README (#18497 ) Update Elasticsearch README with information on how to start a deployment. Also make some cosmetic changes to the [Elasticsearch docs](https://python.langchain.com/docs/integrations/vectorstores/elasticsearch). Follow up on https://github.com/langchain-ai/langchain/pull/17467	6 months ago
Hech	6a08134661	community[patch], langchain[minor]: Add retriever self_query and score_threshold in DingoDB (#18106 )	6 months ago
Mikhail Khludnev	d039dcb6ba	nvidia-trt[patch]: add TritonTensorRTLLM(verbose_client=False) (#16848 ) - Description: adding verbose flag to TritonTensorRTLLM, - Issue: nope, - Dependencies: not any, - Twitter handle:	6 months ago
Asaf Joseph Gardin	27441555d0	ai21[patch]: AI21 Labs Contextual Answers support (#18270 ) Description: Added support for AI21 Labs model - Contextual Answers Dependencies: ai21, ai21-tokenizer Twitter handle: https://github.com/AI21Labs --------- Co-authored-by: Asaf Gardin <asafg@ai21.com> Co-authored-by: Erick Friis <erick@langchain.dev>	6 months ago
Erick Friis	e169ee8863	anthropic[patch]: handle lists in function calling (#18609 )	6 months ago
Erick Friis	1831733c2e	anthropic[patch]: fix argument integration test (#18605 )	6 months ago
Yudhajit Sinha	4570b477b9	community[patch]: Invoke callback prior to yielding token (titan_takeoff) (#18560 ) ## PR title community[patch]: Invoke callback prior to yielding token ## PR message - Description: Invoke callback prior to yielding token in _stream_ method in llms/titan_takeoff. - Issue: #16913 - Dependencies: None	6 months ago
Tomaz Bratanic	ea51cdaede	Remove neo4j bloom labels from graph schema (#18564 ) Neo4j tools use particular node labels and relationship types to store metadata, but are irrelevant for text2cypher or graph generation, so we want to ignore them in the schema representation.	6 months ago
Erick Friis	e1924b3e93	core[patch]: deprecate hwchase17/langchain-hub, address path traversal (#18600 ) Deprecates the old langchain-hub repository. Does not deprecate the new https://smith.langchain.com/hub @PinkDraconian has correctly raised that in the event someone is loading unsanitized user input into the `try_load_from_hub` function, they have the ability to load files from other locations in github than the hwchase17/langchain-hub repository. This PR adds some more path checking to that function and deprecates the functionality in favor of the hub built into LangSmith.	6 months ago
Jib	fc35262356	langchain-mongodb: add unit tests for MongoDBChatMessageHistory (#18599 ) ## Description Adding in Unit Test variation for `MongoDBChatMessageHistory` package Follow-up to #18590 - [x] Add tests and docs: Unit test is what's being added - [x] Lint and test: Run `make format`, `make lint` and `make test` from the root of the package(s) you've modified. See contribution guidelines for more: https://python.langchain.com/docs/contributing/	6 months ago
Erick Friis	48e303ea10	airbyte[patch]: release 0.1.1, python 3.9 compat (#18597 )	6 months ago
Jib	9da1e0cf34	mongodb[patch]: Migrate MongoDBChatMessageHistory (#18590 ) ## Description Migrate the `MongoDBChatMessageHistory` to the managed `langchain-mongodb` partner-package ## Dependencies None ## Twitter handle @mongodb ## tests and docs - [x] Migrate existing integration test - [x ]~ Convert existing integration test to a unit test~ Creation is out of scope for this ticket - [x ] ~Considering delaying work until #17470 merges to leverage the `MockCollection` object. ~ - [x] Lint and test: Run `make format`, `make lint` and `make test` from the root of the package(s) you've modified. See contribution guidelines for more: https://python.langchain.com/docs/contributing/ --------- Co-authored-by: Erick Friis <erick@langchain.dev>	6 months ago
Jib	f92f7d2e03	mongodb[minor]: Add MongoDB LLM Cache (#17470 ) # Description - Description: Adding MongoDB LLM Caching Layer abstraction - Issue: N/A - Dependencies: None - Twitter handle: @mongodb Checklist: - [x] PR title: Please title your PR "package: description", where "package" is whichever of langchain, community, core, experimental, etc. is being modified. Use "docs: ..." for purely docs changes, "templates: ..." for template changes, "infra: ..." for CI changes. - Example: "community: add foobar LLM" - [x] PR Message (above) - [x] Pass lint and test: Run `make format`, `make lint` and `make test` from the root of the package(s) you've modified to check that you're passing lint and testing. See contribution guidelines for more information on how to write/run tests, lint, etc: https://python.langchain.com/docs/contributing/ - [ ] Add tests and docs: If you're adding a new integration, please include 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. It lives in `docs/docs/integrations` directory. Additional guidelines: - Make sure optional dependencies are imported within a function. - Please do not add dependencies to pyproject.toml files (even optional ones) unless they are required for unit tests. - Most PRs should not touch more than one package. - Changes should be backwards compatible. - If you are adding something to community, do not re-import it in langchain. If no one reviews your PR within a few days, please @-mention one of @baskaryan, @efriis, @eyurtsev, @hwchase17. --------- Co-authored-by: Jib <jib@byblack.us>	6 months ago
Tomaz Bratanic	353248838d	Add precedence for input params over env variables in neo4j integration (#18581 ) input parameters take precedence over env variables	6 months ago
Christophe Bornet	c8a171a154	community: Implement lazy_load() for GithubFileLoader (#18584 )	6 months ago
Leonid Kuligin	04d134df17	marked MatchingEngine as deprecated (#18585 ) Thank you for contributing to LangChain! - [ ] PR title: "community: deprecate vectorstores.MatchingEngine" - [ ] PR message: - Description: announced a deprecation since this integration has been moved to langchain_google_vertexai	6 months ago
Erick Friis	4ac2cb4adc	anthropic[minor]: add tool calling (#18554 )	6 months ago
Bagatur	5fc67ca2c7	langchain[patch]: Release 0.1.11 (#18558 )	6 months ago
Erick Friis	68c1878380	anthropic[patch]: model type string (#18510 )	6 months ago
Erick Friis	25c7d52140	anthropic[patch]: multimodal (#18517 ) - anthropic[minor]: claude 3 - x - x --------- Co-authored-by: William FH <13333726+hinthornw@users.noreply.github.com>	6 months ago
Erick Friis	343438e872	community[patch]: deprecate community fireworks (#18544 )	6 months ago
William FH	ca1d42785d	Evals wording (#18542 )	6 months ago
Bagatur	dd07eddf24	core[patch]: Release 0.1.29 (#18530 )	6 months ago
William FH	30ccc009e6	[Evals] Support list examples by dataset version tag (#18534 ) previously only supported by timestamp	6 months ago
aditya thomas	5c387a173f	docs: update to docstrings of ChatAnthropic class (#18493 ) Description: Update docstrings of ChatAnthropic class Issue: Change to ChatAnthropic from ChatAnthropicMessages Dependencies: None Lint and test: `make format`, `make lint` and `make test` passed	6 months ago
Erick Friis	24f9c700f2	anthropic[minor]: claude 3 (#18508 )	6 months ago
William FH	1eec67e8fe	Evaluate on Version (#18471 )	6 months ago
Harrison Chase	73d653324f	[Evals] Session-level feedback (#18463 ) Co-authored-by: William Fu-Hinthorn <13333726+hinthornw@users.noreply.github.com>	6 months ago
Scott Nath	b051bba1a9	community: Add you.com tool, add async to retriever, add async testing, add You tool doc (#18032 ) - Description: finishes adding the you.com functionality including: - add async functions to utility and retriever - add the You.com Tool - add async testing for utility, retriever, and tool - add a tool integration notebook page - Dependencies: any dependencies required for this change - Twitter handle: @scottnath	6 months ago
mackong	b89d9fc177	langchain[patch]: add tools renderer for various non-openai agents (#18307 ) - Description: add tools_renderer for various non-openai agents, make tools can be render in different ways for your LLM. - Issue: N/A - Dependencies: N/A --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	6 months ago
William De Vena	a63cee04ac	nvidia-trt[patch]: Invoke callback prior to yielding token (#18446 ) ## PR title nvidia-trt[patch]: Invoke callback prior to yielding ## PR message - Description: Invoke on_llm_new_token callback prior to yielding token in _stream method. - Issue: https://github.com/langchain-ai/langchain/issues/16913 - Dependencies: None	6 months ago
William De Vena	275877980e	community[patch]: Invoke callback prior to yielding token (#18447 ) ## PR title community[patch]: Invoke callback prior to yielding token ## PR message Description: Invoke callback prior to yielding token in _stream method in llms/vertexai. Issue: https://github.com/langchain-ai/langchain/issues/16913 Dependencies: None	6 months ago
William De Vena	67375e96e0	community[patch]: Invoke callback prior to yielding token (#18448 ) ## PR title community[patch]: Invoke callback prior to yielding token ## PR message - Description: Invoke callback prior to yielding token in _stream method in llms/tongyi. - Issue: https://github.com/langchain-ai/langchain/issues/16913 - Dependencies: None	6 months ago
William De Vena	2087cbae64	community[patch]: Invoke callback prior to yielding token (#18449 ) ## PR title community[patch]: Invoke callback prior to yielding token ## PR message - Description: Invoke callback prior to yielding token in _stream method in chat_models/perplexity. - Issue: https://github.com/langchain-ai/langchain/issues/16913 - Dependencies: None	6 months ago
William De Vena	eb04d0d3e2	community[patch]: Invoke callback prior to yielding token (#18452 ) ## PR title community[patch]: Invoke callback prior to yielding token ## PR message - Description: Invoke callback prior to yielding token in _stream and _astream methods in llms/anthropic. - Issue: https://github.com/langchain-ai/langchain/issues/16913 - Dependencies: None	6 months ago
William De Vena	371bec79bc	community[patch]: Invoke callback prior to yielding token (#18454 ) ## PR title community[patch]: Invoke callback prior to yielding token ## PR message - Description: Invoke callback prior to yielding token in _stream and _astream methods in llms/baidu_qianfan_endpoint. - Issue: https://github.com/langchain-ai/langchain/issues/16913 - Dependencies: None	6 months ago
Aayush Kataria	7c2f3f6f95	community[minor]: Adding Azure Cosmos Mongo vCore Vector DB Cache (#16856 ) Description: This pull request introduces several enhancements for Azure Cosmos Vector DB, primarily focused on improving caching and search capabilities using Azure Cosmos MongoDB vCore Vector DB. Here's a summary of the changes: - AzureCosmosDBSemanticCache: Added a new cache implementation called AzureCosmosDBSemanticCache, which utilizes Azure Cosmos MongoDB vCore Vector DB for efficient caching of semantic data. Added comprehensive test cases for AzureCosmosDBSemanticCache to ensure its correctness and robustness. These tests cover various scenarios and edge cases to validate the cache's behavior. - HNSW Vector Search: Added HNSW vector search functionality in the CosmosDB Vector Search module. This enhancement enables more efficient and accurate vector searches by utilizing the HNSW (Hierarchical Navigable Small World) algorithm. Added corresponding test cases to validate the HNSW vector search functionality in both AzureCosmosDBSemanticCache and AzureCosmosDBVectorSearch. These tests ensure the correctness and performance of the HNSW search algorithm. - LLM Caching Notebook - The notebook now includes a comprehensive example showcasing the usage of the AzureCosmosDBSemanticCache. This example highlights how the cache can be employed to efficiently store and retrieve semantic data. Additionally, the example provides default values for all parameters used within the AzureCosmosDBSemanticCache, ensuring clarity and ease of understanding for users who are new to the cache implementation. @hwchase17,@baskaryan, @eyurtsev,	6 months ago
Erick Friis	f96dd57501	langchain[patch]: release 0.1.10 (#18410 )	6 months ago
Erick Friis	1fd1ac8e95	community[patch]: release 0.0.25 (#18408 )	6 months ago
Sourav Pradhan	50abeb7ed9	community[patch]: fix Chroma add_images (#17964 ) ### Description Fixed a small bug in chroma.py add_images(), previously whenever we are not passing metadata the documents is containing the base64 of the uris passed, but when we are passing the metadata the documents is containing normal string uris which should not be the case. ### Issue In add_images() method when we are calling upsert() we have to use "b64_texts" instead of normal string "uris". ### Twitter handle https://twitter.com/whitepegasus01	6 months ago
Kate Silverstein	b7c71e2e07	community[minor]: llamafile embeddings support (#17976 ) * Description: adds `LlamafileEmbeddings` class implementation for generating embeddings using [llamafile](https://github.com/Mozilla-Ocho/llamafile)-based models. Includes related unit tests and notebook showing example usage. * Issue: N/A * Dependencies: N/A	7 months ago
Mateusz Szewczyk	9298a0b941	langchain_ibm[patch] update docstring, dependencies, tests (#18386 ) - Description: Update docstring, dependencies, tests, README - Dependencies: [ibm-watsonx-ai](https://pypi.org/project/ibm-watsonx-ai/), - Tag maintainer: : Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally -> ✅ Please make sure integration_tests passing locally -> ✅ --------- Co-authored-by: Erick Friis <erick@langchain.dev>	7 months ago
Jib	c2b1abe91b	mongodb[patch]: Set delete_many only if count_documents is not 0 (#18402 ) - [x] PR message: *Delete this entire checklist* and replace with - Description: Remove the assert statement on the `count_documents` in setup_class. It should just delete if there are documents present - Issue: the issue # Crashes on class setup - Dependencies: None - Twitter handle: @mongodb - [x] Add tests and docs: If you're adding a new integration, please include 1. N/A - [ ] Lint and test: Run `make format`, `make lint` and `make test` from the root of the package(s) you've modified. See contribution guidelines for more: https://python.langchain.com/docs/contributing/ Additional guidelines: - Make sure optional dependencies are imported within a function. - Please do not add dependencies to pyproject.toml files (even optional ones) unless they are required for unit tests. - Most PRs should not touch more than one package. - Changes should be backwards compatible. - If you are adding something to community, do not re-import it in langchain. If no one reviews your PR within a few days, please @-mention one of baskaryan, efriis, eyurtsev, hwchase17. Co-authored-by: Jib <jib@byblack.us>	7 months ago
Tomaz Bratanic	f6bfb969ba	community[patch]: Add an option for indexed generic label when import neo4j graph documents (#18122 ) Current implementation doesn't have an indexed property that would optimize the import. I have added a `baseEntityLabel` parameter that allows you to add a secondary node label, which has an indexed id `property`. By default, the behaviour is identical to previous version. Since multi-labeled nodes are terrible for text2cypher, I removed the secondary label from schema representation object and string, which is used in text2cypher.	7 months ago
Arun Sathiya	4adac20d7b	community[patch]: Make cohere_api_key a SecretStr (#12188 ) This PR makes `cohere_api_key` in `llms/cohere` a SecretStr, so that the API Key is not leaked when `Cohere.cohere_api_key` is represented as a string. --------- Signed-off-by: Arun <arun@arun.blog> Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com>	7 months ago
Petteri Johansson	6c1989d292	community[minor], langchain[minor], docs: Gremlin Graph Store and QA Chain (#17683 ) - Description: New feature: Gremlin graph-store and QA chain (including docs). Compatible with Azure CosmosDB. - Dependencies: no changes	7 months ago
Ather Fawaz	a5ccf5d33c	community[minor]: Add support for Perplexity chat model(#17024 ) - Description: This PR adds support for [Perplexity AI APIs](https://blog.perplexity.ai/blog/introducing-pplx-api). - Issues: None - Dependencies: None - Twitter handle: [@atherfawaz](https://twitter.com/AtherFawaz) --------- Co-authored-by: Bagatur <baskaryan@gmail.com> Co-authored-by: Erick Friis <erick@langchain.dev>	7 months ago
Rodrigo Nogueira	3438d2cbcc	community[minor]: add maritalk chat (#17675 ) Description: Adds the MariTalk chat that is based on a LLM specially trained for Portuguese. Twitter handle: @MaritacaAI	7 months ago
sarahberenji	08fa38d56d	community[patch]: the syntax error for Redis generated query (#17717 ) To fix the reported error: https://github.com/langchain-ai/langchain/discussions/17397	7 months ago
certified-dodo	43e3244573	community[patch]: Fix MongoDBAtlasVectorSearch max_marginal_relevance_search (#17971 ) Description: * `self._embedding_key` is accessed after deletion, breaking `max_marginal_relevance_search` search * Introduced in: `e135e5257c` * Updated but still persists in: `ce22e10c4b` Issue: https://github.com/langchain-ai/langchain/issues/17963 Co-authored-by: Bagatur <baskaryan@gmail.com>	7 months ago
Nikita Titov	9f2ab37162	community[patch]: don't try to parse json in case of errored response (#18317 ) Related issue: #13896. In case Ollama is behind a proxy, proxy error responses cannot be viewed. You aren't even able to check response code. For example, if your Ollama has basic access authentication and it's not passed, `JSONDecodeError` will overwrite the truth response error. <details> <summary><b>Log now:</b></summary> ``` { "name": "JSONDecodeError", "message": "Expecting value: line 1 column 1 (char 0)", "stack": "--------------------------------------------------------------------------- JSONDecodeError Traceback (most recent call last) File /opt/miniforge3/envs/.gpt/lib/python3.10/site-packages/requests/models.py:971, in Response.json(self, kwargs) 970 try: --> 971 return complexjson.loads(self.text, kwargs) 972 except JSONDecodeError as e: 973 # Catch JSON-related errors and raise as requests.JSONDecodeError 974 # This aliases json.JSONDecodeError and simplejson.JSONDecodeError File /opt/miniforge3/envs/.gpt/lib/python3.10/json/__init__.py:346, in loads(s, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, kw) 343 if (cls is None and object_hook is None and 344 parse_int is None and parse_float is None and 345 parse_constant is None and object_pairs_hook is None and not kw): --> 346 return _default_decoder.decode(s) 347 if cls is None: File /opt/miniforge3/envs/.gpt/lib/python3.10/json/decoder.py:337, in JSONDecoder.decode(self, s, _w) 333 \"\"\"Return the Python representation of ``s`` (a ``str`` instance 334 containing a JSON document). 335 336 \"\"\" --> 337 obj, end = self.raw_decode(s, idx=_w(s, 0).end()) 338 end = _w(s, end).end() File /opt/miniforge3/envs/.gpt/lib/python3.10/json/decoder.py:355, in JSONDecoder.raw_decode(self, s, idx) 354 except StopIteration as err: --> 355 raise JSONDecodeError(\"Expecting value\", s, err.value) from None 356 return obj, end JSONDecodeError: Expecting value: line 1 column 1 (char 0) During handling of the above exception, another exception occurred: JSONDecodeError Traceback (most recent call last) Cell In[3], line 1 ----> 1 print(translate_func().invoke('text')) File /opt/miniforge3/envs/.gpt/lib/python3.10/site-packages/langchain_core/runnables/base.py:2053, in RunnableSequence.invoke(self, input, config) 2051 try: 2052 for i, step in enumerate(self.steps): -> 2053 input = step.invoke( 2054 input, 2055 # mark each step as a child run 2056 patch_config( 2057 config, callbacks=run_manager.get_child(f\"seq:step:{i+1}\") 2058 ), 2059 ) 2060 # finish the root run 2061 except BaseException as e: File /opt/miniforge3/envs/.gpt/lib/python3.10/site-packages/langchain_core/language_models/chat_models.py:165, in BaseChatModel.invoke(self, input, config, stop, kwargs) 154 def invoke( 155 self, 156 input: LanguageModelInput, (...) 160 kwargs: Any, 161 ) -> BaseMessage: 162 config = ensure_config(config) 163 return cast( 164 ChatGeneration, --> 165 self.generate_prompt( 166 [self._convert_input(input)], 167 stop=stop, 168 callbacks=config.get(\"callbacks\"), 169 tags=config.get(\"tags\"), 170 metadata=config.get(\"metadata\"), 171 run_name=config.get(\"run_name\"), 172 kwargs, 173 ).generations[0][0], 174 ).message File /opt/miniforge3/envs/.gpt/lib/python3.10/site-packages/langchain_core/language_models/chat_models.py:543, in BaseChatModel.generate_prompt(self, prompts, stop, callbacks, kwargs) 535 def generate_prompt( 536 self, 537 prompts: List[PromptValue], (...) 540 kwargs: Any, 541 ) -> LLMResult: 542 prompt_messages = [p.to_messages() for p in prompts] --> 543 return self.generate(prompt_messages, stop=stop, callbacks=callbacks, kwargs) File /opt/miniforge3/envs/.gpt/lib/python3.10/site-packages/langchain_core/language_models/chat_models.py:407, in BaseChatModel.generate(self, messages, stop, callbacks, tags, metadata, run_name, kwargs) 405 if run_managers: 406 run_managers[i].on_llm_error(e, response=LLMResult(generations=[])) --> 407 raise e 408 flattened_outputs = [ 409 LLMResult(generations=[res.generations], llm_output=res.llm_output) 410 for res in results 411 ] 412 llm_output = self._combine_llm_outputs([res.llm_output for res in results]) File /opt/miniforge3/envs/.gpt/lib/python3.10/site-packages/langchain_core/language_models/chat_models.py:397, in BaseChatModel.generate(self, messages, stop, callbacks, tags, metadata, run_name, kwargs) 394 for i, m in enumerate(messages): 395 try: 396 results.append( --> 397 self._generate_with_cache( 398 m, 399 stop=stop, 400 run_manager=run_managers[i] if run_managers else None, 401 kwargs, 402 ) 403 ) 404 except BaseException as e: 405 if run_managers: File /opt/miniforge3/envs/.gpt/lib/python3.10/site-packages/langchain_core/language_models/chat_models.py:576, in BaseChatModel._generate_with_cache(self, messages, stop, run_manager, kwargs) 572 raise ValueError( 573 \"Asked to cache, but no cache found at `langchain.cache`.\" 574 ) 575 if new_arg_supported: --> 576 return self._generate( 577 messages, stop=stop, run_manager=run_manager, kwargs 578 ) 579 else: 580 return self._generate(messages, stop=stop, kwargs) File /opt/miniforge3/envs/.gpt/lib/python3.10/site-packages/langchain_community/chat_models/ollama.py:250, in ChatOllama._generate(self, messages, stop, run_manager, kwargs) 226 def _generate( 227 self, 228 messages: List[BaseMessage], (...) 231 kwargs: Any, 232 ) -> ChatResult: 233 \"\"\"Call out to Ollama's generate endpoint. 234 235 Args: (...) 247 ]) 248 \"\"\" --> 250 final_chunk = self._chat_stream_with_aggregation( 251 messages, 252 stop=stop, 253 run_manager=run_manager, 254 verbose=self.verbose, 255 kwargs, 256 ) 257 chat_generation = ChatGeneration( 258 message=AIMessage(content=final_chunk.text), 259 generation_info=final_chunk.generation_info, 260 ) 261 return ChatResult(generations=[chat_generation]) File /opt/miniforge3/envs/.gpt/lib/python3.10/site-packages/langchain_community/chat_models/ollama.py:183, in ChatOllama._chat_stream_with_aggregation(self, messages, stop, run_manager, verbose, kwargs) 174 def _chat_stream_with_aggregation( 175 self, 176 messages: List[BaseMessage], (...) 180 kwargs: Any, 181 ) -> ChatGenerationChunk: 182 final_chunk: Optional[ChatGenerationChunk] = None --> 183 for stream_resp in self._create_chat_stream(messages, stop, kwargs): 184 if stream_resp: 185 chunk = _chat_stream_response_to_chat_generation_chunk(stream_resp) File /opt/miniforge3/envs/.gpt/lib/python3.10/site-packages/langchain_community/chat_models/ollama.py:156, in ChatOllama._create_chat_stream(self, messages, stop, kwargs) 147 def _create_chat_stream( 148 self, 149 messages: List[BaseMessage], 150 stop: Optional[List[str]] = None, 151 kwargs: Any, 152 ) -> Iterator[str]: 153 payload = { 154 \"messages\": self._convert_messages_to_ollama_messages(messages), 155 } --> 156 yield from self._create_stream( 157 payload=payload, stop=stop, api_url=f\"{self.base_url}/api/chat/\", kwargs 158 ) File /opt/miniforge3/envs/.gpt/lib/python3.10/site-packages/langchain_community/llms/ollama.py:234, in _OllamaCommon._create_stream(self, api_url, payload, stop, kwargs) 228 raise OllamaEndpointNotFoundError( 229 \"Ollama call failed with status code 404. \" 230 \"Maybe your model is not found \" 231 f\"and you should pull the model with `ollama pull {self.model}`.\" 232 ) 233 else: --> 234 optional_detail = response.json().get(\"error\") 235 raise ValueError( 236 f\"Ollama call failed with status code {response.status_code}.\" 237 f\" Details: {optional_detail}\" 238 ) 239 return response.iter_lines(decode_unicode=True) File /opt/miniforge3/envs/.gpt/lib/python3.10/site-packages/requests/models.py:975, in Response.json(self, kwargs) 971 return complexjson.loads(self.text, kwargs) 972 except JSONDecodeError as e: 973 # Catch JSON-related errors and raise as requests.JSONDecodeError 974 # This aliases json.JSONDecodeError and simplejson.JSONDecodeError --> 975 raise RequestsJSONDecodeError(e.msg, e.doc, e.pos) JSONDecodeError: Expecting value: line 1 column 1 (char 0)" } ``` </details> <details> <summary><b>Log after a fix:</b></summary> ``` { "name": "ValueError", "message": "Ollama call failed with status code 401. Details: <html>\r <head><title>401 Authorization Required</title></head>\r <body>\r <center><h1>401 Authorization Required</h1></center>\r <hr><center>nginx/1.18.0 (Ubuntu)</center>\r </body>\r </html>\r ", "stack": "--------------------------------------------------------------------------- ValueError Traceback (most recent call last) Cell In[2], line 1 ----> 1 print(translate_func().invoke('text')) File /opt/miniforge3/envs/.gpt/lib/python3.10/site-packages/langchain_core/runnables/base.py:2053, in RunnableSequence.invoke(self, input, config) 2051 try: 2052 for i, step in enumerate(self.steps): -> 2053 input = step.invoke( 2054 input, 2055 # mark each step as a child run 2056 patch_config( 2057 config, callbacks=run_manager.get_child(f\"seq:step:{i+1}\") 2058 ), 2059 ) 2060 # finish the root run 2061 except BaseException as e: File /opt/miniforge3/envs/.gpt/lib/python3.10/site-packages/langchain_core/language_models/chat_models.py:165, in BaseChatModel.invoke(self, input, config, stop, kwargs) 154 def invoke( 155 self, 156 input: LanguageModelInput, (...) 160 kwargs: Any, 161 ) -> BaseMessage: 162 config = ensure_config(config) 163 return cast( 164 ChatGeneration, --> 165 self.generate_prompt( 166 [self._convert_input(input)], 167 stop=stop, 168 callbacks=config.get(\"callbacks\"), 169 tags=config.get(\"tags\"), 170 metadata=config.get(\"metadata\"), 171 run_name=config.get(\"run_name\"), 172 kwargs, 173 ).generations[0][0], 174 ).message File /opt/miniforge3/envs/.gpt/lib/python3.10/site-packages/langchain_core/language_models/chat_models.py:543, in BaseChatModel.generate_prompt(self, prompts, stop, callbacks, kwargs) 535 def generate_prompt( 536 self, 537 prompts: List[PromptValue], (...) 540 kwargs: Any, 541 ) -> LLMResult: 542 prompt_messages = [p.to_messages() for p in prompts] --> 543 return self.generate(prompt_messages, stop=stop, callbacks=callbacks, kwargs) File /opt/miniforge3/envs/.gpt/lib/python3.10/site-packages/langchain_core/language_models/chat_models.py:407, in BaseChatModel.generate(self, messages, stop, callbacks, tags, metadata, run_name, kwargs) 405 if run_managers: 406 run_managers[i].on_llm_error(e, response=LLMResult(generations=[])) --> 407 raise e 408 flattened_outputs = [ 409 LLMResult(generations=[res.generations], llm_output=res.llm_output) 410 for res in results 411 ] 412 llm_output = self._combine_llm_outputs([res.llm_output for res in results]) File /opt/miniforge3/envs/.gpt/lib/python3.10/site-packages/langchain_core/language_models/chat_models.py:397, in BaseChatModel.generate(self, messages, stop, callbacks, tags, metadata, run_name, kwargs) 394 for i, m in enumerate(messages): 395 try: 396 results.append( --> 397 self._generate_with_cache( 398 m, 399 stop=stop, 400 run_manager=run_managers[i] if run_managers else None, 401 kwargs, 402 ) 403 ) 404 except BaseException as e: 405 if run_managers: File /opt/miniforge3/envs/.gpt/lib/python3.10/site-packages/langchain_core/language_models/chat_models.py:576, in BaseChatModel._generate_with_cache(self, messages, stop, run_manager, kwargs) 572 raise ValueError( 573 \"Asked to cache, but no cache found at `langchain.cache`.\" 574 ) 575 if new_arg_supported: --> 576 return self._generate( 577 messages, stop=stop, run_manager=run_manager, kwargs 578 ) 579 else: 580 return self._generate(messages, stop=stop, kwargs) File /opt/miniforge3/envs/.gpt/lib/python3.10/site-packages/langchain_community/chat_models/ollama.py:250, in ChatOllama._generate(self, messages, stop, run_manager, kwargs) 226 def _generate( 227 self, 228 messages: List[BaseMessage], (...) 231 kwargs: Any, 232 ) -> ChatResult: 233 \"\"\"Call out to Ollama's generate endpoint. 234 235 Args: (...) 247 ]) 248 \"\"\" --> 250 final_chunk = self._chat_stream_with_aggregation( 251 messages, 252 stop=stop, 253 run_manager=run_manager, 254 verbose=self.verbose, 255 kwargs, 256 ) 257 chat_generation = ChatGeneration( 258 message=AIMessage(content=final_chunk.text), 259 generation_info=final_chunk.generation_info, 260 ) 261 return ChatResult(generations=[chat_generation]) File /storage/gpt-project/Repos/repo_nikita/gpt_lib/langchain/ollama.py:328, in ChatOllamaCustom._chat_stream_with_aggregation(self, messages, stop, run_manager, verbose, kwargs) 319 def _chat_stream_with_aggregation( 320 self, 321 messages: List[BaseMessage], (...) 325 kwargs: Any, 326 ) -> ChatGenerationChunk: 327 final_chunk: Optional[ChatGenerationChunk] = None --> 328 for stream_resp in self._create_chat_stream(messages, stop, kwargs): 329 if stream_resp: 330 chunk = _chat_stream_response_to_chat_generation_chunk(stream_resp) File /storage/gpt-project/Repos/repo_nikita/gpt_lib/langchain/ollama.py:301, in ChatOllamaCustom._create_chat_stream(self, messages, stop, kwargs) 292 def _create_chat_stream( 293 self, 294 messages: List[BaseMessage], 295 stop: Optional[List[str]] = None, 296 kwargs: Any, 297 ) -> Iterator[str]: 298 payload = { 299 \"messages\": self._convert_messages_to_ollama_messages(messages), 300 } --> 301 yield from self._create_stream( 302 payload=payload, stop=stop, api_url=f\"{self.base_url}/api/chat\", kwargs 303 ) File /storage/gpt-project/Repos/repo_nikita/gpt_lib/langchain/ollama.py:134, in _OllamaCommonCustom._create_stream(self, api_url, payload, stop, **kwargs) 132 else: 133 optional_detail = response.text --> 134 raise ValueError( 135 f\"Ollama call failed with status code {response.status_code}.\" 136 f\" Details: {optional_detail}\" 137 ) 138 return response.iter_lines(decode_unicode=True) ValueError: Ollama call failed with status code 401. Details: <html>\r <head><title>401 Authorization Required</title></head>\r <body>\r <center><h1>401 Authorization Required</h1></center>\r <hr><center>nginx/1.18.0 (Ubuntu)</center>\r </body>\r </html>\r " } ``` </details> The same is true for timeout errors or when you simply mistyped in `base_url` arg and get response from some other service, for instance. Real Ollama errors are still clearly readable: ``` ValueError: Ollama call failed with status code 400. Details: {"error":"invalid options: unknown_option"} ``` --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	7 months ago
Yudhajit Sinha	e2b901c35b	community[patch]: chat message histrory mypy fix (#18250 ) Description: Fixed type: ignore's for mypy for chat_message_histories(streamlit) Adresses #17048 Planning to add more based on reviews	7 months ago
Gabriel Altay	b9416dc96a	docs: update pinecone README to use PineconeVectorStore (#18170 )	7 months ago
Hemslo Wang	58a2abf089	community[patch]: fix RecursiveUrlLoader metadata_extractor return type (#18193 ) Description: Fix `metadata_extractor` type for `RecursiveUrlLoader`, the default `_metadata_extractor` returns `dict` instead of `str`. Issue: N/A Dependencies: N/A Twitter handle: N/A Signed-off-by: Hemslo Wang <hemslo.wang@gmail.com>	7 months ago
Maxime Perrin	98380cff9b	community[patch]: removing "response_mode" parameter in llama_index retriever (#18180 ) - Description: Removing this line ```python response = index.query(query, response_mode="no_text", self.query_kwargs) ``` to ```python response = index.query(query, self.query_kwargs) ``` Since llama index query does not support response_mode anymore : ``` \| TypeError: BaseQueryEngine.query() got an unexpected keyword argument 'response_mode'```` - Twitter handle: @maximeperrin_ --------- Co-authored-by: Maxime Perrin <mperrin@doing.fr>	7 months ago
Christophe Bornet	177f51c7bd	community: Use default load() implementation in doc loaders (#18385 ) Following https://github.com/langchain-ai/langchain/pull/18289	7 months ago
William De Vena	42341bc787	infra: fake model invoke callback prior to yielding token (#18286 ) ## PR title core[patch]: Invoke callback prior to yielding ## PR message Description: Invoke on_llm_new_token callback prior to yielding token in _stream and _astream methods. Issue: https://github.com/langchain-ai/langchain/issues/16913 Dependencies: None Twitter handle: None	7 months ago
mwmajewsk	e192f6b6eb	community[patch]: fix, better error message in deeplake vectoriser (#18397 ) If the document loader recieves Pathlib path instead of str, it reads the file correctly, but the problem begins when the document is added to Deeplake. This problem arises from casting the path to str in the metadata. ```python deeplake = True fname = Path('./lorem_ipsum.txt') loader = TextLoader(fname, encoding="utf-8") docs = loader.load_and_split() text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=100) chunks= text_splitter.split_documents(docs) if deeplake: db = DeepLake(dataset_path=ds_path, embedding=embeddings, token=activeloop_token) db.add_documents(chunks) else: db = Chroma.from_documents(docs, embeddings) ``` So using this snippet of code the error message for deeplake looks like this: ``` [part of error message omitted] Traceback (most recent call last): File "/home/mwm/repositories/sources/fixing_langchain/main.py", line 53, in <module> db.add_documents(chunks) File "/home/mwm/repositories/sources/langchain/libs/core/langchain_core/vectorstores.py", line 139, in add_documents return self.add_texts(texts, metadatas, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/mwm/repositories/sources/langchain/libs/community/langchain_community/vectorstores/deeplake.py", line 258, in add_texts return self.vectorstore.add( ^^^^^^^^^^^^^^^^^^^^^ File "/home/mwm/anaconda3/envs/langchain/lib/python3.11/site-packages/deeplake/core/vectorstore/deeplake_vectorstore.py", line 226, in add return self.dataset_handler.add( ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/mwm/anaconda3/envs/langchain/lib/python3.11/site-packages/deeplake/core/vectorstore/dataset_handlers/client_side_dataset_handler.py", line 139, in add dataset_utils.extend_or_ingest_dataset( File "/home/mwm/anaconda3/envs/langchain/lib/python3.11/site-packages/deeplake/core/vectorstore/vector_search/dataset/dataset.py", line 544, in extend_or_ingest_dataset extend( File "/home/mwm/anaconda3/envs/langchain/lib/python3.11/site-packages/deeplake/core/vectorstore/vector_search/dataset/dataset.py", line 505, in extend dataset.extend(batched_processed_tensors, progressbar=False) File "/home/mwm/anaconda3/envs/langchain/lib/python3.11/site-packages/deeplake/core/dataset/dataset.py", line 3247, in extend raise SampleExtendError(str(e)) from e.__cause__ deeplake.util.exceptions.SampleExtendError: Failed to append a sample to the tensor 'metadata'. See more details in the traceback. If you wish to skip the samples that cause errors, please specify `ignore_errors=True`. ``` Which is does not explain the error well enough. The same error for chroma looks like this ``` During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/mwm/repositories/sources/fixing_langchain/main.py", line 56, in <module> db = Chroma.from_documents(docs, embeddings) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/mwm/repositories/sources/langchain/libs/community/langchain_community/vectorstores/chroma.py", line 778, in from_documents return cls.from_texts( ^^^^^^^^^^^^^^^ File "/home/mwm/repositories/sources/langchain/libs/community/langchain_community/vectorstores/chroma.py", line 736, in from_texts chroma_collection.add_texts( File "/home/mwm/repositories/sources/langchain/libs/community/langchain_community/vectorstores/chroma.py", line 309, in add_texts raise ValueError(e.args[0] + "\n\n" + msg) ValueError: Expected metadata value to be a str, int, float or bool, got lorem_ipsum.txt which is a <class 'pathlib.PosixPath'> Try filtering complex metadata from the document using langchain_community.vectorstores.utils.filter_complex_metadata. ``` Which is way more user friendly, so I just added information about possible mismatch of the type in the error message, the same way it is covered in chroma https://github.com/langchain-ai/langchain/blob/master/libs/community/langchain_community/vectorstores/chroma.py#L224	7 months ago
Daniel Chico	7d962278f6	community[patch]: type ignore fixes (#18395 ) Related to #17048	7 months ago
Christophe Bornet	69be82c86d	community[patch]: Implement lazy_load() for CSVLoader (#18391 ) Covered by `test_csv_loader.py`	7 months ago
Bagatur	c54d6eb5da	fireworks[patch]: support "any" tool_choice (#18343 ) per https://readme.fireworks.ai/docs/function-calling	7 months ago
Erick Friis	6afb135baa	astradb: move to langchain-datastax repo (#18354 )	7 months ago
Guangdong Liu	760a16ff32	community[patch]: Fix ChatModel for sparkllm Bug. (#18375 ) PR message: *Delete this entire checklist* and replace with - Description: fix sparkllm paramer error - Issue: close #18370 - Dependencies: change `IFLYTEK_SPARK_APP_URL` to `IFLYTEK_SPARK_API_URL` - Twitter handle: No	7 months ago
Yujie Qian	cbb65741a7	community[patch]: Voyage AI updates default model and batch size (#17655 ) - Description: update the default model and batch size in VoyageEmbeddings - Issue: N/A - Dependencies: N/A - Twitter handle: N/A --------- Co-authored-by: fodizoltan <zoltan@conway.expert>	7 months ago
Shengsheng Huang	ae471a7dcb	community[minor]: add BigDL-LLM integrations (#17953 ) - Description: [`bigdl-llm`](https://github.com/intel-analytics/BigDL) is a library for running LLM on Intel XPU (from Laptop to GPU to Cloud) using INT4/FP4/INT8/FP8 with very low latency (for any PyTorch model). This PR adds bigdl-llm integrations to langchain. - Issue: NA - Dependencies: `bigdl-llm` library - Contribution maintainer: @shane-huang Examples added: - docs/docs/integrations/llms/bigdl.ipynb	7 months ago
Ethan Yang	f61cb8d407	community[minor]: Add openvino backend support (#11591 ) - Description: add openvino backend support by HuggingFace Optimum Intel, - Dependencies: “optimum[openvino]”, --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	7 months ago
Leonid Ganeline	a89f007947	docs: `runnable` module description (#17966 ) Added a module description. Added `batch` description.	7 months ago
RadhikaBansal97	8bafd2df5e	community[patch]: Change github endpoint in GithubLoader (#17622 ) Description- - Changed the GitHub endpoint as existing was not working and giving 404 not found error - Also the existing function was failing if file_filter is not passed as the tree api return all paths including directory as well, and when get_file_content was iterating over these path, the function was failing for directory as the api was returning list of files inside the directory, so added a condition to ignore the paths if it a directory - Fixes this issue - https://github.com/langchain-ai/langchain/issues/17453 Co-authored-by: Radhika Bansal <Radhika.Bansal@veritas.com>	7 months ago
Yufei (Benny) Chen	2b93206f02	fireworks[patch]: Fix fireworks async stream (#18372 ) - Description: Fix the async stream issue with Fireworks - Dependencies: fireworks >= 0.13.0 ``` tests/integration_tests/test_chat_models.py .......... [ 45%] tests/integration_tests/test_compile.py . [ 50%] tests/integration_tests/test_embeddings.py .. [ 59%] tests/integration_tests/test_llms.py ......... [100%] ``` ``` tests/unit_tests/test_embeddings.py . [ 16%] tests/unit_tests/test_imports.py . [ 33%] tests/unit_tests/test_llms.py .... [100%] ``` --------- Co-authored-by: Erick Friis <erick@langchain.dev>	7 months ago
William FH	1deb8cadd5	Add dataset version info (#18299 )	7 months ago
Anush	9d663f31fa	community[patch]: FastEmbed to latest (#18040 ) ## Description Updates the `langchain_community.embeddings.fastembed` provider as per the recent updates to [`FastEmbed`](https://github.com/qdrant/fastembed) library.	7 months ago
Erick Friis	3c8a115e21	fireworks[patch]: remove custom async and stream implementations (#18363 )	7 months ago
Bagatur	f220af3dce	docs: text splitters readme (#18359 )	7 months ago
Bagatur	0d7fb5f60a	langchain[patch]: langchain-text-splitters dep (#18357 )	7 months ago
Eugene Yurtsev	51b661cfe8	community[patch]: BaseLoader load method should just delegate to lazy_load (#18289 ) load() should just reference lazy_load()	7 months ago
Bagatur	5efb5c099f	text-splitters[minor], langchain[minor], community[patch], templates, docs: langchain-text-splitters 0.0.1 (#18346 )	7 months ago
Nuno Campos	7891934173	Fix missing labels (#18356 ) Thank you for contributing to LangChain! - [ ] PR title: "package: description" - Where "package" is whichever of langchain, community, core, experimental, etc. is being modified. Use "docs: ..." for purely docs changes, "templates: ..." for template changes, "infra: ..." for CI changes. - Example: "community: add foobar LLM" - [ ] PR message: *Delete this entire checklist* and replace with - Description: a description of the change - Issue: the issue # it fixes, if applicable - Dependencies: any dependencies required for this change - Twitter handle: if your PR gets announced, and you'd like a mention, we'll gladly shout you out! - [ ] Add tests and docs: If you're adding a new integration, please include 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. It lives in `docs/docs/integrations` directory. - [ ] Lint and test: Run `make format`, `make lint` and `make test` from the root of the package(s) you've modified. See contribution guidelines for more: https://python.langchain.com/docs/contributing/ Additional guidelines: - Make sure optional dependencies are imported within a function. - Please do not add dependencies to pyproject.toml files (even optional ones) unless they are required for unit tests. - Most PRs should not touch more than one package. - Changes should be backwards compatible. - If you are adding something to community, do not re-import it in langchain. If no one reviews your PR within a few days, please @-mention one of baskaryan, efriis, eyurtsev, hwchase17.	7 months ago
William FH	fdab931fd3	[Core] Patch: rm dumpd of outputs from runnables/base (#18295 ) It obstructs evaluations when your return a pydantic object.	7 months ago
William FH	f481cbb32d	fireworks[patch]: Fix fireworks bind tools (#18352 ) Co-authored-by: Erick Friis <erick@langchain.dev>	7 months ago

... 2 3 4 5 6 ...

3440 Commits (f1313339ac6f45bcf0afcf572d372c7e3660c0af)