4765 Commits (46cbf0e4aa9483ee753960870f5402c1781f2a53)

Author	SHA1	Message	Date
ccurme	46cbf0e4aa	anthropic[patch]: use core output parsers for structured output (#23776) ... Also add to standard tests for structured output.	3 months ago
kiarina	dc396835ed	langchain_anthropic: add stop_reason in ChatAnthropic stream result (#23689) ... `ChatAnthropic` can get `stop_reason` from the resulting `AIMessage` in `invoke` and `ainvoke`, but not in `stream` and `astream`. This is a different behavior from `ChatOpenAI`. It is possible to get `stop_reason` from `stream` as well, since it is needed to determine the next action after the LLM call. This would be easier to handle in situations where only `stop_reason` is needed. - Issue: NA - Dependencies: NA - Twitter handle: https://x.com/kiarina37	3 months ago
maang-h	e4e28a6ff5	community[patch]: Fix MiniMaxChat validate_environment error (#23770) ... - **Description:** Fix some issues in MiniMaxChat - Fix `minimax_api_host` not in `values` error - Remove `minimax_group_id` from reading environment variables, the `minimax_group_id` no longer use in MiniMaxChat - Invoke callback prior to yielding token, the issus #16913	3 months ago
SN	acc457f645	core[patch]: fix nested sections for mustache templating (#23747) ... The prompt template variable detection only worked for singly-nested sections because we just kept track of whether we were in a section and then set that to false as soon as we encountered an end block. i.e. the following: ``` {{#outerSection}} {{variableThatShouldntShowUp}} {{#nestedSection}} {{nestedVal}} {{/nestedSection}} {{anotherVariableThatShouldntShowUp}} {{/outerSection}} ``` Would yield `['outerSection', 'anotherVariableThatShouldntShowUp']` as input_variables (whereas it should just yield `['outerSection']`). This fixes that by keeping track of the current depth and using a stack.	3 months ago
Eugene Yurtsev	46ff0f7a3c	community[patch]: Update @root_validators to use explicit pre=True or pre=False (#23737)	3 months ago
Igor Drozdov	b664dbcc36	feat(community): add support for tool_calls response (#23765) ... When `model_kwargs={"tools": tools}` are passed to `ChatLiteLLM`, they are executed, but the response is not recognized correctly Let's add `tool_calls` to the `additional_kwargs` Thank you for contributing to LangChain! ## ChatAnthropic I used the following example to verify the output of llm with tools: ```python from langchain_core.pydantic_v1 import BaseModel, Field from langchain_anthropic import ChatAnthropic class GetWeather(BaseModel): '''Get the current weather in a given location''' location: str = Field(..., description="The city and state, e.g. San Francisco, CA") class GetPopulation(BaseModel): '''Get the current population in a given location''' location: str = Field(..., description="The city and state, e.g. San Francisco, CA") llm = ChatAnthropic(model="claude-3-sonnet-20240229") llm_with_tools = llm.bind_tools([GetWeather, GetPopulation]) ai_msg = llm_with_tools.invoke("Which city is hotter today and which is bigger: LA or NY?") print(ai_msg.tool_calls) ``` I get the following response: ```json [{'name': 'GetWeather', 'args': {'location': 'Los Angeles, CA'}, 'id': 'toolu_01UfDA89knrhw3vFV9X47neT'}, {'name': 'GetWeather', 'args': {'location': 'New York, NY'}, 'id': 'toolu_01NrYVRYae7m7z7tBgyPb3Gd'}, {'name': 'GetPopulation', 'args': {'location': 'Los Angeles, CA'}, 'id': 'toolu_01EPFEpDgzL6vV2dTpD9SVP5'}, {'name': 'GetPopulation', 'args': {'location': 'New York, NY'}, 'id': 'toolu_01B5J6tPJXgwwfhQX9BHP2dt'}] ``` ## LiteLLM Based on https://litellm.vercel.app/docs/completion/function_call ```python from langchain_core.pydantic_v1 import BaseModel, Field from langchain_core.utils.function_calling import convert_to_openai_tool import litellm class GetWeather(BaseModel): '''Get the current weather in a given location''' location: str = Field(..., description="The city and state, e.g. San Francisco, CA") class GetPopulation(BaseModel): '''Get the current population in a given location''' location: str = Field(..., description="The city and state, e.g. San Francisco, CA") prompt = "Which city is hotter today and which is bigger: LA or NY?" tools = [convert_to_openai_tool(GetWeather), convert_to_openai_tool(GetPopulation)] response = litellm.completion(model="claude-3-sonnet-20240229", messages=[{'role': 'user', 'content': prompt}], tools=tools) print(response.choices[0].message.tool_calls) ``` ```python [ChatCompletionMessageToolCall(function=Function(arguments='{"location": "Los Angeles, CA"}', name='GetWeather'), id='toolu_01HeDWV5vP7BDFfytH5FJsja', type='function'), ChatCompletionMessageToolCall(function=Function(arguments='{"location": "New York, NY"}', name='GetWeather'), id='toolu_01EiLesUSEr3YK1DaE2jxsQv', type='function'), ChatCompletionMessageToolCall(function=Function(arguments='{"location": "Los Angeles, CA"}', name='GetPopulation'), id='toolu_01Xz26zvkBDRxEUEWm9pX6xa', type='function'), ChatCompletionMessageToolCall(function=Function(arguments='{"location": "New York, NY"}', name='GetPopulation'), id='toolu_01SDqKnsLjvUXuBsgAZdEEpp', type='function')] ``` ## ChatLiteLLM When I try the following ```python from langchain_core.pydantic_v1 import BaseModel, Field from langchain_core.utils.function_calling import convert_to_openai_tool from langchain_community.chat_models import ChatLiteLLM class GetWeather(BaseModel): '''Get the current weather in a given location''' location: str = Field(..., description="The city and state, e.g. San Francisco, CA") class GetPopulation(BaseModel): '''Get the current population in a given location''' location: str = Field(..., description="The city and state, e.g. San Francisco, CA") prompt = "Which city is hotter today and which is bigger: LA or NY?" tools = [convert_to_openai_tool(GetWeather), convert_to_openai_tool(GetPopulation)] llm = ChatLiteLLM(model="claude-3-sonnet-20240229", model_kwargs={"tools": tools}) ai_msg = llm.invoke(prompt) print(ai_msg) print(ai_msg.tool_calls) ``` ```python content="Okay, let's find out the current weather and populations for Los Angeles and New York City:" response_metadata={'token_usage': Usage(prompt_tokens=329, completion_tokens=193, total_tokens=522), 'model': 'claude-3-sonnet-20240229', 'finish_reason': 'tool_calls'} id='run-748b7a84-84f4-497e-bba1-320bd4823937-0' [] ``` --- When I apply the changes of this PR, the output is ```json [{'name': 'GetWeather', 'args': {'location': 'Los Angeles, CA'}, 'id': 'toolu_017D2tGjiaiakB1HadsEFZ4e'}, {'name': 'GetWeather', 'args': {'location': 'New York, NY'}, 'id': 'toolu_01WrDpJfVqLkPejWzonPCbLW'}, {'name': 'GetPopulation', 'args': {'location': 'Los Angeles, CA'}, 'id': 'toolu_016UKyYrVAV9Pz99iZGgGU7V'}, {'name': 'GetPopulation', 'args': {'location': 'New York, NY'}, 'id': 'toolu_01Sgv1imExFX1oiR1Cw88zKy'}] ``` If no one reviews your PR within a few days, please @-mention one of baskaryan, efriis, eyurtsev, ccurme, vbarda, hwchase17. Co-authored-by: Igor Drozdov <idrozdov@gitlab.com>	3 months ago
Eugene Yurtsev	338cef35b4	community[patch]: update @root_validator in utilities namespace (#23768) ... Update all utilities to use `pre=True` or `pre=False` https://github.com/langchain-ai/langchain/issues/22819	3 months ago
wenngong	ee5eedfa04	partners: support reading HuggingFace params from env (#23309) ... Description: 1. partners/HuggingFace module support reading params from env. Not adjust langchain_community/.../huggingfaceXX modules since they are deprecated. 2. pydantic 2 @root_validator migration. Issue: #22448 #22819 --------- Co-authored-by: gongwn1 <gongwn1@lenovo.com>	3 months ago
antonpibm	ffde8a6a09	Milvus vectorstore: fix pass ids as argument after upsert (#23761) ... **Description**: Milvus vectorstore supports both `add_documents` via the base class and `upsert` method which deletes and re-adds documents based on their ids **Issue**: Due to mismatch in the interfaces the ids used by `upsert` are neglected in `add_documents`, as `ids` are passed as argument in `upsert` but via `kwargs` is `add_documents` This caused exceptions and inconsistency in the DB, tested with `auto_id=False` **Fix**: pass `ids` via `kwargs` to `add_documents`	3 months ago
Eugene Yurtsev	d084172b63	community[patch]: root validator set explicit pre=False or pre=True (#23764) ... See issue: https://github.com/langchain-ai/langchain/issues/22819	3 months ago
mattthomps1	cc55823486	docs: updated PPLX model (#23723) ... Description: updated pplx docs to reference a currently [supported model](https://docs.perplexity.ai/docs/model-cards). pplx-70b-online ->llama-3-sonar-small-32k-online --------- Co-authored-by: Chester Curme <chester.curme@gmail.com>	3 months ago
Jacob Lee	7791d92711	community[patch]: Fix requests alias for load_tools (#23734) ... CC @baskaryan	3 months ago
Eugene Yurtsev	f24e38876a	community[patch]: Update root_validators to use explicit pre=True or pre=False (#23736)	3 months ago
Yannick Stephan	5b1de2ae93	mistralai: Fixed streaming in MistralAI with ainvoke and callbacks (#22000) ... # Fix streaming in mistral with ainvoke - [x] **PR title** - [x] **PR message** - [x] **Add tests and docs**: 1. [x] Added a test for the fixed integration. 2. [x] An example notebook showing its use. It lives in `docs/docs/integrations` directory. - [x] **Lint and test**: Ran `make format`, `make lint` and `make test` from the root of the package(s) I've modified. Hello * I Identified an issue in the mistral package where the callback streaming (see on_llm_new_token) was not functioning correctly when the streaming parameter was set to True and call with `ainvoke`. * The root cause of the problem was the streaming not taking into account. ( I think it's an oversight ) * To resolve the issue, I added the `streaming` attribut. * Now, the callback with streaming works as expected when the streaming parameter is set to True. ## How to reproduce ``` from langchain_mistralai.chat_models import ChatMistralAI chain = ChatMistralAI(streaming=True) # Add a callback chain.ainvoke(..) # Oberve on_llm_new_token # Now, the callback is given as streaming tokens, before it was in grouped format. ``` Co-authored-by: Erick Friis <erick@langchain.dev>	3 months ago
Eugene Yurtsev	5d2262af34	community[patch]: Update root_validators to use pre=True or pre=False (#23731) ... Update root_validators in preparation for pydantic 2 migration.	3 months ago
Eugene Yurtsev	ebcee4f610	core[patch]: Add versionadded to get_by_ids (#23728)	3 months ago
Eugene Yurtsev	e800f6bb57	core[minor]: Create BaseMedia object (#23639) ... This PR implements a BaseContent object from which Document and Blob objects will inherit proposed here: https://github.com/langchain-ai/langchain/pull/23544 Alternative: Create a base object that only has an identifier and no metadata. For now decided against it, since that refactor can be done at a later time. It also feels a bit odd since our IDs are optional at the moment. --------- Co-authored-by: Erick Friis <erick@langchain.dev>	3 months ago
Chip Davis	04bc5f1a95	partners[azure]: fix having openai_api_base set for other packages (#22068) ... This fix is for #21726. When having other packages installed that require the `openai_api_base` environment variable, users are not able to instantiate the AzureChatModels or AzureEmbeddings. This PR adds a new value `ignore_openai_api_base` which is a bool. When set to True, it sets `openai_api_base` to `None` Two new tests were added for the `test_azure` and a new file `test_azure_embeddings` A different approach may be better for this. If you can think of better logic, let me know and I can adjust it. --------- Co-authored-by: Erick Friis <erick@langchain.dev>	3 months ago
Nuno Campos	b36e95caa9	core[patch]: use async messages where possible (#23718) ... Fix #23716 Thank you for contributing to LangChain! - [ ] **PR title**: "package: description" - Where "package" is whichever of langchain, community, core, experimental, etc. is being modified. Use "docs: ..." for purely docs changes, "templates: ..." for template changes, "infra: ..." for CI changes. - Example: "community: add foobar LLM" - [ ] **PR message**: ***Delete this entire checklist*** and replace with - **Description:** a description of the change - **Issue:** the issue # it fixes, if applicable - **Dependencies:** any dependencies required for this change - **Twitter handle:** if your PR gets announced, and you'd like a mention, we'll gladly shout you out! - [ ] **Add tests and docs**: If you're adding a new integration, please include 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. It lives in `docs/docs/integrations` directory. - [ ] **Lint and test**: Run `make format`, `make lint` and `make test` from the root of the package(s) you've modified. See contribution guidelines for more: https://python.langchain.com/docs/contributing/ Additional guidelines: - Make sure optional dependencies are imported within a function. - Please do not add dependencies to pyproject.toml files (even optional ones) unless they are required for unit tests. - Most PRs should not touch more than one package. - Changes should be backwards compatible. - If you are adding something to community, do not re-import it in langchain. If no one reviews your PR within a few days, please @-mention one of baskaryan, efriis, eyurtsev, ccurme, vbarda, hwchase17. --------- Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com>	3 months ago
Spyros Avlonitis	8cfb2fa1b7	core[minor]: Add maxsize for InMemoryCache (#23405) ... This PR introduces a maxsize parameter for the InMemoryCache class, allowing users to specify the maximum number of items to store in the cache. If the cache exceeds the specified maximum size, the oldest items are removed. Additionally, comprehensive unit tests have been added to ensure all functionalities are thoroughly tested. The tests are written using pytest and cover both synchronous and asynchronous methods. Twitter: @spyrosavl --------- Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com>	3 months ago
maang-h	96af8f31ae	community[patch]: Invoke callback prior to yielding token (#23638) ... - **Description:** Invoke callback prior to yielding token in stream and astream methods for ChatZhipuAI. - **Issue:** the issue #16913	3 months ago
Eugene Yurtsev	b5aef4cf97	core[patch]: Fix llm string representation for serializable models (#23416) ... Fix LLM string representation for serializable objects. Fix for issue: https://github.com/langchain-ai/langchain/issues/23257 The llm string of serializable chat models is the serialized representation of the object. LangChain serialization dumps some basic information about non serializable objects including their repr() which includes an object id. This means that if a chat model has any non serializable fields (e.g., a cache), then any new instantiation of the those fields will change the llm representation of the chat model and cause chat misses. i.e., re-instantiating a postgres cache would result in cache misses!	3 months ago
nobbbbby	3904f2cd40	core: fix NameError (#23658) ... **Description:** In the chat_models module of the language model, the import statement for BaseModel has been moved from the conditionally imported section to the main import area, fixing `NameError `. **Issue:** fix `NameError `	3 months ago
Jordy Jackson Antunes da Rocha	a50eabbd48	experimental: LLMGraphTransformer add missing conditional adding restrictions to prompts for LLM that do not support function calling (#22793) ... - Description: Modified the prompt created by the function `create_unstructured_prompt` (which is called for LLMs that do not support function calling) by adding conditional checks that verify if restrictions on entity types and rel_types should be added to the prompt. If the user provides a sufficiently large text, the current prompt **may** fail to produce results in some LLMs. I have first seen this issue when I implemented a custom LLM class that did not support Function Calling and used Gemini 1.5 Pro, but I was able to replicate this issue using OpenAI models. By loading a sufficiently large text ```python from langchain_community.llms import Ollama from langchain_openai import ChatOpenAI, OpenAI from langchain_core.prompts import PromptTemplate import re from langchain_experimental.graph_transformers import LLMGraphTransformer from langchain_core.documents import Document with open("texto-longo.txt", "r") as file: full_text = file.read() partial_text = full_text[:4000] documents = [Document(page_content=partial_text)] # cropped to fit GPT 3.5 context window ``` And using the chat class (that has function calling) ```python chat_openai = ChatOpenAI(model="gpt-3.5-turbo", model_kwargs={"seed": 42}) chat_gpt35_transformer = LLMGraphTransformer(llm=chat_openai) graph_from_chat_gpt35 = chat_gpt35_transformer.convert_to_graph_documents(documents) ``` It works: ``` >>> print(graph_from_chat_gpt35[0].nodes) [Node(id="Jesu, Joy of Man's Desiring", type='Music'), Node(id='Godel', type='Person'), Node(id='Johann Sebastian Bach', type='Person'), Node(id='clever way of encoding the complicated expressions as numbers', type='Concept')] ``` But if you try to use the non-chat LLM class (that does not support function calling) ```python openai = OpenAI( model="gpt-3.5-turbo-instruct", max_tokens=1000, ) gpt35_transformer = LLMGraphTransformer(llm=openai) graph_from_gpt35 = gpt35_transformer.convert_to_graph_documents(documents) ``` It uses the prompt that has issues and sometimes does not produce any result ``` >>> print(graph_from_gpt35[0].nodes) [] ``` After implementing the changes, I was able to use both classes more consistently: ```shell >>> chat_gpt35_transformer = LLMGraphTransformer(llm=chat_openai) >>> graph_from_chat_gpt35 = chat_gpt35_transformer.convert_to_graph_documents(documents) >>> print(graph_from_chat_gpt35[0].nodes) [Node(id="Jesu, Joy Of Man'S Desiring", type='Music'), Node(id='Johann Sebastian Bach', type='Person'), Node(id='Godel', type='Person')] >>> gpt35_transformer = LLMGraphTransformer(llm=openai) >>> graph_from_gpt35 = gpt35_transformer.convert_to_graph_documents(documents) >>> print(graph_from_gpt35[0].nodes) [Node(id='I', type='Pronoun'), Node(id="JESU, JOY OF MAN'S DESIRING", type='Song'), Node(id='larger memory', type='Memory'), Node(id='this nice tree structure', type='Structure'), Node(id='how you can do it all with the numbers', type='Process'), Node(id='JOHANN SEBASTIAN BACH', type='Composer'), Node(id='type of structure', type='Characteristic'), Node(id='that', type='Pronoun'), Node(id='we', type='Pronoun'), Node(id='worry', type='Verb')] ``` The results are a little inconsistent because the GPT 3.5 model may produce incomplete json due to the token limit, but that could be solved (or mitigated) by checking for a complete json when parsing it.	3 months ago
Eugene Yurtsev	4f1821db3e	core[minor]: Add get_by_ids to vectorstore interface (#23594) ... This PR adds a part of the indexing API proposed in this RFC https://github.com/langchain-ai/langchain/pull/23544/files. It allows rolling out `get_by_ids` which should be uncontroversial to existing vectorstores without introducing new abstractions. The semantics for this method depend on the ability of identifying returned documents using the new optional ID field on documents: https://github.com/langchain-ai/langchain/pull/23411 Alternatives are: 1. Relax the sequence requirement ```python def get_by_ids(self, ids: Iterable[str], /) -> Iterable[Document]: ``` Rejected: - implementations are more likley to start batching with bad defaults - users would need to call list() or we'd need to introduce another convenience method 2. Support more kwargs ```python def get_by_ids(self, ids: Sequence[str], /, **kwargs) -> List[Document]: ... ``` Rejected: - No need for `batch` parameter since IDs is a sequence - Output cannot be customized since `Document` is fixed. (e.g., parameters could be useful to grab extra metadata like the vector that was indexed with the Document or to project a part of the document)	3 months ago
Valentin	bf402f902e	community: Fix LanceDB similarity search bug (#23591) ... **Description:** LanceDB didn't allow querying the database using similarity score thresholds because the metrics value was missing. This PR simply fixes that bug. **Issue:** not applicable **Dependencies:** none **Twitter handle:** not available --------- Co-authored-by: ccurme <chester.curme@gmail.com>	3 months ago
Bagatur	389a568f9a	standard-tests[patch]: add anthropic format integration test (#23717)	3 months ago
Rafael Pereira	4b9517db85	Jira: Allow Jira access using only the token (#23708) ... - **Description:** At the moment the Jira wrapper only accepts the the usage of the Username and Password/Token at the same time. However Jira allows the connection using only is useful for enterprise context. Co-authored-by: rpereira <rafael.pereira@criticalsoftware.com>	3 months ago
Tim Van Wassenhove	24916c6703	community: Register pandas df in duckdb when creating vector_store (#23690) ... - **Description:** Register pandas df in duckdb when creating vector_store - **Issue:** Resolves #23308 - **Dependencies:** None - **Twitter handle:** @timvw Co-authored-by: Tim Van Wassenhove <tim.van.wassenhove@telenetgroup.be>	3 months ago
Bagatur	29aa9d6750	groq[patch]: Release 0.1.6 (#23655)	3 months ago
Bagatur	f2d0c13a15	fireworks[patch]: Release 0.1.4 (#23654)	3 months ago
Bagatur	9a5e35d1ba	mistralai[patch]: Release 0.1.9 (#23653)	3 months ago
Mateusz Szewczyk	a78ccb993c	ibm: Add support for Chat Models (#22979)	3 months ago
Bagatur	af2c05e5f3	openai[patch]: Release 0.1.13 (#23651)	3 months ago
Bagatur	b63c7f10bc	anthropic[patch]: Release 0.1.17 (#23650)	3 months ago
Bagatur	fc8fd49328	openai, anthropic, ...: with_structured_output to pass in explicit tool choice (#23645) ... ...community, mistralai, groq, fireworks part of #23644	3 months ago
Bagatur	81064017a9	docs: azure openai docstring (#23643) ... part of #22296	3 months ago
Bagatur	381aedcc61	docs: standardize azure openai page (#23642) ... part of #22296	3 months ago
Vadym Barda	e8d77002ea	core: add RemoveMessage (#23636) ... This change adds a new message type `RemoveMessage`. This will enable `langgraph` users to manually modify graph state (or have the graph nodes modify the state) to remove messages by `id` Examples: * allow users to delete messages from state by calling ```python graph.update_state(config, values=[RemoveMessage(id=state.values[-1].id)]) ``` * allow nodes to delete messages ```python graph.add_node("delete_messages", lambda state: [RemoveMessage(id=state[-1].id)]) ```	3 months ago
ccurme	8fce8c6771	community: fix extended tests (#23640)	3 months ago
ccurme	5d93916665	openai[patch]: release 0.1.12 (#23641)	3 months ago
Jacob Lee	a032583b17	docs[patch]: Update diagrams (#23613)	3 months ago
ccurme	390ee8d971	standard-tests: add test for structured output (#23631) ... - add test for structured output - fix bug with structured output for Azure - better testing on Groq (break out Mixtral + Llama3 and add xfails where needed)	3 months ago
j pradhan	5f21eab491	community:perplexity[patch]: standardize init args (#21794) ... updated request_timeout default alias value per related docstring. Related to [20085](https://github.com/langchain-ai/langchain/issues/20085) Thank you for contributing to LangChain! --------- Co-authored-by: ccurme <chester.curme@gmail.com>	3 months ago
mackong	11483b0fb8	community[patch]: set tool name for tongyi&qianfan llm (#22889) ... - **Description:** The name of ToolMessage is default to None, which makes tool message send to LLM likes ```json {"role": "tool", "tool_call_id": "", "content": "{\"time\": \"12:12\"}", "name": null} ``` But the name seems essential for some LLMs like TongYi Qwen. so we need to set the name use agent_action's tool value. - **Issue:** N/A - **Dependencies:** N/A	3 months ago
Leonid Ganeline	e4caa41aa9	community: docstrings `toolkits` (#23616) ... Added missed docstrings. Formatted docstrings to the consistent form.	3 months ago
ccurme	adf2dc13de	community: fix lint (#23611)	3 months ago
Leonid Ganeline	75a44fe951	core: `chat_*` docstrings (#23412) ... Added missed docstrings. Formatted docstrings to the consistent form.	3 months ago
Bagatur	3b1fcb2a65	chroma[patch]: Release 0.1.2 (#23604)	3 months ago
Eugene Yurtsev	68f348357e	community[patch]: Test InMemoryVectorStore with RWAPI test suite (#23603) ... Add standard test suite to InMemoryVectorStore implementation.	3 months ago