langchain

Commit Graph

Author	SHA1	Message	Date
Erick Friis	007c5a85d5	multiple: use modern installer in poetry (#23998 )	2 months ago
Bagatur	a0c2281540	infra: update mypy 1.10, ruff 0.5 (#23721 ) ```python """python scripts/update_mypy_ruff.py""" import glob import tomllib from pathlib import Path import toml import subprocess import re ROOT_DIR = Path(__file__).parents[1] def main(): for path in glob.glob(str(ROOT_DIR / "libs/*/pyproject.toml"), recursive=True): print(path) with open(path, "rb") as f: pyproject = tomllib.load(f) try: pyproject["tool"]["poetry"]["group"]["typing"]["dependencies"]["mypy"] = ( "^1.10" ) pyproject["tool"]["poetry"]["group"]["lint"]["dependencies"]["ruff"] = ( "^0.5" ) except KeyError: continue with open(path, "w") as f: toml.dump(pyproject, f) cwd = "/".join(path.split("/")[:-1]) completed = subprocess.run( "poetry lock --no-update; poetry install --with typing; poetry run mypy . --no-color", cwd=cwd, shell=True, capture_output=True, text=True, ) logs = completed.stdout.split("\n") to_ignore = {} for l in logs: if re.match("^(.)\:(\d+)\: error:.\[(.)\]", l): path, line_no, error_type = re.match( "^(.)\:(\d+)\: error:.\[(.*)\]", l ).groups() if (path, line_no) in to_ignore: to_ignore[(path, line_no)].append(error_type) else: to_ignore[(path, line_no)] = [error_type] print(len(to_ignore)) for (error_path, line_no), error_types in to_ignore.items(): all_errors = ", ".join(error_types) full_path = f"{cwd}/{error_path}" try: with open(full_path, "r") as f: file_lines = f.readlines() except FileNotFoundError: continue file_lines[int(line_no) - 1] = ( file_lines[int(line_no) - 1][:-1] + f" # type: ignore[{all_errors}]\n" ) with open(full_path, "w") as f: f.write("".join(file_lines)) subprocess.run( "poetry run ruff format .; poetry run ruff --select I --fix .", cwd=cwd, shell=True, capture_output=True, text=True, ) if __name__ == "__main__": main() ```	3 months ago
Jordy Jackson Antunes da Rocha	a50eabbd48	experimental: LLMGraphTransformer add missing conditional adding restrictions to prompts for LLM that do not support function calling (#22793 ) - Description: Modified the prompt created by the function `create_unstructured_prompt` (which is called for LLMs that do not support function calling) by adding conditional checks that verify if restrictions on entity types and rel_types should be added to the prompt. If the user provides a sufficiently large text, the current prompt may fail to produce results in some LLMs. I have first seen this issue when I implemented a custom LLM class that did not support Function Calling and used Gemini 1.5 Pro, but I was able to replicate this issue using OpenAI models. By loading a sufficiently large text ```python from langchain_community.llms import Ollama from langchain_openai import ChatOpenAI, OpenAI from langchain_core.prompts import PromptTemplate import re from langchain_experimental.graph_transformers import LLMGraphTransformer from langchain_core.documents import Document with open("texto-longo.txt", "r") as file: full_text = file.read() partial_text = full_text[:4000] documents = [Document(page_content=partial_text)] # cropped to fit GPT 3.5 context window ``` And using the chat class (that has function calling) ```python chat_openai = ChatOpenAI(model="gpt-3.5-turbo", model_kwargs={"seed": 42}) chat_gpt35_transformer = LLMGraphTransformer(llm=chat_openai) graph_from_chat_gpt35 = chat_gpt35_transformer.convert_to_graph_documents(documents) ``` It works: ``` >>> print(graph_from_chat_gpt35[0].nodes) [Node(id="Jesu, Joy of Man's Desiring", type='Music'), Node(id='Godel', type='Person'), Node(id='Johann Sebastian Bach', type='Person'), Node(id='clever way of encoding the complicated expressions as numbers', type='Concept')] ``` But if you try to use the non-chat LLM class (that does not support function calling) ```python openai = OpenAI( model="gpt-3.5-turbo-instruct", max_tokens=1000, ) gpt35_transformer = LLMGraphTransformer(llm=openai) graph_from_gpt35 = gpt35_transformer.convert_to_graph_documents(documents) ``` It uses the prompt that has issues and sometimes does not produce any result ``` >>> print(graph_from_gpt35[0].nodes) [] ``` After implementing the changes, I was able to use both classes more consistently: ```shell >>> chat_gpt35_transformer = LLMGraphTransformer(llm=chat_openai) >>> graph_from_chat_gpt35 = chat_gpt35_transformer.convert_to_graph_documents(documents) >>> print(graph_from_chat_gpt35[0].nodes) [Node(id="Jesu, Joy Of Man'S Desiring", type='Music'), Node(id='Johann Sebastian Bach', type='Person'), Node(id='Godel', type='Person')] >>> gpt35_transformer = LLMGraphTransformer(llm=openai) >>> graph_from_gpt35 = gpt35_transformer.convert_to_graph_documents(documents) >>> print(graph_from_gpt35[0].nodes) [Node(id='I', type='Pronoun'), Node(id="JESU, JOY OF MAN'S DESIRING", type='Song'), Node(id='larger memory', type='Memory'), Node(id='this nice tree structure', type='Structure'), Node(id='how you can do it all with the numbers', type='Process'), Node(id='JOHANN SEBASTIAN BACH', type='Composer'), Node(id='type of structure', type='Characteristic'), Node(id='that', type='Pronoun'), Node(id='we', type='Pronoun'), Node(id='worry', type='Verb')] ``` The results are a little inconsistent because the GPT 3.5 model may produce incomplete json due to the token limit, but that could be solved (or mitigated) by checking for a complete json when parsing it.	3 months ago
ccurme	62b16fcc6b	experimental: release 0.0.62 (#23507 )	3 months ago
Tomaz Bratanic	22fa32e164	LLM Graph transformer dealing with empty strings (#23368 ) Pydantic allows empty strings: ``` from langchain.pydantic_v1 import Field, BaseModel class Property(BaseModel): """A single property consisting of key and value""" key: str = Field(..., description="key") value: str = Field(..., description="value") x = Property(key="", value="") ``` Which can produce errors downstream. We simply ignore those records	3 months ago
Brace Sproul	abe7566d7d	core[minor]: BaseChatModel with_structured_output implementation (#22859 )	3 months ago
Raviraj	858ce264ef	SemanticChunker : Feature Addition ("Semantic Splitting with gradient") (#22895 ) ```SemanticChunker``` currently provide three methods to split the texts semantically: - percentile - standard_deviation - interquartile I propose new method ```gradient```. In this method, the gradient of distance is used to split chunks along with the percentile method (technically) . This method is useful when chunks are highly correlated with each other or specific to a domain e.g. legal or medical. The idea is to apply anomaly detection on gradient array so that the distribution become wider and easy to identify boundaries in highly semantic data. I have tested this merge on a set of 10 domain specific documents (mostly legal). Details : - Issue: Improvement - Dependencies: NA - Twitter handle: [x.com/prajapat_ravi](https://x.com/prajapat_ravi) @hwchase17 --------- Co-authored-by: Raviraj Prajapat <raviraj.prajapat@sirionlabs.com> Co-authored-by: isaac hershenson <ihershenson@hmc.edu>	3 months ago
Tomaz Bratanic	1c661fd849	Improve llm graph transformer docstring (#22939 )	3 months ago
Erick Friis	c374c98389	experimental: release 0.0.61 (#22924 )	3 months ago
Istvan/Nebulinq	513e491ce9	experimental: LLMGraphTransformer - added relationship properties. (#21856 ) - Description: The generated relationships in the graph had no properties, but the Relationship class was properly defined with properties. This made it very difficult to transform conditional sentences into a graph. Adding properties to relationships can solve this issue elegantly. The changes expand on the existing LLMGraphTransformer implementation but add the possibility to define allowed relationship properties like this: LLMGraphTransformer(llm=llm, relationship_properties=["Condition", "Time"],) - Issue: no issue found - Dependencies: n/a - Twitter handle: @IstvanSpace -Quick Test ================================================================= from dotenv import load_dotenv import os from langchain_community.graphs import Neo4jGraph from langchain_experimental.graph_transformers import LLMGraphTransformer from langchain_openai import ChatOpenAI from langchain_core.prompts import ChatPromptTemplate from langchain_core.documents import Document load_dotenv() os.environ["NEO4J_URI"] = os.getenv("NEO4J_URI") os.environ["NEO4J_USERNAME"] = os.getenv("NEO4J_USERNAME") os.environ["NEO4J_PASSWORD"] = os.getenv("NEO4J_PASSWORD") graph = Neo4jGraph() llm = ChatOpenAI(temperature=0, model_name="gpt-4o") llm_transformer = LLMGraphTransformer(llm=llm) #text = "Harry potter likes pies, but only if it rains outside" text = "Jack has a dog named Max. Jack only walks Max if it is sunny outside." documents = [Document(page_content=text)] llm_transformer_props = LLMGraphTransformer( llm=llm, relationship_properties=["Condition"], ) graph_documents_props = llm_transformer_props.convert_to_graph_documents(documents) print(f"Nodes:{graph_documents_props[0].nodes}") print(f"Relationships:{graph_documents_props[0].relationships}") graph.add_graph_documents(graph_documents_props) --------- Co-authored-by: Istvan Lorincz <istvan.lorincz@pm.me> Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com>	3 months ago
Eugene Yurtsev	c72bcda4f2	community[major], experimental[patch]: Remove Python REPL from community (#22904 ) Remove the REPL from community, and suggest an alternative import from langchain_experimental. Fix for this issue: https://github.com/langchain-ai/langchain/issues/14345 This is not a bug in the code or an actual security risk. The python REPL itself is behaving as expected. The PR is done to appease blanket security policies that are just looking for the presence of exec in the code. --------- Co-authored-by: Erick Friis <erick@langchain.dev>	3 months ago
Kagura Chen	57783c5e55	Fix: lint errors and update Field alias in models.py and AutoSelectionScorer initialization (#22846 ) This PR addresses several lint errors in the core package of LangChain. Specifically, the following issues were fixed: 1.Unexpected keyword argument "required" for "Field" [call-arg] 2.tests/integration_tests/chains/test_cpal.py:263: error: Unexpected keyword argument "narrative_input" for "QueryModel" [call-arg]	3 months ago
Eugene Yurtsev	a766815a99	experimental[patch]/docs[patch]: Update links to security docs (#22864 ) Minor update to newest version of security docs (content should be identical).	3 months ago
Eugene Yurtsev	ce0b0f22a1	experimental[major]: Force users to opt-in into code that relies on the python repl (#22860 ) This should make it obvious that a few of the agents in langchain experimental rely on the python REPL as a tool under the hood, and will force users to opt-in.	3 months ago
Karim Lalani	276be6cdd4	[experimental][llms][OllamaFunctions] tool calling related fixes (#22339 ) Fixes issues with tool calling to handle tool objects correctly. Added support to handle ToolMessage correctly. Added additional checks for error conditions. --------- Co-authored-by: ccurme <chester.curme@gmail.com>	3 months ago
Christophe Bornet	d04e899b56	ci: add testing with Python 3.12 (#22813 ) We need to use a different version of numpy for py3.8 and py3.12 in pyproject. And so do projects that use that Python version range and import langchain. - Twitter handle: _cbornet	3 months ago
Erick Friis	a24a9c6427	multiple: get rid of pyproject extras (#22581 ) They cause `poetry lock` to take a ton of time, and `uv pip install` can resolve the constraints from these toml files in trivial time (addressing problem with #19153) This allows us to properly upgrade lockfile dependencies moving forward, which revealed some issues that were either fixed or type-ignored (see file comments)	4 months ago
Mohammad Mohtashim	7fcef2556c	[Experimental]: Async agenerate method ollama functions (#21682 ) - Description: : Added Async method for Generate for OllamaFunctions which was missing and was raising errors for the users. - Issue: #21422	4 months ago
Bagatur	48fba40fce	experimental[patch]: Release 0.0.60 (#22497 )	4 months ago
liugz18	8fd231086e	experimental[patch]: Fix graph_transformers llms #21482 (#22417 ) Fix AttributeError on calling LLMGraphTransformer.convert_to_graph_documents #21482 since raw_schema is always a str @baskaryan	4 months ago
Karim Lalani	a1899439fc	[experimental][llms][ollama_functions] Update OllamaFunctions to send `tool_calls` attribute (#21625 ) Update OllamaFunctions to return `tool_calls` for AIMessages when used for tool calling.	4 months ago
Bagatur	50186da0a1	infra: rm unused # noqa violations (#22049 ) Updating #21137	4 months ago
mochi	63284ffebf	experimental[patch], docs: refine notebook for MyScale `SelfQueryRetriever` (#22016 ) - Description: upgrade model to `gpt-4o`	4 months ago
Tomaz Bratanic	a43515ca65	experimental[patch]: Pass enum only to openai in llm graph transformer (#21860 ) Some models like Groq return bad request if you pass in `enum` parameter in tool definition	4 months ago
Erick Friis	2d3f4e1a16	experimental: release 0.0.59 (#21835 )	4 months ago
Erick Friis	c77d2f2b06	multiple: core 0.2 nonbreaking dep, check_diff community->langchain dep (#21646 ) 0.2 is not a breaking release for core (but it is for langchain and community) To keep the core+langchain+community packages in sync at 0.2, we will relax deps throughout the ecosystem to tolerate `langchain-core` 0.2	4 months ago
Tomaz Bratanic	89ff6a3d3b	Add sentiment and confidence levels to diffbotgraphtransformer (#21590 ) Co-authored-by: Erick Friis <erickfriis@gmail.com> Co-authored-by: Erick Friis <erick@langchain.dev>	4 months ago
Erick Friis	83eecd54fe	experimental: 0.2 relax (#21468 )	4 months ago
Eugene Yurtsev	f92006de3c	multiple: langchain 0.2 in master (#21191 ) 0.2rc migrations - [x] Move memory - [x] Move remaining retrievers - [x] graph_qa chains - [x] some dependency from evaluation code potentially on math utils - [x] Move openapi chain from `langchain.chains.api.openapi` to `langchain_community.chains.openapi` - [x] Migrate `langchain.chains.ernie_functions` to `langchain_community.chains.ernie_functions` - [x] migrate `langchain/chains/llm_requests.py` to `langchain_community.chains.llm_requests` - [x] Moving `langchain_community.cross_enoders.base:BaseCrossEncoder` -> `langchain_community.retrievers.document_compressors.cross_encoder:BaseCrossEncoder` (namespace not ideal, but it needs to be moved to `langchain` to avoid circular deps) - [x] unit tests langchain -- add pytest.mark.community to some unit tests that will stay in langchain - [x] unit tests community -- move unit tests that depend on community to community - [x] mv integration tests that depend on community to community - [x] mypy checks Other todo - [x] Make deprecation warnings not noisy (need to use warn deprecated and check that things are implemented properly) - [x] Update deprecation messages with timeline for code removal (likely we actually won't be removing things until 0.4 release) -- will give people more time to transition their code. - [ ] Add information to deprecation warning to show users how to migrate their code base using langchain-cli - [ ] Remove any unnecessary requirements in langchain (e.g., is SQLALchemy required?) --------- Co-authored-by: Erick Friis <erick@langchain.dev>	4 months ago
Erick Friis	bbdf0f8801	experimental[patch]: core and langchain dep (#21402 )	5 months ago
Erick Friis	e4aca0d052	experimental[patch]: release 0.0.58 (#21397 )	5 months ago
Tomaz Bratanic	0bf7596839	Add simple node properties to llm graph transformer (#21369 ) Add support for simple node properties in llm graph transformer. Linter and dynamic pydantic classes aren't friends, hence I added two ignores	5 months ago
Tomaz Bratanic	ad3fd44a7f	experimental: Fix llm graph transformer bug (#21362 )	5 months ago
Tomaz Bratanic	5b6d1a907d	Add the extract types to diffbot graph transformer (#21315 ) Before you could only extract triples (diffbot calls it facts) from diffbot to avoid isolated nodes. However, sometimes isolated nodes can still be useful like for prefiltering, so we want to allow users to extract them if they want. Default behaviour is unchanged.	5 months ago
ccurme	6da3d92b42	(all): update removal in deprecation warnings from 0.2 to 0.3 (#21265 ) We are pushing out the removal of these to 0.3. `find . -type f -name "*.py" -exec sed -i '' 's/removal="0\.2/removal="0.3/g' {} +`	5 months ago
Liu Xiaodong	3b473d10f2	experimental: clean python repl input（experimental：Added code for PythonREPL） (#20930 ) Update python.py（experimental：Added code for PythonREPL） Added code for PythonREPL, defining a static method 'sanitize_input' that takes the string 'query' as input and returns a sanitizing string. The purpose of this method is to remove unwanted characters from the input string, Specifically: 1. Delete the whitespace at the beginning and end of the string (' \s'). 2. Remove the quotation marks (`` ` ``) at the beginning and end of the string. 3. Remove the keyword "python" at the beginning of the string (case insensitive) because the user may have typed it. This method uses regular expressions (regex) to implement sanitizing. It all started with this code： from langchain.agents import Tool from langchain_experimental.utilities import PythonREPL python_repl = PythonREPL() repl_tool = Tool( name="python_repl", description="Remove redundant formatting marks at the beginning and end of source code from input.Use a Python shell to execute python commands. If you want to see the output of a value, you should print it out with `print(...)`.", func=python_repl.run, ) When I call the agent to write a piece of code for me and execute it with the defined code, I must get an error: SyntaxError('invalid syntax', ('<string>', 1, 1,'In', 1, 2)) After checking, I found that pythonREPL has less formatting of input code than the soon-to-be deprecated pythonREPL tool, so I added this step to it, so that no matter what code I ask the agent to write for me, it can be executed smoothly and get the output result. I have tried modifying the prompt words to solve this problem before, but it did not work, and by adding a simple format check, the problem is well resolved. <img width="1271" alt="image" src="https://github.com/langchain-ai/langchain/assets/164149097/c49a685f-d246-4b11-b655-fd952fc2f04c"> --------- Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com> Co-authored-by: Bagatur <baskaryan@gmail.com>	5 months ago
Tomaz Bratanic	7860e4c649	experimental[patch]: Add support for non-function calling LLMs in llm graph transformers (#21014 )	5 months ago
Karim Lalani	2ddac9a7c3	experimental[minor]: Add bind_tools and with_structured_output functions to OllamaFunctions (#20881 ) Implemented bind_tools for OllamaFunctions. Made OllamaFunctions sub class of ChatOllama. Implemented with_structured_output for OllamaFunctions. integration unit test has been updated. notebook has been updated. --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	5 months ago
Michael Schock	5e60d65917	experimental[patch]: return from HuggingGPT task executor task.run() exception (#20219 ) Description: Fixes a bug in the HuggingGPT task execution logic here: except Exception as e: self.status = "failed" self.message = str(e) self.status = "completed" self.save_product() where a caught exception effectively just sets `self.message` and can then throw an exception if, e.g., `self.product` is not defined. Issue: None that I'm aware of. Dependencies: None Twitter handle: https://twitter.com/michaeljschock Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com>	5 months ago
Michael Schock	63a07f52df	experimental[patch]: remove \n from AutoGPT feedback_tool exit check (#20132 )	5 months ago
GustavoSept	c2d09a5186	experimental[patch]: Makes regex customizable in text_splitter.py (SemanticChunker class) (#20485 ) - Description: Currently, the regex is static (`r"(?<=[.?!])\s+"`), which is only useful for certain use cases. The current change only moves this to be a parameter of split_text(). Which adds flexibility without making it more complex (as the default regex is still the same). - Issue: Not applicable (I searched, no one seems to have created this issue yet). - Dependencies: None. _If no one reviews your PR within a few days, please @-mention one of baskaryan, efriis, eyurtsev, hwchase17._ --------- Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com> Co-authored-by: Bagatur <baskaryan@gmail.com>	5 months ago
Oleksandr Yaremchuk	9428923bab	experimental[minor]: upgrade the prompt injection model (#20783 ) - Description: In January, Laiyer.ai became part of ProtectAI, which means the model became owned by ProtectAI. In addition to that, yesterday, we released a new version of the model addressing issues the Langchain's community and others mentioned to us about false-positives. The new model has a better accuracy compared to the previous version, and we thought the Langchain community would benefit from using the [latest version of the model](https://huggingface.co/protectai/deberta-v3-base-prompt-injection-v2). - Issue: N/A - Dependencies: N/A - Twitter handle: @alex_yaremchuk	5 months ago
ccurme	c010ec8b71	patch: deprecate (a)get_relevant_documents (#20477 ) - `.get_relevant_documents(query)` -> `.invoke(query)` - `.get_relevant_documents(query=query)` -> `.invoke(query)` - `.get_relevant_documents(query, callbacks=callbacks)` -> `.invoke(query, config={"callbacks": callbacks})` - `.get_relevant_documents(query, kwargs)` -> `.invoke(query, kwargs)` --------- Co-authored-by: Erick Friis <erick@langchain.dev>	5 months ago
Bagatur	1c7b3c75a7	community[patch], experimental[patch]: support tool-calling sql and p… (#20639 ) d agents	5 months ago
Leonid Ganeline	95dc90609e	experimental[patch]: `prompts` import fix (#20534 ) Replaced `from langchain.prompts` with `from langchain_core.prompts` where it is appropriate. Most of the changes go to `langchain_experimental` Similar to #20348	5 months ago
ccurme	38faa74c23	community[patch]: update use of deprecated llm methods (#20393 ) .predict and .predict_messages for BaseLanguageModel and BaseChatModel	5 months ago
Leonid Ganeline	e512d3c6a6	langchain: `callbacks` imports fix (#20348 ) Replaced all `from langchain.callbacks` into `from langchain_core.callbacks` . Changes in the `langchain` and `langchain_experimental` --------- Co-authored-by: Erick Friis <erick@langchain.dev>	5 months ago
Tomaz Bratanic	a1b105ac00	experimental[patch]: Skip pydantic validation for llm graph transformer and fix JSON response where possible (#19915 ) LLMs might sometimes return invalid response for LLM graph transformer. Instead of failing due to pydantic validation, we skip it and manually check and optionally fix error where we can, so that more information gets extracted	5 months ago
Bagatur	2d83505be9	experimental[patch]: Release 0.0.57 (#20243 )	5 months ago
Erick Friis	f0d5b59962	core[patch]: remove requests (#19891 ) Removes required usage of `requests` from `langchain-core`, all of which has been deprecated. - removes Tracer V1 implementations - removes old `try_load_from_hub` github-based hub implementations Removal done in a way where imports will still succeed, and usage will fail with a `RuntimeError`.	6 months ago

1 2 3 4 5 ...

291 Commits (1d7a3ae7ce1caf1958ff19301d0e9acf02ecce0c)