langchain

Commit Graph

Author	SHA1	Message	Date
James Braza	3833882ab7	Removing extra `StdOutCallbackHandler` overridden methods (#14136 ) Unnecessarily overridden methods: - Give the idea the subclass is doing something special (when it isn't) - Block CTRL-click to the actual method This PR removes some unnecessarily overridden methods in `StdOutCallbackHandler` Supercedes https://github.com/langchain-ai/langchain/pull/12858	10 months ago
Bob Lin	ac449f186b	Update docs to use new usage in openai>1.0.0 (#14163 ) ### Description Use new [APIs](https://github.com/openai/openai-python/blob/main/api.md#finetuning) ### Twitter handle [lin_bob57617](https://twitter.com/lin_bob57617)	10 months ago
James Braza	052e23be3e	Added Python `logging` tracer (#14190 ) This PR creates a logging handler and adds a simple unit test of it Supercedes https://github.com/langchain-ai/langchain/pull/12862 --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	10 months ago
Bob Lin	1ea48a31da	Update fallback cases (#14164 ) ### Description The `RateLimitError` initialization method has changed after openai v1, and the usage of `patch` needs to be changed. ### Twitter handle [lin_bob57617](https://twitter.com/lin_bob57617)	10 months ago
Bob Lin	62505043be	Closed #14069 (#14166 ) ### Description Fix #14069 ### Twitter handle [lin_bob57617](https://twitter.com/lin_bob57617)	10 months ago
Yong woo Song	9938086df0	Fix Html2TextTransformer for shallow copy (#14197 ) <!-- Thank you for contributing to LangChain! Replace this entire comment with: - Description: a description of the change, - Issue: the issue # it fixes (if applicable), - Dependencies: any dependencies required for this change, - Tag maintainer: for a quicker response, tag the relevant maintainer (see below), - Twitter handle: we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out! Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. It lives in `docs/extras` directory. If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17. --> Hi, There is some unintended behavior in Html2TextTransformer. The current code is directly modifying the original documents that are passed as arguments to the function. Therefore, not only the return of the function but also the input variables are being modified simultaneously. To resolve this, I added unit test code as well. reference link: [Shallow vs Deep Copying of Python Objects](https://realpython.com/copying-python-objects/) Thanks! ☺️	10 months ago
h3l	818252b1f8	Fix: (issue #14127 ) Volc Engine MaaS import error (#14194 ) - Description: fix Volc Engine MaaS import error - Issue: [the issue # it fixes (if applicable),](https://github.com/langchain-ai/langchain/issues/14127) - Dependencies: None - Tag maintainer: @baskaryan - Twitter handle: Co-authored-by: lvzhong <lvzhong@bytedance.com>	10 months ago
Leonid Ganeline	6ae0194dc7	docs: `integrations/toolkits/office365` notebook update (#14188 ) Added more descriptions and authentication details.	10 months ago
Bagatur	0bdb434383	langchain[patch]: Release langchain 0.0.345 (#14184 )	10 months ago
Bagatur	15c04a5670	core[patch]: Release 0.0.9 (#14182 )	10 months ago
James Braza	bdb6ae2ed3	core[patch]: `BaseTracer` helper method for `Run` lookup (#14139 ) I observed the same run ID extraction logic is repeated many times in `BaseTracer`. This PR creates a helper method for DRY code.	10 months ago
Harutaka Kawamura	41ee3be95f	langchain[patch]: Support passing parameters to `llms.Databricks` and `llms.Mlflow` (#14100 ) Before, we need to use `params` to pass extra parameters: ```python from langchain.llms import Databricks Databricks(..., params={"temperature": 0.0}) ``` Now, we can directly specify extra params: ```python from langchain.llms import Databricks Databricks(..., temperature=0.0) ```	10 months ago
Abdul	82102c99b3	langchain[patch]: Running SQLDatabaseChain adds prefix "SQLQuery:\n" (#14058 ) - Issue: https://github.com/langchain-ai/langchain/issues/12077 --------- Co-authored-by: Abdul Kader Maliyakkal <maliyakk@amazon.com>	10 months ago
Samuel Kemp	fd781c89cc	langchain[minor]: add azure ai data document loader (#13404 ) This PR adds an "Azure AI data" document loader, which allows Azure AI users to load their registered data assets as a document object in langchain. --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	10 months ago
James Braza	24385a00de	core[minor], langchain[patch], experimental[patch]: Added missing `py.typed` to `langchain_core` (#14143 ) See PR title. From what I can see, `poetry` will auto-include this. Please let me know if I am missing something here. --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	10 months ago
quantum00549	f7c257553d	langchain[patch]: fixed a bug that was causing the streaming transfer to not work… (#10827 ) … properly Fixed a bug that was causing the streaming transfer to not work properly. - Description: 1、The on_llm_new_token method in the streaming callback can now be called properly in streaming transfer mode. 2、In streaming transfer mode, LLM can now correctly output the complete response instead of just the first token. - Tag maintainer: @wangxuqi - **Twitter handle: @kGX7XJjuYxzX9Km --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	10 months ago
Eugene Yurtsev	6d0209e0aa	Improve file system blob loader and generic loader (#14004 ) * Add support for passing a specific file to the file system blob loader * Allow specifying a class parameter for the parser for the generic loader ```python class AudioLoader(GenericLoader): @staticmethod def get_parser(kwargs): return MyAudioParser(kwargs): ``` The intent of the GenericLoader is to provide on-ramps from different sources (e.g., web, s3, file system). An alternative is to use pipelining syntax or creating a Pipeline ``` FileSystemBlobLoader(...) \| MyAudioParser ``` --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	10 months ago
Erick Friis	700428593a	fix broken api docs links (#14154 )	10 months ago
Bagatur	340b42d8ee	docs[minor]: lcel why page (#14089 )	10 months ago
Lance Martin	cbe4753e1a	Update Open CLIP embd (#14155 ) Prior default model required a large amt of RAM and often crashed Jupyter ntbk kernel.	10 months ago
Erick Friis	b01d9d27d9	docs[patch]: docs local build (#14152 )	10 months ago
Alex Kira	0caef3cde7	Change RunnableMap to RunnableParallel for consistency (#14142 ) - Description: Change instances of RunnableMap to RunnableParallel, as that should be the one used going forward. This makes it consistent across the codebase.	10 months ago
Erick Friis	96f6b90349	templates[patch]: relock templates (#14149 )	10 months ago
Martin Jul	e3a7c96a8e	docs[patch]: Fix minor typos (casing) in quickstart (#14138 ) Fix casing of API and LangChain in the description text for the LangServe example server.	10 months ago
Erick Friis	8cf4cb9e48	docs[patch]: Fix templates/index (#14146 )	10 months ago
Amyh102	b6d26d3f9f	infra[patch]: Add unit tests for Huggingface dataset loader (#14053 ) - Description: Add unit tests for huggingface dataset loader and sample huggingface dataset for future tests. Updates dependencies for `datasets` module. - Adds coverage for [previous pull request](https://github.com/langchain-ai/langchain/pull/13864) - Tag maintainer: @hwchase17 --------- Co-authored-by: Amy Han <amyhan@Amys-Air.lan> Co-authored-by: Amy Han <amyhan@Amys-MacBook-Air.local> Co-authored-by: Bagatur <baskaryan@gmail.com>	10 months ago
Alex Kira	6eb40db353	docs[patch]: Add getting started section to LCEL doc (#14045 ) ### Description: Doc addition for LCEL introduction. Adds a more basic starter guide for using LCEL. --------- Co-authored-by: Alex Kira <akira@Alexs-MBP.local.tld> Co-authored-by: Bagatur <baskaryan@gmail.com>	10 months ago
Govinda Totla	62a3473ac0	docs[patch]: add text_splitter.py test (#14025 ) Description: Add HTMLHeaderTextSplitter unit test Dependencies: none	10 months ago
Bagatur	7d5341dbd3	docs[patch]: add contribs to readme (#14137 )	10 months ago
axiangcoding	1b36ddf16c	docs[patch]: add deprecated note for ErnieChatBot (#14061 ) - Description: just a little change of ErnieChatBot class description, sugguesting user to use more suitable class - Issue: none, - Dependencies: none, - Tag maintainer: @baskaryan , - Twitter handle: none	10 months ago
Alex Kira	1757258b2a	docs[patch]: Add mermaid JS theme dependency to docusaurus (#14051 ) - Description: Add mermaid JS dependency and configs to documentation. Allows inline doc diagrams in markdown. - Dependencies: NPM package @docusaurus/theme-mermaid	10 months ago
Devin Dahoon Kim	32da0a4d71	langchain[patch]: use async_embed_with_retry in _aget_len_safe_embeddings (#14110 ) Description `embed_with_retry` is for sync operations and not for async operations. Use `async_embed_with_retry` for appropriate async operations. I'm using `OpenAIEmbedding(http_client=httpx.AsyncClient())` with only async operations. However, I got an error when I use `embedding.aembed_documents` because `embed_with_retry` uses sync OpenAI client with async http client.	10 months ago
lijie	371bcb7580	langchain[patch]: set maxsplit when parse python function docstring (#14121 ) Description when the desc of arg in python docstring contains ":", the `_parse_python_function_docstring` will raise ValueError: too many values to unpack (expected 2). A sample desc would be: """ Args: error_arg: this is an arg with an additional ":" symbol """ So, set `maxsplit` parameter to fix it.	10 months ago
Harrison Chase	ae646701c4	Harrison/ibm (#14133 ) Co-authored-by: Mateusz Szewczyk <139469471+MateuszOssGit@users.noreply.github.com>	10 months ago
Eugene Yurtsev	943aa01c14	Improve indexing performance for Postgres (remote database) for refresh for async API (#14132 ) This PR speeds up the indexing api on the async path by batching the uid updates in the sql record manager (which may be remote).	10 months ago
William FH	528fc76d6a	Update Prompt Format Error (#14044 ) The number of times I try to format a string (especially in lcel) is embarrassingly high. Think this may be more actionable than the default error message. Now I get nice helpful errors ``` KeyError: "Input to ChatPromptTemplate is missing variable 'input'. Expected: ['input'] Received: ['dialogue']" ```	10 months ago
William FH	71c2e184b4	[Nits] Evaluation - Some Rendering Improvements (#14097 ) - Improve rendering of aggregate results at the end - flatten reference if present	10 months ago
Bob Lin	f15859bd86	docs[patch]: Update discord.ipynb (#14099 ) ### Description Now if `example` in Message is False, it will not be displayed. Update the output in this document. ```python In [22]: m = HumanMessage(content="Text") In [23]: m Out[23]: HumanMessage(content='Text') In [24]: m = HumanMessage(content="Text", example=True) In [25]: m Out[25]: HumanMessage(content='Text', example=True) ``` ### Twitter handle [lin_bob57617](https://twitter.com/lin_bob57617)	10 months ago
Lance Martin	b07a5a9509	Template for Ollama + Multi-query retriever (#14092 )	10 months ago
Bob Lin	75312c3694	docs[patch]: Update facebook.ipynb (#14102 ) ### Description Openai version 1.0.0 and later no longer supports the usage of camel case, So [the APIs](https://github.com/openai/openai-python/blob/main/api.md#finetuning) needs to be modified. ### Twitter handle [lin_bob57617](https://twitter.com/lin_bob57617)	10 months ago
Erick Friis	a3ae8e0a41	templates[patch]: opensearch readme update (#14103 )	10 months ago
Ean Yang	ac1c8634a8	docs[patch] Update invalid guides link (#14106 )	10 months ago
Mark Scannell	9b0e46dcf0	Improve indexing performance for Postgres (remote database) for refresh (#14126 ) Description: By combining the document timestamp refresh within a single call to update(), this enables batching of multiple documents in a single SQL statement. This is important for non-local databases where tens of milliseconds has a huge impact on performance when doing document-by-document SQL statements. Issue: #11935 Dependencies: None Tag maintainer: @eyurtsev	10 months ago
Erick Friis	b161f302ff	docs[patch]: local docs build <5s (#14096 )	10 months ago
Hubert Yuan	80ed588733	docs[patch]: Update metaphor_search.ipynb (#14093 ) - Description: Touch up of the documentation page for Metaphor Search Tool integration. Removes documentation for old built-in tool wrapper. --------- Co-authored-by: Erick Friis <erick@langchain.dev>	10 months ago
Jacob Lee	3328507f11	langchain[patch], experimental[minor]: Adds OllamaFunctions wrapper (#13330 ) CC @baskaryan @hwchase17 @jmorganca Having a bit of trouble importing `langchain_experimental` from a notebook, will figure it out tomorrow ~Ah and also is blocked by #13226~ --------- Co-authored-by: Lance Martin <lance@langchain.dev> Co-authored-by: Bagatur <baskaryan@gmail.com>	10 months ago
Bagatur	4063bf144a	langchain[patch]: release 0.0.344 (#14095 )	10 months ago
Bagatur	efce352d6b	core[patch]: release 0.0.8 (#14086 )	10 months ago
Harutaka Kawamura	0d08a692a3	langchain[minor]: Migrate mlflow and databricks classes to deployments APIs. (#13699 ) ## Description Related to https://github.com/mlflow/mlflow/pull/10420. MLflow AI gateway will be deprecated and replaced by the `mlflow.deployments` module. Happy to split this PR if it's too large. ``` pip install git+https://github.com/langchain-ai/langchain.git@refs/pull/13699/merge#subdirectory=libs/langchain ``` ## Dependencies Install mlflow from https://github.com/mlflow/mlflow/pull/10420: ``` pip install git+https://github.com/mlflow/mlflow.git@refs/pull/10420/merge ``` ## Testing plan The following code works fine on local and databricks: <details><summary>Click</summary> <p> ```python """ Setup ----- mlflow deployments start-server --config-path examples/gateway/openai/config.yaml databricks secrets create-scope <scope> databricks secrets put-secret <scope> openai-api-key --string-value $OPENAI_API_KEY Run --- python /path/to/this/file.py secrets/<scope>/openai-api-key """ from langchain.chat_models import ChatMlflow, ChatDatabricks from langchain.embeddings import MlflowEmbeddings, DatabricksEmbeddings from langchain.llms import Databricks, Mlflow from langchain.schema.messages import HumanMessage from langchain.chains.loading import load_chain from mlflow.deployments import get_deploy_client import uuid import sys import tempfile from langchain.chains import LLMChain from langchain.prompts import PromptTemplate ############################### # MLflow ############################### chat = ChatMlflow( target_uri="http://127.0.0.1:5000", endpoint="chat", params={"temperature": 0.1} ) print(chat([HumanMessage(content="hello")])) embeddings = MlflowEmbeddings(target_uri="http://127.0.0.1:5000", endpoint="embeddings") print(embeddings.embed_query("hello")[:3]) print(embeddings.embed_documents(["hello", "world"])[0][:3]) llm = Mlflow( target_uri="http://127.0.0.1:5000", endpoint="completions", params={"temperature": 0.1}, ) print(llm("I am")) llm_chain = LLMChain( llm=llm, prompt=PromptTemplate( input_variables=["adjective"], template="Tell me a {adjective} joke", ), ) print(llm_chain.run(adjective="funny")) # serialization/deserialization with tempfile.TemporaryDirectory() as tmpdir: print(tmpdir) path = f"{tmpdir}/llm.yaml" llm_chain.save(path) loaded_chain = load_chain(path) print(loaded_chain("funny")) ############################### # Databricks ############################### secret = sys.argv[1] client = get_deploy_client("databricks") # External - chat name = f"chat-{uuid.uuid4()}" client.create_endpoint( name=name, config={ "served_entities": [ { "name": "test", "external_model": { "name": "gpt-4", "provider": "openai", "task": "llm/v1/chat", "openai_config": { "openai_api_key": "{{" + secret + "}}", }, }, } ], }, ) try: chat = ChatDatabricks( target_uri="databricks", endpoint=name, params={"temperature": 0.1} ) print(chat([HumanMessage(content="hello")])) finally: client.delete_endpoint(endpoint=name) # External - embeddings name = f"embeddings-{uuid.uuid4()}" client.create_endpoint( name=name, config={ "served_entities": [ { "name": "test", "external_model": { "name": "text-embedding-ada-002", "provider": "openai", "task": "llm/v1/embeddings", "openai_config": { "openai_api_key": "{{" + secret + "}}", }, }, } ], }, ) try: embeddings = DatabricksEmbeddings(target_uri="databricks", endpoint=name) print(embeddings.embed_query("hello")[:3]) print(embeddings.embed_documents(["hello", "world"])[0][:3]) finally: client.delete_endpoint(endpoint=name) # External - completions name = f"completions-{uuid.uuid4()}" client.create_endpoint( name=name, config={ "served_entities": [ { "name": "test", "external_model": { "name": "gpt-3.5-turbo-instruct", "provider": "openai", "task": "llm/v1/completions", "openai_config": { "openai_api_key": "{{" + secret + "}}", }, }, } ], }, ) try: llm = Databricks( endpoint_name=name, model_kwargs={"temperature": 0.1}, ) print(llm("I am")) finally: client.delete_endpoint(endpoint=name) # Foundation model - chat chat = ChatDatabricks( endpoint="databricks-llama-2-70b-chat", params={"temperature": 0.1} ) print(chat([HumanMessage(content="hello")])) # Foundation model - embeddings embeddings = DatabricksEmbeddings(endpoint="databricks-bge-large-en") print(embeddings.embed_query("hello")[:3]) # Foundation model - completions llm = Databricks( endpoint_name="databricks-mpt-7b-instruct", model_kwargs={"temperature": 0.1} ) print(llm("hello")) llm_chain = LLMChain( llm=llm, prompt=PromptTemplate( input_variables=["adjective"], template="Tell me a {adjective} joke", ), ) print(llm_chain.run(adjective="funny")) # serialization/deserialization with tempfile.TemporaryDirectory() as tmpdir: print(tmpdir) path = f"{tmpdir}/llm.yaml" llm_chain.save(path) loaded_chain = load_chain(path) print(loaded_chain("funny")) ``` Output: ``` content='Hello! How can I assist you today?' [-0.025058426, -0.01938856, -0.027781019] [-0.025058426, -0.01938856, -0.027781019] sorry, but I cannot continue the sentence as it is incomplete. Can you please provide more information or context? Sure, here's a classic one for you: Why don't scientists trust atoms? Because they make up everything! /var/folders/dz/cd_nvlf14g9g__n3ph0d_0pm0000gp/T/tmpx_4no6ad {'adjective': 'funny', 'text': "Sure, here's a classic one for you:\n\nWhy don't scientists trust atoms?\n\nBecause they make up everything!"} content='Hello! How can I assist you today?' [-0.025058426, -0.01938856, -0.027781019] [-0.025058426, -0.01938856, -0.027781019] a 23 year old female and I am currently studying for my master's degree content="\nHello! It's nice to meet you. Is there something I can help you with or would you like to chat for a bit?" [0.051055908203125, 0.007221221923828125, 0.003879547119140625] [0.051055908203125, 0.007221221923828125, 0.003879547119140625] hello back Well, I don't really know many jokes, but I do know this funny story... /var/folders/dz/cd_nvlf14g9g__n3ph0d_0pm0000gp/T/tmp7_ds72ex {'adjective': 'funny', 'text': " Well, I don't really know many jokes, but I do know this funny story..."} ``` </p> </details> The existing workflow doesn't break: <details><summary>click</summary> <p> ```python import uuid import mlflow from mlflow.models import ModelSignature from mlflow.types.schema import ColSpec, Schema class MyModel(mlflow.pyfunc.PythonModel): def predict(self, context, model_input): return str(uuid.uuid4()) with mlflow.start_run(): mlflow.pyfunc.log_model( "model", python_model=MyModel(), pip_requirements=["mlflow==2.8.1", "cloudpickle<3"], signature=ModelSignature( inputs=Schema( [ ColSpec("string", "prompt"), ColSpec("string", "stop"), ] ), outputs=Schema( [ ColSpec(name=None, type="string"), ] ), ), registered_model_name=f"lang-{uuid.uuid4()}", ) # Manually create a serving endpoint with the registered model and run from langchain.llms import Databricks llm = Databricks(endpoint_name="<name>") llm("hello") # 9d0b2491-3d13-487c-bc02-1287f06ecae7 ``` </p> </details> ## Follow-up tasks (This PR is too large. I'll file a separate one for follow-up tasks.) - Update `docs/docs/integrations/providers/mlflow_ai_gateway.mdx` and `docs/docs/integrations/providers/databricks.md`. --------- Signed-off-by: harupy <17039389+harupy@users.noreply.github.com> Co-authored-by: Bagatur <baskaryan@gmail.com>	10 months ago
Tyler Hutcherson	dc31714ec5	templates[patch]: Rag redis template dependency update (#13614 ) - Description: Update RAG Redis template readme and dependencies.	10 months ago

1 2 3 4 5 ...

6181 Commits (3833882ab74ea82f9ba7c5de21ffdd225646a591) All Branches Search

6181 Commits (3833882ab74ea82f9ba7c5de21ffdd225646a591)

All Branches