You cannot select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

Go to file

Eugene Yurtsev 7dd6b32991 core[minor]: Add InMemoryRateLimiter (#21992 ) This PR introduces the following Runnables: 1. BaseRateLimiter: an abstraction for specifying a time based rate limiter as a Runnable 2. InMemoryRateLimiter: Provides an in-memory implementation of a rate limiter ## Example ```python from langchain_core.runnables import InMemoryRateLimiter, RunnableLambda from datetime import datetime foo = InMemoryRateLimiter(requests_per_second=0.5) def meow(x): print(datetime.now().strftime("%H:%M:%S.%f")) return x chain = foo \| meow for _ in range(10): print(chain.invoke('hello')) ``` Produces: ``` 17:12:07.530151 hello 17:12:09.537932 hello 17:12:11.548375 hello 17:12:13.558383 hello 17:12:15.568348 hello 17:12:17.578171 hello 17:12:19.587508 hello 17:12:21.597877 hello 17:12:23.607707 hello 17:12:25.617978 hello ``` ![image](https://github.com/user-attachments/assets/283af59f-e1e1-408b-8e75-d3910c3c44cc) ## Interface The rate limiter uses the following interface for acquiring a token: ```python class BaseRateLimiter(Runnable[Input, Output], abc.ABC): @abc.abstractmethod def acquire(self, *, blocking: bool = True) -> bool: """Attempt to acquire the necessary tokens for the rate limiter.``` ``` The flag `blocking` has been added to the abstraction to allow supporting streaming (which is easier if blocking=False). ## Limitations - The rate limiter is not designed to work across different processes. It is an in-memory rate limiter, but it is thread safe. - The rate limiter only supports time-based rate limiting. It does not take into account the size of the request or any other factors. - The current implementation does not handle streaming inputs well and will consume all inputs even if the rate limit has been reached. Better support for streaming inputs will be added in the future. - When the rate limiter is combined with another runnable via a RunnableSequence, usage of .batch() or .abatch() will only respect the average rate limit. There will be bursty behavior as .batch() and .abatch() wait for each step to complete before starting the next step. One way to mitigate this is to use batch_as_completed() or abatch_as_completed(). ## Bursty behavior in `batch` and `abatch` When the rate limiter is combined with another runnable via a RunnableSequence, usage of .batch() or .abatch() will only respect the average rate limit. There will be bursty behavior as .batch() and .abatch() wait for each step to complete before starting the next step. This becomes a problem if users are using `batch` and `abatch` with many inputs (e.g., 100). In this case, there will be a burst of 100 inputs into the batch of the rate limited runnable. 1. Using a RunnableBinding The API would look like: ```python from langchain_core.runnables import InMemoryRateLimiter, RunnableLambda rate_limiter = InMemoryRateLimiter(requests_per_second=0.5) def meow(x): return x rate_limited_meow = RunnableLambda(meow).with_rate_limiter(rate_limiter) ``` 2. Another option is to add some init option to RunnableSequence that changes `.batch()` to be depth first (e.g., by delegating to `batch_as_completed`) ```python RunnableSequence(first=rate_limiter, last=model, how='batch-depth-first') ``` Pros: Does not require Runnable Binding Cons: Feels over-complicated		2 days ago
.devcontainer	community[minor]: Add ApertureDB as a vectorstore (#24088 )	1 week ago
.github	infra: add unstructured api key to release (#24638 )	2 days ago
cookbook	Cookbook: Add Visual RAG example using VDMS (#24353 )	5 days ago
docker	community[minor]: Add VDMS vectorstore (#19551 )	4 months ago
docs	community[minor]: Add ScrapingAnt Loader Community Integration (#24514 )	2 days ago
libs	core[minor]: Add InMemoryRateLimiter (#21992 )	2 days ago
scripts	infra: update mypy 1.10, ruff 0.5 (#23721 )	3 weeks ago
templates	community[patch]: deprecate langchain_community Chroma in favor of langchain_chroma (#24474 )	5 days ago
.gitattributes	Update dev container (#6189 )	1 year ago
.gitignore	community[minor]: Add support for metadata indexing policy in Cassandra vector store (#22548 )	2 months ago
.readthedocs.yaml	infra: update rtd yaml (#17502 )	5 months ago
CITATION.cff	rename repo namespace to langchain-ai (#11259 )	10 months ago
LICENSE	Library Licenses (#13300 )	8 months ago
MIGRATE.md	Update main readme (#13298 )	9 months ago
Makefile	docs: revamp ChatOpenAI (#22253 )	2 months ago
README.md	docs: rm discord (#23985 )	3 weeks ago
SECURITY.md	Updated security policy (#19089 )	4 months ago
poetry.lock	core: release 0.2.14, remove poetry 1.7 incompatible flag from root (#24137 )	2 weeks ago
poetry.toml	multiple: use modern installer in poetry (#23998 )	3 weeks ago
pyproject.toml	core: release 0.2.14, remove poetry 1.7 incompatible flag from root (#24137 )	2 weeks ago

README.md

Unescape Escape

🦜️🔗 LangChain

⚡ Build context-aware reasoning applications ⚡

Looking for the JS/TS library? Check out LangChain.js.

To help you ship LangChain apps to production faster, check out LangSmith. LangSmith is a unified developer platform for building, testing, and monitoring LLM applications. Fill out this form to speak with our sales team.

Quick Install

With pip:

pip install langchain

With conda:

conda install langchain -c conda-forge

🤔 What is LangChain?

LangChain is a framework for developing applications powered by large language models (LLMs).

For these applications, LangChain simplifies the entire application lifecycle:

Open-source libraries: Build your applications using LangChain's open-source building blocks, components, and third-party integrations. Use LangGraph to build stateful agents with first-class streaming and human-in-the-loop support.
Productionization: Inspect, monitor, and evaluate your apps with LangSmith so that you can constantly optimize and deploy with confidence.
Deployment: Turn your LangGraph applications into production-ready APIs and Assistants with LangGraph Cloud.

Open-source libraries

langchain-core: Base abstractions and LangChain Expression Language.
langchain-community: Third party integrations.
- Some integrations have been further split into partner packages that only rely on langchain-core. Examples include langchain_openai and langchain_anthropic.
langchain: Chains, agents, and retrieval strategies that make up an application's cognitive architecture.
LangGraph: A library for building robust and stateful multi-actor applications with LLMs by modeling steps as edges and nodes in a graph. Integrates smoothly with LangChain, but can be used without it.

Productionization:

LangSmith: A developer platform that lets you debug, test, evaluate, and monitor chains built on any LLM framework and seamlessly integrates with LangChain.

Deployment:

LangGraph Cloud: Turn your LangGraph applications into production-ready APIs and Assistants.

🧱 What can you build with LangChain?

❓ Question answering with RAG

Documentation
End-to-end Example: Chat LangChain and repo

🧱 Extracting structured output

Documentation
End-to-end Example: SQL Llama2 Template

🤖 Chatbots

Documentation
End-to-end Example: Web LangChain (web researcher chatbot) and repo

And much more! Head to the Tutorials section of the docs for more.

🚀 How does LangChain help?

The main value props of the LangChain libraries are:

Components: composable building blocks, tools and integrations for working with language models. Components are modular and easy-to-use, whether you are using the rest of the LangChain framework or not
Off-the-shelf chains: built-in assemblages of components for accomplishing higher-level tasks

Off-the-shelf chains make it easy to get started. Components make it easy to customize existing chains and build new ones.

LangChain Expression Language (LCEL)

LCEL is the foundation of many of LangChain's components, and is a declarative way to compose chains. LCEL was designed from day 1 to support putting prototypes in production, with no code changes, from the simplest “prompt + LLM” chain to the most complex chains.

Overview: LCEL and its benefits
Interface: The standard Runnable interface for LCEL objects
Primitives: More on the primitives LCEL includes
Cheatsheet: Quick overview of the most common usage patterns

Components

Components fall into the following modules:

📃 Model I/O

This includes prompt management, prompt optimization, a generic interface for chat models and LLMs, and common utilities for working with model outputs.

📚 Retrieval

Retrieval Augmented Generation involves loading data from a variety of sources, preparing it, then searching over (a.k.a. retrieving from) it for use in the generation step.

🤖 Agents

Agents allow an LLM autonomy over how a task is accomplished. Agents make decisions about which Actions to take, then take that Action, observe the result, and repeat until the task is complete. LangChain provides a standard interface for agents, along with LangGraph for building custom agents.

📖 Documentation

Please see here for full documentation, which includes:

Introduction: Overview of the framework and the structure of the docs.
Tutorials: If you're looking to build something specific or are more of a hands-on learner, check out our tutorials. This is the best place to get started.
How-to guides: Answers to “How do I….?” type questions. These guides are goal-oriented and concrete; they're meant to help you complete a specific task.
Conceptual guide: Conceptual explanations of the key parts of the framework.
API Reference: Thorough documentation of every class and method.

🌐 Ecosystem

🦜🛠️ LangSmith: Trace and evaluate your language model applications and intelligent agents to help you move from prototype to production.
🦜🕸️ LangGraph: Create stateful, multi-actor applications with LLMs. Integrates smoothly with LangChain, but can be used without it.
🦜🏓 LangServe: Deploy LangChain runnables and chains as REST APIs.

💁 Contributing

As an open-source project in a rapidly developing field, we are extremely open to contributions, whether it be in the form of a new feature, improved infrastructure, or better documentation.

For detailed information on how to contribute, see here.

README.md Unescape Escape