mirror of https://github.com/hwchase17/langchain
core[minor]: RFC Add astream_events to Runnables (#16172)
This PR adds `astream_events` method to Runnables to make it easier to
stream data from arbitrary chains.
* Streaming only works properly in async right now
* One should use `astream()` with if mixing in imperative code as might
be done with tool implementations
* Astream_log has been modified with minimal additive changes, so no
breaking changes are expected
* Underlying callback code / tracing code should be refactored at some
point to handle things more consistently (OK for now)
- ~~[ ] verify event for on_retry~~ does not work until we implement
streaming for retry
- ~~[ ] Any rrenaming? Should we rename "event" to "hook"?~~
- [ ] Any other feedback from community?
- [x] throw NotImplementedError for `RunnableEach` for now
## Example
See this [Example
Notebook](dbbc7fa0d6/docs/docs/modules/agents/how_to/streaming_events.ipynb)
for an example with streaming in the context of an Agent
## Event Hooks Reference
Here is a reference table that shows some events that might be emitted
by the various Runnable objects.
Definitions for some of the Runnable are included after the table.
| event | name | chunk | input | output |
|----------------------|------------------|---------------------------------|-----------------------------------------------|-------------------------------------------------|
| on_chat_model_start | [model name] | | {"messages": [[SystemMessage,
HumanMessage]]} | |
| on_chat_model_stream | [model name] | AIMessageChunk(content="hello")
| | |
| on_chat_model_end | [model name] | | {"messages": [[SystemMessage,
HumanMessage]]} | {"generations": [...], "llm_output": None, ...} |
| on_llm_start | [model name] | | {'input': 'hello'} | |
| on_llm_stream | [model name] | 'Hello' | | |
| on_llm_end | [model name] | | 'Hello human!' |
| on_chain_start | format_docs | | | |
| on_chain_stream | format_docs | "hello world!, goodbye world!" | | |
| on_chain_end | format_docs | | [Document(...)] | "hello world!,
goodbye world!" |
| on_tool_start | some_tool | | {"x": 1, "y": "2"} | |
| on_tool_stream | some_tool | {"x": 1, "y": "2"} | | |
| on_tool_end | some_tool | | | {"x": 1, "y": "2"} |
| on_retriever_start | [retriever name] | | {"query": "hello"} | |
| on_retriever_chunk | [retriever name] | {documents: [...]} | | |
| on_retriever_end | [retriever name] | | {"query": "hello"} |
{documents: [...]} |
| on_prompt_start | [template_name] | | {"question": "hello"} | |
| on_prompt_end | [template_name] | | {"question": "hello"} |
ChatPromptValue(messages: [SystemMessage, ...]) |
Here are declarations associated with the events shown above:
`format_docs`:
```python
def format_docs(docs: List[Document]) -> str:
'''Format the docs.'''
return ", ".join([doc.page_content for doc in docs])
format_docs = RunnableLambda(format_docs)
```
`some_tool`:
```python
@tool
def some_tool(x: int, y: str) -> dict:
'''Some_tool.'''
return {"x": x, "y": y}
```
`prompt`:
```python
template = ChatPromptTemplate.from_messages(
[("system", "You are Cat Agent 007"), ("human", "{question}")]
).with_config({"run_name": "my_template", "tags": ["my_template"]})
```
pull/16003/head^2
Eugene Yurtsev
5 months ago
committed by
GitHub
parent
f175bf7d7b
commit
177af65dc4
@ -0,0 +1,350 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "b69e747b-4e79-4caf-8f8b-c6e70275a31d",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Event Streaming\n",
|
||||
"\n",
|
||||
"**NEW** This is a new API only works with recent versions of langchain-core!\n",
|
||||
"\n",
|
||||
"In this notebook, we'll see how to use `astream_events` to stream **token by token** from LLM calls used within the tools invoked by the agent. \n",
|
||||
"\n",
|
||||
"We will **only** stream tokens from LLMs used within tools and from no other LLMs (just to show that we can)! \n",
|
||||
"\n",
|
||||
"Feel free to adapt this example to the needs of your application.\n",
|
||||
"\n",
|
||||
"Our agent will use the OpenAI tools API for tool invocation, and we'll provide the agent with two tools:\n",
|
||||
"\n",
|
||||
"1. `where_cat_is_hiding`: A tool that uses an LLM to tell us where the cat is hiding\n",
|
||||
"2. `tell_me_a_joke_about`: A tool that can use an LLM to tell a joke about the given topic\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"## ⚠️ Beta API ⚠️ ##\n",
|
||||
"\n",
|
||||
"Event Streaming is a **beta** API, and may change a bit based on feedback.\n",
|
||||
"\n",
|
||||
"Keep in mind the following constraints (repeated in tools section):\n",
|
||||
"\n",
|
||||
"* streaming only works properly if using `async`\n",
|
||||
"* propagate callbacks if definning custom functions / runnables\n",
|
||||
"* If creating a tool that uses an LLM, make sure to use `.astream()` on the LLM rather than `.ainvoke` to ask the LLM to stream tokens.\n",
|
||||
"\n",
|
||||
"## Event Hooks Reference\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"Here is a reference table that shows some events that might be emitted by the various Runnable objects.\n",
|
||||
"Definitions for some of the Runnable are included after the table.\n",
|
||||
"\n",
|
||||
"⚠️ When streaming the inputs for the runnable will not be available until the input stream has been entirely consumed This means that the inputs will be available at for the corresponding `end` hook rather than `start` event.\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"| event | name | chunk | input | output |\n",
|
||||
"|----------------------|------------------|---------------------------------|-----------------------------------------------|-------------------------------------------------|\n",
|
||||
"| on_chat_model_start | [model name] | | {\"messages\": [[SystemMessage, HumanMessage]]} | |\n",
|
||||
"| on_chat_model_stream | [model name] | AIMessageChunk(content=\"hello\") | | |\n",
|
||||
"| on_chat_model_end | [model name] | | {\"messages\": [[SystemMessage, HumanMessage]]} | {\"generations\": [...], \"llm_output\": None, ...} |\n",
|
||||
"| on_llm_start | [model name] | | {'input': 'hello'} | |\n",
|
||||
"| on_llm_stream | [model name] | 'Hello' | | |\n",
|
||||
"| on_llm_end | [model name] | | 'Hello human!' |\n",
|
||||
"| on_chain_start | format_docs | | | |\n",
|
||||
"| on_chain_stream | format_docs | \"hello world!, goodbye world!\" | | |\n",
|
||||
"| on_chain_end | format_docs | | [Document(...)] | \"hello world!, goodbye world!\" |\n",
|
||||
"| on_tool_start | some_tool | | {\"x\": 1, \"y\": \"2\"} | |\n",
|
||||
"| on_tool_stream | some_tool | {\"x\": 1, \"y\": \"2\"} | | |\n",
|
||||
"| on_tool_end | some_tool | | | {\"x\": 1, \"y\": \"2\"} |\n",
|
||||
"| on_retriever_start | [retriever name] | | {\"query\": \"hello\"} | |\n",
|
||||
"| on_retriever_chunk | [retriever name] | {documents: [...]} | | |\n",
|
||||
"| on_retriever_end | [retriever name] | | {\"query\": \"hello\"} | {documents: [...]} |\n",
|
||||
"| on_prompt_start | [template_name] | | {\"question\": \"hello\"} | |\n",
|
||||
"| on_prompt_end | [template_name] | | {\"question\": \"hello\"} | ChatPromptValue(messages: [SystemMessage, ...]) |\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"Here are declarations associated with the events shown above:\n",
|
||||
"\n",
|
||||
"`format_docs`:\n",
|
||||
"\n",
|
||||
"```python\n",
|
||||
"def format_docs(docs: List[Document]) -> str:\n",
|
||||
" '''Format the docs.'''\n",
|
||||
" return \", \".join([doc.page_content for doc in docs])\n",
|
||||
"\n",
|
||||
"format_docs = RunnableLambda(format_docs)\n",
|
||||
"```\n",
|
||||
"\n",
|
||||
"`some_tool`:\n",
|
||||
"\n",
|
||||
"```python\n",
|
||||
"@tool\n",
|
||||
"def some_tool(x: int, y: str) -> dict:\n",
|
||||
" '''Some_tool.'''\n",
|
||||
" return {\"x\": x, \"y\": y}\n",
|
||||
"```\n",
|
||||
"\n",
|
||||
"`prompt`:\n",
|
||||
"\n",
|
||||
"```python\n",
|
||||
"template = ChatPromptTemplate.from_messages(\n",
|
||||
" [(\"system\", \"You are Cat Agent 007\"), (\"human\", \"{question}\")]\n",
|
||||
").with_config({\"run_name\": \"my_template\", \"tags\": [\"my_template\"]})\n",
|
||||
"```\n",
|
||||
"\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 6,
|
||||
"id": "29205bef-2288-48e9-9067-f19072277a97",
|
||||
"metadata": {
|
||||
"tags": []
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from langchain import hub\n",
|
||||
"from langchain.agents import AgentExecutor, create_openai_tools_agent\n",
|
||||
"from langchain.tools import tool\n",
|
||||
"from langchain_core.callbacks import Callbacks\n",
|
||||
"from langchain_core.prompts import ChatPromptTemplate\n",
|
||||
"from langchain_openai import ChatOpenAI"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "d6b0fafa-ce3b-489b-bf1d-d37b87f4819e",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Create the model\n",
|
||||
"\n",
|
||||
"**Attention** For older versions of langchain, we must set `streaming=True`"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 7,
|
||||
"id": "fa3c3761-a1cd-4118-8559-ea4d8857d394",
|
||||
"metadata": {
|
||||
"tags": []
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"model = ChatOpenAI(temperature=0, streaming=True)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "b76e1a3b-2983-42d9-ac12-4a0f32cd4a24",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Tools\n",
|
||||
"\n",
|
||||
"We define two tools that rely on a chat model to generate output!\n",
|
||||
"\n",
|
||||
"Please note a few different things:\n",
|
||||
"\n",
|
||||
"1. The tools are **async**\n",
|
||||
"1. The model is invoked using **.astream()** to force the output to stream\n",
|
||||
"1. For older langchain versions you should set `streaming=True` on the model!\n",
|
||||
"1. We attach tags to the model so that we can filter on said tags in our callback handler\n",
|
||||
"1. The tools accept callbacks and propagate them to the model as a runtime argument"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 8,
|
||||
"id": "c767f760-fe52-47e5-9c2a-622f03507aaf",
|
||||
"metadata": {
|
||||
"tags": []
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"@tool\n",
|
||||
"async def where_cat_is_hiding(callbacks: Callbacks) -> str: # <--- Accept callbacks\n",
|
||||
" \"\"\"Where is the cat hiding right now?\"\"\"\n",
|
||||
" chunks = [\n",
|
||||
" chunk\n",
|
||||
" async for chunk in model.astream(\n",
|
||||
" \"Give one up to three word answer about where the cat might be hiding in the house right now.\",\n",
|
||||
" {\n",
|
||||
" \"tags\": [\"tool_llm\"],\n",
|
||||
" \"callbacks\": callbacks,\n",
|
||||
" }, # <--- Propagate callbacks and assign a tag to this model\n",
|
||||
" )\n",
|
||||
" ]\n",
|
||||
" return \"\".join(chunk.content for chunk in chunks)\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"@tool\n",
|
||||
"async def tell_me_a_joke_about(\n",
|
||||
" topic: str, callbacks: Callbacks\n",
|
||||
") -> str: # <--- Accept callbacks\n",
|
||||
" \"\"\"Tell a joke about a given topic.\"\"\"\n",
|
||||
" template = ChatPromptTemplate.from_messages(\n",
|
||||
" [\n",
|
||||
" (\"system\", \"You are Cat Agent 007. You are funny and know many jokes.\"),\n",
|
||||
" (\"human\", \"Tell me a long joke about {topic}\"),\n",
|
||||
" ]\n",
|
||||
" )\n",
|
||||
" chain = template | model.with_config({\"tags\": [\"tool_llm\"]})\n",
|
||||
" chunks = [\n",
|
||||
" chunk\n",
|
||||
" async for chunk in chain.astream({\"topic\": topic}, {\"callbacks\": callbacks})\n",
|
||||
" ]\n",
|
||||
" return \"\".join(chunk.content for chunk in chunks)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "cba476f8-29da-4c2c-9134-186871caf7ae",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Initialize the Agent"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 9,
|
||||
"id": "0bab4488-bf4c-461f-b41e-5e60310fe0f2",
|
||||
"metadata": {
|
||||
"tags": []
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"input_variables=['agent_scratchpad', 'input'] input_types={'chat_history': typing.List[typing.Union[langchain_core.messages.ai.AIMessage, langchain_core.messages.human.HumanMessage, langchain_core.messages.chat.ChatMessage, langchain_core.messages.system.SystemMessage, langchain_core.messages.function.FunctionMessage, langchain_core.messages.tool.ToolMessage]], 'agent_scratchpad': typing.List[typing.Union[langchain_core.messages.ai.AIMessage, langchain_core.messages.human.HumanMessage, langchain_core.messages.chat.ChatMessage, langchain_core.messages.system.SystemMessage, langchain_core.messages.function.FunctionMessage, langchain_core.messages.tool.ToolMessage]]} messages=[SystemMessagePromptTemplate(prompt=PromptTemplate(input_variables=[], template='You are a helpful assistant')), MessagesPlaceholder(variable_name='chat_history', optional=True), HumanMessagePromptTemplate(prompt=PromptTemplate(input_variables=['input'], template='{input}')), MessagesPlaceholder(variable_name='agent_scratchpad')]\n",
|
||||
"[SystemMessagePromptTemplate(prompt=PromptTemplate(input_variables=[], template='You are a helpful assistant')), MessagesPlaceholder(variable_name='chat_history', optional=True), HumanMessagePromptTemplate(prompt=PromptTemplate(input_variables=['input'], template='{input}')), MessagesPlaceholder(variable_name='agent_scratchpad')]\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# Get the prompt to use - you can modify this!\n",
|
||||
"prompt = hub.pull(\"hwchase17/openai-tools-agent\")\n",
|
||||
"print(prompt)\n",
|
||||
"print(prompt.messages)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 10,
|
||||
"id": "1762f4e1-402a-4bfb-af26-eb5b7b8f56bd",
|
||||
"metadata": {
|
||||
"tags": []
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"tools = [tell_me_a_joke_about, where_cat_is_hiding]\n",
|
||||
"agent = create_openai_tools_agent(model.with_config({\"tags\": [\"agent\"]}), tools, prompt)\n",
|
||||
"executor = AgentExecutor(agent=agent, tools=tools)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "841271d7-1de1-41a9-9387-bb04368537f1",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Stream the output\n",
|
||||
"\n",
|
||||
"The streamed output is shown with a `|` as the delimiter between tokens. "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 11,
|
||||
"id": "a5d94bd8-4a55-4527-b21a-4245a38c7c26",
|
||||
"metadata": {
|
||||
"tags": []
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"/home/eugene/src/langchain/libs/core/langchain_core/_api/beta_decorator.py:86: LangChainBetaWarning: This API is in beta and may change in the future.\n",
|
||||
" warn_beta(\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"--\n",
|
||||
"Starting tool: where_cat_is_hiding with inputs: {}\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"|Under| the| bed|.||\n",
|
||||
"\n",
|
||||
"Ended tool: where_cat_is_hiding\n",
|
||||
"--\n",
|
||||
"Starting tool: tell_me_a_joke_about with inputs: {'topic': 'under the bed'}\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"|Sure|,| here|'s| a| long| joke| about| what|'s| hiding| under| the| bed|:\n",
|
||||
"\n",
|
||||
"|Once| upon| a| time|,| there| was| a| mis|chie|vous| little| boy| named| Tim|my|.| Tim|my| had| always| been| afraid| of| what| might| be| lurking| under| his| bed| at| night|.| Every| evening|,| he| would| ti|pt|oe| into| his| room|,| turn| off| the| lights|,| and| then| make| a| daring| leap| onto| his| bed|,| ensuring| that| nothing| could| grab| his| ankles|.\n",
|
||||
"\n",
|
||||
"|One| night|,| Tim|my|'s| parents| decided| to| play| a| prank| on| him|.| They| hid| a| remote|-controlled| toy| monster| under| his| bed|,| complete| with| glowing| eyes| and| a| grow|ling| sound| effect|.| As| Tim|my| settled| into| bed|,| his| parents| quietly| sn|uck| into| his| room|,| ready| to| give| him| the| scare| of| a| lifetime|.\n",
|
||||
"\n",
|
||||
"|Just| as| Tim|my| was| about| to| drift| off| to| sleep|,| he| heard| a| faint| grow|l| coming| from| under| his| bed|.| His| eyes| widened| with| fear|,| and| his| heart| started| racing|.| He| must|ered| up| the| courage| to| peek| under| the| bed|,| and| to| his| surprise|,| he| saw| a| pair| of| glowing| eyes| staring| back| at| him|.\n",
|
||||
"\n",
|
||||
"|Terr|ified|,| Tim|my| jumped| out| of| bed| and| ran| to| his| parents|,| screaming|,| \"|There|'s| a| monster| under| my| bed|!| Help|!\"\n",
|
||||
"\n",
|
||||
"|His| parents|,| trying| to| st|ifle| their| laughter|,| rushed| into| his| room|.| They| pretended| to| be| just| as| scared| as| Tim|my|,| and| together|,| they| brav|ely| approached| the| bed|.| Tim|my|'s| dad| grabbed| a| bro|om|stick|,| ready| to| defend| his| family| against| the| imaginary| monster|.\n",
|
||||
"\n",
|
||||
"|As| they| got| closer|,| the| \"|monster|\"| under| the| bed| started| to| move|.| Tim|my|'s| mom|,| unable| to| contain| her| laughter| any| longer|,| pressed| a| button| on| the| remote| control|,| causing| the| toy| monster| to| sc|urry| out| from| under| the| bed|.| Tim|my|'s| fear| quickly| turned| into| confusion|,| and| then| into| laughter| as| he| realized| it| was| all| just| a| prank|.\n",
|
||||
"\n",
|
||||
"|From| that| day| forward|,| Tim|my| learned| that| sometimes| the| things| we| fear| the| most| are| just| fig|ments| of| our| imagination|.| And| as| for| what|'s| hiding| under| his| bed|?| Well|,| it|'s| just| dust| b|unn|ies| and| the| occasional| missing| sock|.| Nothing| to| be| afraid| of|!\n",
|
||||
"\n",
|
||||
"|Remember|,| laughter| is| the| best| monster| repell|ent|!||\n",
|
||||
"\n",
|
||||
"Ended tool: tell_me_a_joke_about\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"async for event in executor.astream_events(\n",
|
||||
" {\"input\": \"where is the cat hiding? Tell me a joke about that location?\"},\n",
|
||||
" include_tags=[\"tool_llm\"],\n",
|
||||
" include_types=[\"tool\"],\n",
|
||||
"):\n",
|
||||
" hook = event[\"event\"]\n",
|
||||
" if hook == \"on_chat_model_stream\":\n",
|
||||
" print(event[\"data\"][\"chunk\"].content, end=\"|\")\n",
|
||||
" elif hook in {\"on_chat_model_start\", \"on_chat_model_end\"}:\n",
|
||||
" print()\n",
|
||||
" print()\n",
|
||||
" elif hook == \"on_tool_start\":\n",
|
||||
" print(\"--\")\n",
|
||||
" print(\n",
|
||||
" f\"Starting tool: {event['name']} with inputs: {event['data'].get('input')}\"\n",
|
||||
" )\n",
|
||||
" elif hook == \"on_tool_end\":\n",
|
||||
" print(f\"Ended tool: {event['name']}\")\n",
|
||||
" else:\n",
|
||||
" pass"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3 (ipykernel)",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.11.4"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
}
|
||||
@ -0,0 +1,133 @@
|
||||
"""Module contains typedefs that are used with runnables."""
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any, Dict, List
|
||||
|
||||
from typing_extensions import NotRequired, TypedDict
|
||||
|
||||
|
||||
class EventData(TypedDict, total=False):
|
||||
"""Data associated with a streaming event."""
|
||||
|
||||
input: Any
|
||||
"""The input passed to the runnable that generated the event.
|
||||
|
||||
Inputs will sometimes be available at the *START* of the runnable, and
|
||||
sometimes at the *END* of the runnable.
|
||||
|
||||
If a runnable is able to stream its inputs, then its input by definition
|
||||
won't be known until the *END* of the runnable when it has finished streaming
|
||||
its inputs.
|
||||
"""
|
||||
output: Any
|
||||
"""The output of the runnable that generated the event.
|
||||
|
||||
Outputs will only be available at the *END* of the runnable.
|
||||
|
||||
For most runnables, this field can be inferred from the `chunk` field,
|
||||
though there might be some exceptions for special cased runnables (e.g., like
|
||||
chat models), which may return more information.
|
||||
"""
|
||||
chunk: Any
|
||||
"""A streaming chunk from the output that generated the event.
|
||||
|
||||
chunks support addition in general, and adding them up should result
|
||||
in the output of the runnable that generated the event.
|
||||
"""
|
||||
|
||||
|
||||
class StreamEvent(TypedDict):
|
||||
"""A streaming event.
|
||||
|
||||
Schema of a streaming event which is produced from the astream_events method.
|
||||
|
||||
Example:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
from langchain_core.runnables import RunnableLambda
|
||||
|
||||
async def reverse(s: str) -> str:
|
||||
return s[::-1]
|
||||
|
||||
chain = RunnableLambda(func=reverse)
|
||||
|
||||
events = [event async for event in chain.astream_events("hello")]
|
||||
|
||||
# will produce the following events (run_id has been omitted for brevity):
|
||||
[
|
||||
{
|
||||
"data": {"input": "hello"},
|
||||
"event": "on_chain_start",
|
||||
"metadata": {},
|
||||
"name": "reverse",
|
||||
"tags": [],
|
||||
},
|
||||
{
|
||||
"data": {"chunk": "olleh"},
|
||||
"event": "on_chain_stream",
|
||||
"metadata": {},
|
||||
"name": "reverse",
|
||||
"tags": [],
|
||||
},
|
||||
{
|
||||
"data": {"output": "olleh"},
|
||||
"event": "on_chain_end",
|
||||
"metadata": {},
|
||||
"name": "reverse",
|
||||
"tags": [],
|
||||
},
|
||||
]
|
||||
"""
|
||||
|
||||
event: str
|
||||
"""Event names are of the format: on_[runnable_type]_(start|stream|end).
|
||||
|
||||
Runnable types are one of:
|
||||
* llm - used by non chat models
|
||||
* chat_model - used by chat models
|
||||
* prompt -- e.g., ChatPromptTemplate
|
||||
* tool -- from tools defined via @tool decorator or inheriting from Tool/BaseTool
|
||||
* chain - most Runnables are of this type
|
||||
|
||||
Further, the events are categorized as one of:
|
||||
* start - when the runnable starts
|
||||
* stream - when the runnable is streaming
|
||||
* end - when the runnable ends
|
||||
|
||||
start, stream and end are associated with slightly different `data` payload.
|
||||
|
||||
Please see the documentation for `EventData` for more details.
|
||||
"""
|
||||
name: str
|
||||
"""The name of the runnable that generated the event."""
|
||||
run_id: str
|
||||
"""An randomly generated ID to keep track of the execution of the given runnable.
|
||||
|
||||
Each child runnable that gets invoked as part of the execution of a parent runnable
|
||||
is assigned its own unique ID.
|
||||
"""
|
||||
tags: NotRequired[List[str]]
|
||||
"""Tags associated with the runnable that generated this event.
|
||||
|
||||
Tags are always inherited from parent runnables.
|
||||
|
||||
Tags can either be bound to a runnable using `.with_config({"tags": ["hello"]})`
|
||||
or passed at run time using `.astream_events(..., {"tags": ["hello"]})`.
|
||||
"""
|
||||
metadata: NotRequired[Dict[str, Any]]
|
||||
"""Metadata associated with the runnable that generated this event.
|
||||
|
||||
Metadata can either be bound to a runnable using
|
||||
|
||||
`.with_config({"metadata": { "foo": "bar" }})`
|
||||
|
||||
or passed at run time using
|
||||
|
||||
`.astream_events(..., {"metadata": {"foo": "bar"}})`.
|
||||
"""
|
||||
data: EventData
|
||||
"""Event data.
|
||||
|
||||
The contents of the event data depend on the event type.
|
||||
"""
|
||||
File diff suppressed because it is too large
Load Diff
Loading…
Reference in New Issue