mirror of https://github.com/hwchase17/langchain
You cannot select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
1161 lines
42 KiB
Plaintext
1161 lines
42 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "raw",
|
|
"id": "366a0e68-fd67-4fe5-a292-5c33733339ea",
|
|
"metadata": {},
|
|
"source": [
|
|
"---\n",
|
|
"sidebar_position: 1\n",
|
|
"title: Interface\n",
|
|
"---"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "9a9acd2e",
|
|
"metadata": {},
|
|
"source": [
|
|
"To make it as easy as possible to create custom chains, we've implemented a [\"Runnable\"](https://api.python.langchain.com/en/stable/runnables/langchain_core.runnables.base.Runnable.html#langchain_core.runnables.base.Runnable) protocol. The `Runnable` protocol is implemented for most components. \n",
|
|
"This is a standard interface, which makes it easy to define custom chains as well as invoke them in a standard way. \n",
|
|
"The standard interface includes:\n",
|
|
"\n",
|
|
"- [`stream`](#stream): stream back chunks of the response\n",
|
|
"- [`invoke`](#invoke): call the chain on an input\n",
|
|
"- [`batch`](#batch): call the chain on a list of inputs\n",
|
|
"\n",
|
|
"These also have corresponding async methods:\n",
|
|
"\n",
|
|
"- [`astream`](#async-stream): stream back chunks of the response async\n",
|
|
"- [`ainvoke`](#async-invoke): call the chain on an input async\n",
|
|
"- [`abatch`](#async-batch): call the chain on a list of inputs async\n",
|
|
"- [`astream_log`](#async-stream-intermediate-steps): stream back intermediate steps as they happen, in addition to the final response\n",
|
|
"\n",
|
|
"The **input type** and **output type** varies by component:\n",
|
|
"\n",
|
|
"| Component | Input Type | Output Type |\n",
|
|
"| --- | --- | --- |\n",
|
|
"| Prompt | Dictionary | PromptValue |\n",
|
|
"| ChatModel | Single string, list of chat messages or a PromptValue | ChatMessage |\n",
|
|
"| LLM | Single string, list of chat messages or a PromptValue | String |\n",
|
|
"| OutputParser | The output of an LLM or ChatModel | Depends on the parser |\n",
|
|
"| Retriever | Single string | List of Documents |\n",
|
|
"| Tool | Single string or dictionary, depending on the tool | Depends on the tool |\n",
|
|
"\n",
|
|
"\n",
|
|
"All runnables expose input and output **schemas** to inspect the inputs and outputs:\n",
|
|
"- [`input_schema`](#input-schema): an input Pydantic model auto-generated from the structure of the Runnable\n",
|
|
"- [`output_schema`](#output-schema): an output Pydantic model auto-generated from the structure of the Runnable\n",
|
|
"\n",
|
|
"Let's take a look at these methods. To do so, we'll create a super simple PromptTemplate + ChatModel chain."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 1,
|
|
"id": "466b65b3",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"from langchain_core.prompts import ChatPromptTemplate\n",
|
|
"from langchain_openai import ChatOpenAI\n",
|
|
"\n",
|
|
"model = ChatOpenAI()\n",
|
|
"prompt = ChatPromptTemplate.from_template(\"tell me a joke about {topic}\")\n",
|
|
"chain = prompt | model"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "5cccdf0b-2d89-4f74-9530-bf499610e9a5",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Input Schema\n",
|
|
"\n",
|
|
"A description of the inputs accepted by a Runnable.\n",
|
|
"This is a Pydantic model dynamically generated from the structure of any Runnable.\n",
|
|
"You can call `.schema()` on it to obtain a JSONSchema representation."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 13,
|
|
"id": "25e146d4-60da-40a2-9026-b5dfee106a3f",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"{'title': 'PromptInput',\n",
|
|
" 'type': 'object',\n",
|
|
" 'properties': {'topic': {'title': 'Topic', 'type': 'string'}}}"
|
|
]
|
|
},
|
|
"execution_count": 13,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"# The input schema of the chain is the input schema of its first part, the prompt.\n",
|
|
"chain.input_schema.schema()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 16,
|
|
"id": "ad130546-4c14-4f6c-95af-c56ea19b12ac",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"{'title': 'PromptInput',\n",
|
|
" 'type': 'object',\n",
|
|
" 'properties': {'topic': {'title': 'Topic', 'type': 'string'}}}"
|
|
]
|
|
},
|
|
"execution_count": 16,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"prompt.input_schema.schema()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 15,
|
|
"id": "49d34744-d6db-4fdf-a0d6-261522b7f251",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"{'title': 'ChatOpenAIInput',\n",
|
|
" 'anyOf': [{'type': 'string'},\n",
|
|
" {'$ref': '#/definitions/StringPromptValue'},\n",
|
|
" {'$ref': '#/definitions/ChatPromptValueConcrete'},\n",
|
|
" {'type': 'array',\n",
|
|
" 'items': {'anyOf': [{'$ref': '#/definitions/AIMessage'},\n",
|
|
" {'$ref': '#/definitions/HumanMessage'},\n",
|
|
" {'$ref': '#/definitions/ChatMessage'},\n",
|
|
" {'$ref': '#/definitions/SystemMessage'},\n",
|
|
" {'$ref': '#/definitions/FunctionMessage'}]}}],\n",
|
|
" 'definitions': {'StringPromptValue': {'title': 'StringPromptValue',\n",
|
|
" 'description': 'String prompt value.',\n",
|
|
" 'type': 'object',\n",
|
|
" 'properties': {'text': {'title': 'Text', 'type': 'string'},\n",
|
|
" 'type': {'title': 'Type',\n",
|
|
" 'default': 'StringPromptValue',\n",
|
|
" 'enum': ['StringPromptValue'],\n",
|
|
" 'type': 'string'}},\n",
|
|
" 'required': ['text']},\n",
|
|
" 'AIMessage': {'title': 'AIMessage',\n",
|
|
" 'description': 'A Message from an AI.',\n",
|
|
" 'type': 'object',\n",
|
|
" 'properties': {'content': {'title': 'Content', 'type': 'string'},\n",
|
|
" 'additional_kwargs': {'title': 'Additional Kwargs', 'type': 'object'},\n",
|
|
" 'type': {'title': 'Type',\n",
|
|
" 'default': 'ai',\n",
|
|
" 'enum': ['ai'],\n",
|
|
" 'type': 'string'},\n",
|
|
" 'example': {'title': 'Example', 'default': False, 'type': 'boolean'}},\n",
|
|
" 'required': ['content']},\n",
|
|
" 'HumanMessage': {'title': 'HumanMessage',\n",
|
|
" 'description': 'A Message from a human.',\n",
|
|
" 'type': 'object',\n",
|
|
" 'properties': {'content': {'title': 'Content', 'type': 'string'},\n",
|
|
" 'additional_kwargs': {'title': 'Additional Kwargs', 'type': 'object'},\n",
|
|
" 'type': {'title': 'Type',\n",
|
|
" 'default': 'human',\n",
|
|
" 'enum': ['human'],\n",
|
|
" 'type': 'string'},\n",
|
|
" 'example': {'title': 'Example', 'default': False, 'type': 'boolean'}},\n",
|
|
" 'required': ['content']},\n",
|
|
" 'ChatMessage': {'title': 'ChatMessage',\n",
|
|
" 'description': 'A Message that can be assigned an arbitrary speaker (i.e. role).',\n",
|
|
" 'type': 'object',\n",
|
|
" 'properties': {'content': {'title': 'Content', 'type': 'string'},\n",
|
|
" 'additional_kwargs': {'title': 'Additional Kwargs', 'type': 'object'},\n",
|
|
" 'type': {'title': 'Type',\n",
|
|
" 'default': 'chat',\n",
|
|
" 'enum': ['chat'],\n",
|
|
" 'type': 'string'},\n",
|
|
" 'role': {'title': 'Role', 'type': 'string'}},\n",
|
|
" 'required': ['content', 'role']},\n",
|
|
" 'SystemMessage': {'title': 'SystemMessage',\n",
|
|
" 'description': 'A Message for priming AI behavior, usually passed in as the first of a sequence\\nof input messages.',\n",
|
|
" 'type': 'object',\n",
|
|
" 'properties': {'content': {'title': 'Content', 'type': 'string'},\n",
|
|
" 'additional_kwargs': {'title': 'Additional Kwargs', 'type': 'object'},\n",
|
|
" 'type': {'title': 'Type',\n",
|
|
" 'default': 'system',\n",
|
|
" 'enum': ['system'],\n",
|
|
" 'type': 'string'}},\n",
|
|
" 'required': ['content']},\n",
|
|
" 'FunctionMessage': {'title': 'FunctionMessage',\n",
|
|
" 'description': 'A Message for passing the result of executing a function back to a model.',\n",
|
|
" 'type': 'object',\n",
|
|
" 'properties': {'content': {'title': 'Content', 'type': 'string'},\n",
|
|
" 'additional_kwargs': {'title': 'Additional Kwargs', 'type': 'object'},\n",
|
|
" 'type': {'title': 'Type',\n",
|
|
" 'default': 'function',\n",
|
|
" 'enum': ['function'],\n",
|
|
" 'type': 'string'},\n",
|
|
" 'name': {'title': 'Name', 'type': 'string'}},\n",
|
|
" 'required': ['content', 'name']},\n",
|
|
" 'ChatPromptValueConcrete': {'title': 'ChatPromptValueConcrete',\n",
|
|
" 'description': 'Chat prompt value which explicitly lists out the message types it accepts.\\nFor use in external schemas.',\n",
|
|
" 'type': 'object',\n",
|
|
" 'properties': {'messages': {'title': 'Messages',\n",
|
|
" 'type': 'array',\n",
|
|
" 'items': {'anyOf': [{'$ref': '#/definitions/AIMessage'},\n",
|
|
" {'$ref': '#/definitions/HumanMessage'},\n",
|
|
" {'$ref': '#/definitions/ChatMessage'},\n",
|
|
" {'$ref': '#/definitions/SystemMessage'},\n",
|
|
" {'$ref': '#/definitions/FunctionMessage'}]}},\n",
|
|
" 'type': {'title': 'Type',\n",
|
|
" 'default': 'ChatPromptValueConcrete',\n",
|
|
" 'enum': ['ChatPromptValueConcrete'],\n",
|
|
" 'type': 'string'}},\n",
|
|
" 'required': ['messages']}}}"
|
|
]
|
|
},
|
|
"execution_count": 15,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"model.input_schema.schema()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "5059a5dc-d544-4add-85bd-78a3f2b78b9a",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Output Schema\n",
|
|
"\n",
|
|
"A description of the outputs produced by a Runnable.\n",
|
|
"This is a Pydantic model dynamically generated from the structure of any Runnable.\n",
|
|
"You can call `.schema()` on it to obtain a JSONSchema representation."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 17,
|
|
"id": "a0e41fd3-77d8-4911-af6a-d4d3aad5f77b",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"{'title': 'ChatOpenAIOutput',\n",
|
|
" 'anyOf': [{'$ref': '#/definitions/HumanMessage'},\n",
|
|
" {'$ref': '#/definitions/AIMessage'},\n",
|
|
" {'$ref': '#/definitions/ChatMessage'},\n",
|
|
" {'$ref': '#/definitions/FunctionMessage'},\n",
|
|
" {'$ref': '#/definitions/SystemMessage'}],\n",
|
|
" 'definitions': {'HumanMessage': {'title': 'HumanMessage',\n",
|
|
" 'description': 'A Message from a human.',\n",
|
|
" 'type': 'object',\n",
|
|
" 'properties': {'content': {'title': 'Content', 'type': 'string'},\n",
|
|
" 'additional_kwargs': {'title': 'Additional Kwargs', 'type': 'object'},\n",
|
|
" 'type': {'title': 'Type',\n",
|
|
" 'default': 'human',\n",
|
|
" 'enum': ['human'],\n",
|
|
" 'type': 'string'},\n",
|
|
" 'example': {'title': 'Example', 'default': False, 'type': 'boolean'}},\n",
|
|
" 'required': ['content']},\n",
|
|
" 'AIMessage': {'title': 'AIMessage',\n",
|
|
" 'description': 'A Message from an AI.',\n",
|
|
" 'type': 'object',\n",
|
|
" 'properties': {'content': {'title': 'Content', 'type': 'string'},\n",
|
|
" 'additional_kwargs': {'title': 'Additional Kwargs', 'type': 'object'},\n",
|
|
" 'type': {'title': 'Type',\n",
|
|
" 'default': 'ai',\n",
|
|
" 'enum': ['ai'],\n",
|
|
" 'type': 'string'},\n",
|
|
" 'example': {'title': 'Example', 'default': False, 'type': 'boolean'}},\n",
|
|
" 'required': ['content']},\n",
|
|
" 'ChatMessage': {'title': 'ChatMessage',\n",
|
|
" 'description': 'A Message that can be assigned an arbitrary speaker (i.e. role).',\n",
|
|
" 'type': 'object',\n",
|
|
" 'properties': {'content': {'title': 'Content', 'type': 'string'},\n",
|
|
" 'additional_kwargs': {'title': 'Additional Kwargs', 'type': 'object'},\n",
|
|
" 'type': {'title': 'Type',\n",
|
|
" 'default': 'chat',\n",
|
|
" 'enum': ['chat'],\n",
|
|
" 'type': 'string'},\n",
|
|
" 'role': {'title': 'Role', 'type': 'string'}},\n",
|
|
" 'required': ['content', 'role']},\n",
|
|
" 'FunctionMessage': {'title': 'FunctionMessage',\n",
|
|
" 'description': 'A Message for passing the result of executing a function back to a model.',\n",
|
|
" 'type': 'object',\n",
|
|
" 'properties': {'content': {'title': 'Content', 'type': 'string'},\n",
|
|
" 'additional_kwargs': {'title': 'Additional Kwargs', 'type': 'object'},\n",
|
|
" 'type': {'title': 'Type',\n",
|
|
" 'default': 'function',\n",
|
|
" 'enum': ['function'],\n",
|
|
" 'type': 'string'},\n",
|
|
" 'name': {'title': 'Name', 'type': 'string'}},\n",
|
|
" 'required': ['content', 'name']},\n",
|
|
" 'SystemMessage': {'title': 'SystemMessage',\n",
|
|
" 'description': 'A Message for priming AI behavior, usually passed in as the first of a sequence\\nof input messages.',\n",
|
|
" 'type': 'object',\n",
|
|
" 'properties': {'content': {'title': 'Content', 'type': 'string'},\n",
|
|
" 'additional_kwargs': {'title': 'Additional Kwargs', 'type': 'object'},\n",
|
|
" 'type': {'title': 'Type',\n",
|
|
" 'default': 'system',\n",
|
|
" 'enum': ['system'],\n",
|
|
" 'type': 'string'}},\n",
|
|
" 'required': ['content']}}}"
|
|
]
|
|
},
|
|
"execution_count": 17,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"# The output schema of the chain is the output schema of its last part, in this case a ChatModel, which outputs a ChatMessage\n",
|
|
"chain.output_schema.schema()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "daf2b2b2",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Stream"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 20,
|
|
"id": "bea9639d",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Why don't bears wear shoes?\n",
|
|
"\n",
|
|
"Because they already have bear feet!"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"for s in chain.stream({\"topic\": \"bears\"}):\n",
|
|
" print(s.content, end=\"\", flush=True)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "cbf1c782",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Invoke"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 21,
|
|
"id": "470e483f",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"AIMessage(content=\"Why don't bears wear shoes?\\n\\nBecause they already have bear feet!\")"
|
|
]
|
|
},
|
|
"execution_count": 21,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"chain.invoke({\"topic\": \"bears\"})"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "88f0c279",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Batch"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 22,
|
|
"id": "9685de67",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"[AIMessage(content=\"Why don't bears wear shoes?\\n\\nBecause they have bear feet!\"),\n",
|
|
" AIMessage(content=\"Why don't cats play poker in the wild?\\n\\nToo many cheetahs!\")]"
|
|
]
|
|
},
|
|
"execution_count": 22,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"chain.batch([{\"topic\": \"bears\"}, {\"topic\": \"cats\"}])"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "2434ab15",
|
|
"metadata": {},
|
|
"source": [
|
|
"You can set the number of concurrent requests by using the `max_concurrency` parameter"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 23,
|
|
"id": "a08522f6",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"[AIMessage(content=\"Why don't bears wear shoes? \\n\\nBecause they have bear feet!\"),\n",
|
|
" AIMessage(content=\"Why don't cats play poker in the wild?\\n\\nToo many cheetahs!\")]"
|
|
]
|
|
},
|
|
"execution_count": 23,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"chain.batch([{\"topic\": \"bears\"}, {\"topic\": \"cats\"}], config={\"max_concurrency\": 5})"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "b960cbfe",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Async Stream"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 24,
|
|
"id": "ea35eee4",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Sure, here's a bear-themed joke for you:\n",
|
|
"\n",
|
|
"Why don't bears wear shoes?\n",
|
|
"\n",
|
|
"Because they already have bear feet!"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"async for s in chain.astream({\"topic\": \"bears\"}):\n",
|
|
" print(s.content, end=\"\", flush=True)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "04cb3324",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Async Invoke"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 25,
|
|
"id": "ef8c9b20",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"AIMessage(content=\"Why don't bears wear shoes? \\n\\nBecause they have bear feet!\")"
|
|
]
|
|
},
|
|
"execution_count": 25,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"await chain.ainvoke({\"topic\": \"bears\"})"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "3da288d5",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Async Batch"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 26,
|
|
"id": "eba2a103",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"[AIMessage(content=\"Why don't bears wear shoes?\\n\\nBecause they have bear feet!\")]"
|
|
]
|
|
},
|
|
"execution_count": 26,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"await chain.abatch([{\"topic\": \"bears\"}])"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "f9cef104",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Async Stream Intermediate Steps\n",
|
|
"\n",
|
|
"All runnables also have a method `.astream_log()` which is used to stream (as they happen) all or part of the intermediate steps of your chain/sequence. \n",
|
|
"\n",
|
|
"This is useful to show progress to the user, to use intermediate results, or to debug your chain.\n",
|
|
"\n",
|
|
"You can stream all steps (default) or include/exclude steps by name, tags or metadata.\n",
|
|
"\n",
|
|
"This method yields [JSONPatch](https://jsonpatch.com) ops that when applied in the same order as received build up the RunState.\n",
|
|
"\n",
|
|
"```python\n",
|
|
"class LogEntry(TypedDict):\n",
|
|
" id: str\n",
|
|
" \"\"\"ID of the sub-run.\"\"\"\n",
|
|
" name: str\n",
|
|
" \"\"\"Name of the object being run.\"\"\"\n",
|
|
" type: str\n",
|
|
" \"\"\"Type of the object being run, eg. prompt, chain, llm, etc.\"\"\"\n",
|
|
" tags: List[str]\n",
|
|
" \"\"\"List of tags for the run.\"\"\"\n",
|
|
" metadata: Dict[str, Any]\n",
|
|
" \"\"\"Key-value pairs of metadata for the run.\"\"\"\n",
|
|
" start_time: str\n",
|
|
" \"\"\"ISO-8601 timestamp of when the run started.\"\"\"\n",
|
|
"\n",
|
|
" streamed_output_str: List[str]\n",
|
|
" \"\"\"List of LLM tokens streamed by this run, if applicable.\"\"\"\n",
|
|
" final_output: Optional[Any]\n",
|
|
" \"\"\"Final output of this run.\n",
|
|
" Only available after the run has finished successfully.\"\"\"\n",
|
|
" end_time: Optional[str]\n",
|
|
" \"\"\"ISO-8601 timestamp of when the run ended.\n",
|
|
" Only available after the run has finished.\"\"\"\n",
|
|
"\n",
|
|
"\n",
|
|
"class RunState(TypedDict):\n",
|
|
" id: str\n",
|
|
" \"\"\"ID of the run.\"\"\"\n",
|
|
" streamed_output: List[Any]\n",
|
|
" \"\"\"List of output chunks streamed by Runnable.stream()\"\"\"\n",
|
|
" final_output: Optional[Any]\n",
|
|
" \"\"\"Final output of the run, usually the result of aggregating (`+`) streamed_output.\n",
|
|
" Only available after the run has finished successfully.\"\"\"\n",
|
|
"\n",
|
|
" logs: Dict[str, LogEntry]\n",
|
|
" \"\"\"Map of run names to sub-runs. If filters were supplied, this list will\n",
|
|
" contain only the runs that matched the filters.\"\"\"\n",
|
|
"```"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "a146a5df-25be-4fa2-a7e4-df8ebe55a35e",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Streaming JSONPatch chunks\n",
|
|
"\n",
|
|
"This is useful eg. to stream the `JSONPatch` in an HTTP server, and then apply the ops on the client to rebuild the run state there. See [LangServe](https://github.com/langchain-ai/langserve) for tooling to make it easier to build a webserver from any Runnable."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 29,
|
|
"id": "21c9019e",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"----------------------------------------\n",
|
|
"RunLogPatch({'op': 'replace',\n",
|
|
" 'path': '',\n",
|
|
" 'value': {'final_output': None,\n",
|
|
" 'id': 'e2f2cc72-eb63-4d20-8326-237367482efb',\n",
|
|
" 'logs': {},\n",
|
|
" 'streamed_output': []}})\n",
|
|
"----------------------------------------\n",
|
|
"RunLogPatch({'op': 'add',\n",
|
|
" 'path': '/logs/Docs',\n",
|
|
" 'value': {'end_time': None,\n",
|
|
" 'final_output': None,\n",
|
|
" 'id': '8da492cc-4492-4e74-b8b0-9e60e8693390',\n",
|
|
" 'metadata': {},\n",
|
|
" 'name': 'Docs',\n",
|
|
" 'start_time': '2023-10-19T17:50:13.526',\n",
|
|
" 'streamed_output_str': [],\n",
|
|
" 'tags': ['map:key:context', 'FAISS'],\n",
|
|
" 'type': 'retriever'}})\n",
|
|
"----------------------------------------\n",
|
|
"RunLogPatch({'op': 'add',\n",
|
|
" 'path': '/logs/Docs/final_output',\n",
|
|
" 'value': {'documents': [Document(page_content='harrison worked at kensho')]}},\n",
|
|
" {'op': 'add',\n",
|
|
" 'path': '/logs/Docs/end_time',\n",
|
|
" 'value': '2023-10-19T17:50:13.713'})\n",
|
|
"----------------------------------------\n",
|
|
"RunLogPatch({'op': 'add', 'path': '/streamed_output/-', 'value': ''})\n",
|
|
"----------------------------------------\n",
|
|
"RunLogPatch({'op': 'add', 'path': '/streamed_output/-', 'value': 'H'})\n",
|
|
"----------------------------------------\n",
|
|
"RunLogPatch({'op': 'add', 'path': '/streamed_output/-', 'value': 'arrison'})\n",
|
|
"----------------------------------------\n",
|
|
"RunLogPatch({'op': 'add', 'path': '/streamed_output/-', 'value': ' worked'})\n",
|
|
"----------------------------------------\n",
|
|
"RunLogPatch({'op': 'add', 'path': '/streamed_output/-', 'value': ' at'})\n",
|
|
"----------------------------------------\n",
|
|
"RunLogPatch({'op': 'add', 'path': '/streamed_output/-', 'value': ' Kens'})\n",
|
|
"----------------------------------------\n",
|
|
"RunLogPatch({'op': 'add', 'path': '/streamed_output/-', 'value': 'ho'})\n",
|
|
"----------------------------------------\n",
|
|
"RunLogPatch({'op': 'add', 'path': '/streamed_output/-', 'value': '.'})\n",
|
|
"----------------------------------------\n",
|
|
"RunLogPatch({'op': 'add', 'path': '/streamed_output/-', 'value': ''})\n",
|
|
"----------------------------------------\n",
|
|
"RunLogPatch({'op': 'replace',\n",
|
|
" 'path': '/final_output',\n",
|
|
" 'value': {'output': 'Harrison worked at Kensho.'}})\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"from langchain_community.vectorstores import FAISS\n",
|
|
"from langchain_core.output_parsers import StrOutputParser\n",
|
|
"from langchain_core.runnables import RunnablePassthrough\n",
|
|
"from langchain_openai import OpenAIEmbeddings\n",
|
|
"\n",
|
|
"template = \"\"\"Answer the question based only on the following context:\n",
|
|
"{context}\n",
|
|
"\n",
|
|
"Question: {question}\n",
|
|
"\"\"\"\n",
|
|
"prompt = ChatPromptTemplate.from_template(template)\n",
|
|
"\n",
|
|
"vectorstore = FAISS.from_texts(\n",
|
|
" [\"harrison worked at kensho\"], embedding=OpenAIEmbeddings()\n",
|
|
")\n",
|
|
"retriever = vectorstore.as_retriever()\n",
|
|
"\n",
|
|
"retrieval_chain = (\n",
|
|
" {\n",
|
|
" \"context\": retriever.with_config(run_name=\"Docs\"),\n",
|
|
" \"question\": RunnablePassthrough(),\n",
|
|
" }\n",
|
|
" | prompt\n",
|
|
" | model\n",
|
|
" | StrOutputParser()\n",
|
|
")\n",
|
|
"\n",
|
|
"async for chunk in retrieval_chain.astream_log(\n",
|
|
" \"where did harrison work?\", include_names=[\"Docs\"]\n",
|
|
"):\n",
|
|
" print(\"-\" * 40)\n",
|
|
" print(chunk)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "19570f36-7126-4fe2-b209-0cc6178b4582",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Streaming the incremental RunState\n",
|
|
"\n",
|
|
"You can simply pass `diff=False` to get incremental values of `RunState`. \n",
|
|
"You get more verbose output with more repetitive parts."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 31,
|
|
"id": "5c26b731-b4eb-4967-a42a-dec813249ecb",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"----------------------------------------------------------------------\n",
|
|
"RunLog({'final_output': None,\n",
|
|
" 'id': 'afe66178-d75f-4c2d-b348-b1d144239cd6',\n",
|
|
" 'logs': {},\n",
|
|
" 'streamed_output': []})\n",
|
|
"----------------------------------------------------------------------\n",
|
|
"RunLog({'final_output': None,\n",
|
|
" 'id': 'afe66178-d75f-4c2d-b348-b1d144239cd6',\n",
|
|
" 'logs': {'Docs': {'end_time': None,\n",
|
|
" 'final_output': None,\n",
|
|
" 'id': '88d51118-5756-4891-89c5-2f6a5e90cc26',\n",
|
|
" 'metadata': {},\n",
|
|
" 'name': 'Docs',\n",
|
|
" 'start_time': '2023-10-19T17:52:15.438',\n",
|
|
" 'streamed_output_str': [],\n",
|
|
" 'tags': ['map:key:context', 'FAISS'],\n",
|
|
" 'type': 'retriever'}},\n",
|
|
" 'streamed_output': []})\n",
|
|
"----------------------------------------------------------------------\n",
|
|
"RunLog({'final_output': None,\n",
|
|
" 'id': 'afe66178-d75f-4c2d-b348-b1d144239cd6',\n",
|
|
" 'logs': {'Docs': {'end_time': '2023-10-19T17:52:15.738',\n",
|
|
" 'final_output': {'documents': [Document(page_content='harrison worked at kensho')]},\n",
|
|
" 'id': '88d51118-5756-4891-89c5-2f6a5e90cc26',\n",
|
|
" 'metadata': {},\n",
|
|
" 'name': 'Docs',\n",
|
|
" 'start_time': '2023-10-19T17:52:15.438',\n",
|
|
" 'streamed_output_str': [],\n",
|
|
" 'tags': ['map:key:context', 'FAISS'],\n",
|
|
" 'type': 'retriever'}},\n",
|
|
" 'streamed_output': []})\n",
|
|
"----------------------------------------------------------------------\n",
|
|
"RunLog({'final_output': None,\n",
|
|
" 'id': 'afe66178-d75f-4c2d-b348-b1d144239cd6',\n",
|
|
" 'logs': {'Docs': {'end_time': '2023-10-19T17:52:15.738',\n",
|
|
" 'final_output': {'documents': [Document(page_content='harrison worked at kensho')]},\n",
|
|
" 'id': '88d51118-5756-4891-89c5-2f6a5e90cc26',\n",
|
|
" 'metadata': {},\n",
|
|
" 'name': 'Docs',\n",
|
|
" 'start_time': '2023-10-19T17:52:15.438',\n",
|
|
" 'streamed_output_str': [],\n",
|
|
" 'tags': ['map:key:context', 'FAISS'],\n",
|
|
" 'type': 'retriever'}},\n",
|
|
" 'streamed_output': ['']})\n",
|
|
"----------------------------------------------------------------------\n",
|
|
"RunLog({'final_output': None,\n",
|
|
" 'id': 'afe66178-d75f-4c2d-b348-b1d144239cd6',\n",
|
|
" 'logs': {'Docs': {'end_time': '2023-10-19T17:52:15.738',\n",
|
|
" 'final_output': {'documents': [Document(page_content='harrison worked at kensho')]},\n",
|
|
" 'id': '88d51118-5756-4891-89c5-2f6a5e90cc26',\n",
|
|
" 'metadata': {},\n",
|
|
" 'name': 'Docs',\n",
|
|
" 'start_time': '2023-10-19T17:52:15.438',\n",
|
|
" 'streamed_output_str': [],\n",
|
|
" 'tags': ['map:key:context', 'FAISS'],\n",
|
|
" 'type': 'retriever'}},\n",
|
|
" 'streamed_output': ['', 'H']})\n",
|
|
"----------------------------------------------------------------------\n",
|
|
"RunLog({'final_output': None,\n",
|
|
" 'id': 'afe66178-d75f-4c2d-b348-b1d144239cd6',\n",
|
|
" 'logs': {'Docs': {'end_time': '2023-10-19T17:52:15.738',\n",
|
|
" 'final_output': {'documents': [Document(page_content='harrison worked at kensho')]},\n",
|
|
" 'id': '88d51118-5756-4891-89c5-2f6a5e90cc26',\n",
|
|
" 'metadata': {},\n",
|
|
" 'name': 'Docs',\n",
|
|
" 'start_time': '2023-10-19T17:52:15.438',\n",
|
|
" 'streamed_output_str': [],\n",
|
|
" 'tags': ['map:key:context', 'FAISS'],\n",
|
|
" 'type': 'retriever'}},\n",
|
|
" 'streamed_output': ['', 'H', 'arrison']})\n",
|
|
"----------------------------------------------------------------------\n",
|
|
"RunLog({'final_output': None,\n",
|
|
" 'id': 'afe66178-d75f-4c2d-b348-b1d144239cd6',\n",
|
|
" 'logs': {'Docs': {'end_time': '2023-10-19T17:52:15.738',\n",
|
|
" 'final_output': {'documents': [Document(page_content='harrison worked at kensho')]},\n",
|
|
" 'id': '88d51118-5756-4891-89c5-2f6a5e90cc26',\n",
|
|
" 'metadata': {},\n",
|
|
" 'name': 'Docs',\n",
|
|
" 'start_time': '2023-10-19T17:52:15.438',\n",
|
|
" 'streamed_output_str': [],\n",
|
|
" 'tags': ['map:key:context', 'FAISS'],\n",
|
|
" 'type': 'retriever'}},\n",
|
|
" 'streamed_output': ['', 'H', 'arrison', ' worked']})\n",
|
|
"----------------------------------------------------------------------\n",
|
|
"RunLog({'final_output': None,\n",
|
|
" 'id': 'afe66178-d75f-4c2d-b348-b1d144239cd6',\n",
|
|
" 'logs': {'Docs': {'end_time': '2023-10-19T17:52:15.738',\n",
|
|
" 'final_output': {'documents': [Document(page_content='harrison worked at kensho')]},\n",
|
|
" 'id': '88d51118-5756-4891-89c5-2f6a5e90cc26',\n",
|
|
" 'metadata': {},\n",
|
|
" 'name': 'Docs',\n",
|
|
" 'start_time': '2023-10-19T17:52:15.438',\n",
|
|
" 'streamed_output_str': [],\n",
|
|
" 'tags': ['map:key:context', 'FAISS'],\n",
|
|
" 'type': 'retriever'}},\n",
|
|
" 'streamed_output': ['', 'H', 'arrison', ' worked', ' at']})\n",
|
|
"----------------------------------------------------------------------\n",
|
|
"RunLog({'final_output': None,\n",
|
|
" 'id': 'afe66178-d75f-4c2d-b348-b1d144239cd6',\n",
|
|
" 'logs': {'Docs': {'end_time': '2023-10-19T17:52:15.738',\n",
|
|
" 'final_output': {'documents': [Document(page_content='harrison worked at kensho')]},\n",
|
|
" 'id': '88d51118-5756-4891-89c5-2f6a5e90cc26',\n",
|
|
" 'metadata': {},\n",
|
|
" 'name': 'Docs',\n",
|
|
" 'start_time': '2023-10-19T17:52:15.438',\n",
|
|
" 'streamed_output_str': [],\n",
|
|
" 'tags': ['map:key:context', 'FAISS'],\n",
|
|
" 'type': 'retriever'}},\n",
|
|
" 'streamed_output': ['', 'H', 'arrison', ' worked', ' at', ' Kens']})\n",
|
|
"----------------------------------------------------------------------\n",
|
|
"RunLog({'final_output': None,\n",
|
|
" 'id': 'afe66178-d75f-4c2d-b348-b1d144239cd6',\n",
|
|
" 'logs': {'Docs': {'end_time': '2023-10-19T17:52:15.738',\n",
|
|
" 'final_output': {'documents': [Document(page_content='harrison worked at kensho')]},\n",
|
|
" 'id': '88d51118-5756-4891-89c5-2f6a5e90cc26',\n",
|
|
" 'metadata': {},\n",
|
|
" 'name': 'Docs',\n",
|
|
" 'start_time': '2023-10-19T17:52:15.438',\n",
|
|
" 'streamed_output_str': [],\n",
|
|
" 'tags': ['map:key:context', 'FAISS'],\n",
|
|
" 'type': 'retriever'}},\n",
|
|
" 'streamed_output': ['', 'H', 'arrison', ' worked', ' at', ' Kens', 'ho']})\n",
|
|
"----------------------------------------------------------------------\n",
|
|
"RunLog({'final_output': None,\n",
|
|
" 'id': 'afe66178-d75f-4c2d-b348-b1d144239cd6',\n",
|
|
" 'logs': {'Docs': {'end_time': '2023-10-19T17:52:15.738',\n",
|
|
" 'final_output': {'documents': [Document(page_content='harrison worked at kensho')]},\n",
|
|
" 'id': '88d51118-5756-4891-89c5-2f6a5e90cc26',\n",
|
|
" 'metadata': {},\n",
|
|
" 'name': 'Docs',\n",
|
|
" 'start_time': '2023-10-19T17:52:15.438',\n",
|
|
" 'streamed_output_str': [],\n",
|
|
" 'tags': ['map:key:context', 'FAISS'],\n",
|
|
" 'type': 'retriever'}},\n",
|
|
" 'streamed_output': ['', 'H', 'arrison', ' worked', ' at', ' Kens', 'ho', '.']})\n",
|
|
"----------------------------------------------------------------------\n",
|
|
"RunLog({'final_output': None,\n",
|
|
" 'id': 'afe66178-d75f-4c2d-b348-b1d144239cd6',\n",
|
|
" 'logs': {'Docs': {'end_time': '2023-10-19T17:52:15.738',\n",
|
|
" 'final_output': {'documents': [Document(page_content='harrison worked at kensho')]},\n",
|
|
" 'id': '88d51118-5756-4891-89c5-2f6a5e90cc26',\n",
|
|
" 'metadata': {},\n",
|
|
" 'name': 'Docs',\n",
|
|
" 'start_time': '2023-10-19T17:52:15.438',\n",
|
|
" 'streamed_output_str': [],\n",
|
|
" 'tags': ['map:key:context', 'FAISS'],\n",
|
|
" 'type': 'retriever'}},\n",
|
|
" 'streamed_output': ['',\n",
|
|
" 'H',\n",
|
|
" 'arrison',\n",
|
|
" ' worked',\n",
|
|
" ' at',\n",
|
|
" ' Kens',\n",
|
|
" 'ho',\n",
|
|
" '.',\n",
|
|
" '']})\n",
|
|
"----------------------------------------------------------------------\n",
|
|
"RunLog({'final_output': {'output': 'Harrison worked at Kensho.'},\n",
|
|
" 'id': 'afe66178-d75f-4c2d-b348-b1d144239cd6',\n",
|
|
" 'logs': {'Docs': {'end_time': '2023-10-19T17:52:15.738',\n",
|
|
" 'final_output': {'documents': [Document(page_content='harrison worked at kensho')]},\n",
|
|
" 'id': '88d51118-5756-4891-89c5-2f6a5e90cc26',\n",
|
|
" 'metadata': {},\n",
|
|
" 'name': 'Docs',\n",
|
|
" 'start_time': '2023-10-19T17:52:15.438',\n",
|
|
" 'streamed_output_str': [],\n",
|
|
" 'tags': ['map:key:context', 'FAISS'],\n",
|
|
" 'type': 'retriever'}},\n",
|
|
" 'streamed_output': ['',\n",
|
|
" 'H',\n",
|
|
" 'arrison',\n",
|
|
" ' worked',\n",
|
|
" ' at',\n",
|
|
" ' Kens',\n",
|
|
" 'ho',\n",
|
|
" '.',\n",
|
|
" '']})\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"async for chunk in retrieval_chain.astream_log(\n",
|
|
" \"where did harrison work?\", include_names=[\"Docs\"], diff=False\n",
|
|
"):\n",
|
|
" print(\"-\" * 70)\n",
|
|
" print(chunk)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "7006f1aa",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Parallelism\n",
|
|
"\n",
|
|
"Let's take a look at how LangChain Expression Language supports parallel requests. \n",
|
|
"For example, when using a `RunnableParallel` (often written as a dictionary) it executes each element in parallel."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 32,
|
|
"id": "0a1c409d",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"from langchain_core.runnables import RunnableParallel\n",
|
|
"\n",
|
|
"chain1 = ChatPromptTemplate.from_template(\"tell me a joke about {topic}\") | model\n",
|
|
"chain2 = (\n",
|
|
" ChatPromptTemplate.from_template(\"write a short (2 line) poem about {topic}\")\n",
|
|
" | model\n",
|
|
")\n",
|
|
"combined = RunnableParallel(joke=chain1, poem=chain2)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 43,
|
|
"id": "08044c0a",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"CPU times: user 54.3 ms, sys: 0 ns, total: 54.3 ms\n",
|
|
"Wall time: 2.29 s\n"
|
|
]
|
|
},
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"AIMessage(content=\"Why don't bears wear shoes?\\n\\nBecause they already have bear feet!\")"
|
|
]
|
|
},
|
|
"execution_count": 43,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"%%time\n",
|
|
"chain1.invoke({\"topic\": \"bears\"})"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 44,
|
|
"id": "22c56804",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"CPU times: user 7.8 ms, sys: 0 ns, total: 7.8 ms\n",
|
|
"Wall time: 1.43 s\n"
|
|
]
|
|
},
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"AIMessage(content=\"In wild embrace,\\nNature's strength roams with grace.\")"
|
|
]
|
|
},
|
|
"execution_count": 44,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"%%time\n",
|
|
"chain2.invoke({\"topic\": \"bears\"})"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 45,
|
|
"id": "4fff4cbb",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"CPU times: user 167 ms, sys: 921 µs, total: 168 ms\n",
|
|
"Wall time: 1.56 s\n"
|
|
]
|
|
},
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"{'joke': AIMessage(content=\"Why don't bears wear shoes?\\n\\nBecause they already have bear feet!\"),\n",
|
|
" 'poem': AIMessage(content=\"Fierce and wild, nature's might,\\nBears roam the woods, shadows of the night.\")}"
|
|
]
|
|
},
|
|
"execution_count": 45,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"%%time\n",
|
|
"combined.invoke({\"topic\": \"bears\"})"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "80164216-0abd-439b-8407-409539e104b6",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Parallelism on batches\n",
|
|
"\n",
|
|
"Parallelism can be combined with other runnables.\n",
|
|
"Let's try to use parallelism with batches."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 40,
|
|
"id": "f67d2268-c766-441b-8d64-57b8219ccb34",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"CPU times: user 159 ms, sys: 3.66 ms, total: 163 ms\n",
|
|
"Wall time: 1.34 s\n"
|
|
]
|
|
},
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"[AIMessage(content=\"Why don't bears wear shoes?\\n\\nBecause they already have bear feet!\"),\n",
|
|
" AIMessage(content=\"Sure, here's a cat joke for you:\\n\\nWhy don't cats play poker in the wild?\\n\\nBecause there are too many cheetahs!\")]"
|
|
]
|
|
},
|
|
"execution_count": 40,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"%%time\n",
|
|
"chain1.batch([{\"topic\": \"bears\"}, {\"topic\": \"cats\"}])"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 41,
|
|
"id": "83c8d511-9563-403e-9c06-cae986cf5dee",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"CPU times: user 165 ms, sys: 0 ns, total: 165 ms\n",
|
|
"Wall time: 1.73 s\n"
|
|
]
|
|
},
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"[AIMessage(content=\"Silent giants roam,\\nNature's strength, love's emblem shown.\"),\n",
|
|
" AIMessage(content='Whiskers aglow, paws tiptoe,\\nGraceful hunters, hearts aglow.')]"
|
|
]
|
|
},
|
|
"execution_count": 41,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"%%time\n",
|
|
"chain2.batch([{\"topic\": \"bears\"}, {\"topic\": \"cats\"}])"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 42,
|
|
"id": "07a81230-8db8-4b96-bdcb-99ae1d171f2f",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"CPU times: user 507 ms, sys: 125 ms, total: 632 ms\n",
|
|
"Wall time: 1.49 s\n"
|
|
]
|
|
},
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"[{'joke': AIMessage(content=\"Why don't bears wear shoes?\\n\\nBecause they already have bear feet!\"),\n",
|
|
" 'poem': AIMessage(content=\"Majestic bears roam,\\nNature's wild guardians of home.\")},\n",
|
|
" {'joke': AIMessage(content=\"Sure, here's a cat joke for you:\\n\\nWhy did the cat sit on the computer?\\n\\nBecause it wanted to keep an eye on the mouse!\"),\n",
|
|
" 'poem': AIMessage(content='Whiskers twitch, eyes gleam,\\nGraceful creatures, feline dream.')}]"
|
|
]
|
|
},
|
|
"execution_count": 42,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"%%time\n",
|
|
"combined.batch([{\"topic\": \"bears\"}, {\"topic\": \"cats\"}])"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "5218cafd-370a-4e3a-85a0-452e570092fe",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": []
|
|
}
|
|
],
|
|
"metadata": {
|
|
"kernelspec": {
|
|
"display_name": "Python 3 (ipykernel)",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.9.1"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 5
|
|
}
|