mirror of
https://github.com/hwchase17/langchain
synced 2024-11-10 01:10:59 +00:00
docs: tool artifacts how to (#24198)
This commit is contained in:
Bagatur
GitHub
parent
5c3e2612da
commit
9224027e45
@ -197,6 +197,7 @@ LangChain [Tools](/docs/concepts/#tools) contain a description of the tool (to p
|
||||
- [How to: disable parallel tool calling](/docs/how_to/tool_choice)
|
||||
- [How to: access the `RunnableConfig` object within a custom tool](/docs/how_to/tool_configure)
|
||||
- [How to: stream events from child runs within a custom tool](/docs/how_to/tool_stream_events)
|
||||
- [How to: return extra artifacts from a tool](/docs/how_to/tool_artifacts/)
|
||||
|
||||
### Multimodal
|
||||
|
||||
|
||||
388
docs/docs/how_to/tool_artifacts.ipynb
Normal file
388
docs/docs/how_to/tool_artifacts.ipynb
Normal file
@ -0,0 +1,388 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "503e36ae-ca62-4f8a-880c-4fe78ff5df93",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# How to return extra artifacts from a tool\n",
|
||||
"\n",
|
||||
":::info Prerequisites\n",
|
||||
"This guide assumes familiarity with the following concepts:\n",
|
||||
"\n",
|
||||
"- [Tools](/docs/concepts/#tools)\n",
|
||||
"- [Function/tool calling](/docs/concepts/#functiontool-calling)\n",
|
||||
"\n",
|
||||
":::\n",
|
||||
"\n",
|
||||
"Tools are utilities that can be called by a model, and whose outputs are designed to be fed back to a model. Sometimes, however, there are artifacts of a tool's execution that we want to make accessible to downstream components in our chain or agent, but that we don't want to expose to the model itself. For example if a tool returns a custom object, a dataframe or an image, we may want to pass some metadata about this output to the model without passing the actual output to the model. At the same time, we may want to be able to access this full output elsewhere, for example in downstream tools.\n",
|
||||
"\n",
|
||||
"The Tool and [ToolMessage](https://api.python.langchain.com/en/latest/messages/langchain_core.messages.tool.ToolMessage.html) interfaces make it possible to distinguish between the parts of the tool output meant for the model (this is the ToolMessage.content) and those parts which are meant for use outside the model (ToolMessage.artifact).\n",
|
||||
"\n",
|
||||
":::info Requires ``langchain-core >= 0.2.18``\n",
|
||||
"\n",
|
||||
"This functionality was added in ``langchain-core == 0.2.18``. Please make sure your package is up to date.\n",
|
||||
"\n",
|
||||
":::\n",
|
||||
"\n",
|
||||
"## Defining the tool\n",
|
||||
"\n",
|
||||
"If we want our tool to distinguish between message content and other artifacts, we need to specify `response_format=\"content_and_artifact\"` when defining our tool and make sure that we return a tuple of (content, artifact):"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "762b9199-885f-4946-9c98-cc54d72b0d76",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"%pip install -qU \"langchain-core>=0.2.18\""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"id": "b9eb179d-1f41-4748-9866-b3d3e8c73cd0",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import random\n",
|
||||
"from typing import List, Tuple\n",
|
||||
"\n",
|
||||
"from langchain_core.tools import tool\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"@tool(response_format=\"content_and_artifact\")\n",
|
||||
"def generate_random_ints(min: int, max: int, size: int) -> Tuple[str, List[int]]:\n",
|
||||
" \"\"\"Generate size random ints in the range [min, max].\"\"\"\n",
|
||||
" array = [random.randint(min, max) for _ in range(size)]\n",
|
||||
" content = f\"Successfully generated array of {size} random ints in [{min}, {max}].\"\n",
|
||||
" return content, array"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "0ab05d25-af4a-4e5a-afe2-f090416d7ee7",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Invoking the tool with ToolCall\n",
|
||||
"\n",
|
||||
"If we directly invoke our tool with just the tool arguments, you'll notice that we only get back the content part of the Tool output:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 8,
|
||||
"id": "5e7d5e77-3102-4a59-8ade-e4e699dd1817",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"'Successfully generated array of 10 random ints in [0, 9].'"
|
||||
]
|
||||
},
|
||||
"execution_count": 8,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"generate_random_ints.invoke({\"min\": 0, \"max\": 9, \"size\": 10})"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "30db7228-f04c-489e-afda-9a572eaa90a1",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"In order to get back both the content and the artifact, we need to invoke our model with a ToolCall (which is just a dictionary with \"name\", \"args\", \"id\" and \"type\" keys), which has additional info needed to generate a ToolMessage like the tool call ID:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 4,
|
||||
"id": "da1d939d-a900-4b01-92aa-d19011a6b034",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"ToolMessage(content='Successfully generated array of 10 random ints in [0, 9].', tool_call_id='123', artifact=[7, 0, 5, 3, 1, 7, 9, 3, 1, 0])"
|
||||
]
|
||||
},
|
||||
"execution_count": 4,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"generate_random_ints.invoke(\n",
|
||||
" {\n",
|
||||
" \"name\": \"generate_random_ints\",\n",
|
||||
" \"args\": {\"min\": 0, \"max\": 9, \"size\": 10},\n",
|
||||
" \"id\": \"123\",\n",
|
||||
" \"type\": \"tool_call\",\n",
|
||||
" }\n",
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "a3cfc03d-020b-42c7-b0f8-c824af19e45e",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Using with a model\n",
|
||||
"\n",
|
||||
"With a [tool-calling model](/docs/how_to/tool_calling/), we can easily use a model to call our Tool and generate ToolMessages:\n",
|
||||
"\n",
|
||||
"```{=mdx}\n",
|
||||
"import ChatModelTabs from \"@theme/ChatModelTabs\";\n",
|
||||
"\n",
|
||||
"<ChatModelTabs\n",
|
||||
" customVarName=\"llm\"\n",
|
||||
"/>\n",
|
||||
"```"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 5,
|
||||
"id": "74de0286-b003-4b48-9cdd-ecab435515ca",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# | echo: false\n",
|
||||
"# | output: false\n",
|
||||
"\n",
|
||||
"from langchain_anthropic import ChatAnthropic\n",
|
||||
"\n",
|
||||
"llm = ChatAnthropic(model=\"claude-3-5-sonnet-20240620\", temperature=0)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 6,
|
||||
"id": "8a67424b-d19c-43df-ac7b-690bca42146c",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"[{'name': 'generate_random_ints',\n",
|
||||
" 'args': {'min': 1, 'max': 24, 'size': 6},\n",
|
||||
" 'id': 'toolu_014wkiiCjbnJzUiR7fJXnCCY',\n",
|
||||
" 'type': 'tool_call'}]"
|
||||
]
|
||||
},
|
||||
"execution_count": 6,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"llm_with_tools = llm.bind_tools([generate_random_ints])\n",
|
||||
"\n",
|
||||
"ai_msg = llm_with_tools.invoke(\"generate 6 positive ints less than 25\")\n",
|
||||
"ai_msg.tool_calls"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 7,
|
||||
"id": "00c4e906-3ca8-41e8-a0be-65cb0db7d574",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"ToolMessage(content='Successfully generated array of 6 random ints in [1, 24].', tool_call_id='toolu_014wkiiCjbnJzUiR7fJXnCCY', artifact=[9, 13, 10, 16, 23, 11])"
|
||||
]
|
||||
},
|
||||
"execution_count": 7,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"generate_random_ints.invoke(ai_msg.tool_calls[0])"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "ddef2690-70de-4542-ab20-2337f77f3e46",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"If we just pass in the tool call args, we'll only get back the content:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 7,
|
||||
"id": "f4a6c9a6-0ffc-4b0e-a59f-f3c3d69d824d",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"'Successfully generated array of 6 random ints in [1, 24].'"
|
||||
]
|
||||
},
|
||||
"execution_count": 7,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"generate_random_ints.invoke(ai_msg.tool_calls[0][\"args\"])"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "98d6443b-ff41-4d91-8523-b6274fc74ee5",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"If we wanted to declaratively create a chain, we could do this:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 10,
|
||||
"id": "eb55ec23-95a4-464e-b886-d9679bf3aaa2",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"[ToolMessage(content='Successfully generated array of 1 random ints in [1, 5].', tool_call_id='toolu_01UZiQLczkDx3ELv27ureuCP', artifact=[1])]"
|
||||
]
|
||||
},
|
||||
"execution_count": 10,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"from operator import attrgetter\n",
|
||||
"\n",
|
||||
"chain = llm_with_tools | attrgetter(\"tool_calls\") | generate_random_ints.map()\n",
|
||||
"\n",
|
||||
"chain.invoke(\"give me a random number between 1 and 5\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "4df46be2-babb-4bfe-a641-91cd3d03ffaf",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Creating from BaseTool class\n",
|
||||
"\n",
|
||||
"If you want to create a BaseTool object directly, instead of decorating a function with `@tool`, you can do so like this:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 20,
|
||||
"id": "9a9129e1-6aee-4a10-ad57-62ef3bf0276c",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from langchain_core.tools import BaseTool\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"class GenerateRandomFloats(BaseTool):\n",
|
||||
" name: str = \"generate_random_floats\"\n",
|
||||
" description: str = \"Generate size random floats in the range [min, max].\"\n",
|
||||
" response_format: str = \"content_and_artifact\"\n",
|
||||
"\n",
|
||||
" ndigits: int = 2\n",
|
||||
"\n",
|
||||
" def _run(self, min: float, max: float, size: int) -> Tuple[str, List[float]]:\n",
|
||||
" range_ = max - min\n",
|
||||
" array = [\n",
|
||||
" round(min + (range_ * random.random()), ndigits=self.ndigits)\n",
|
||||
" for _ in range(size)\n",
|
||||
" ]\n",
|
||||
" content = f\"Generated {size} floats in [{min}, {max}], rounded to {self.ndigits} decimals.\"\n",
|
||||
" return content, array\n",
|
||||
"\n",
|
||||
" # Optionally define an equivalent async method\n",
|
||||
"\n",
|
||||
" # async def _arun(self, min: float, max: float, size: int) -> Tuple[str, List[float]]:\n",
|
||||
" # ..."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 21,
|
||||
"id": "d7322619-f420-4b29-8ee5-023e693d0179",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"'Generated 3 floats in [0.1, 3.3333], rounded to 4 decimals.'"
|
||||
]
|
||||
},
|
||||
"execution_count": 21,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"rand_gen = GenerateRandomFloats(ndigits=4)\n",
|
||||
"rand_gen.invoke({\"min\": 0.1, \"max\": 3.3333, \"size\": 3})"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 22,
|
||||
"id": "0892f277-23a6-4bb8-a0e9-59f533ac9750",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"ToolMessage(content='Generated 3 floats in [0.1, 3.3333], rounded to 4 decimals.', tool_call_id='123', artifact=[0.7306, 1.8991, 0.1615])"
|
||||
]
|
||||
},
|
||||
"execution_count": 22,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"rand_gen.invoke(\n",
|
||||
" {\n",
|
||||
" \"name\": \"generate_random_floats\",\n",
|
||||
" \"args\": {\"min\": 0.1, \"max\": 3.3333, \"size\": 3},\n",
|
||||
" \"id\": \"123\",\n",
|
||||
" \"type\": \"tool_call\",\n",
|
||||
" }\n",
|
||||
")"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "poetry-venv-311",
|
||||
"language": "python",
|
||||
"name": "poetry-venv-311"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.11.9"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
}
|
||||
Loading…
Reference in New Issue
Block a user