Archives/langchain
2
0
Fork 1
mirror of https://github.com/hwchase17/langchain synced 2024-11-02 09:40:22 +00:00
Code Issues Packages Projects Releases Wiki Activity

docs: how to guide tool calling using prompts (#21827)

Browse Source 
Update tool calling using prompts.

- Add required concepts
- Update names of tool invoking function.
- Add doc-string to function, and add information about `config` (which
users often forget)
- Remove steps that show how to use single function only. This makes the
how-to guide a bit shorter and more to the point.
- Add diagram from another how-to guide that shows how the thing works
overall.
This commit is contained in:
Eugene Yurtsev 2024-05-17 16:46:59 -04:00 committed by GitHub
parent e5046cbd72
commit 2656bfe941
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
2 changed files with 279 additions and 174 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
docs/docs/how_to/index.mdx
View File

@ -169,7 +169,7 @@ LangChain Tools contain a description of the tool (to pass to the language model
- [How to: create custom tools](/docs/how_to/custom_tools)
- [How to: use built-in tools and built-in toolkits](/docs/how_to/tools_builtin)
- [How to: use a chat model to call tools](/docs/how_to/tool_calling/)
- [How to: use tools with LLMs that do not support tool calling natively](/docs/how_to/tools_prompting)
- [How to: use add ad-hoc tool calling capability to LLMs and chat models](/docs/how_to/tools_prompting)
- [How to: convert LangChain tools to OpenAI functions](/docs/how_to/tools_as_openai_functions)
- [How to: add a human in the loop to tool usage](/docs/how_to/tools_human)
- [How to: handle errors when calling tools](/docs/how_to/tools_error)

451
docs/docs/how_to/tools_prompting.ipynb
View File

@ -15,9 +15,30 @@
"id": "14b94240",
"metadata": {},
"source": [
"# How to use tools without function calling\n",
"# How to add ad-hoc tool calling capability to LLMs and Chat Models\n",
"\n",
"In this guide we'll build a Chain that does not rely on any special model APIs (like tool calling, which we showed in the [Quickstart](/docs/how_to/tool_calling)) and instead just prompts the model directly to invoke tools."
":::{.callout-caution}\n",
"\n",
"Some models have been fine-tuned for tool calling and provide a dedicated API for tool calling. Generally, such models are better at tool calling than non-fine-tuned models, and are recommended for use cases that require tool calling. Please see the [how to use a chat model to call tools](/docs/how_to/tool_calling/) guide for more information.\n",
"\n",
":::\n",
"\n",
":::info Prerequisites\n",
"\n",
"This guide assumes familiarity with the following concepts:\n",
"\n",
"- [LangChain Tools](/docs/concepts/#tools)\n",
"- [Function/tool calling](https://python.langchain.com/v0.2/docs/concepts/#functiontool-calling)\n",
"- [Chat models](/docs/concepts/#chat-models)\n",
"- [LLMs](/docs/concepts/#llms)\n",
"\n",
":::\n",
"\n",
"In this guide, we'll see how to add **ad-hoc** tool calling support to a chat model. This is an alternative method to invoke tools if you're using a model that does not natively support [tool calling](/docs/how_to/tool_calling/).\n",
"\n",
"We'll do this by simply writing a prompt that will get the model to invoke the appropriate tools. Here's a diagram of the logic:\n",
"\n",
"![chain](../../static/img/tool_chain.svg)"
]
},
{
@ -37,34 +58,58 @@
"metadata": {},
"outputs": [],
"source": [
"%pip install --upgrade --quiet langchain langchain-openai"
"%pip install --upgrade --quiet langchain langchain-community"
]
},
{
"cell_type": "markdown",
"id": "5e727d22-f861-4eee-882a-688f8efc885e",
"id": "897bc01e-cc2b-4400-8a64-db4aa56085d3",
"metadata": {},
"source": [
"And set these environment variables:"
"If you'd like to use LangSmith, uncomment the below:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "527ef906-0104-4872-b4e5-f371cf73feba",
"execution_count": 26,
"id": "5efb4170-b95b-4d29-8f57-09509f3ba6df",
"metadata": {},
"outputs": [],
"source": [
"import getpass\n",
"import os\n",
"\n",
"os.environ[\"OPENAI_API_KEY\"] = getpass.getpass()\n",
"\n",
"# If you'd like to use LangSmith, uncomment the below:\n",
"# os.environ[\"LANGCHAIN_TRACING_V2\"] = \"true\"\n",
"# os.environ[\"LANGCHAIN_API_KEY\"] = getpass.getpass()"
]
},
{
"cell_type": "markdown",
"id": "7ec6409b-21e5-4d0a-8a46-c4ef0b055dd3",
"metadata": {},
"source": [
"You can select any of the given models for this how-to guide. Keep in mind that most of these models already [support native tool calling](/docs/integrations/chat/), so using the prompting strategy shown here doesn't make sense for these models, and instead you should follow the [how to use a chat model to call tools](/docs/how_to/tool_calling/) guide.\n",
"\n",
"```{=mdx}\n",
"import ChatModelTabs from \"@theme/ChatModelTabs\";\n",
"\n",
"<ChatModelTabs openaiParams={`model=\"gpt-4\"`} />\n",
"```\n",
"\n",
"To illustrate the idea, we'll use `phi3` via Ollama, which does **NOT** have native support for tool calling. If you'd like to use `Ollama` as well follow [these instructions](/docs/integrations/chat/ollama/)."
]
},
{
"cell_type": "code",
"execution_count": 24,
"id": "424be968-2806-4d1a-a6aa-5499ae20fac5",
"metadata": {},
"outputs": [],
"source": [
"from langchain_community.llms import Ollama\n",
"\n",
"model = Ollama(model=\"phi3\")"
]
},
{
"cell_type": "markdown",
"id": "68946881",
@ -72,66 +117,75 @@
"source": [
"## Create a tool\n",
"\n",
"First, we need to create a tool to call. For this example, we will create a custom tool from a function. For more information on all details related to creating custom tools, please see [this guide](/docs/how_to/custom_tools)."
"First, let's create an `add` and `multiply` tools. For more information on creating custom tools, please see [this guide](/docs/how_to/custom_tools)."
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "90187d07",
"metadata": {},
"outputs": [],
"source": [
"from langchain_core.tools import tool\n",
"\n",
"\n",
"@tool\n",
"def multiply(first_int: int, second_int: int) -> int:\n",
" \"\"\"Multiply two integers together.\"\"\"\n",
" return first_int * second_int"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "d7009e1a",
"execution_count": 4,
"id": "4548e6fa-0f9b-4d7a-8fa5-66cec0350e5f",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"--\n",
"multiply\n",
"multiply(first_int: int, second_int: int) -> int - Multiply two integers together.\n",
"{'first_int': {'title': 'First Int', 'type': 'integer'}, 'second_int': {'title': 'Second Int', 'type': 'integer'}}\n"
"Multiply two numbers together.\n",
"{'x': {'title': 'X', 'type': 'number'}, 'y': {'title': 'Y', 'type': 'number'}}\n",
"--\n",
"add\n",
"Add two numbers.\n",
"{'x': {'title': 'X', 'type': 'integer'}, 'y': {'title': 'Y', 'type': 'integer'}}\n"
]
}
],
"source": [
"print(multiply.name)\n",
"print(multiply.description)\n",
"print(multiply.args)"
"from langchain_core.tools import tool\n",
"\n",
"\n",
"@tool\n",
"def multiply(x: float, y: float) -> float:\n",
" \"\"\"Multiply two numbers together.\"\"\"\n",
" return x * y\n",
"\n",
"\n",
"@tool\n",
"def add(x: int, y: int) -> int:\n",
" \"Add two numbers.\"\n",
" return x + y\n",
"\n",
"\n",
"tools = [multiply, add]\n",
"\n",
"# Let's inspect the tools\n",
"for t in tools:\n",
" print(\"--\")\n",
" print(t.name)\n",
" print(t.description)\n",
" print(t.args)"
]
},
{
"cell_type": "code",
"execution_count": 3,
"execution_count": 5,
"id": "be77e780",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"20"
"20.0"
]
},
"execution_count": 3,
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"multiply.invoke({\"first_int\": 4, \"second_int\": 5})"
"multiply.invoke({\"x\": 4, \"y\": 5})"
]
},
{
@ -146,48 +200,85 @@
},
{
"cell_type": "code",
"execution_count": 4,
"id": "c64818f0-9364-423c-922e-bdfb8f01e726",
"execution_count": 6,
"id": "2063b564-25ca-4729-a45f-ba4633175b04",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'multiply: multiply(first_int: int, second_int: int) -> int - Multiply two integers together.'"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
"name": "stdout",
"output_type": "stream",
"text": [
"multiply(x: float, y: float) -> float - Multiply two numbers together.\n",
"add(x: int, y: int) -> int - Add two numbers.\n"
]
}
],
"source": [
"from langchain_core.output_parsers import JsonOutputParser\n",
"from langchain_core.prompts import ChatPromptTemplate\n",
"from langchain_core.tools import render_text_description\n",
"\n",
"rendered_tools = render_text_description([multiply])\n",
"rendered_tools"
"rendered_tools = render_text_description(tools)\n",
"print(rendered_tools)"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "63552d4d-8bd6-4aca-8805-56e236f6552d",
"execution_count": 17,
"id": "f02f1dce-76e7-4ca9-9bac-5af496131fe1",
"metadata": {},
"outputs": [],
"source": [
"from langchain_core.prompts import ChatPromptTemplate\n",
"\n",
"system_prompt = f\"\"\"You are an assistant that has access to the following set of tools. Here are the names and descriptions for each tool:\n",
"system_prompt = f\"\"\"\\\n",
"You are an assistant that has access to the following set of tools. \n",
"Here are the names and descriptions for each tool:\n",
"\n",
"{rendered_tools}\n",
"\n",
"Given the user input, return the name and input of the tool to use. Return your response as a JSON blob with 'name' and 'arguments' keys.\"\"\"\n",
"Given the user input, return the name and input of the tool to use. \n",
"Return your response as a JSON blob with 'name' and 'arguments' keys.\n",
"\n",
"The `arguments` should be a dictionary, with keys corresponding \n",
"to the argument names and the values corresponding to the requested values.\n",
"\"\"\"\n",
"\n",
"prompt = ChatPromptTemplate.from_messages(\n",
" [(\"system\", system_prompt), (\"user\", \"{input}\")]\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 18,
"id": "f8623e03-60eb-4439-b57b-ecbcebc61b58",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"{\n",
" \"name\": \"add\",\n",
" \"arguments\": {\n",
" \"x\": 3,\n",
" \"y\": 1132\n",
" }\n",
"}\n"
]
}
],
"source": [
"chain = prompt | model\n",
"message = chain.invoke({\"input\": \"what's 3 plus 1132\"})\n",
"\n",
"# Let's take a look at the output from the model\n",
"# if the model is an LLM (not a chat model), the output will be a string.\n",
"if isinstance(message, str):\n",
" print(message)\n",
"else: # Otherwise it's a chat model\n",
" print(message.content)"
]
},
{
"cell_type": "markdown",
"id": "14df2cd5-b6fa-4b10-892d-e8692c7931e5",
@ -200,156 +291,153 @@
},
{
"cell_type": "code",
"execution_count": 7,
"execution_count": 19,
"id": "f129f5bd-127c-4c95-8f34-8f437da7ca8f",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"{'name': 'multiply', 'arguments': {'first_int': 13, 'second_int': 4}}"
"{'name': 'multiply', 'arguments': {'x': 13.0, 'y': 4.0}}"
]
},
"execution_count": 7,
"execution_count": 19,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from langchain_core.output_parsers import JsonOutputParser\n",
"from langchain_openai import ChatOpenAI\n",
"\n",
"model = ChatOpenAI(model=\"gpt-3.5-turbo\", temperature=0)\n",
"chain = prompt | model | JsonOutputParser()\n",
"chain.invoke({\"input\": \"what's thirteen times 4\"})"
]
},
{
"cell_type": "markdown",
"id": "e1f08255-f146-4f4a-be43-5c21c1d3ae83",
"metadata": {},
"source": [
":::{.callout-important}\n",
"\n",
"🎉 Amazing! 🎉 We now instructed our model on how to **request** that a tool be invoked.\n",
"\n",
"Now, let's create some logic to actually run the tool!\n",
":::"
]
},
{
"cell_type": "markdown",
"id": "8e29dd4c-8eb5-457f-92d1-8add076404dc",
"metadata": {},
"source": [
"## Invoking the tool\n",
"## Invoking the tool 🏃\n",
"\n",
"We can invoke the tool as part of the chain by passing along the model-generated \"arguments\" to it:"
"Now that the model can request that a tool be invoked, we need to write a function that can actually invoke \n",
"the tool.\n",
"\n",
"The function will select the appropriate tool by name, and pass to it the arguments chosen by the model."
]
},
{
"cell_type": "code",
"execution_count": 8,
"execution_count": 20,
"id": "faee95e0-4095-4310-991f-9e9465c6738e",
"metadata": {},
"outputs": [],
"source": [
"from typing import Any, Dict, Optional, TypedDict\n",
"\n",
"from langchain_core.runnables import RunnableConfig\n",
"\n",
"\n",
"class ToolCallRequest(TypedDict):\n",
" \"\"\"A typed dict that shows the inputs into the invoke_tool function.\"\"\"\n",
"\n",
" name: str\n",
" arguments: Dict[str, Any]\n",
"\n",
"\n",
"def invoke_tool(\n",
" tool_call_request: ToolCallRequest, config: Optional[RunnableConfig] = None\n",
"):\n",
" \"\"\"A function that we can use the perform a tool invocation.\n",
"\n",
" Args:\n",
" tool_call_request: a dict that contains the keys name and arguments.\n",
" The name must match the name of a tool that exists.\n",
" The arguments are the arguments to that tool.\n",
" config: This is configuration information that LangChain uses that contains\n",
" things like callbacks, metadata, etc.See LCEL documentation about RunnableConfig.\n",
"\n",
" Returns:\n",
" output from the requested tool\n",
" \"\"\"\n",
" tool_name_to_tool = {tool.name: tool for tool in tools}\n",
" name = tool_call_request[\"name\"]\n",
" requested_tool = tool_name_to_tool[name]\n",
" return requested_tool.invoke(tool_call_request[\"arguments\"], config=config)"
]
},
{
"cell_type": "markdown",
"id": "f4957532-9e0c-47f6-bb62-0fd789ac1d3e",
"metadata": {},
"source": [
"Let's test this out 🧪!"
]
},
{
"cell_type": "code",
"execution_count": 21,
"id": "d0ea3b2a-8fb2-4016-83c8-a5d3e78fedbc",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"15.0"
]
},
"execution_count": 21,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"invoke_tool({\"name\": \"multiply\", \"arguments\": {\"x\": 3, \"y\": 5}})"
]
},
{
"cell_type": "markdown",
"id": "715af6e1-935d-4bc0-a3d2-646ecf8a329b",
"metadata": {},
"source": [
"## Let's put it together\n",
"\n",
"Let's put it together into a chain that creates a calculator with add and multiplication capabilities."
]
},
{
"cell_type": "code",
"execution_count": 22,
"id": "0555b384-fde6-4404-86e0-7ea199003d58",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"52"
"53.83784653"
]
},
"execution_count": 8,
"execution_count": 22,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from operator import itemgetter\n",
"\n",
"chain = prompt | model | JsonOutputParser() | itemgetter(\"arguments\") | multiply\n",
"chain.invoke({\"input\": \"what's thirteen times 4\"})"
]
},
{
"cell_type": "markdown",
"id": "8d60b2cb-6ce0-48fc-8d18-d2337161a53d",
"metadata": {},
"source": [
"## Choosing from multiple tools\n",
"\n",
"Suppose we have multiple tools we want the chain to be able to choose from:"
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "95c86d32-ee45-4c87-a28c-14eff19b49e9",
"metadata": {},
"outputs": [],
"source": [
"@tool\n",
"def add(first_int: int, second_int: int) -> int:\n",
" \"Add two integers.\"\n",
" return first_int + second_int\n",
"\n",
"\n",
"@tool\n",
"def exponentiate(base: int, exponent: int) -> int:\n",
" \"Exponentiate the base to the exponent power.\"\n",
" return base**exponent"
]
},
{
"cell_type": "markdown",
"id": "748405ff-4c85-4bd7-82e1-30458b5a4106",
"metadata": {},
"source": [
"With function calling, we can do this like so:"
]
},
{
"cell_type": "markdown",
"id": "eb3aa89e-40e1-45ec-b1f3-ab28cfc8e42d",
"metadata": {},
"source": [
"If we want to run the model selected tool, we can do so using a function that returns the tool based on the model output. Specifically, our function will action return it's own subchain that gets the \"arguments\" part of the model output and passes it to the chosen tool:"
]
},
{
"cell_type": "code",
"execution_count": 13,
"id": "db254773-5b8e-43d0-aabe-c21566c154cd",
"metadata": {},
"outputs": [],
"source": [
"tools = [add, exponentiate, multiply]\n",
"\n",
"\n",
"def tool_chain(model_output):\n",
" tool_map = {tool.name: tool for tool in tools}\n",
" chosen_tool = tool_map[model_output[\"name\"]]\n",
" return itemgetter(\"arguments\") | chosen_tool"
]
},
{
"cell_type": "code",
"execution_count": 14,
"id": "ad9f5cff-b86a-45fc-9ce4-b0aa9025a378",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"1135"
]
},
"execution_count": 14,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"rendered_tools = render_text_description(tools)\n",
"system_prompt = f\"\"\"You are an assistant that has access to the following set of tools. Here are the names and descriptions for each tool:\n",
"\n",
"{rendered_tools}\n",
"\n",
"Given the user input, return the name and input of the tool to use. Return your response as a JSON blob with 'name' and 'arguments' keys.\"\"\"\n",
"\n",
"prompt = ChatPromptTemplate.from_messages(\n",
" [(\"system\", system_prompt), (\"user\", \"{input}\")]\n",
")\n",
"\n",
"chain = prompt | model | JsonOutputParser() | tool_chain\n",
"chain.invoke({\"input\": \"what's 3 plus 1132\"})"
"chain = prompt | model | JsonOutputParser() | invoke_tool\n",
"chain.invoke({\"input\": \"what's thirteen times 4.14137281\"})"
]
},
{
@ -364,19 +452,19 @@
},
{
"cell_type": "code",
"execution_count": 15,
"execution_count": 23,
"id": "45404406-859d-4caa-8b9d-5838162c80a0",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"{'name': 'add',\n",
" 'arguments': {'first_int': 3, 'second_int': 1132},\n",
" 'output': 1135}"
"{'name': 'multiply',\n",
" 'arguments': {'x': 13, 'y': 4.14137281},\n",
" 'output': 53.83784653}"
]
},
"execution_count": 15,
"execution_count": 23,
"metadata": {},
"output_type": "execute_result"
}
@ -385,9 +473,26 @@
"from langchain_core.runnables import RunnablePassthrough\n",
"\n",
"chain = (\n",
" prompt | model | JsonOutputParser() | RunnablePassthrough.assign(output=tool_chain)\n",
" prompt | model | JsonOutputParser() | RunnablePassthrough.assign(output=invoke_tool)\n",
")\n",
"chain.invoke({\"input\": \"what's 3 plus 1132\"})"
"chain.invoke({\"input\": \"what's thirteen times 4.14137281\"})"
]
},
{
"cell_type": "markdown",
"id": "1797fe82-ea35-4cba-834a-1caf9740d184",
"metadata": {},
"source": [
"## What's next?\n",
"\n",
"This how-to guide shows the \"happy path\" when the model correctly outputs all the required tool information.\n",
"\n",
"In reality, if you're using more complex tools, you will start encountering errors from the model, especially for models that have not been fine tuned for tool calling and for less capable models.\n",
"\n",
"You will need to be prepared to add strategies to improve the output from the model; e.g.,\n",
"\n",
"1. Provide few shot examples.\n",
"2. Add error handling (e.g., catch the exception and feed it back to the LLM to ask it to correct its previous output)."
]
}
],
@ -407,7 +512,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.4"
"version": "3.11.4"
}
},
"nbformat": 4,