"We use the term tool calling interchangeably with function calling. Although\n",
"We use the term \"tool calling\" interchangeably with \"function calling\". Although\n",
"function calling is sometimes meant to refer to invocations of a single function,\n",
"we treat all models as though they can return multiple tool or function calls in \n",
"each message.\n",
":::\n",
"\n",
":::tip\n",
"See [here](/docs/integrations/chat/) for a list of all models that support tool calling.\n",
":::\n",
"```\n",
"\n",
"Tool calling allows a model to respond to a given prompt by generating output that \n",
@ -86,12 +90,14 @@
"LangChain implements standard interfaces for defining tools, passing them to LLMs, \n",
"and representing tool calls.\n",
"\n",
"## Passing tools to LLMs\n",
"## Request: Passing tools to model\n",
"\n",
"Chat models supporting tool calling features implement a `.bind_tools` method, which \n",
"receives a list of LangChain [tool objects](https://api.python.langchain.com/en/latest/tools/langchain_core.tools.BaseTool.html#langchain_core.tools.BaseTool) \n",
"and binds them to the chat model in its expected format. Subsequent invocations of the \n",
"chat model will include tool schemas in its calls to the LLM.\n",
"For a model to be able to invoke tools, you need to pass tool schemas to it when making a chat request.\n",
"LangChain ChatModels supporting tool calling features implement a `.bind_tools` method, which \n",
"receives a list of LangChain [tool objects](https://api.python.langchain.com/en/latest/tools/langchain_core.tools.BaseTool.html#langchain_core.tools.BaseTool), Pydantic classes, or JSON Schemas and binds them to the chat model in the provider-specific expected format. Subsequent invocations of the \n",
"bound chat model will include tool schemas in every call to the model API.\n",
"\n",
"### Defining tool schemas: LangChain Tool\n",
"\n",
"For example, we can define the schema for custom tools using the `@tool` decorator \n",
"on Python functions:"
@ -99,7 +105,7 @@
},
{
"cell_type": "code",
"execution_count": 22,
"execution_count": 1,
"id": "841dca72-1b57-4a42-8e22-da4835c4cfe0",
"metadata": {},
"outputs": [],
@ -109,13 +115,23 @@
"\n",
"@tool\n",
"def add(a: int, b: int) -> int:\n",
" \"\"\"Adds a and b.\"\"\"\n",
" \"\"\"Adds a and b.\n",
"\n",
" Args:\n",
" a: first int\n",
" b: second int\n",
" \"\"\"\n",
" return a + b\n",
"\n",
"\n",
"@tool\n",
"def multiply(a: int, b: int) -> int:\n",
" \"\"\"Multiplies a and b.\"\"\"\n",
" \"\"\"Multiplies a and b.\n",
"\n",
" Args:\n",
" a: first int\n",
" b: second int\n",
" \"\"\"\n",
" return a * b\n",
"\n",
"\n",
@ -127,12 +143,14 @@
"id": "48058b7d-048d-48e6-a272-3931ad7ad146",
"metadata": {},
"source": [
"Or below, we define the schema using Pydantic:\n"
"### Defining tool schemas: Pydantic class\n",
"\n",
"We can equivalently define the schema using Pydantic. Pydantic is useful when your tool inputs are more complex:"
]
},
{
"cell_type": "code",
"execution_count": 23,
"execution_count": 2,
"id": "fca56328-85e4-4839-97b7-b5dc55920602",
"metadata": {},
"outputs": [],
@ -142,21 +160,21 @@
"\n",
"# Note that the docstrings here are crucial, as they will be passed along\n",
"# to the model along with the class name.\n",
"class Add(BaseModel):\n",
"class add(BaseModel):\n",
" \"\"\"Add two integers together.\"\"\"\n",
"\n",
" a: int = Field(..., description=\"First integer\")\n",
" b: int = Field(..., description=\"Second integer\")\n",
"\n",
"\n",
"class Multiply(BaseModel):\n",
"class multiply(BaseModel):\n",
" \"\"\"Multiply two integers together.\"\"\"\n",
"\n",
" a: int = Field(..., description=\"First integer\")\n",
" b: int = Field(..., description=\"Second integer\")\n",
"\n",
"\n",
"tools = [Add, Multiply]"
"tools = [add, multiply]"
]
},
{
@ -175,6 +193,8 @@
"/>\n",
"```\n",
"\n",
"### Binding tool schemas\n",
"\n",
"We can use the `bind_tools()` method to handle converting\n",
"`Multiply` to a \"tool\" and binding it to the model (i.e.,\n",
"passing it in each time the model is invoked)."
@ -182,7 +202,7 @@
},
{
"cell_type": "code",
"execution_count": 67,
"execution_count": 3,
"id": "44eb8327-a03d-4c7c-945e-30f13f455346",
"metadata": {},
"outputs": [],
@ -197,7 +217,7 @@
},
{
"cell_type": "code",
"execution_count": 68,
"execution_count": 4,
"id": "af2a83ac-e43f-43ce-b107-9ed8376bfb75",
"metadata": {},
"outputs": [],
@ -205,17 +225,45 @@
"llm_with_tools = llm.bind_tools(tools)"
]
},
{
"cell_type": "markdown",
"id": "3dd0e53f-d48d-4952-b53e-faf97ebf8831",
"metadata": {},
"source": [
"## Request: Forcing a tool call\n",
"\n",
"When you just use `bind_tools(tools)`, the model can choose whether to return one tool call, multiple tool calls, or no tool calls at all. Some models support a `tool_choice` parameter that gives you some ability to force the model to call a tool. For models that support this, you can pass in the name of the tool you want the model to always call `tool_choice=\"xyz_tool_name\"`. Or you can pass in `tool_choice=\"any\"` to force the model to call at least one tool, without specifying which tool specifically.\n",
"\n",
"```{=mdx}\n",
":::note\n",
"Currently `tool_choice=\"any\"` functionality is supported by OpenAI, MistralAI, FireworksAI, and Groq.\n",
"\n",
"Currently Anthropic does not support `tool_choice` at all.\n",
":::\n",
"```\n",
"\n",
"If we wanted our model to always call the multiply tool we could do:\n",
"as a list of [tool call](https://api.python.langchain.com/en/latest/messages/langchain_core.messages.tool.ToolCall.html#langchain_core.messages.tool.ToolCall) \n",
"as a list of [ToolCall](https://api.python.langchain.com/en/latest/messages/langchain_core.messages.tool.ToolCall.html#langchain_core.messages.tool.ToolCall) \n",
"objects in the `.tool_calls` attribute. A `ToolCall` is a typed dict that includes a \n",
"tool name, dict of argument values, and (optionally) an identifier. Messages with no \n",
"tool calls default to an empty list for this attribute.\n",
"If we're using the model-generated tool invocations to actually call tools and want to pass the tool results back to the model, we can do so using `ToolMessage`s."
]
},
{
"cell_type": "code",
"execution_count": 117,
"execution_count": 13,
"id": "48049192-be28-42ab-9a44-d897924e67cd",
"metadata": {},
"outputs": [
@ -495,12 +543,12 @@
"data": {
"text/plain": [
"[HumanMessage(content='What is 3 * 12? Also, what is 11 + 49?'),\n",
"For more complex tool use it's very useful to add few-shot examples to the prompt. We can do this by adding `AIMessage`s with `ToolCall`s and corresponding `ToolMessage`s to our prompt.\n",
"For more complex tool use it's very useful to add few-shot examples to the prompt. We can do this by adding `AIMessage`s with `ToolCall`s and corresponding `ToolMessage`s to our prompt. \n",
"\n",
"```{=mdx}\n",
":::note\n",
"For most models it's important that the ToolCall and ToolMessage ids line up, so that each AIMessage with ToolCalls is followed by ToolMessages with corresponding ids.\n",
"```\n",
"\n",
"For example, even with some special instructions our model can get tripped up by order of operations:"
Bagatur
4 weeks ago