forked from Archives/langchain
add docs for routing chains
Harrison Chase
2 years ago
parent
71a0940435
commit
aba405a570
@ -0,0 +1,171 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "5436020b",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Routing Chains\n",
|
||||
"Some applications will require not just a predetermined chain of calls to LLMs/other tools, but potentially an unknown chain that depends on the user input. In these types of chains, there is a \"router\" LLM chain which has access to a suite of tools. Depending on the user input, the router can then decide which, if any, of these tools to call.\n",
|
||||
"\n",
|
||||
"These types of chains are called Routing Chains. When used correctly these can be extremely powerful. The purpose of this notebook is to go through the core abstractions at the heart of Routing Chains and enable you to build applications with them."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "efb8e2b6",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Terminology\n",
|
||||
"\n",
|
||||
"Before going through any code, let's align on some terminology.\n",
|
||||
"- Tool: A function that performs a specific duty. This can be things like: Google Search, Database lookup, Python REPL. The interface for a tool is currently a function that is expected to have a string as an input, with a string as an output.\n",
|
||||
"- Tool Input: The input string to a tool.\n",
|
||||
"- Observation: The output from calling a tool on a particular input.\n",
|
||||
"- Router: The object responsible for deciding which tools to call and when. Exposes a `route` method, which takes in a string and returns a Router Output.\n",
|
||||
"- Router Output: The object returned from calling `Router.route` on a string. Consists of:\n",
|
||||
" - The tool to use\n",
|
||||
" - The input to that tool\n",
|
||||
" - A log of the router's thinking. This is not required, as it's only used for logging purposes, but it can be helpful for understanding why the router is acting why it is\n",
|
||||
"- Routing Chain: A chain which is made up of a router and suite of tools. When passed a string, the Routing Chain will iterative call tools as needed until it arrives at a Final Answer.\n",
|
||||
"- Final Answer: The final output of a Routing Chain."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "d582004a",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Tools\n",
|
||||
"When constructing your own Routing Chain, you will need to provide it with a list of tools that it can use. This is done with a list of ToolConfigs. The ToolConfig is used not only to create the Routing Chain, but is also sometimes used to create the router itself (often, the router logic depends on the tools available). \n",
|
||||
"\n",
|
||||
"```python\n",
|
||||
"class ToolConfig(NamedTuple):\n",
|
||||
" \"\"\"Configuration for tools.\"\"\"\n",
|
||||
"\n",
|
||||
" tool_name: str\n",
|
||||
" tool: Callable[[str], str]\n",
|
||||
" # Needed to construct some routers.\n",
|
||||
" tool_description: Optional[str] = None\n",
|
||||
"```\n",
|
||||
"\n",
|
||||
"The two required components of a ToolConfig are the name and then the tool itself. A tool description is optional, as it is needed for some routers but not all."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "2558a02d",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Router\n",
|
||||
"A central piece of this chain is the router. The router is responsible for taking user input and deciding which tools, if any, to use. Although it doesn't necessarily have to be backed by a language model (LLM), for pretty much all current use cases it is. LLMs make great routers because they are really good at understanding human intent, which makes them perfect for choosing which tools to use (and for interpreting the output of those tools).\n",
|
||||
"\n",
|
||||
"Below is the interface we expect routers to expose, along with the RouterOutput definition.\n",
|
||||
"\n",
|
||||
"```python\n",
|
||||
"\n",
|
||||
"class RouterOutput(NamedTuple):\n",
|
||||
" \"\"\"Output of a router.\"\"\"\n",
|
||||
"\n",
|
||||
" tool: str\n",
|
||||
" tool_input: str\n",
|
||||
" log: str\n",
|
||||
" \n",
|
||||
"\n",
|
||||
"class Router(ABC):\n",
|
||||
" \"\"\"Chain responsible for deciding the action to take.\"\"\"\n",
|
||||
"\n",
|
||||
" @abstractmethod\n",
|
||||
" def route(self, text: str) -> RouterOutput:\n",
|
||||
" \"\"\"Given input, decided how to route it.\n",
|
||||
"\n",
|
||||
" Args:\n",
|
||||
" text: input string\n",
|
||||
"\n",
|
||||
" Returns:\n",
|
||||
" RouterOutput specifying what tool to use.\n",
|
||||
" \"\"\"\n",
|
||||
"\n",
|
||||
" @property\n",
|
||||
" @abstractmethod\n",
|
||||
" def observation_prefix(self) -> str:\n",
|
||||
" \"\"\"Prefix to append the observation with before calling the router again.\"\"\"\n",
|
||||
"\n",
|
||||
" @property\n",
|
||||
" @abstractmethod\n",
|
||||
" def router_prefix(self) -> str:\n",
|
||||
" \"\"\"Prefix to prepend the router call with.\"\"\"\n",
|
||||
"\n",
|
||||
" @property\n",
|
||||
" def finish_tool_name(self) -> str:\n",
|
||||
" \"\"\"Name of the tool to use to finish the chain.\"\"\"\n",
|
||||
" return \"Final Answer\"\n",
|
||||
"\n",
|
||||
" @property\n",
|
||||
" def starter_string(self) -> str:\n",
|
||||
" \"\"\"Put this string after user input but before first router call.\"\"\"\n",
|
||||
" return \"\\n\"\n",
|
||||
"```"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "be30fc0d",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Routing Chains\n",
|
||||
"The above are definitions for Routers and ToolConfigs. These are useful if you want to create your own Router. In practice, you will most likely use an existing RoutingChain. Below we cover a few different types of RoutingChains. Please select the one that best suites your needs.\n",
|
||||
"\n",
|
||||
"### ReAct on Docstore\n",
|
||||
"To construct this RoutingChain, please provide a `Docstore` object along with the LLM to use as the router:\n",
|
||||
"\n",
|
||||
"```python\n",
|
||||
"chain = ReActChain(llm, docstore)\n",
|
||||
"```\n",
|
||||
"\n",
|
||||
"### Self-Ask with Search\n",
|
||||
"To construct this RoutingChain, please provide a `Search` object along with the LLM to use as the router.\n",
|
||||
"\n",
|
||||
"```python\n",
|
||||
"chain = SelfAskWithSearch(llm, search)\n",
|
||||
"```\n",
|
||||
"\n",
|
||||
"### MRKL Zero Shot\n",
|
||||
"To construct this RoutingChain, a `tool_description` must be specified for each tool. You may provide any number of tools, along side the LLM to use as a router.\n",
|
||||
"\n",
|
||||
"```python\n",
|
||||
"chain = MRKLChain.from_tools(llm, tools)\n",
|
||||
"```"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "1ee765cc",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": []
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3 (ipykernel)",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.7.6"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
}
|
||||
Loading…
Reference in New Issue