cr

docs/examples/demos/custom_routing_chains.ipynb

@@ -0,0 +1,123 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "f6766398",
+   "metadata": {},
+   "source": [
+    "# Custom Routing Chains\n",
+    "\n",
+    "This covers how to implement a custom routing chain. That problem really reduces to how to implement a custom router. This also acts as a design doc of sorts for routers."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "f909b220",
+   "metadata": {},
+   "source": [
+    "## Terminology\n",
+    "\n",
+    "Before going through any code, let's align on some terminology.\n",
+    "- Tool: A function that performs a specific duty. This can be things like: Google Search, Database lookup, Python REPL. The interface for a tool is currently a function that is expected to have a string as an input, with a string as an output.\n",
+    "- Tool Input: The input string to a tool.\n",
+    "- Observation: The output from calling a tool on a particular input.\n",
+    "- Router: The object responsible for deciding which tools to call and when. Exposes a `route` method, which takes in a string and returns a Router Output.\n",
+    "- Router Output: The object returned from calling `Router.route` on a string. Consists of:\n",
+    "    - The tool to use\n",
+    "    - The input to that tool\n",
+    "    - A log of the router's thinking.\n",
+    "- Routing Chain: A chain which is made up of a router and suite of tools. When passed a string, the Routing Chain will iterative call tools as needed until it arrives at a Final Answer.\n",
+    "- Final Answer: The final output of a Routing Chain."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "7e14d6e8",
+   "metadata": {},
+   "source": [
+    "## Router\n",
+    "A central piece of this chain is the router. The router is responsible for taking user input and deciding which tools, if any, to use. Although it doesn't necessarily have to be backed by a language model (LLM), for pretty much all current use cases it is. LLMs make great routers because they are really good at understanding human intent, which makes them perfect for choosing which tools to use (and for interpreting the output of those tools).\n",
+    "\n",
+    "Below is the interface we expect routers to expose, along with the RouterOutput definition.\n",
+    "\n",
+    "```python\n",
+    "\n",
+    "class RouterOutput(NamedTuple):\n",
+    "    \"\"\"Output of a router.\"\"\"\n",
+    "\n",
+    "    tool: str\n",
+    "    tool_input: str\n",
+    "    log: str\n",
+    "        \n",
+    "\n",
+    "class Router(ABC):\n",
+    "    \"\"\"Chain responsible for deciding the action to take.\"\"\"\n",
+    "\n",
+    "    @abstractmethod\n",
+    "    def route(self, text: str) -> RouterOutput:\n",
+    "        \"\"\"Given input, decided how to route it.\n",
+    "\n",
+    "        Args:\n",
+    "            text: input string\n",
+    "\n",
+    "        Returns:\n",
+    "            RouterOutput specifying what tool to use.\n",
+    "        \"\"\"\n",
+    "\n",
+    "    @property\n",
+    "    @abstractmethod\n",
+    "    def observation_prefix(self) -> str:\n",
+    "        \"\"\"Prefix to append the observation with before calling the router again.\"\"\"\n",
+    "\n",
+    "    @property\n",
+    "    @abstractmethod\n",
+    "    def router_prefix(self) -> str:\n",
+    "        \"\"\"Prefix to prepend the router call with.\"\"\"\n",
+    "\n",
+    "    @property\n",
+    "    def finish_tool_name(self) -> str:\n",
+    "        \"\"\"Name of the tool to use to finish the chain.\"\"\"\n",
+    "        return \"Final Answer\"\n",
+    "\n",
+    "    @property\n",
+    "    def starter_string(self) -> str:\n",
+    "        \"\"\"Put this string after user input but before first router call.\"\"\"\n",
+    "        return \"\\n\"\n",
+    "```"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "8f4730a3",
+   "metadata": {},
+   "source": [
+    "In order to understand why the router interface is what it is, let's take a look at how it is used in the RoutingChain class:\n",
+    "\n",
+    "```python\n",
+    "\n",
+    "```"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.7.6"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

docs/examples/demos/routing_chains.ipynb

@@ -8,32 +8,25 @@
     "# Routing Chains\n",
     "Some applications will require not just a predetermined chain of calls to LLMs/other tools, but potentially an unknown chain that depends on the user input. In these types of chains, there is a \"router\" LLM chain which has access to a suite of tools. Depending on the user input, the router can then decide which, if any, of these tools to call.\n",
     "\n",
-    "These types of chains are called Routing Chains. When used correctly these can be extremely powerful. The purpose of this notebook is to go through the core abstractions at the heart of Routing Chains and enable you to build applications with them."
+    "These types of chains are called Routing Chains. When used correctly these can be extremely powerful. The purpose of this notebook is to show you how to easily use routing chains through the simplest, highest level API. If you want more low level control over various components, check out the documentation for custom routing chains."
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "efb8e2b6",
+   "id": "4b21ae68",
    "metadata": {},
    "source": [
-    "## Terminology\n",
+    "## Concepts\n",
     "\n",
-    "Before going through any code, let's align on some terminology.\n",
-    "- Tool: A function that performs a specific duty. This can be things like: Google Search, Database lookup, Python REPL. The interface for a tool is currently a function that is expected to have a string as an input, with a string as an output.\n",
-    "- Tool Input: The input string to a tool.\n",
-    "- Observation: The output from calling a tool on a particular input.\n",
-    "- Router: The object responsible for deciding which tools to call and when. Exposes a `route` method, which takes in a string and returns a Router Output.\n",
-    "- Router Output: The object returned from calling `Router.route` on a string. Consists of:\n",
-    "    - The tool to use\n",
-    "    - The input to that tool\n",
-    "    - A log of the router's thinking. This is not required, as it's only used for logging purposes, but it can be helpful for understanding why the router is acting why it is\n",
-    "- Routing Chain: A chain which is made up of a router and suite of tools. When passed a string, the Routing Chain will iterative call tools as needed until it arrives at a Final Answer.\n",
-    "- Final Answer: The final output of a Routing Chain."
+    "In order to understand routing chains, you should understand the following concepts:\n",
+    "- Tool: A function that performs a specific duty. This can be things like: Google Search, Database lookup, Python REPL, other chains. The interface for a tool is currently a function that is expected to have a string as an input, with a string as an output.\n",
+    "- LLM: The language model responsible for doing the router.\n",
+    "- RouterType: The type of the router to use. This should be a string (see more on the allowed router types below). Because this notebook focuses on the simplest, highest level API, this only covers using the standard supported routers. If you want to implement a custom router, see the documentation for custom routing chains."
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "d582004a",
+   "id": "c9677868",
    "metadata": {},
    "source": [
     "## Tools\n",
@@ -57,91 +50,22 @@
    "id": "2558a02d",
    "metadata": {},
    "source": [
-    "## Router\n",
-    "A central piece of this chain is the router. The router is responsible for taking user input and deciding which tools, if any, to use. Although it doesn't necessarily have to be backed by a language model (LLM), for pretty much all current use cases it is. LLMs make great routers because they are really good at understanding human intent, which makes them perfect for choosing which tools to use (and for interpreting the output of those tools).\n",
-    "\n",
-    "Below is the interface we expect routers to expose, along with the RouterOutput definition.\n",
-    "\n",
-    "```python\n",
-    "\n",
-    "class RouterOutput(NamedTuple):\n",
-    "    \"\"\"Output of a router.\"\"\"\n",
-    "\n",
-    "    tool: str\n",
-    "    tool_input: str\n",
-    "    log: str\n",
-    "        \n",
-    "\n",
-    "class Router(ABC):\n",
-    "    \"\"\"Chain responsible for deciding the action to take.\"\"\"\n",
-    "\n",
-    "    @abstractmethod\n",
-    "    def route(self, text: str) -> RouterOutput:\n",
-    "        \"\"\"Given input, decided how to route it.\n",
-    "\n",
-    "        Args:\n",
-    "            text: input string\n",
-    "\n",
-    "        Returns:\n",
-    "            RouterOutput specifying what tool to use.\n",
-    "        \"\"\"\n",
-    "\n",
-    "    @property\n",
-    "    @abstractmethod\n",
-    "    def observation_prefix(self) -> str:\n",
-    "        \"\"\"Prefix to append the observation with before calling the router again.\"\"\"\n",
-    "\n",
-    "    @property\n",
-    "    @abstractmethod\n",
-    "    def router_prefix(self) -> str:\n",
-    "        \"\"\"Prefix to prepend the router call with.\"\"\"\n",
-    "\n",
-    "    @property\n",
-    "    def finish_tool_name(self) -> str:\n",
-    "        \"\"\"Name of the tool to use to finish the chain.\"\"\"\n",
-    "        return \"Final Answer\"\n",
-    "\n",
-    "    @property\n",
-    "    def starter_string(self) -> str:\n",
-    "        \"\"\"Put this string after user input but before first router call.\"\"\"\n",
-    "        return \"\\n\"\n",
-    "```"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "be30fc0d",
-   "metadata": {},
-   "source": [
-    "## Routing Chains\n",
-    "The above are definitions for Routers and ToolConfigs. These are useful if you want to create your own Router. In practice, you will most likely use an existing RoutingChain. Below we cover a few different types of RoutingChains. Please select the one that best suites your needs.\n",
-    "\n",
-    "### ReAct on Docstore\n",
-    "To construct this RoutingChain, please provide a `Docstore` object along with the LLM to use as the router:\n",
+    "## Loading the chains\n",
     "\n",
     "```python\n",
-    "chain = ReActChain(llm, docstore)\n",
-    "```\n",
-    "\n",
-    "### Self-Ask with Search\n",
-    "To construct this RoutingChain, please provide a `Search` object along with the LLM to use as the router.\n",
+    "from langchain.routing_chains import load_chain\n",
     "\n",
-    "```python\n",
-    "chain = SelfAskWithSearch(llm, search)\n",
-    "```\n",
-    "\n",
-    "### MRKL Zero Shot\n",
-    "To construct this RoutingChain, a `tool_description` must be specified for each tool. You may provide any number of tools, along side the LLM to use as a router.\n",
-    "\n",
-    "```python\n",
-    "chain = MRKLChain.from_tools(llm, tools)\n",
+    "tools: List[ToolConfig] = [...]\n",
+    "llm: LLM = OpenAI(temperature=0)\n",
+    "router_type: str = \"zero-shot\"\n",
+    "chain = load_chain(tools, llm, router_type, verbose=True)\n",
     "```"
    ]
   },
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "1ee765cc",
+   "id": "d3254c6c",
    "metadata": {},
    "outputs": [],
    "source": []

langchain/routing_chains/routing_chain.py

@@ -48,23 +48,40 @@ class RoutingChain(Chain, BaseModel):
         return [self.output_key]
 
     def _call(self, inputs: Dict[str, str]) -> Dict[str, str]:
+        # Construct a mapping of tool name to tool for easy lookup
         name_to_tool_map = {tc.tool_name: tc.tool for tc in self.tool_configs}
+        # Construct the initial string to pass into the router. This is made up
+        # of the user input, the special starter string, and then the router prefix.
+        # The starter string is a special string that may be used by a router to
+        # immediately follow the user input. The router prefix is a string that
+        # prompts the router to start routing.
         starter_string = (
             inputs[self.input_key]
             + self.router.starter_string
             + self.router.router_prefix
         )
+        # We use the ChainedInput class to iteratively add to the input over time.
         chained_input = ChainedInput(starter_string, verbose=self.verbose)
+        # We construct a mapping from each tool to a color, used for logging.
         color_mapping = get_color_mapping(
             [c.tool_name for c in self.tool_configs], excluded_colors=["green"]
         )
+        # We now enter the router loop (until it returns something).
         while True:
+            # Call the router to see what to do.
             output = self.router.route(chained_input.input)
+            # Add the log to the Chained Input.
             chained_input.add(output.log, color="green")
+            # If the tool chosen is the finishing tool, then we end and return.
             if output.tool == self.router.finish_tool_name:
                 return {self.output_key: output.tool_input}
+            # Otherwise we lookup the tool
             chain = name_to_tool_map[output.tool]
+            # We then call the tool on the tool input to get an observation
             observation = chain(output.tool_input)
+            # We then log the observation
             chained_input.add(f"\n{self.router.observation_prefix}")
             chained_input.add(observation, color=color_mapping[output.tool])
+            # We then add the router prefix into the prompt to get the router to start
+            # thinking, and start the loop all over.
             chained_input.add(f"\n{self.router.router_prefix}")