Harrison/llm final stuff (#332)

README.md

@@ -31,14 +31,25 @@ Please see [here](https://langchain.readthedocs.io/en/latest/?) for full documen
 There are four main areas that LangChain is designed to help with.
 These are, in increasing order of complexity:
 
-1. LLM and Prompts
-2. Chains
-3. Agents
-4. Memory
+**📃 LLMs and Prompts:**
+
+This includes prompt management, prompt optimization, generic interface for all LLMs, and common utilities for working with LLMs.
+
+**🔗 Chains:**
+
+Chains go beyond just a single LLM call, and are sequences of calls (whether to an LLM or a different utility). LangChain provides a standard interface for chains, lots of integrations with other tools, and end-to-end chains for common applications.
+
+**🤖 Agents:**
+
+Agents involve an LLM making decisions about which Actions to take, taking that Action, seeing an Observation, and repeating that until done. LangChain provides a standard interface for agents, a selection of agents to choose from, and examples of end to end agents.
+
+**🧠 Memory:**
+
+Memory is the concept of persisting state between calls of a chain/agent. LangChain provides a standard interface for memory, a collection of memory implementations, and examples of chains/agents that use memory.
 
 For more information on these concepts, please see our [full documentation](https://langchain.readthedocs.io/en/latest/?).
 
-## 🤖 Contributing
+## 💁 Contributing
 
 As an open source project in a rapidly developing field, we are extremely open
 to contributions, whether it be in the form of a new feature, improved infra, or better documentation.

docs/examples/chains/summarize.ipynb

@@ -131,7 +131,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 8,
+   "execution_count": 6,
    "id": "ef28e1d4",
    "metadata": {},
    "outputs": [],
@@ -148,7 +148,7 @@
     {
      "data": {
       "text/plain": [
-       "' In response to Russian aggression in Ukraine, the US and its allies have imposed economic sanctions, cut off access to technology, seized assets of Russian oligarchs, and closed American airspace to Russian flights. The US is also providing military, economic, and humanitarian assistance to Ukraine, mobilizing ground forces, air squadrons, and ship deployments, and releasing 30 million barrels of oil from its Strategic Petroleum Reserve. President Biden has also passed the American Rescue Plan, Bipartisan Infrastructure Law, and Bipartisan Innovation Act to provide economic relief and create jobs.'"
+       "\" In response to Vladimir Putin's aggression in Ukraine, the US and its allies have taken action to hold him accountable, including economic sanctions, cutting off access to technology, and seizing the assets of Russian oligarchs. They are also providing military, economic, and humanitarian assistance to the Ukrainians, and releasing 60 million barrels of oil from reserves around the world. President Biden has passed several laws to provide economic relief to Americans and create jobs, and is making sure taxpayer dollars support American jobs and businesses.\""
       ]
      },
      "execution_count": 9,

docs/examples/prompts.rst

@@ -1,10 +1,35 @@
-Prompts
-=======
+LLMs & Prompts
+==============
+
+The examples here all highlight how to work with LLMs and prompts.
+
+**LLMs**
+
+`LLM Functionality <prompts/llm_functionality.ipynb>`_: A walkthrough of all the functionality the standard LLM interface exposes.
+
+`LLM Serialization <prompts/llm_serialization.ipynb>`_: A walkthrough of how to serialize LLMs to and from disk.
+
+`Custom LLM <prompts/custom_llm.ipynb>`_: How to create and use a custom LLM class, in case you have an LLM not from one of the standard providers (including one that you host yourself).
+
+
+**Prompts**
+
+`Prompt Management <prompts/prompt_management.ipynb>`_: A walkthrough of all the functionality LangChain supports for working with prompts.
+
+`Prompt Serialization <prompts/prompt_serialization.ipynb>`_: A walkthrough of how to serialize prompts to and from disk.
+
+`Few Shot Examples <prompts/few_shot_examples.ipynb>`_: How to include examples in the prompt.
+
+`Generate Examples <prompts/generate_examples.ipynb>`_: How to use existing examples to generate more examples.
+
+`Custom Example Selector <prompts/custom_example_selector.ipynb>`_: How to create and use a custom ExampleSelector (the class responsible for choosing which examples to use in a prompt).
+
+`Custom Prompt Template <prompts/custom_prompt_template.ipynb>`_: How to create and use a custom PromptTemplate, the logic that decides how input variables get formatted into a prompt.
 
-The examples here all highlight how to work with prompts.
 
 .. toctree::
    :maxdepth: 1
    :glob:
+   :hidden:
 
    prompts/*

docs/examples/prompts/llm.json

@@ -0,0 +1,11 @@
+{
+    "model_name": "text-davinci-003",
+    "temperature": 0.7,
+    "max_tokens": 256,
+    "top_p": 1.0,
+    "frequency_penalty": 0.0,
+    "presence_penalty": 0.0,
+    "n": 1,
+    "best_of": 1,
+    "_type": "openai"
+}

docs/examples/prompts/llm.yaml

@@ -0,0 +1,9 @@
+_type: openai
+best_of: 1
+frequency_penalty: 0.0
+max_tokens: 256
+model_name: text-davinci-003
+n: 1
+presence_penalty: 0.0
+temperature: 0.7
+top_p: 1.0

docs/examples/prompts/llm_functionality.ipynb

@@ -0,0 +1,222 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "060f7bc9",
+   "metadata": {},
+   "source": [
+    "# LLM Functionality\n",
+    "\n",
+    "This notebook goes over all the different features of the LLM class in LangChain.\n",
+    "\n",
+    "We will work with an OpenAI LLM wrapper, although these functionalities should exist for all LLM types."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "5bddaa9a",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.llms import OpenAI"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "id": "f6bed875",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "llm = OpenAI(model_name=\"text-ada-001\", n=2, best_of=2)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "edb2f14e",
+   "metadata": {},
+   "source": [
+    "**Generate Text:** The most basic functionality an LLM has is just the ability to call it, passing in a string and getting back a string."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "c29ba285",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "'\\n\\nWhy did the chicken cross the road?\\n\\nTo get to the other side.'"
+      ]
+     },
+     "execution_count": 3,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "llm(\"Tell me a joke\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "1f4a350a",
+   "metadata": {},
+   "source": [
+    "**Generate:** More broadly, you can call it with a list of inputs, getting back a more complete response than just the text. This complete response includes things like multiple top responses, as well as LLM provider specific information"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "id": "a586c9ca",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "llm_result = llm.generate([\"Tell me a joke\", \"Tell me a poem\"])"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "id": "22470289",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "[Generation(text='\\n\\nWhy did the chicken cross the road?\\n\\nTo get to the other side.'),\n",
+       " Generation(text='\\n\\nWhy did the chicken cross the road?\\n\\nTo get to the other side!')]"
+      ]
+     },
+     "execution_count": 5,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "# Results of the first input\n",
+    "llm_result.generations[0]"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "id": "a1e72553",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "[Generation(text='\\n\\nIn the eyes of the moon\\n\\nI have seen a face\\n\\nThat I will never forget.\\n\\nThe light that I see\\n\\nIs like a fire in my heart.\\n\\nEvery letter I write\\n\\nWill be the last word\\n\\nOf my love for this person.\\n\\nThe darkness that I feel\\n\\nIs like a weight on my heart.'),\n",
+       " Generation(text=\"\\n\\nA rose by the side of the road\\n\\nIs all I need to find my way\\n\\nTo the place I've been searching for\\n\\nAnd my heart is singing with joy\\n\\nWhen I look at this rose\\n\\nIt reminds me of the love I've found\\n\\nAnd I know that wherever I go\\n\\nI'll always find my rose by the side of the road.\")]"
+      ]
+     },
+     "execution_count": 6,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "# Results of the second input\n",
+    "llm_result.generations[1]"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "id": "90c52536",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "{'token_usage': <OpenAIObject at 0x10b4f0d10> JSON: {\n",
+       "   \"completion_tokens\": 199,\n",
+       "   \"prompt_tokens\": 8,\n",
+       "   \"total_tokens\": 207\n",
+       " }}"
+      ]
+     },
+     "execution_count": 7,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "# Provider specific info\n",
+    "llm_result.llm_output"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "92f6e7a5",
+   "metadata": {},
+   "source": [
+    "**Number of Tokens:** You can also estimate how many tokens a piece of text will be in that model. This is useful because models have a context length (and cost more for more tokens), which means you need to be aware of how long the text you are passing in is.\n",
+    "\n",
+    "Notice that by default the tokens are estimated using a HuggingFace tokenizer."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "id": "acfd9200",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "None of PyTorch, TensorFlow >= 2.0, or Flax have been found. Models won't be available and only tokenizers, configuration and file/data utilities can be used.\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "3"
+      ]
+     },
+     "execution_count": 8,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "llm.get_num_tokens(\"what a joke\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "68ff3688",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.8"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

docs/examples/prompts/llm_serialization.ipynb

@@ -0,0 +1,166 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "73f9bf40",
+   "metadata": {},
+   "source": [
+    "# LLM Serialization\n",
+    "\n",
+    "This notebook walks how to write and read an LLM Configuration to and from disk. This is useful if you want to save the configuration for a given LLM (eg the provider, the temperature, etc)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "9c9fb6ff",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain.llms import OpenAI\n",
+    "from langchain.llms.loading import load_llm"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "88ce018b",
+   "metadata": {},
+   "source": [
+    "### Loading\n",
+    "First, lets go over loading a LLM from disk. LLMs can be saved on disk in two formats: json or yaml. No matter the extension, they are loaded in the same way."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "id": "f12b28f3",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "{\r\n",
+      "    \"model_name\": \"text-davinci-003\",\r\n",
+      "    \"temperature\": 0.7,\r\n",
+      "    \"max_tokens\": 256,\r\n",
+      "    \"top_p\": 1,\r\n",
+      "    \"frequency_penalty\": 0,\r\n",
+      "    \"presence_penalty\": 0,\r\n",
+      "    \"n\": 1,\r\n",
+      "    \"best_of\": 1,\r\n",
+      "    \"_type\": \"openai\"\r\n",
+      "}"
+     ]
+    }
+   ],
+   "source": [
+    "!cat llm.json"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "9ab709fc",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "llm = load_llm(\"llm.json\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "id": "095b1d56",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "_type: openai\r\n",
+      "best_of: 1\r\n",
+      "frequency_penalty: 0\r\n",
+      "max_tokens: 256\r\n",
+      "model_name: text-davinci-003\r\n",
+      "n: 1\r\n",
+      "presence_penalty: 0\r\n",
+      "temperature: 0.7\r\n",
+      "top_p: 1\r\n"
+     ]
+    }
+   ],
+   "source": [
+    "!cat llm.yaml"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "id": "8cafaafe",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "llm = load_llm(\"llm.yaml\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "ab3e4223",
+   "metadata": {},
+   "source": [
+    "### Saving\n",
+    "If you want to go from a LLM in memory to a serialized version of it, you can do so easily by calling the `.save` method. Again, this supports both json and yaml."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "id": "b38f685d",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "llm.save(\"llm.json\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "id": "b7365503",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "llm.save(\"llm.yaml\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "0e494851",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.8"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

langchain/chains/llm.py

@@ -51,18 +51,34 @@ class LLMChain(Chain, BaseModel):
         """
         return [self.output_key]
 
+    def apply(self, input_list: List[Dict[str, Any]]) -> List[Dict[str, str]]:
+        """Utilize the LLM generate method for speed gains."""
+        stop = None
+        if "stop" in input_list[0]:
+            stop = input_list[0]["stop"]
+        prompts = []
+        for inputs in input_list:
+            selected_inputs = {k: inputs[k] for k in self.prompt.input_variables}
+            prompt = self.prompt.format(**selected_inputs)
+            if self.verbose:
+                langchain.logger.log_llm_inputs(selected_inputs, prompt)
+            if "stop" in inputs and inputs["stop"] != stop:
+                raise ValueError(
+                    "If `stop` is present in any inputs, should be present in all."
+                )
+            prompts.append(prompt)
+        response = self.llm.generate(prompts, stop=stop)
+        outputs = []
+        for generation in response.generations:
+            # Get the text of the top generated string.
+            response_str = generation[0].text
+            if self.verbose:
+                langchain.logger.log_llm_response(response_str)
+            outputs.append({self.output_key: response_str})
+        return outputs
+
     def _call(self, inputs: Dict[str, Any]) -> Dict[str, str]:
-        selected_inputs = {k: inputs[k] for k in self.prompt.input_variables}
-        prompt = self.prompt.format(**selected_inputs)
-        if self.verbose:
-            langchain.logger.log_llm_inputs(selected_inputs, prompt)
-        kwargs = {}
-        if "stop" in inputs:
-            kwargs["stop"] = inputs["stop"]
-        response = self.llm(prompt, **kwargs)
-        if self.verbose:
-            langchain.logger.log_llm_response(response)
-        return {self.output_key: response}
+        return self.apply([inputs])[0]
 
     def predict(self, **kwargs: Any) -> str:
         """Format prompt with kwargs and pass to LLM.