diff --git a/README.md b/README.md index 38d09b92..d7eb1f56 100644 --- a/README.md +++ b/README.md @@ -45,8 +45,11 @@ However, there are still some challenges going from that to an application runni - LLM: A large language model, in particular a text-to-text model. - Prompt: The input to a language model. Typically this is not simply a hardcoded string but rather a combination of a template, some examples, and user input. - Prompt Template: An object responsible for constructing the final prompt to pass to a LLM. +- Examples: Datapoints that can be included in the prompt in order to give the model more context what to do. +- Few Shot Prompt Template: A subclass of the PromptTemplate class that uses examples. +- Example Selector: A class responsible to selecting examples to use dynamically (depending on user input) in a few shot prompt. -**Problems solved** +**Problems Solved** - Switching costs: by exposing a standard interface for all the top LLM providers, LangChain makes it easy to switch from one provider to another, whether it be for production use cases or just for testing stuff out. - Prompt management: managing your prompts is easy when you only have one simple one, but can get tricky when you have a bunch or when they start to get more complex. LangChain provides a standard way for storing, constructing, and referencing prompts. - Prompt optimization: despite the underlying models getting better and better, there is still currently a need for carefully constructing prompts. @@ -59,7 +62,7 @@ LangChain provides several parts to help with that. - Tools: APIs designed for assisting with a particular use case (search, databases, Python REPL, etc). Prompt templates, LLMs, and chains can also be considered tools. - Chains: A combination of multiple tools in a deterministic manner. -**Problems solved** +**Problems Solved** - Standard interface for working with Chains - Easy way to construct chains of LLMs - Lots of integrations with other tools that you may want to use in conjunction with LLMs @@ -75,13 +78,24 @@ Depending on the user input, the agent can then decide which, if any, of these t - Agent: An LLM-powered class responsible for determining which tools to use and in what order. -**Problems solved** +**Problems Solved** - Standard agent interfaces - A selection of powerful agents to choose from - Common chains that can be used as tools ### Memory -Coming soon. +By default, Chains and Agents are stateless, meaning that they treat each incoming query independently. +In some applications (chatbots being a GREAT example) it is highly important to remember previous interactions, +both at a short term but also at a long term level. The concept of "Memory" exists to do exactly that. + +**Key Concepts** +- Memory: A class that can be added to an Agent or Chain to (1) pull in memory variables before calling that chain/agent, and (2) create new memories after the chain/agent finishes. +- Memory Variables: Variables returned from a Memory class, to be passed into the chain/agent along with the user input. + +**Problems Solved** +- Standard memory interfaces +- A collection of common memory implementations to choose from +- Common chains/agents that use memory (e.g. chatbots) ## 🤖 Developer Guide diff --git a/docs/examples/agents/custom_agent.ipynb b/docs/examples/agents/custom_agent.ipynb new file mode 100644 index 00000000..b97921e4 --- /dev/null +++ b/docs/examples/agents/custom_agent.ipynb @@ -0,0 +1,232 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "ba5f8741", + "metadata": {}, + "source": [ + "# Custom Agent\n", + "\n", + "This notebook goes through how to create your own custom agent.\n", + "\n", + "An agent consists of three parts:\n", + " \n", + " - Tools: The tools the agent has available to use.\n", + " - LLMChain: The LLMChain that produces the text that is parsed in a certain way to determine which action to take.\n", + " - The agent class itself: this parses the output of the LLMChain to determin which action to take.\n", + " \n", + " \n", + "In this notebook we walk through two types of custom agents. The first type shows how to create a custom LLMChain, but still use an existing agent class to parse the output. The second shows how to create a custom agent class." + ] + }, + { + "cell_type": "markdown", + "id": "6064f080", + "metadata": {}, + "source": [ + "### Custom LLMChain\n", + "\n", + "The first way to create a custom agent is to use an existing Agent class, but use a custom LLMChain. This is the simplest way to create a custom Agent. It is highly reccomended that you work with the `ZeroShotAgent`, as at the moment that is by far the most generalizable one. \n", + "\n", + "Most of the work in creating the custom LLMChain comes down to the prompt. Because we are using an existing agent class to parse the output, it is very important that the prompt say to produce text in that format. However, besides those instructions, you can customize the prompt as you wish.\n", + "\n", + "To ensure that the prompt contains the appropriate instructions, we will utilize a helper method on that class. The helper method for the `ZeroShotAgent` takes the following arguments:\n", + "\n", + "- tools: List of tools the agent will have access to, used to format the prompt.\n", + "- prefix: String to put before the list of tools.\n", + "- suffix: String to put after the list of tools.\n", + "- input_variables: List of input variables the final prompt will expect.\n", + "\n", + "For this exercise, we will give our agent access to Google Search, and we will customize it in that we will have it answer as a pirate." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "9af9734e", + "metadata": {}, + "outputs": [], + "source": [ + "from langchain.agents import ZeroShotAgent, Tool\n", + "from langchain import OpenAI, SerpAPIChain, LLMChain" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "becda2a1", + "metadata": {}, + "outputs": [], + "source": [ + "search = SerpAPIChain()\n", + "tools = [\n", + " Tool(\n", + " name = \"Search\",\n", + " func=search.run,\n", + " description=\"useful for when you need to answer questions about current events\"\n", + " )\n", + "]" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "339b1bb8", + "metadata": {}, + "outputs": [], + "source": [ + "prefix = \"\"\"Answer the following questions as best you can, but speaking as a pirate might speak. You have access to the following tools:\"\"\"\n", + "suffix = \"\"\"Begin! Remember to speak as a pirate when giving your final answer. Use lots of \"Args\"\n", + "\n", + "Question: {input}\"\"\"\n", + "\n", + "prompt = ZeroShotAgent.create_prompt(\n", + " tools, \n", + " prefix=prefix, \n", + " suffix=suffix, \n", + " input_variables=[\"input\"]\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "59db7b58", + "metadata": {}, + "source": [ + "In case we are curious, we can now take a look at the final prompt template to see what it looks like when its all put together." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "e21d2098", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Answer the following questions as best you can, but speaking as a pirate might speak. You have access to the following tools:\n", + "\n", + "Search: useful for when you need to answer questions about current events\n", + "\n", + "Use the following format:\n", + "\n", + "Question: the input question you must answer\n", + "Thought: you should always think about what to do\n", + "Action: the action to take, should be one of [Search]\n", + "Action Input: the input to the action\n", + "Observation: the result of the action\n", + "... (this Thought/Action/Action Input/Observation can repeat N times)\n", + "Thought: I now know the final answer\n", + "Final Answer: the final answer to the original input question\n", + "\n", + "Begin! Remember to speak as a pirate when giving your final answer. Use lots of \"Args\"\n", + "\n", + "Question: {input}\n" + ] + } + ], + "source": [ + "print(prompt.template)" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "9b1cc2a2", + "metadata": {}, + "outputs": [], + "source": [ + "llm_chain = LLMChain(llm=OpenAI(temperature=0), prompt=prompt)" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "e4f5092f", + "metadata": {}, + "outputs": [], + "source": [ + "agent = ZeroShotAgent(llm_chain=llm_chain, tools=tools, verbose=True)" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "653b1617", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "\n", + "\u001b[1m> Entering new chain...\u001b[0m\n", + "How many people live in canada?\n", + "Thought:\u001b[32;1m\u001b[1;3m I should look this up\n", + "Action: Search\n", + "Action Input: How many people live in canada\u001b[0m\n", + "Observation: \u001b[36;1m\u001b[1;3mThe current population of Canada is 38,533,678 as of Friday, November 25, 2022, based on Worldometer elaboration of the latest United Nations data. · Canada 2020 ...\u001b[0m\n", + "Thought:\u001b[32;1m\u001b[1;3m I now know the final answer\n", + "Final Answer: Arrr, there be 38,533,678 people in Canada\u001b[0m\n", + "\u001b[1m> Finished chain.\u001b[0m\n" + ] + }, + { + "data": { + "text/plain": [ + "'Arrr, there be 38,533,678 people in Canada'" + ] + }, + "execution_count": 7, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "agent.run(\"How many people live in canada?\")" + ] + }, + { + "cell_type": "markdown", + "id": "90171b2b", + "metadata": {}, + "source": [ + "### Custom Agent Class\n", + "\n", + "Coming soon." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "adefb4c2", + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.6" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/docs/examples/memory.rst b/docs/examples/memory.rst new file mode 100644 index 00000000..c5c064f8 --- /dev/null +++ b/docs/examples/memory.rst @@ -0,0 +1,11 @@ +Memory +====== + +The examples here are all related to working with the concept of Memory in LangChain. + +.. toctree:: + :maxdepth: 1 + :glob: + :caption: Memory + + memory/* \ No newline at end of file diff --git a/docs/examples/memory/adding_memory.ipynb b/docs/examples/memory/adding_memory.ipynb new file mode 100644 index 00000000..b6a23364 --- /dev/null +++ b/docs/examples/memory/adding_memory.ipynb @@ -0,0 +1,175 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "00695447", + "metadata": {}, + "source": [ + "# Adding Memory To an LLMChain\n", + "\n", + "This notebook goes over how to use the Memory class with an LLMChain. For the purposes of this walkthrough, we will add the `ConversationBufferMemory` class, although this can be any memory class." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "9f1aaf47", + "metadata": {}, + "outputs": [], + "source": [ + "from langchain.chains.conversation.memory import ConversationBufferMemory\n", + "from langchain import OpenAI, LLMChain, PromptTemplate" + ] + }, + { + "cell_type": "markdown", + "id": "4b066ced", + "metadata": {}, + "source": [ + "The most important step is setting up the prompt correctly. In the below prompt, we have two input keys: one for the actual input, another for the input from the Memory class. Importantly, we make sure the keys in the PromptTemplate and the ConversationBufferMemory match up (`chat_history`)." + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "e5501eda", + "metadata": {}, + "outputs": [], + "source": [ + "template = \"\"\"You are a chatbot having a conversation with a human.\n", + "\n", + "{chat_history}\n", + "Human: {human_input}\n", + "Chatbot:\"\"\"\n", + "\n", + "prompt = PromptTemplate(\n", + " input_variables=[\"chat_history\", \"human_input\"], \n", + " template=template\n", + ")\n", + "memory = ConversationBufferMemory(memory_key=\"chat_history\")" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "f6566275", + "metadata": {}, + "outputs": [], + "source": [ + "llm_chain = LLMChain(\n", + " llm=OpenAI(), \n", + " prompt=prompt, \n", + " verbose=True, \n", + " memory=memory,\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "e2b189dc", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "\n", + "\u001b[1m> Entering new chain...\u001b[0m\n", + "Prompt after formatting:\n", + "\u001b[32;1m\u001b[1;3mYou are a chatbot having a conversation with a human.\n", + "\n", + "\n", + "Human: Hi there my friend\n", + "Chatbot:\u001b[0m\n", + "\n", + "\u001b[1m> Finished chain.\u001b[0m\n" + ] + }, + { + "data": { + "text/plain": [ + "' Hi there!'" + ] + }, + "execution_count": 4, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "llm_chain.predict(human_input=\"Hi there my friend\")" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "a902729f", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "\n", + "\u001b[1m> Entering new chain...\u001b[0m\n", + "Prompt after formatting:\n", + "\u001b[32;1m\u001b[1;3mYou are a chatbot having a conversation with a human.\n", + "\n", + "\n", + "Human: Hi there my friend\n", + "AI: Hi there!\n", + "Human: Not to bad - how are you?\n", + "Chatbot:\u001b[0m\n", + "\n", + "\u001b[1m> Finished chain.\u001b[0m\n" + ] + }, + { + "data": { + "text/plain": [ + "\"\\n\\nI'm doing well, thanks for asking. How about you?\"" + ] + }, + "execution_count": 5, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "llm_chain.predict(human_input=\"Not to bad - how are you?\")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "ae5309bb", + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.6" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/docs/examples/memory/agent_with_memory.ipynb b/docs/examples/memory/agent_with_memory.ipynb new file mode 100644 index 00000000..fa93a7e0 --- /dev/null +++ b/docs/examples/memory/agent_with_memory.ipynb @@ -0,0 +1,325 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "fa6802ac", + "metadata": {}, + "source": [ + "# Adding Memory to an Agent\n", + "\n", + "This notebook goes over adding memory to an Agent. Before going through this notebook, please walkthrough the following notebooks, as this will build on top of both of them:\n", + "\n", + "- [Adding memory to an LLM Chain](adding_memory.ipynb)\n", + "- [Custom Agents](../agents/custom_agent.ipynb)\n", + "\n", + "In order to add a memory to an agent we are going to the the following steps:\n", + "\n", + "1. We are going to create an LLMChain with memory.\n", + "2. We are going to use that LLMChain to create a custom Agent.\n", + "\n", + "For the purposes of this exercise, we are going to create a simple custom Agent that has access to a search tool and utilizes the `ConversationBufferMemory` class." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "8db95912", + "metadata": {}, + "outputs": [], + "source": [ + "from langchain.agents import ZeroShotAgent, Tool\n", + "from langchain.chains.conversation.memory import ConversationBufferMemory\n", + "from langchain import OpenAI, SerpAPIChain, LLMChain" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "97ad8467", + "metadata": {}, + "outputs": [], + "source": [ + "search = SerpAPIChain()\n", + "tools = [\n", + " Tool(\n", + " name = \"Search\",\n", + " func=search.run,\n", + " description=\"useful for when you need to answer questions about current events\"\n", + " )\n", + "]" + ] + }, + { + "cell_type": "markdown", + "id": "4ad2e708", + "metadata": {}, + "source": [ + "Notice the usage of the `chat_history` variable in the PromptTemplate, which matches up with the dynamic key name in the ConversationBufferMemory." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "e3439cd6", + "metadata": {}, + "outputs": [], + "source": [ + "prefix = \"\"\"Have a conversation with a human, answering the following questions as best you can. You have access to the following tools:\"\"\"\n", + "suffix = \"\"\"Begin!\"\n", + "\n", + "{chat_history}\n", + "Question: {input}\"\"\"\n", + "\n", + "prompt = ZeroShotAgent.create_prompt(\n", + " tools, \n", + " prefix=prefix, \n", + " suffix=suffix, \n", + " input_variables=[\"input\", \"chat_history\"]\n", + ")\n", + "memory = ConversationBufferMemory(memory_key=\"chat_history\")" + ] + }, + { + "cell_type": "markdown", + "id": "0021675b", + "metadata": {}, + "source": [ + "We can now construct the LLMChain, with the Memory object, and then create the agent." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "c56a0e73", + "metadata": {}, + "outputs": [], + "source": [ + "llm_chain = LLMChain(llm=OpenAI(temperature=0), prompt=prompt, memory=memory)\n", + "agent = ZeroShotAgent(llm_chain=llm_chain, tools=tools, verbose=True)" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "ca4bc1fb", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "\n", + "\u001b[1m> Entering new chain...\u001b[0m\n", + "How many people live in canada?\n", + "Thought:\u001b[32;1m\u001b[1;3m I should look up how many people live in canada\n", + "Action: Search\n", + "Action Input: \"How many people live in canada?\"\u001b[0m\n", + "Observation: \u001b[36;1m\u001b[1;3mThe current population of Canada is 38,533,678 as of Friday, November 25, 2022, based on Worldometer elaboration of the latest United Nations data. · Canada 2020 ...\u001b[0m\n", + "Thought:\u001b[32;1m\u001b[1;3m I now know the final answer\n", + "Final Answer: The current population of Canada is 38,533,678 as of Friday, November 25, 2022, based on Worldometer elaboration of the latest United Nations data.\u001b[0m\n", + "\u001b[1m> Finished chain.\u001b[0m\n" + ] + }, + { + "data": { + "text/plain": [ + "'The current population of Canada is 38,533,678 as of Friday, November 25, 2022, based on Worldometer elaboration of the latest United Nations data.'" + ] + }, + "execution_count": 5, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "agent.run(\"How many people live in canada?\")" + ] + }, + { + "cell_type": "markdown", + "id": "45627664", + "metadata": {}, + "source": [ + "To test the memory of this agent, we can ask a followup question that relies on information in the previous exchange to be answered correctly." + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "eecc0462", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "\n", + "\u001b[1m> Entering new chain...\u001b[0m\n", + "what is their national anthem called?\n", + "Thought:\u001b[32;1m\u001b[1;3m\n", + "AI: I should look up the name of Canada's national anthem\n", + "Action: Search\n", + "Action Input: \"What is the name of Canada's national anthem?\"\u001b[0m\n", + "Observation: \u001b[36;1m\u001b[1;3mAfter 100 years of tradition, O Canada was proclaimed Canada's national anthem in 1980. The music for O Canada was composed in 1880 by Calixa ...\u001b[0m\n", + "Thought:\u001b[32;1m\u001b[1;3m\n", + "AI: I now know the final answer\n", + "Final Answer: After 100 years of tradition, O Canada was proclaimed Canada's national anthem in 1980. The music for O Canada was composed in 1880 by Calixa Lavallée.\u001b[0m\n", + "\u001b[1m> Finished chain.\u001b[0m\n" + ] + }, + { + "data": { + "text/plain": [ + "\"After 100 years of tradition, O Canada was proclaimed Canada's national anthem in 1980. The music for O Canada was composed in 1880 by Calixa Lavallée.\"" + ] + }, + "execution_count": 6, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "agent.run(\"what is their national anthem called?\")" + ] + }, + { + "cell_type": "markdown", + "id": "cc3d0aa4", + "metadata": {}, + "source": [ + "We can see that the agent remembered that the previous question was about Canada, and properly asked Google Search what the name of Canada's national anthem was.\n", + "\n", + "For fun, let's compare this to an agent that does NOT have memory." + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "3359d043", + "metadata": {}, + "outputs": [], + "source": [ + "prefix = \"\"\"Have a conversation with a human, answering the following questions as best you can. You have access to the following tools:\"\"\"\n", + "suffix = \"\"\"Begin!\"\n", + "\n", + "Question: {input}\"\"\"\n", + "\n", + "prompt = ZeroShotAgent.create_prompt(\n", + " tools, \n", + " prefix=prefix, \n", + " suffix=suffix, \n", + " input_variables=[\"input\"]\n", + ")\n", + "llm_chain = LLMChain(llm=OpenAI(temperature=0), prompt=prompt)\n", + "agent_without_memory = ZeroShotAgent(llm_chain=llm_chain, tools=tools, verbose=True)" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "id": "970d23df", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "\n", + "\u001b[1m> Entering new chain...\u001b[0m\n", + "How many people live in canada?\n", + "Thought:\u001b[32;1m\u001b[1;3m I should look up how many people live in canada\n", + "Action: Search\n", + "Action Input: \"How many people live in canada?\"\u001b[0m\n", + "Observation: \u001b[36;1m\u001b[1;3mThe current population of Canada is 38,533,678 as of Friday, November 25, 2022, based on Worldometer elaboration of the latest United Nations data. · Canada 2020 ...\u001b[0m\n", + "Thought:\u001b[32;1m\u001b[1;3m I now know the final answer\n", + "Final Answer: The current population of Canada is 38,533,678\u001b[0m\n", + "\u001b[1m> Finished chain.\u001b[0m\n" + ] + }, + { + "data": { + "text/plain": [ + "'The current population of Canada is 38,533,678'" + ] + }, + "execution_count": 10, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "agent_without_memory.run(\"How many people live in canada?\")" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "id": "d9ea82f0", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "\n", + "\u001b[1m> Entering new chain...\u001b[0m\n", + "what is their national anthem called?\n", + "Thought:\u001b[32;1m\u001b[1;3m I should probably look this up\n", + "Action: Search\n", + "Action Input: \"What is the national anthem of [country]\"\u001b[0m\n", + "Observation: \u001b[36;1m\u001b[1;3mMost nation states have an anthem, defined as \"a song, as of praise, devotion, or patriotism\"; most anthems are either marches or hymns in style.\u001b[0m\n", + "Thought:\u001b[32;1m\u001b[1;3m I now know the final answer\n", + "Final Answer: The national anthem is called \"the national anthem.\"\u001b[0m\n", + "\u001b[1m> Finished chain.\u001b[0m\n" + ] + }, + { + "data": { + "text/plain": [ + "'The national anthem is called \"the national anthem.\"'" + ] + }, + "execution_count": 11, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "agent_without_memory.run(\"what is their national anthem called?\")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "5b1f9223", + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.6" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/docs/examples/memory/custom_memory.ipynb b/docs/examples/memory/custom_memory.ipynb new file mode 100644 index 00000000..91b5a847 --- /dev/null +++ b/docs/examples/memory/custom_memory.ipynb @@ -0,0 +1,295 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "94e33ebe", + "metadata": {}, + "source": [ + "# Custom Memory\n", + "Although there are a few predefined types of memory in LangChain, it is highly possible you will want to add your own type of memory that is optimal for your application. This notebook covers how to do that." + ] + }, + { + "cell_type": "markdown", + "id": "bdfd0305", + "metadata": {}, + "source": [ + "For this notebook, we will add a custom memory type to `ConversationChain`. In order to add a custom memory class, we need to import the base memory class and subclass it." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "6d787ef2", + "metadata": {}, + "outputs": [], + "source": [ + "from langchain import OpenAI, ConversationChain\n", + "from langchain.chains.base import Memory\n", + "from pydantic import BaseModel\n", + "from typing import List, Dict, Any" + ] + }, + { + "cell_type": "markdown", + "id": "9489e5e1", + "metadata": {}, + "source": [ + "In this example, we will write a custom memory class that uses spacy to extract entities and save information about them in a simple hash table. Then, during the conversation, we will look at the input text, extract any entities, and put any information about them into the context.\n", + "\n", + "* Please note that this implementation is pretty simple and brittle and probably not useful in a production setting. Its purpose is to showcase that you can add custom memory implementations.\n", + "\n", + "For this, we will need spacy." + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "12bbed4e", + "metadata": {}, + "outputs": [], + "source": [ + "# !pip install spacy\n", + "# !python -m spacy download en_core_web_lg" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "ff065f58", + "metadata": {}, + "outputs": [], + "source": [ + "import spacy\n", + "nlp = spacy.load('en_core_web_lg')" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "id": "1d45d429", + "metadata": {}, + "outputs": [], + "source": [ + "class SpacyEntityMemory(Memory, BaseModel):\n", + " \"\"\"Memory class for storing information about entities.\"\"\"\n", + "\n", + " # Define dictionary to store information about entities.\n", + " entities: dict = {}\n", + " # Define key to pass information about entities into prompt.\n", + " memory_key: str = \"entities\"\n", + "\n", + " @property\n", + " def memory_variables(self) -> List[str]:\n", + " \"\"\"Define the variables we are providing to the prompt.\"\"\"\n", + " return [self.memory_key]\n", + "\n", + " def load_memory_variables(self, inputs: Dict[str, Any]) -> Dict[str, str]:\n", + " \"\"\"Load the memory variables, in this case the entity key.\"\"\"\n", + " # Get the input text and run through spacy\n", + " doc = nlp(inputs[list(inputs.keys())[0]])\n", + " # Extract known information about entities, if they exist.\n", + " entities = [self.entities[str(ent)] for ent in doc.ents if str(ent) in self.entities]\n", + " # Return combined information about entities to put into context.\n", + " return {self.memory_key: \"\\n\".join(entities)}\n", + "\n", + " def save_context(self, inputs: Dict[str, Any], outputs: Dict[str, str]) -> None:\n", + " \"\"\"Save context from this conversation to buffer.\"\"\"\n", + " # Get the input text and run through spacy\n", + " text = inputs[list(inputs.keys())[0]]\n", + " doc = nlp(text)\n", + " # For each entity that was mentioned, save this information to the dictionary.\n", + " for ent in doc.ents:\n", + " ent_str = str(ent)\n", + " if ent_str in self.entities:\n", + " self.entities[ent_str] += f\"\\n{text}\"\n", + " else:\n", + " self.entities[ent_str] = text" + ] + }, + { + "cell_type": "markdown", + "id": "429ba264", + "metadata": {}, + "source": [ + "We now define a prompt that takes in information about entities as well as user input" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "id": "c05159b6", + "metadata": {}, + "outputs": [], + "source": [ + "from langchain.prompts.prompt import PromptTemplate\n", + "\n", + "template = \"\"\"The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know. You are provided with information about entities the Human mentions, if relevant.\n", + "\n", + "Relevant entity information:\n", + "{entities}\n", + "\n", + "Conversation:\n", + "Human: {input}\n", + "AI:\"\"\"\n", + "prompt = PromptTemplate(\n", + " input_variables=[\"entities\", \"input\"], template=template\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "db611041", + "metadata": {}, + "source": [ + "And now we put it all together!" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "id": "f08dc8ed", + "metadata": {}, + "outputs": [], + "source": [ + "llm = OpenAI(temperature=0)\n", + "conversation = ConversationChain(llm=llm, prompt=prompt, verbose=True, memory=SpacyEntityMemory())" + ] + }, + { + "cell_type": "markdown", + "id": "92a5f685", + "metadata": {}, + "source": [ + "In the first example, with no prior knowledge about Harrison, the \"Relevant entity information\" section is empty." + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "id": "5b96e836", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "\n", + "\u001b[1m> Entering new chain...\u001b[0m\n", + "Prompt after formatting:\n", + "\u001b[32;1m\u001b[1;3mThe following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know. You are provided with information about entities the Human mentions, if relevant.\n", + "\n", + "Relevant entity information:\n", + "\n", + "\n", + "Conversation:\n", + "Human: Harrison likes machine learning\n", + "AI:\u001b[0m\n", + "\n", + "\u001b[1m> Finished chain.\u001b[0m\n" + ] + }, + { + "data": { + "text/plain": [ + "\"\\n\\nThat's really interesting! I'm sure he has a lot of fun with it.\"" + ] + }, + "execution_count": 11, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "conversation.predict(input=\"Harrison likes machine learning\")" + ] + }, + { + "cell_type": "markdown", + "id": "b1faa743", + "metadata": {}, + "source": [ + "Now in the second example, we can see that it pulls in information about Harrison." + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "id": "4bca7070", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "\n", + "\u001b[1m> Entering new chain...\u001b[0m\n", + "Prompt after formatting:\n", + "\u001b[32;1m\u001b[1;3mThe following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know. You are provided with information about entities the Human mentions, if relevant.\n", + "\n", + "Relevant entity information:\n", + "Harrison likes machine learning\n", + "\n", + "Conversation:\n", + "Human: What do you think Harrison's favorite subject in college was?\n", + "AI:\u001b[0m\n", + "\n", + "\u001b[1m> Finished chain.\u001b[0m\n" + ] + }, + { + "data": { + "text/plain": [ + "\" Harrison's favorite subject in college was machine learning.\"" + ] + }, + "execution_count": 12, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "conversation.predict(input=\"What do you think Harrison's favorite subject in college was?\")" + ] + }, + { + "cell_type": "markdown", + "id": "58b856e3", + "metadata": {}, + "source": [ + "Again, please note that this implementation is pretty simple and brittle and probably not useful in a production setting. Its purpose is to showcase that you can add custom memory implementations." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a1994600", + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.6" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/docs/examples/prompts/custom_example_selector.ipynb b/docs/examples/prompts/custom_example_selector.ipynb new file mode 100644 index 00000000..f8550d08 --- /dev/null +++ b/docs/examples/prompts/custom_example_selector.ipynb @@ -0,0 +1,176 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "f897c784", + "metadata": {}, + "source": [ + "# Custom ExampleSelector\n", + "\n", + "This notebook goes over how to implement a custom ExampleSelector. ExampleSelectors are used to select examples to use in few shot prompts.\n", + "\n", + "An ExampleSelector must implement two methods:\n", + "\n", + "1. An `add_example` method which takes in an example and adds it into the ExampleSelector\n", + "2. A `select_examples` method which takes in input variables (which are meant to be user input) and returns a list of examples to use in the few shot prompt.\n", + "\n", + "\n", + "Let's implement a custom ExampleSelector that just selects two examples at random." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "1a945da1", + "metadata": {}, + "outputs": [], + "source": [ + "from langchain.prompts.example_selector.base import BaseExampleSelector\n", + "from typing import Dict, List\n", + "import numpy as np" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "62cf0ad7", + "metadata": {}, + "outputs": [], + "source": [ + "class CustomExampleSelector(BaseExampleSelector):\n", + " \n", + " def __init__(self, examples: List[Dict[str, str]]):\n", + " self.examples = examples\n", + " \n", + " def add_example(self, example: Dict[str, str]) -> None:\n", + " \"\"\"Add new example to store for a key.\"\"\"\n", + " self.examples.append(example)\n", + "\n", + " def select_examples(self, input_variables: Dict[str, str]) -> List[dict]:\n", + " \"\"\"Select which examples to use based on the inputs.\"\"\"\n", + " return np.random.choice(self.examples, size=2, replace=False)" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "242d3213", + "metadata": {}, + "outputs": [], + "source": [ + "examples = [{\"foo\": \"1\"}, {\"foo\": \"2\"}, {\"foo\": \"3\"}]\n", + "example_selector = CustomExampleSelector(examples)" + ] + }, + { + "cell_type": "markdown", + "id": "2a038065", + "metadata": {}, + "source": [ + "Let's now try it out! We can select some examples and try adding examples." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "74fbbef5", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "array([{'foo': '2'}, {'foo': '3'}], dtype=object)" + ] + }, + "execution_count": 4, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "example_selector.select_examples({\"foo\": \"foo\"})" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "9bbb5421", + "metadata": {}, + "outputs": [], + "source": [ + "example_selector.add_example({\"foo\": \"4\"})" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "c0eb9f22", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[{'foo': '1'}, {'foo': '2'}, {'foo': '3'}, {'foo': '4'}]" + ] + }, + "execution_count": 6, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "example_selector.examples" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "cc39b1e3", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "array([{'foo': '1'}, {'foo': '4'}], dtype=object)" + ] + }, + "execution_count": 7, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "example_selector.select_examples({\"foo\": \"foo\"})" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "1739dd96", + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.6" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/docs/examples/prompts/custom_llm.ipynb b/docs/examples/prompts/custom_llm.ipynb new file mode 100644 index 00000000..fd9e6a7a --- /dev/null +++ b/docs/examples/prompts/custom_llm.ipynb @@ -0,0 +1,153 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "9e9b7651", + "metadata": {}, + "source": [ + "# Custom LLM\n", + "\n", + "This notebook goes over how to create a custom LLM wrapper, in case you want to use your own LLM or a different wrapper than one that is supported in LangChain.\n", + "\n", + "There is only one required thing that a custom LLM needs to implement:\n", + "\n", + "1. A `__call__` method that takes in a string, some optional stop words, and returns a string\n", + "\n", + "There is a second optional thing it can implement:\n", + "\n", + "1. An `_identifying_params` property that is used to help with printing of this class. Should return a dictionary.\n", + "\n", + "Let's implement a very simple custom LLM that just returns the first N characters of the input." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "a65696a0", + "metadata": {}, + "outputs": [], + "source": [ + "from langchain.llms.base import LLM\n", + "from typing import Optional, List, Mapping, Any" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "d5ceff02", + "metadata": {}, + "outputs": [], + "source": [ + "class CustomLLM(LLM):\n", + " \n", + " def __init__(self, n: int):\n", + " self.n = n\n", + " \n", + " def __call__(self, prompt: str, stop: Optional[List[str]] = None) -> str:\n", + " if stop is not None:\n", + " raise ValueError(\"stop kwargs are not permitted.\")\n", + " return prompt[:self.n]\n", + " \n", + " @property\n", + " def _identifying_params(self) -> Mapping[str, Any]:\n", + " \"\"\"Get the identifying parameters.\"\"\"\n", + " return {\"n\": self.n}" + ] + }, + { + "cell_type": "markdown", + "id": "714dede0", + "metadata": {}, + "source": [ + "We can now use this as an any other LLM." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "10e5ece6", + "metadata": {}, + "outputs": [], + "source": [ + "llm = CustomLLM(n=10)" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "8cd49199", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'This is a '" + ] + }, + "execution_count": 4, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "llm(\"This is a foobar thing\")" + ] + }, + { + "cell_type": "markdown", + "id": "bbfebea1", + "metadata": {}, + "source": [ + "We can also print the LLM and see its custom print." + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "9c33fa19", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\u001b[1mCustomLLM\u001b[0m\n", + "Params: {'n': 10}\n" + ] + } + ], + "source": [ + "print(llm)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "6dac3f47", + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.6" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/docs/examples/prompts/custom_prompt_template.ipynb b/docs/examples/prompts/custom_prompt_template.ipynb new file mode 100644 index 00000000..820d4f09 --- /dev/null +++ b/docs/examples/prompts/custom_prompt_template.ipynb @@ -0,0 +1,116 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "a37d9694", + "metadata": {}, + "source": [ + "# Custom Prompt Template\n", + "\n", + "This notebook goes over how to create a custom prompt template, in case you want to create your own methodology for creating prompts.\n", + "\n", + "The only two requirements for all prompt templates are:\n", + "\n", + "1. They have a `input_variables` attribute that exposes what input variables this prompt template expects.\n", + "2. They expose a `format` method which takes in keyword arguments corresponding to the expected `input_variables` and returns the formatted prompt.\n", + "\n", + "Let's imagine that we want to create a prompt template that takes in input variables and formats them into the template AFTER capitalizing them. " + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "26f796e5", + "metadata": {}, + "outputs": [], + "source": [ + "from langchain.prompts import BasePromptTemplate\n", + "from pydantic import BaseModel" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "27919e96", + "metadata": {}, + "outputs": [], + "source": [ + "class CustomPromptTemplate(BasePromptTemplate, BaseModel):\n", + " template: str\n", + " \n", + " def format(self, **kwargs) -> str:\n", + " capitalized_kwargs = {k: v.upper() for k, v in kwargs.items()}\n", + " return self.template.format(**capitalized_kwargs)\n", + " " + ] + }, + { + "cell_type": "markdown", + "id": "76d1d84d", + "metadata": {}, + "source": [ + "We can now see that when we use this, the input variables get formatted." + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "id": "eed1ff28", + "metadata": {}, + "outputs": [], + "source": [ + "prompt = CustomPromptTemplate(input_variables=[\"foo\"], template=\"Capitalized: {foo}\")" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "id": "94892a3c", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'Capitalized: LOWERCASE'" + ] + }, + "execution_count": 9, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "prompt.format(foo=\"lowercase\")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "d3d9a7c7", + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.6" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/docs/examples/prompts/prompt_management.ipynb b/docs/examples/prompts/prompt_management.ipynb index 6d11f6df..5d0b4731 100644 --- a/docs/examples/prompts/prompt_management.ipynb +++ b/docs/examples/prompts/prompt_management.ipynb @@ -602,7 +602,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.10.4" + "version": "3.7.6" } }, "nbformat": 4, diff --git a/docs/explanation/core_concepts.md b/docs/explanation/core_concepts.md index be4b8e2e..eafb6db0 100644 --- a/docs/explanation/core_concepts.md +++ b/docs/explanation/core_concepts.md @@ -29,3 +29,9 @@ They vary greatly in complexity and are combination of generic, highly configura ## Agents As opposed to a chain, whether the steps to be taken are known ahead of time, agents use an LLM to determine which tools to call and in what order. + +## Memory +By default, Chains and Agents are stateless, meaning that they treat each incoming query independently. +In some applications (chatbots being a GREAT example) it is highly important to remember previous interactions, +both at a short term but also at a long term level. The concept of "Memory" exists to do exactly that. + diff --git a/docs/getting_started/memory.ipynb b/docs/getting_started/memory.ipynb index 402ac39c..6b297f3c 100644 --- a/docs/getting_started/memory.ipynb +++ b/docs/getting_started/memory.ipynb @@ -290,10 +290,20 @@ "conversation_with_summary.predict(input=\"Very cool -- what is the scope of the project?\")" ] }, + { + "cell_type": "markdown", + "id": "5c8735cc", + "metadata": {}, + "source": [ + "### More Resources on Memory\n", + "\n", + "This just scratches the surface of what you can do with memory. For more examples on things like how to implement custom memory classes, how to add memory to a custom LLM chain and how to use memory with and agent, please see the [How-To: Memory](../../examples/memory) section." + ] + }, { "cell_type": "code", "execution_count": null, - "id": "0eb11bd0", + "id": "436dda66", "metadata": {}, "outputs": [], "source": [] diff --git a/docs/index.rst b/docs/index.rst index a4f80dfc..5dc98488 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -30,7 +30,7 @@ However, there are still some challenges going from that to an application runni - Prompt: The input to a language model. Typically this is not simply a hardcoded string but rather a combination of a template, some examples, and user input. - Prompt Template: An object responsible for constructing the final prompt to pass to a LLM. -*Problems solved* +*Problems Solved* - Switching costs: by exposing a standard interface for all the top LLM providers, LangChain makes it easy to switch from one provider to another, whether it be for production use cases or just for testing stuff out. - Prompt management: managing your prompts is easy when you only have one simple one, but can get tricky when you have a bunch or when they start to get more complex. LangChain provides a standard way for storing, constructing, and referencing prompts. @@ -46,7 +46,7 @@ LangChain provides several parts to help with that. - Tools: APIs designed for assisting with a particular use case (search, databases, Python REPL, etc). Prompt templates, LLMs, and chains can also be considered tools. - Chains: A combination of multiple tools in a deterministic manner. -*Problems solved* +*Problems Solved* - Standard interface for working with Chains - Easy way to construct chains of LLMs @@ -65,7 +65,7 @@ Depending on the user input, the agent can then decide which, if any, of these t - Agent: An LLM-powered class responsible for determining which tools to use and in what order. -*Problems solved* +*Problems Solved* - Standard agent interfaces - A selection of powerful agents to choose from @@ -73,7 +73,20 @@ Depending on the user input, the agent can then decide which, if any, of these t **🧠 Memory** -Coming soon. +By default, Chains and Agents are stateless, meaning that they treat each incoming query independently. +In some applications (chatbots being a GREAT example) it is highly important to remember previous interactions, +both at a short term but also at a long term level. The concept of "Memory" exists to do exactly that. + +*Key Concepts* + +- Memory: A class that can be added to an Agent or Chain to (1) pull in memory variables before calling that chain/agent, and (2) create new memories after the chain/agent finishes. +- Memory Variables: Variables returned from a Memory class, to be passed into the chain/agent along with the user input. + +*Problems Solved* + +- Standard memory interfaces +- A collection of common memory implementations to choose from +- Common chains/agents that use memory (e.g. chatbots) Documentation Structure ======================= @@ -107,6 +120,7 @@ Start here if you haven't used LangChain before. examples/integrations.rst examples/chains.rst examples/agents.rst + examples/memory.rst examples/model_laboratory.ipynb More elaborate examples and walk-throughs of particular diff --git a/langchain/VERSION b/langchain/VERSION index fe04e7f6..818944f5 100644 --- a/langchain/VERSION +++ b/langchain/VERSION @@ -1 +1 @@ -0.0.20 +0.0.22 diff --git a/langchain/agents/__init__.py b/langchain/agents/__init__.py index 2acd684b..d5376947 100644 --- a/langchain/agents/__init__.py +++ b/langchain/agents/__init__.py @@ -1,7 +1,7 @@ """Routing chains.""" from langchain.agents.agent import Agent from langchain.agents.loading import initialize_agent -from langchain.agents.mrkl.base import MRKLChain +from langchain.agents.mrkl.base import MRKLChain, ZeroShotAgent from langchain.agents.react.base import ReActChain from langchain.agents.self_ask_with_search.base import SelfAskWithSearchChain from langchain.agents.tools import Tool @@ -13,4 +13,5 @@ __all__ = [ "Agent", "Tool", "initialize_agent", + "ZeroShotAgent", ] diff --git a/langchain/agents/agent.py b/langchain/agents/agent.py index c7e82ff5..a3aebe2f 100644 --- a/langchain/agents/agent.py +++ b/langchain/agents/agent.py @@ -83,14 +83,15 @@ class Agent(Chain, BaseModel, ABC): pass @classmethod - def _get_prompt(cls, tools: List[Tool]) -> BasePromptTemplate: + def create_prompt(cls, tools: List[Tool]) -> BasePromptTemplate: + """Create a prompt for this class.""" return cls.prompt @classmethod def from_llm_and_tools(cls, llm: LLM, tools: List[Tool], **kwargs: Any) -> "Agent": """Construct an agent from an LLM and tools.""" cls._validate_tools(tools) - llm_chain = LLMChain(llm=llm, prompt=cls._get_prompt(tools)) + llm_chain = LLMChain(llm=llm, prompt=cls.create_prompt(tools)) return cls(llm_chain=llm_chain, tools=tools, **kwargs) def get_action(self, text: str) -> Action: diff --git a/langchain/agents/mrkl/base.py b/langchain/agents/mrkl/base.py index 839782b9..5474b2a9 100644 --- a/langchain/agents/mrkl/base.py +++ b/langchain/agents/mrkl/base.py @@ -2,11 +2,10 @@ from typing import Any, Callable, List, NamedTuple, Optional, Tuple from langchain.agents.agent import Agent -from langchain.agents.mrkl.prompt import BASE_TEMPLATE +from langchain.agents.mrkl.prompt import FORMAT_INSTRUCTIONS, PREFIX, SUFFIX from langchain.agents.tools import Tool from langchain.llms.base import LLM from langchain.prompts import PromptTemplate -from langchain.prompts.base import BasePromptTemplate FINAL_ANSWER_ACTION = "Final Answer: " @@ -60,11 +59,32 @@ class ZeroShotAgent(Agent): return "Thought:" @classmethod - def _get_prompt(cls, tools: List[Tool]) -> BasePromptTemplate: + def create_prompt( + cls, + tools: List[Tool], + prefix: str = PREFIX, + suffix: str = SUFFIX, + input_variables: Optional[List[str]] = None, + ) -> PromptTemplate: + """Create prompt in the style of the zero shot agent. + + Args: + tools: List of tools the agent will have access to, used to format the + prompt. + prefix: String to put before the list of tools. + suffix: String to put after the list of tools. + input_variables: List of input variables the final prompt will expect. + + Returns: + A PromptTemplate with the template assembled from the pieces here. + """ tool_strings = "\n".join([f"{tool.name}: {tool.description}" for tool in tools]) tool_names = ", ".join([tool.name for tool in tools]) - template = BASE_TEMPLATE.format(tools=tool_strings, tool_names=tool_names) - return PromptTemplate(template=template, input_variables=["input"]) + format_instructions = FORMAT_INSTRUCTIONS.format(tool_names=tool_names) + template = "\n\n".join([prefix, tool_strings, format_instructions, suffix]) + if input_variables is None: + input_variables = ["input"] + return PromptTemplate(template=template, input_variables=input_variables) @classmethod def _validate_tools(cls, tools: List[Tool]) -> None: diff --git a/langchain/agents/mrkl/prompt.py b/langchain/agents/mrkl/prompt.py index 128bdb85..c32310d4 100644 --- a/langchain/agents/mrkl/prompt.py +++ b/langchain/agents/mrkl/prompt.py @@ -1,9 +1,6 @@ # flake8: noqa -BASE_TEMPLATE = """Answer the following questions as best you can. You have access to the following tools: - -{tools} - -Use the following format: +PREFIX = """Answer the following questions as best you can. You have access to the following tools:""" +FORMAT_INSTRUCTIONS = """Use the following format: Question: the input question you must answer Thought: you should always think about what to do @@ -12,8 +9,7 @@ Action Input: the input to the action Observation: the result of the action ... (this Thought/Action/Action Input/Observation can repeat N times) Thought: I now know the final answer -Final Answer: the final answer to the original input question - -Begin! +Final Answer: the final answer to the original input question""" +SUFFIX = """Begin! -Question: {{input}}""" +Question: {input}""" diff --git a/langchain/chains/base.py b/langchain/chains/base.py index cdfae440..8e8005e7 100644 --- a/langchain/chains/base.py +++ b/langchain/chains/base.py @@ -16,11 +16,11 @@ class Memory(BaseModel, ABC): @property @abstractmethod - def dynamic_keys(self) -> List[str]: + def memory_variables(self) -> List[str]: """Input keys this memory class will load dynamically.""" @abstractmethod - def load_dynamic_keys(self, inputs: Dict[str, Any]) -> Dict[str, str]: + def load_memory_variables(self, inputs: Dict[str, Any]) -> Dict[str, str]: """Return key-value pairs given the text input to the chain.""" @abstractmethod @@ -77,7 +77,7 @@ class Chain(BaseModel, ABC): """ if self.memory is not None: - external_context = self.memory.load_dynamic_keys(inputs) + external_context = self.memory.load_memory_variables(inputs) inputs = dict(inputs, **external_context) self._validate_inputs(inputs) if self.verbose: diff --git a/langchain/chains/conversation/base.py b/langchain/chains/conversation/base.py index 9cc7578e..0a1b9de5 100644 --- a/langchain/chains/conversation/base.py +++ b/langchain/chains/conversation/base.py @@ -43,7 +43,7 @@ class ConversationChain(LLMChain, BaseModel): @root_validator() def validate_prompt_input_variables(cls, values: Dict) -> Dict: """Validate that prompt input variables are consistent.""" - memory_keys = values["memory"].dynamic_keys + memory_keys = values["memory"].memory_variables input_key = values["input_key"] if input_key in memory_keys: raise ValueError( diff --git a/langchain/chains/conversation/memory.py b/langchain/chains/conversation/memory.py index 7e822921..b6bf9e26 100644 --- a/langchain/chains/conversation/memory.py +++ b/langchain/chains/conversation/memory.py @@ -10,32 +10,39 @@ from langchain.llms.base import LLM from langchain.prompts.base import BasePromptTemplate +def _get_prompt_input_key(inputs: Dict[str, Any], memory_variables: List[str]) -> str: + # "stop" is a special key that can be passed as input but is not used to + # format the prompt. + prompt_input_keys = list(set(inputs).difference(memory_variables + ["stop"])) + if len(prompt_input_keys) != 1: + raise ValueError(f"One input key expected got {prompt_input_keys}") + return prompt_input_keys[0] + + class ConversationBufferMemory(Memory, BaseModel): """Buffer for storing conversation memory.""" buffer: str = "" - dynamic_key: str = "history" #: :meta private: + memory_key: str = "history" #: :meta private: @property - def dynamic_keys(self) -> List[str]: - """Will always return list of dynamic keys. + def memory_variables(self) -> List[str]: + """Will always return list of memory variables. :meta private: """ - return [self.dynamic_key] + return [self.memory_key] - def load_dynamic_keys(self, inputs: Dict[str, Any]) -> Dict[str, str]: + def load_memory_variables(self, inputs: Dict[str, Any]) -> Dict[str, str]: """Return history buffer.""" - return {self.dynamic_key: self.buffer} + return {self.memory_key: self.buffer} def save_context(self, inputs: Dict[str, Any], outputs: Dict[str, str]) -> None: """Save context from this conversation to buffer.""" - prompt_input_keys = list(set(inputs).difference(self.dynamic_keys)) - if len(prompt_input_keys) != 1: - raise ValueError(f"One input key expected got {prompt_input_keys}") + prompt_input_key = _get_prompt_input_key(inputs, self.memory_variables) if len(outputs) != 1: raise ValueError(f"One output key expected, got {outputs.keys()}") - human = "Human: " + inputs[prompt_input_keys[0]] + human = "Human: " + inputs[prompt_input_key] ai = "AI: " + outputs[list(outputs.keys())[0]] self.buffer += "\n" + "\n".join([human, ai]) @@ -46,19 +53,19 @@ class ConversationSummaryMemory(Memory, BaseModel): buffer: str = "" llm: LLM prompt: BasePromptTemplate = SUMMARY_PROMPT - dynamic_key: str = "history" #: :meta private: + memory_key: str = "history" #: :meta private: @property - def dynamic_keys(self) -> List[str]: - """Will always return list of dynamic keys. + def memory_variables(self) -> List[str]: + """Will always return list of memory variables. :meta private: """ - return [self.dynamic_key] + return [self.memory_key] - def load_dynamic_keys(self, inputs: Dict[str, Any]) -> Dict[str, str]: + def load_memory_variables(self, inputs: Dict[str, Any]) -> Dict[str, str]: """Return history buffer.""" - return {self.dynamic_key: self.buffer} + return {self.memory_key: self.buffer} @root_validator() def validate_prompt_input_variables(cls, values: Dict) -> Dict: @@ -74,12 +81,10 @@ class ConversationSummaryMemory(Memory, BaseModel): def save_context(self, inputs: Dict[str, Any], outputs: Dict[str, str]) -> None: """Save context from this conversation to buffer.""" - prompt_input_keys = list(set(inputs).difference(self.dynamic_keys)) - if len(prompt_input_keys) != 1: - raise ValueError(f"One input key expected got {prompt_input_keys}") + prompt_input_key = _get_prompt_input_key(inputs, self.memory_variables) if len(outputs) != 1: raise ValueError(f"One output key expected, got {outputs.keys()}") - human = "Human: " + inputs[prompt_input_keys[0]] + human = "Human: " + inputs[prompt_input_key] ai = "AI: " + list(outputs.values())[0] new_lines = "\n".join([human, ai]) chain = LLMChain(llm=self.llm, prompt=self.prompt) diff --git a/langchain/llms/base.py b/langchain/llms/base.py index a45ae606..df81242f 100644 --- a/langchain/llms/base.py +++ b/langchain/llms/base.py @@ -11,9 +11,9 @@ class LLM(ABC): """Run the LLM on the given prompt and input.""" @property - @abstractmethod def _identifying_params(self) -> Mapping[str, Any]: """Get the identifying parameters.""" + return {} def __str__(self) -> str: """Get a string representation of the object for printing.""" diff --git a/langchain/prompts/base.py b/langchain/prompts/base.py index 08b6ceca..8d133cb6 100644 --- a/langchain/prompts/base.py +++ b/langchain/prompts/base.py @@ -1,7 +1,8 @@ """BasePrompt schema definition.""" from abc import ABC, abstractmethod -from typing import Any, List, Union, Dict -from pydantic import BaseModel, Field +from typing import Any, Dict, List, Union + +from pydantic import BaseModel, Field, root_validator from langchain.formatting import formatter @@ -27,6 +28,7 @@ def check_valid_template( except KeyError: raise ValueError("Invalid prompt schema.") + class OutputParser(ABC): """Class to parse the output of an LLM call.""" @@ -51,6 +53,16 @@ class BasePromptTemplate(BaseModel, ABC): output_parser: OutputParser = Field(default_factory=DefaultParser) """How to parse the output of calling an LLM on this formatted prompt.""" + @root_validator() + def validate_variable_names(cls, values: Dict) -> Dict: + """Validate variable names do not restricted names.""" + if "stop" in values["input_variables"]: + raise ValueError( + "Cannot have an input variable named 'stop', as it is used internally," + " please rename." + ) + return values + @abstractmethod def format(self, **kwargs: Any) -> str: """Format the prompt with the inputs. diff --git a/langchain/prompts/prompt.py b/langchain/prompts/prompt.py index 6eccaaa3..bb1d331f 100644 --- a/langchain/prompts/prompt.py +++ b/langchain/prompts/prompt.py @@ -10,7 +10,7 @@ from langchain.prompts.base import ( ) -class PromptTemplate(BaseModel, BasePromptTemplate): +class PromptTemplate(BasePromptTemplate, BaseModel): """Schema to represent a prompt for an LLM. Example: diff --git a/tests/unit_tests/agents/test_mrkl.py b/tests/unit_tests/agents/test_mrkl.py index eed47d48..b1841c8d 100644 --- a/tests/unit_tests/agents/test_mrkl.py +++ b/tests/unit_tests/agents/test_mrkl.py @@ -3,7 +3,7 @@ import pytest from langchain.agents.mrkl.base import ZeroShotAgent, get_action_and_input -from langchain.agents.mrkl.prompt import BASE_TEMPLATE +from langchain.agents.mrkl.prompt import FORMAT_INSTRUCTIONS, PREFIX, SUFFIX from langchain.agents.tools import Tool from langchain.prompts import PromptTemplate from tests.unit_tests.llms.fake_llm import FakeLLM @@ -59,8 +59,13 @@ def test_from_chains() -> None: agent = ZeroShotAgent.from_llm_and_tools(FakeLLM(), chain_configs) expected_tools_prompt = "foo: foobar1\nbar: foobar2" expected_tool_names = "foo, bar" - expected_template = BASE_TEMPLATE.format( - tools=expected_tools_prompt, tool_names=expected_tool_names + expected_template = "\n\n".join( + [ + PREFIX, + expected_tools_prompt, + FORMAT_INSTRUCTIONS.format(tool_names=expected_tool_names), + SUFFIX, + ] ) prompt = agent.llm_chain.prompt assert isinstance(prompt, PromptTemplate) diff --git a/tests/unit_tests/chains/test_conversation.py b/tests/unit_tests/chains/test_conversation.py index 4e5cc244..82653986 100644 --- a/tests/unit_tests/chains/test_conversation.py +++ b/tests/unit_tests/chains/test_conversation.py @@ -15,7 +15,7 @@ def test_conversation_chain_works() -> None: """Test that conversation chain works in basic setting.""" llm = FakeLLM() prompt = PromptTemplate(input_variables=["foo", "bar"], template="{foo} {bar}") - memory = ConversationBufferMemory(dynamic_key="foo") + memory = ConversationBufferMemory(memory_key="foo") chain = ConversationChain(llm=llm, prompt=prompt, memory=memory, input_key="bar") chain.run("foo") @@ -32,7 +32,7 @@ def test_conversation_chain_errors_bad_variable() -> None: """Test that conversation chain works in basic setting.""" llm = FakeLLM() prompt = PromptTemplate(input_variables=["foo"], template="{foo}") - memory = ConversationBufferMemory(dynamic_key="foo") + memory = ConversationBufferMemory(memory_key="foo") with pytest.raises(ValueError): ConversationChain(llm=llm, prompt=prompt, memory=memory, input_key="foo") @@ -40,8 +40,8 @@ def test_conversation_chain_errors_bad_variable() -> None: @pytest.mark.parametrize( "memory", [ - ConversationBufferMemory(dynamic_key="baz"), - ConversationSummaryMemory(llm=FakeLLM(), dynamic_key="baz"), + ConversationBufferMemory(memory_key="baz"), + ConversationSummaryMemory(llm=FakeLLM(), memory_key="baz"), ], ) def test_conversation_memory(memory: Memory) -> None: