{ "cells": [ { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "# Zep Memory\n", "\n", "## REACT Agent Chat Message History with Zep - A long-term memory store for LLM applications.\n", "\n", "This notebook demonstrates how to use the [Zep Long-term Memory Store](https://docs.getzep.com/) as memory for your chatbot.\n", "\n", "We'll demonstrate:\n", "\n", "1. Adding conversation history to the Zep memory store.\n", "2. Running an agent and having message automatically added to the store.\n", "3. Viewing the enriched messages.\n", "4. Vector search over the conversation history.\n", "\n", "### More on Zep:\n", "\n", "Zep stores, summarizes, embeds, indexes, and enriches conversational AI chat histories, and exposes them via simple, low-latency APIs.\n", "\n", "Key Features:\n", "\n", "- **Fast!** Zep’s async extractors operate independently of the your chat loop, ensuring a snappy user experience.\n", "- **Long-term memory persistence**, with access to historical messages irrespective of your summarization strategy.\n", "- **Auto-summarization** of memory messages based on a configurable message window. A series of summaries are stored, providing flexibility for future summarization strategies.\n", "- **Hybrid search** over memories and metadata, with messages automatically embedded on creation.\n", "- **Entity Extractor** that automatically extracts named entities from messages and stores them in the message metadata.\n", "- **Auto-token counting** of memories and summaries, allowing finer-grained control over prompt assembly.\n", "- Python and JavaScript SDKs.\n", "\n", "Zep project: [https://github.com/getzep/zep](https://github.com/getzep/zep)\n", "Docs: [https://docs.getzep.com/](https://docs.getzep.com/)\n" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "is_executing": true }, "outputs": [], "source": [ "from langchain.memory.chat_message_histories import ZepChatMessageHistory\n", "from langchain.memory import ConversationBufferMemory\n", "from langchain import OpenAI\n", "from langchain.schema import HumanMessage, AIMessage\n", "from langchain.utilities import WikipediaAPIWrapper\n", "from langchain.agents import initialize_agent, AgentType, Tool\n", "from uuid import uuid4\n", "\n", "\n", "# Set this to your Zep server URL\n", "ZEP_API_URL = \"http://localhost:8000\"\n", "\n", "session_id = str(uuid4()) # This is a unique identifier for the user" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "ExecuteTime": { "end_time": "2023-05-25T15:09:41.762056Z", "start_time": "2023-05-25T15:09:41.755238Z" } }, "outputs": [ { "name": "stdin", "output_type": "stream", "text": [ " ········\n" ] } ], "source": [ "# Provide your OpenAI key\n", "import getpass\n", "\n", "openai_key = getpass.getpass()" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "name": "stdin", "output_type": "stream", "text": [ " ········\n" ] } ], "source": [ "# Provide your Zep API key. Note that this is optional. See https://docs.getzep.com/deployment/auth\n", "\n", "zep_api_key = getpass.getpass()" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "### Initialize the Zep Chat Message History Class and initialize the Agent\n" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "ExecuteTime": { "end_time": "2023-05-25T15:09:41.840440Z", "start_time": "2023-05-25T15:09:41.762277Z" } }, "outputs": [], "source": [ "search = WikipediaAPIWrapper()\n", "tools = [\n", " Tool(\n", " name=\"Search\",\n", " func=search.run,\n", " description=\"useful for when you need to search online for answers. You should ask targeted questions\",\n", " ),\n", "]\n", "\n", "# Set up Zep Chat History\n", "zep_chat_history = ZepChatMessageHistory(\n", " session_id=session_id,\n", " url=ZEP_API_URL,\n", " api_key=zep_api_key\n", ")\n", "\n", "# Use a standard ConversationBufferMemory to encapsulate the Zep chat history\n", "memory = ConversationBufferMemory(\n", " memory_key=\"chat_history\", chat_memory=zep_chat_history\n", ")\n", "\n", "# Initialize the agent\n", "llm = OpenAI(temperature=0, openai_api_key=openai_key)\n", "agent_chain = initialize_agent(\n", " tools,\n", " llm,\n", " agent=AgentType.CONVERSATIONAL_REACT_DESCRIPTION,\n", " verbose=True,\n", " memory=memory,\n", ")" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "### Add some history data\n" ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "ExecuteTime": { "end_time": "2023-05-25T15:09:41.960661Z", "start_time": "2023-05-25T15:09:41.842656Z" } }, "outputs": [], "source": [ "# Preload some messages into the memory. The default message window is 12 messages. We want to push beyond this to demonstrate auto-summarization.\n", "test_history = [\n", " {\"role\": \"human\", \"content\": \"Who was Octavia Butler?\"},\n", " {\n", " \"role\": \"ai\",\n", " \"content\": (\n", " \"Octavia Estelle Butler (June 22, 1947 – February 24, 2006) was an American\"\n", " \" science fiction author.\"\n", " ),\n", " },\n", " {\"role\": \"human\", \"content\": \"Which books of hers were made into movies?\"},\n", " {\n", " \"role\": \"ai\",\n", " \"content\": (\n", " \"The most well-known adaptation of Octavia Butler's work is the FX series\"\n", " \" Kindred, based on her novel of the same name.\"\n", " ),\n", " },\n", " {\"role\": \"human\", \"content\": \"Who were her contemporaries?\"},\n", " {\n", " \"role\": \"ai\",\n", " \"content\": (\n", " \"Octavia Butler's contemporaries included Ursula K. Le Guin, Samuel R.\"\n", " \" Delany, and Joanna Russ.\"\n", " ),\n", " },\n", " {\"role\": \"human\", \"content\": \"What awards did she win?\"},\n", " {\n", " \"role\": \"ai\",\n", " \"content\": (\n", " \"Octavia Butler won the Hugo Award, the Nebula Award, and the MacArthur\"\n", " \" Fellowship.\"\n", " ),\n", " },\n", " {\n", " \"role\": \"human\",\n", " \"content\": \"Which other women sci-fi writers might I want to read?\",\n", " },\n", " {\n", " \"role\": \"ai\",\n", " \"content\": \"You might want to read Ursula K. Le Guin or Joanna Russ.\",\n", " },\n", " {\n", " \"role\": \"human\",\n", " \"content\": (\n", " \"Write a short synopsis of Butler's book, Parable of the Sower. What is it\"\n", " \" about?\"\n", " ),\n", " },\n", " {\n", " \"role\": \"ai\",\n", " \"content\": (\n", " \"Parable of the Sower is a science fiction novel by Octavia Butler,\"\n", " \" published in 1993. It follows the story of Lauren Olamina, a young woman\"\n", " \" living in a dystopian future where society has collapsed due to\"\n", " \" environmental disasters, poverty, and violence.\"\n", " ),\n", " },\n", "]\n", "\n", "for msg in test_history:\n", " zep_chat_history.add_message(\n", " HumanMessage(content=msg[\"content\"])\n", " if msg[\"role\"] == \"human\"\n", " else AIMessage(content=msg[\"content\"])\n", " )" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "### Run the agent\n", "\n", "Doing so will automatically add the input and response to the Zep memory.\n" ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "ExecuteTime": { "end_time": "2023-05-25T15:09:50.485377Z", "start_time": "2023-05-25T15:09:41.962287Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "\n", "\u001B[1m> Entering new chain...\u001B[0m\n", "\u001B[32;1m\u001B[1;3mThought: Do I need to use a tool? No\n", "AI: Parable of the Sower is a prescient novel that speaks to the challenges facing contemporary society, such as climate change, economic inequality, and the rise of authoritarianism. It is a cautionary tale that warns of the dangers of ignoring these issues and the importance of taking action to address them.\u001B[0m\n", "\n", "\u001B[1m> Finished chain.\u001B[0m\n" ] }, { "data": { "text/plain": [ "'Parable of the Sower is a prescient novel that speaks to the challenges facing contemporary society, such as climate change, economic inequality, and the rise of authoritarianism. It is a cautionary tale that warns of the dangers of ignoring these issues and the importance of taking action to address them.'" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "agent_chain.run(\n", " input=\"WWhat is the book's relevance to the challenges facing contemporary society?\"\n", ")" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "### Inspect the Zep memory\n", "\n", "Note the summary, and that the history has been enriched with token counts, UUIDs, and timestamps.\n", "\n", "Summaries are biased towards the most recent messages.\n" ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "ExecuteTime": { "end_time": "2023-05-25T15:09:50.493438Z", "start_time": "2023-05-25T15:09:50.479230Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "The human asks about Octavia Butler and the AI identifies her as an American science fiction author. They continue to discuss her works and the fact that the FX series Kindred is based on one of her novels. The AI also lists Ursula K. Le Guin, Samuel R. Delany, and Joanna Russ as Butler's contemporaries.\n", "\n", "\n", "{'role': 'human', 'content': 'What awards did she win?', 'uuid': 'a4bdc592-71a5-47d0-9c64-230b882aab48', 'created_at': '2023-06-26T23:37:56.383953Z', 'token_count': 8, 'metadata': {'system': {'entities': [], 'intent': 'The subject is asking about the awards someone won, likely referring to a specific individual.'}}}\n", "{'role': 'ai', 'content': 'Octavia Butler won the Hugo Award, the Nebula Award, and the MacArthur Fellowship.', 'uuid': '60cc6e6b-7cd4-4a81-aebc-72ef997286b4', 'created_at': '2023-06-26T23:37:56.389935Z', 'token_count': 21, 'metadata': {'system': {'entities': [{'Label': 'PERSON', 'Matches': [{'End': 14, 'Start': 0, 'Text': 'Octavia Butler'}], 'Name': 'Octavia Butler'}, {'Label': 'WORK_OF_ART', 'Matches': [{'End': 33, 'Start': 19, 'Text': 'the Hugo Award'}], 'Name': 'the Hugo Award'}, {'Label': 'EVENT', 'Matches': [{'End': 81, 'Start': 57, 'Text': 'the MacArthur Fellowship'}], 'Name': 'the MacArthur Fellowship'}], 'intent': 'The subject is stating the accomplishments and awards received by Octavia Butler.'}}}\n", "{'role': 'human', 'content': 'Which other women sci-fi writers might I want to read?', 'uuid': 'b189fc60-1510-4a4b-a503-899481d652de', 'created_at': '2023-06-26T23:37:56.395722Z', 'token_count': 14, 'metadata': {'system': {'entities': [], 'intent': 'The subject is looking for recommendations on women science fiction writers to read.'}}}\n", "{'role': 'ai', 'content': 'You might want to read Ursula K. Le Guin or Joanna Russ.', 'uuid': '4be1ccbb-a915-45d6-9f18-7a0c1cbd9907', 'created_at': '2023-06-26T23:37:56.403596Z', 'token_count': 18, 'metadata': {'system': {'entities': [{'Label': 'ORG', 'Matches': [{'End': 40, 'Start': 23, 'Text': 'Ursula K. Le Guin'}], 'Name': 'Ursula K. Le Guin'}, {'Label': 'PERSON', 'Matches': [{'End': 55, 'Start': 44, 'Text': 'Joanna Russ'}], 'Name': 'Joanna Russ'}], 'intent': 'The subject is suggesting reading material and making a literary recommendation.'}}}\n", "{'role': 'human', 'content': \"Write a short synopsis of Butler's book, Parable of the Sower. What is it about?\", 'uuid': 'ac3c5e3e-26a7-4f3b-aeb0-bba084e22753', 'created_at': '2023-06-26T23:37:56.410662Z', 'token_count': 23, 'metadata': {'system': {'entities': [{'Label': 'ORG', 'Matches': [{'End': 32, 'Start': 26, 'Text': 'Butler'}], 'Name': 'Butler'}, {'Label': 'WORK_OF_ART', 'Matches': [{'End': 61, 'Start': 41, 'Text': 'Parable of the Sower'}], 'Name': 'Parable of the Sower'}], 'intent': 'The subject is asking for a brief overview or summary of the book \"Parable of the Sower\" written by Butler.'}}}\n", "{'role': 'ai', 'content': 'Parable of the Sower is a science fiction novel by Octavia Butler, published in 1993. It follows the story of Lauren Olamina, a young woman living in a dystopian future where society has collapsed due to environmental disasters, poverty, and violence.', 'uuid': '4a463b4c-bcab-473c-bed1-fc56a7a20ae2', 'created_at': '2023-06-26T23:37:56.41764Z', 'token_count': 56, 'metadata': {'system': {'entities': [{'Label': 'GPE', 'Matches': [{'End': 20, 'Start': 15, 'Text': 'Sower'}], 'Name': 'Sower'}, {'Label': 'PERSON', 'Matches': [{'End': 65, 'Start': 51, 'Text': 'Octavia Butler'}], 'Name': 'Octavia Butler'}, {'Label': 'DATE', 'Matches': [{'End': 84, 'Start': 80, 'Text': '1993'}], 'Name': '1993'}, {'Label': 'PERSON', 'Matches': [{'End': 124, 'Start': 110, 'Text': 'Lauren Olamina'}], 'Name': 'Lauren Olamina'}]}}}\n", "{'role': 'human', 'content': \"WWhat is the book's relevance to the challenges facing contemporary society?\", 'uuid': '41bab0c7-5e20-40a4-9303-f82069977c91', 'created_at': '2023-06-26T23:38:03.559642Z', 'token_count': 16, 'metadata': {'system': {'entities': [{'Label': 'ORG', 'Matches': [{'End': 5, 'Start': 0, 'Text': 'WWhat'}], 'Name': 'WWhat'}]}}}\n", "{'role': 'ai', 'content': 'Parable of the Sower is a prescient novel that speaks to the challenges facing contemporary society, such as climate change, economic inequality, and the rise of authoritarianism. It is a cautionary tale that warns of the dangers of ignoring these issues and the importance of taking action to address them.', 'uuid': 'bfd8146a-4632-4c8c-98b6-9468bb624339', 'created_at': '2023-06-26T23:38:03.589312Z', 'token_count': 62, 'metadata': {'system': {'entities': [{'Label': 'GPE', 'Matches': [{'End': 20, 'Start': 15, 'Text': 'Sower'}], 'Name': 'Sower'}]}}}\n" ] } ], "source": [ "def print_messages(messages):\n", " for m in messages:\n", " print(m.to_dict())\n", "\n", "\n", "print(zep_chat_history.zep_summary)\n", "print(\"\\n\")\n", "print_messages(zep_chat_history.zep_messages)" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "### Vector search over the Zep memory\n", "\n", "Zep provides native vector search over historical conversation memory. Embedding happens automatically.\n" ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "ExecuteTime": { "end_time": "2023-05-25T15:09:50.751203Z", "start_time": "2023-05-25T15:09:50.495050Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{'uuid': 'b189fc60-1510-4a4b-a503-899481d652de', 'created_at': '2023-06-26T23:37:56.395722Z', 'role': 'human', 'content': 'Which other women sci-fi writers might I want to read?', 'metadata': {'system': {'entities': [], 'intent': 'The subject is looking for recommendations on women science fiction writers to read.'}}, 'token_count': 14} 0.9119619869747062\n", "{'uuid': '4be1ccbb-a915-45d6-9f18-7a0c1cbd9907', 'created_at': '2023-06-26T23:37:56.403596Z', 'role': 'ai', 'content': 'You might want to read Ursula K. Le Guin or Joanna Russ.', 'metadata': {'system': {'entities': [{'Label': 'ORG', 'Matches': [{'End': 40, 'Start': 23, 'Text': 'Ursula K. Le Guin'}], 'Name': 'Ursula K. Le Guin'}, {'Label': 'PERSON', 'Matches': [{'End': 55, 'Start': 44, 'Text': 'Joanna Russ'}], 'Name': 'Joanna Russ'}], 'intent': 'The subject is suggesting reading material and making a literary recommendation.'}}, 'token_count': 18} 0.8534346954749745\n", "{'uuid': '76ec2a3d-b908-4c23-a55d-71ff92865a7a', 'created_at': '2023-06-26T23:37:56.378345Z', 'role': 'ai', 'content': \"Octavia Butler's contemporaries included Ursula K. Le Guin, Samuel R. Delany, and Joanna Russ.\", 'metadata': {'system': {'entities': [{'Label': 'PERSON', 'Matches': [{'End': 16, 'Start': 0, 'Text': \"Octavia Butler's\"}], 'Name': \"Octavia Butler's\"}, {'Label': 'ORG', 'Matches': [{'End': 58, 'Start': 41, 'Text': 'Ursula K. Le Guin'}], 'Name': 'Ursula K. Le Guin'}, {'Label': 'PERSON', 'Matches': [{'End': 76, 'Start': 60, 'Text': 'Samuel R. Delany'}], 'Name': 'Samuel R. Delany'}, {'Label': 'PERSON', 'Matches': [{'End': 93, 'Start': 82, 'Text': 'Joanna Russ'}], 'Name': 'Joanna Russ'}], 'intent': 'The subject is stating the contemporaries of Octavia Butler, who are also science fiction writers.'}}, 'token_count': 27} 0.8523930955780226\n", "{'uuid': '1feb02c7-63c9-4616-854d-0d97fb590ea5', 'created_at': '2023-06-26T23:37:56.313009Z', 'role': 'human', 'content': 'Who was Octavia Butler?', 'metadata': {'system': {'entities': [{'Label': 'PERSON', 'Matches': [{'End': 22, 'Start': 8, 'Text': 'Octavia Butler'}], 'Name': 'Octavia Butler'}], 'intent': 'The subject is asking about the identity of Octavia Butler, likely seeking information about her background or accomplishments.'}}, 'token_count': 8} 0.8236355436055457\n", "{'uuid': 'ebe4696d-b5fa-4ca0-88c9-da794d9611ab', 'created_at': '2023-06-26T23:37:56.332247Z', 'role': 'ai', 'content': 'Octavia Estelle Butler (June 22, 1947 – February 24, 2006) was an American science fiction author.', 'metadata': {'system': {'entities': [{'Label': 'PERSON', 'Matches': [{'End': 22, 'Start': 0, 'Text': 'Octavia Estelle Butler'}], 'Name': 'Octavia Estelle Butler'}, {'Label': 'DATE', 'Matches': [{'End': 37, 'Start': 24, 'Text': 'June 22, 1947'}], 'Name': 'June 22, 1947'}, {'Label': 'DATE', 'Matches': [{'End': 57, 'Start': 40, 'Text': 'February 24, 2006'}], 'Name': 'February 24, 2006'}, {'Label': 'NORP', 'Matches': [{'End': 74, 'Start': 66, 'Text': 'American'}], 'Name': 'American'}], 'intent': 'The subject is making a statement about the background and profession of Octavia Estelle Butler, an American author.'}}, 'token_count': 31} 0.8206687242257686\n", "{'uuid': '60cc6e6b-7cd4-4a81-aebc-72ef997286b4', 'created_at': '2023-06-26T23:37:56.389935Z', 'role': 'ai', 'content': 'Octavia Butler won the Hugo Award, the Nebula Award, and the MacArthur Fellowship.', 'metadata': {'system': {'entities': [{'Label': 'PERSON', 'Matches': [{'End': 14, 'Start': 0, 'Text': 'Octavia Butler'}], 'Name': 'Octavia Butler'}, {'Label': 'WORK_OF_ART', 'Matches': [{'End': 33, 'Start': 19, 'Text': 'the Hugo Award'}], 'Name': 'the Hugo Award'}, {'Label': 'EVENT', 'Matches': [{'End': 81, 'Start': 57, 'Text': 'the MacArthur Fellowship'}], 'Name': 'the MacArthur Fellowship'}], 'intent': 'The subject is stating the accomplishments and awards received by Octavia Butler.'}}, 'token_count': 21} 0.8194249796585193\n", "{'uuid': '0fa4f336-909d-4880-b01a-8e80e91fa8f2', 'created_at': '2023-06-26T23:37:56.344552Z', 'role': 'human', 'content': 'Which books of hers were made into movies?', 'metadata': {'system': {'entities': [], 'intent': 'The subject is inquiring about which books written by an unknown female author were adapted into movies.'}}, 'token_count': 11} 0.7955105671310818\n", "{'uuid': 'f91de7f2-4b84-4c5a-8a33-a71f38f3a59c', 'created_at': '2023-06-26T23:37:56.368146Z', 'role': 'human', 'content': 'Who were her contemporaries?', 'metadata': {'system': {'entities': [], 'intent': 'The subject is asking about the people who lived during the same time period as a specific individual.'}}, 'token_count': 8} 0.7942358617914813\n", "{'uuid': '4a463b4c-bcab-473c-bed1-fc56a7a20ae2', 'created_at': '2023-06-26T23:37:56.41764Z', 'role': 'ai', 'content': 'Parable of the Sower is a science fiction novel by Octavia Butler, published in 1993. It follows the story of Lauren Olamina, a young woman living in a dystopian future where society has collapsed due to environmental disasters, poverty, and violence.', 'metadata': {'system': {'entities': [{'Label': 'GPE', 'Matches': [{'End': 20, 'Start': 15, 'Text': 'Sower'}], 'Name': 'Sower'}, {'Label': 'PERSON', 'Matches': [{'End': 65, 'Start': 51, 'Text': 'Octavia Butler'}], 'Name': 'Octavia Butler'}, {'Label': 'DATE', 'Matches': [{'End': 84, 'Start': 80, 'Text': '1993'}], 'Name': '1993'}, {'Label': 'PERSON', 'Matches': [{'End': 124, 'Start': 110, 'Text': 'Lauren Olamina'}], 'Name': 'Lauren Olamina'}]}}, 'token_count': 56} 0.7816448549236643\n", "{'uuid': '6161d934-a629-4ba2-8bba-0b0996c93964', 'created_at': '2023-06-26T23:37:56.358632Z', 'role': 'ai', 'content': \"The most well-known adaptation of Octavia Butler's work is the FX series Kindred, based on her novel of the same name.\", 'metadata': {'system': {'entities': [{'Label': 'PERSON', 'Matches': [{'End': 50, 'Start': 34, 'Text': \"Octavia Butler's\"}], 'Name': \"Octavia Butler's\"}, {'Label': 'ORG', 'Matches': [{'End': 65, 'Start': 63, 'Text': 'FX'}], 'Name': 'FX'}, {'Label': 'GPE', 'Matches': [{'End': 80, 'Start': 73, 'Text': 'Kindred'}], 'Name': 'Kindred'}], 'intent': \"The subject is discussing Octavia Butler's work being adapted into a TV series called Kindred.\"}}, 'token_count': 29} 0.7815841371388998\n" ] } ], "source": [ "search_results = zep_chat_history.search(\"who are some famous women sci-fi authors?\")\n", "for r in search_results:\n", " print(r.message, r.dist)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.4" } }, "nbformat": 4, "nbformat_minor": 4 }