langchain/docs/extras/integrations/retrievers/zep_memorystore.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Zep\n",
    "## Retriever Example for [Zep](https://docs.getzep.com/) - A long-term memory store for LLM applications.\n",
    "\n",
    "### More on Zep:\n",
    "\n",
    "Zep stores, summarizes, embeds, indexes, and enriches conversational AI chat histories, and exposes them via simple, low-latency APIs.\n",
    "\n",
    "Key Features:\n",
    "\n",
    "- **Fast!** Zep’s async extractors operate independently of the your chat loop, ensuring a snappy user experience.\n",
    "- **Long-term memory persistence**, with access to historical messages irrespective of your summarization strategy.\n",
    "- **Auto-summarization** of memory messages based on a configurable message window. A series of summaries are stored, providing flexibility for future summarization strategies.\n",
    "- **Hybrid search** over memories and metadata, with messages automatically embedded on creation.\n",
    "- **Entity Extractor** that automatically extracts named entities from messages and stores them in the message metadata.\n",
    "- **Auto-token counting** of memories and summaries, allowing finer-grained control over prompt assembly.\n",
    "- Python and JavaScript SDKs.\n",
    "\n",
    "Zep project: [https://github.com/getzep/zep](https://github.com/getzep/zep)\n",
    "Docs: [https://docs.getzep.com/](https://docs.getzep.com/)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Retriever Example\n",
    "\n",
    "This notebook demonstrates how to search historical chat message histories using the [Zep Long-term Memory Store](https://getzep.github.io/).\n",
    "\n",
    "We'll demonstrate:\n",
    "\n",
    "1. Adding conversation history to the Zep memory store.\n",
    "2. Vector search over the conversation history.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2023-08-11T20:31:12.231459Z",
     "start_time": "2023-08-11T20:31:11.211176Z"
    }
   },
   "outputs": [],
   "source": [
    "import getpass\n",
    "import time\n",
    "from uuid import uuid4\n",
    "\n",
    "from langchain.memory import ZepMemory, CombinedMemory, VectorStoreRetrieverMemory\n",
    "from langchain.schema import HumanMessage, AIMessage\n",
    "\n",
    "# Set this to your Zep server URL\n",
    "ZEP_API_URL = \"http://localhost:8000\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Initialize the Zep Chat Message History Class and add a chat message history to the memory store\n",
    "\n",
    "**NOTE:** Unlike other Retrievers, the content returned by the Zep Retriever is session/user specific. A `session_id` is required when instantiating the Retriever."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-08-11T20:31:12.237545Z",
     "start_time": "2023-08-11T20:31:12.232416Z"
    }
   },
   "outputs": [],
   "source": [
    "# Provide your Zep API key. Note that this is optional. See https://docs.getzep.com/deployment/auth\n",
    "AUTHENTICATE = False\n",
    "\n",
    "zep_api_key = None\n",
    "if AUTHENTICATE:\n",
    "    zep_api_key = getpass.getpass()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2023-08-11T20:31:12.342790Z",
     "start_time": "2023-08-11T20:31:12.235291Z"
    }
   },
   "outputs": [],
   "source": [
    "session_id = str(uuid4())  # This is a unique identifier for the user/session\n",
    "\n",
    "# Initialize the Zep Memory Class\n",
    "zep_memory = ZepMemory(\n",
    "    session_id=session_id, url=ZEP_API_URL, api_key=zep_api_key\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2023-08-11T20:31:14.455269Z",
     "start_time": "2023-08-11T20:31:12.345635Z"
    }
   },
   "outputs": [],
   "source": [
    "# Preload some messages into the memory. The default message window is 12 messages. We want to push beyond this to demonstrate auto-summarization.\n",
    "test_history = [\n",
    "    {\"role\": \"human\", \"content\": \"Who was Octavia Butler?\"},\n",
    "    {\n",
    "        \"role\": \"ai\",\n",
    "        \"content\": (\n",
    "            \"Octavia Estelle Butler (June 22, 1947 – February 24, 2006) was an American\"\n",
    "            \" science fiction author.\"\n",
    "        ),\n",
    "    },\n",
    "    {\"role\": \"human\", \"content\": \"Which books of hers were made into movies?\"},\n",
    "    {\n",
    "        \"role\": \"ai\",\n",
    "        \"content\": (\n",
    "            \"The most well-known adaptation of Octavia Butler's work is the FX series\"\n",
    "            \" Kindred, based on her novel of the same name.\"\n",
    "        ),\n",
    "    },\n",
    "    {\"role\": \"human\", \"content\": \"Who were her contemporaries?\"},\n",
    "    {\n",
    "        \"role\": \"ai\",\n",
    "        \"content\": (\n",
    "            \"Octavia Butler's contemporaries included Ursula K. Le Guin, Samuel R.\"\n",
    "            \" Delany, and Joanna Russ.\"\n",
    "        ),\n",
    "    },\n",
    "    {\"role\": \"human\", \"content\": \"What awards did she win?\"},\n",
    "    {\n",
    "        \"role\": \"ai\",\n",
    "        \"content\": (\n",
    "            \"Octavia Butler won the Hugo Award, the Nebula Award, and the MacArthur\"\n",
    "            \" Fellowship.\"\n",
    "        ),\n",
    "    },\n",
    "    {\n",
    "        \"role\": \"human\",\n",
    "        \"content\": \"Which other women sci-fi writers might I want to read?\",\n",
    "    },\n",
    "    {\n",
    "        \"role\": \"ai\",\n",
    "        \"content\": \"You might want to read Ursula K. Le Guin or Joanna Russ.\",\n",
    "    },\n",
    "    {\n",
    "        \"role\": \"human\",\n",
    "        \"content\": (\n",
    "            \"Write a short synopsis of Butler's book, Parable of the Sower. What is it\"\n",
    "            \" about?\"\n",
    "        ),\n",
    "    },\n",
    "    {\n",
    "        \"role\": \"ai\",\n",
    "        \"content\": (\n",
    "            \"Parable of the Sower is a science fiction novel by Octavia Butler,\"\n",
    "            \" published in 1993. It follows the story of Lauren Olamina, a young woman\"\n",
    "            \" living in a dystopian future where society has collapsed due to\"\n",
    "            \" environmental disasters, poverty, and violence.\"\n",
    "        ),\n",
    "    },\n",
    "]\n",
    "\n",
    "for msg in test_history:\n",
    "    zep_memory.chat_memory.add_message(\n",
    "        HumanMessage(content=msg[\"content\"])\n",
    "        if msg[\"role\"] == \"human\"\n",
    "        else AIMessage(content=msg[\"content\"])\n",
    "    )\n",
    "    \n",
    "time.sleep(2)  # Wait for the messages to be embedded"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Use the Zep Retriever to vector search over the Zep memory\n",
    "\n",
    "Zep provides native vector search over historical conversation memory. Embedding happens automatically.\n",
    "\n",
    "NOTE: Embedding of messages occurs asynchronously, so the first query may not return results. Subsequent queries will return results as the embeddings are generated."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2023-08-11T20:31:14.758738Z",
     "start_time": "2023-08-11T20:31:14.458850Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": "[Document(page_content='Who was Octavia Butler?', metadata={'score': 0.7758688965570713, 'uuid': 'b3322d28-f589-48c7-9daf-5eb092d65976', 'created_at': '2023-08-11T20:31:12.3856Z', 'role': 'human', 'metadata': {'system': {'entities': [{'Label': 'PERSON', 'Matches': [{'End': 22, 'Start': 8, 'Text': 'Octavia Butler'}], 'Name': 'Octavia Butler'}]}}, 'token_count': 8}),\n Document(page_content=\"Octavia Butler's contemporaries included Ursula K. Le Guin, Samuel R. Delany, and Joanna Russ.\", metadata={'score': 0.7602672137411663, 'uuid': '756b7136-0b4c-4664-ad33-c4431670356c', 'created_at': '2023-08-11T20:31:12.420717Z', 'role': 'ai', 'metadata': {'system': {'entities': [{'Label': 'PERSON', 'Matches': [{'End': 16, 'Start': 0, 'Text': \"Octavia Butler's\"}], 'Name': \"Octavia Butler's\"}, {'Label': 'ORG', 'Matches': [{'End': 58, 'Start': 41, 'Text': 'Ursula K. Le Guin'}], 'Name': 'Ursula K. Le Guin'}, {'Label': 'PERSON', 'Matches': [{'End': 76, 'Start': 60, 'Text': 'Samuel R. Delany'}], 'Name': 'Samuel R. Delany'}, {'Label': 'PERSON', 'Matches': [{'End': 93, 'Start': 82, 'Text': 'Joanna Russ'}], 'Name': 'Joanna Russ'}]}}, 'token_count': 27}),\n Document(page_content='You might want to read Ursula K. Le Guin or Joanna Russ.', metadata={'score': 0.7596040989115522, 'uuid': '166d9556-2d48-4237-8a84-5d8a1024d5f4', 'created_at': '2023-08-11T20:31:12.434522Z', 'role': 'ai', 'metadata': {'system': {'entities': [{'Label': 'ORG', 'Matches': [{'End': 40, 'Start': 23, 'Text': 'Ursula K. Le Guin'}], 'Name': 'Ursula K. Le Guin'}, {'Label': 'PERSON', 'Matches': [{'End': 55, 'Start': 44, 'Text': 'Joanna Russ'}], 'Name': 'Joanna Russ'}]}}, 'token_count': 18}),\n Document(page_content='Who were her contemporaries?', metadata={'score': 0.7575531381951208, 'uuid': 'c6a16691-4012-439f-b223-84fd4e79c4cf', 'created_at': '2023-08-11T20:31:12.410336Z', 'role': 'human', 'token_count': 8}),\n Document(page_content='Octavia Estelle Butler (June 22, 1947 – February 24, 2006) was an American science fiction author.', metadata={'score': 0.7546476914454683, 'uuid': '7c093a2a-0099-415a-95c5-615a8026a894', 'created_at': '2023-08-11T20:31:12.399979Z', 'role': 'ai', 'metadata': {'system': {'entities': [{'Label': 'PERSON', 'Matches': [{'End': 22, 'Start': 0, 'Text': 'Octavia Estelle Butler'}], 'Name': 'Octavia Estelle Butler'}, {'Label': 'DATE', 'Matches': [{'End': 37, 'Start': 24, 'Text': 'June 22, 1947'}], 'Name': 'June 22, 1947'}, {'Label': 'DATE', 'Matches': [{'End': 57, 'Start': 40, 'Text': 'February 24, 2006'}], 'Name': 'February 24, 2006'}, {'Label': 'NORP', 'Matches': [{'End': 74, 'Start': 66, 'Text': 'American'}], 'Name': 'American'}]}}, 'token_count': 31})]"
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from langchain.retrievers import ZepRetriever\n",
    "\n",
    "zep_retriever = ZepRetriever(\n",
    "    session_id=session_id,  # Ensure that you provide the session_id when instantiating the Retriever\n",
    "    url=ZEP_API_URL,\n",
    "    top_k=5,\n",
    "    api_key=zep_api_key,\n",
    ")\n",
    "\n",
    "await zep_retriever.aget_relevant_documents(\"Who wrote Parable of the Sower?\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also use the Zep sync API to retrieve results:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2023-08-11T20:31:14.922838Z",
     "start_time": "2023-08-11T20:31:14.751737Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": "[Document(page_content=\"Write a short synopsis of Butler's book, Parable of the Sower. What is it about?\", metadata={'score': 0.8857504413268114, 'uuid': '82f07ab5-9d4b-4db6-aaae-6028e6fd836b', 'created_at': '2023-08-11T20:31:12.437365Z', 'role': 'human', 'metadata': {'system': {'entities': [{'Label': 'ORG', 'Matches': [{'End': 32, 'Start': 26, 'Text': 'Butler'}], 'Name': 'Butler'}, {'Label': 'WORK_OF_ART', 'Matches': [{'End': 61, 'Start': 41, 'Text': 'Parable of the Sower'}], 'Name': 'Parable of the Sower'}]}}, 'token_count': 23}),\n Document(page_content='Who was Octavia Butler?', metadata={'score': 0.7758688965570713, 'uuid': 'b3322d28-f589-48c7-9daf-5eb092d65976', 'created_at': '2023-08-11T20:31:12.3856Z', 'role': 'human', 'metadata': {'system': {'entities': [{'Label': 'PERSON', 'Matches': [{'End': 22, 'Start': 8, 'Text': 'Octavia Butler'}], 'Name': 'Octavia Butler'}]}}, 'token_count': 8}),\n Document(page_content=\"Octavia Butler's contemporaries included Ursula K. Le Guin, Samuel R. Delany, and Joanna Russ.\", metadata={'score': 0.7602672137411663, 'uuid': '756b7136-0b4c-4664-ad33-c4431670356c', 'created_at': '2023-08-11T20:31:12.420717Z', 'role': 'ai', 'metadata': {'system': {'entities': [{'Label': 'PERSON', 'Matches': [{'End': 16, 'Start': 0, 'Text': \"Octavia Butler's\"}], 'Name': \"Octavia Butler's\"}, {'Label': 'ORG', 'Matches': [{'End': 58, 'Start': 41, 'Text': 'Ursula K. Le Guin'}], 'Name': 'Ursula K. Le Guin'}, {'Label': 'PERSON', 'Matches': [{'End': 76, 'Start': 60, 'Text': 'Samuel R. Delany'}], 'Name': 'Samuel R. Delany'}, {'Label': 'PERSON', 'Matches': [{'End': 93, 'Start': 82, 'Text': 'Joanna Russ'}], 'Name': 'Joanna Russ'}]}}, 'token_count': 27}),\n Document(page_content='You might want to read Ursula K. Le Guin or Joanna Russ.', metadata={'score': 0.7596040989115522, 'uuid': '166d9556-2d48-4237-8a84-5d8a1024d5f4', 'created_at': '2023-08-11T20:31:12.434522Z', 'role': 'ai', 'metadata': {'system': {'entities': [{'Label': 'ORG', 'Matches': [{'End': 40, 'Start': 23, 'Text': 'Ursula K. Le Guin'}], 'Name': 'Ursula K. Le Guin'}, {'Label': 'PERSON', 'Matches': [{'End': 55, 'Start': 44, 'Text': 'Joanna Russ'}], 'Name': 'Joanna Russ'}]}}, 'token_count': 18}),\n Document(page_content='Who were her contemporaries?', metadata={'score': 0.7575531381951208, 'uuid': 'c6a16691-4012-439f-b223-84fd4e79c4cf', 'created_at': '2023-08-11T20:31:12.410336Z', 'role': 'human', 'token_count': 8})]"
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "zep_retriever.get_relevant_documents(\"Who wrote Parable of the Sower?\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "outputs": [],
   "source": [],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2023-08-11T20:31:14.923032Z",
     "start_time": "2023-08-11T20:31:14.918181Z"
    }
   }
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}