openai-cookbook/examples/vector_databases/redis/redisqna/redisqna.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Redis as a Context Store with OpenAI Chat\n",
    "This notebook demonstrates how to use Redis as high-speed context memory with ChatGPT.\n",
    "\n",
    "## Prerequisites\n",
    "* Redis instance with the Redis Search and Redis JSON modules\n",
    "* Redis-py client lib\n",
    "* OpenAI Python client lib\n",
    "* OpenAI API key\n",
    "\n",
    "## Installation\n",
    "Install Python modules necessary for the examples."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "vscode": {
     "languageId": "shellscript"
    }
   },
   "outputs": [],
   "source": [
    "! pip install redis openai python-dotenv openai[datalib]"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## OpenAI API Key\n",
    "Create a .env file and add your OpenAI key to it"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "vscode": {
     "languageId": "shellscript"
    }
   },
   "outputs": [],
   "source": [
    "OPENAI_API_KEY=your_key"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## OpenAI Setup\n",
    "Key load + helper function for chat completion"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import openai\n",
    "import os\n",
    "from dotenv import load_dotenv\n",
    "\n",
    "load_dotenv()\n",
    "openai.api_key = os.getenv(\"OPENAI_API_KEY\")\n",
    "\n",
    "def get_completion(prompt, model=\"gpt-3.5-turbo\"):\n",
    "    messages = [{\"role\": \"user\", \"content\": prompt}]\n",
    "    response = openai.ChatCompletion.create(\n",
    "        model=model,\n",
    "        messages=messages,\n",
    "        temperature=0, \n",
    "    )\n",
    "    return response.choices[0].message[\"content\"]"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Experiment - Chat Completion on a Topic outside of the Model's Knowledge Cutoff Date\n",
    "Gpt-3.5-turbo was trained on data up to Sep 2021.  Let's ask it a question about something that is beyond that date.  In this case, the FTX/Sam Bankman-Fried scandal."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "prompt = \"Is Sam Bankman-Fried's company, FTX, considered a well-managed company?\"\n",
    "response = get_completion(prompt)\n",
    "print(response)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Incomplete Information\n",
    "An unfortunate behavior of these AI systems is the system will provide a confident-sounding response - even when the system is not confident with its result.  One way to mitigate this is prompt re-engineering, as seen below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "prompt =\"Is Sam Bankman-Fried's company, FTX, considered a well-managed company?  If you don't know for certain, say unknown.\"\n",
    "response = get_completion(prompt)\n",
    "print(response)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Additional Context\n",
    "Another way to combat incomplete information is to give the system more information such that it can make intelligent decisions vs guessing.  We'll use Redis as the source for that additional context.  We'll pull in business news articles from after the GPT knowledge cut-off date such that the system will have a better understanding of how FTX was actually managed. "
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Start the Redis Stack Docker container"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "vscode": {
     "languageId": "shellscript"
    }
   },
   "outputs": [],
   "source": [
    "! docker compose up -d"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Connect Redis client"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from redis import from_url\n",
    "\n",
    "REDIS_URL = 'redis://localhost:6379'\n",
    "client = from_url(REDIS_URL)\n",
    "client.ping()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create Index\n",
    "[FT.CREATE](https://redis.io/commands/ft.create/)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "b'OK'"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from redis.commands.search.field import TextField, VectorField\n",
    "from redis.commands.search.indexDefinition import IndexDefinition, IndexType\n",
    "\n",
    "schema = [ VectorField('$.vector', \n",
    "            \"FLAT\", \n",
    "            {   \"TYPE\": 'FLOAT32', \n",
    "                \"DIM\": 1536, \n",
    "                \"DISTANCE_METRIC\": \"COSINE\"\n",
    "            },  as_name='vector' ),\n",
    "            TextField('$.content', as_name='content')\n",
    "        ]\n",
    "idx_def = IndexDefinition(index_type=IndexType.JSON, prefix=['doc:'])\n",
    "try: \n",
    "    client.ft('idx').dropindex()\n",
    "except:\n",
    "    pass\n",
    "client.ft('idx').create_index(schema, definition=idx_def)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load Data Files into Redis as JSON Objects with Text and Vector Fields\n",
    "[Redis JSON](https://redis.io/docs/stack/json/)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import openai\n",
    "\n",
    "directory = './assets/'\n",
    "model='text-embedding-3-small'\n",
    "i = 1\n",
    "for file in os.listdir(directory):\n",
    "    with open(os.path.join(directory, file)) as f:\n",
    "        content = f.read()\n",
    "        vector = openai.Embedding.create(input = [content], model = model)['data'][0]['embedding']\n",
    "        client.json().set(f'doc:{i}', '$', {'content': content, 'vector': vector})\n",
    "    i += 1"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Embed the Question and Perform VSS to find the most relevant document\n",
    "[KNN Search](https://redis.io/docs/stack/search/reference/vectors/#knn-search)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Embattled Crypto Exchange FTX Files for Bankruptcy\n",
      "\n",
      "Nov. 11, 2022\n",
      "On Monday, Sam Bankman-Fried, the chief executive of the cryptocurrency exchange FTX, took to Twitter to reassure his customers: “FTX is fine,” he wrote. “Assets are fine.”\n",
      "\n",
      "On Friday, FTX announced that it was filing for bankruptcy, capping an extraordinary week of corporate drama that has upended crypto markets, sent shock waves through an industry struggling to gain mainstream credibility and sparked government investigations that could lead to more damaging revelations or even criminal charges.\n",
      "\n",
      "In a statement on Twitter, the company said that Mr. Bankman-Fried had resigned, with John J. Ray III, a corporate turnaround specialist, taking over as chief executive.\n",
      "\n",
      "The speed of FTX’s downfall has left crypto insiders stunned. Just days ago, Mr. Bankman-Fried was considered one of the smartest leaders in the crypto industry, an influential figure in Washington who was lobbying to shape regulations. And FTX was widely viewed as one of the most stable and responsible companies in the freewheeling, loosely regulated crypto industry.\n",
      "\n",
      "“Here we are, with one of the richest people in the world, his net worth dropping to zero, his business dropping to zero,” said Jared Ellias, a bankruptcy professor at Harvard Law School. “The velocity of this failure is just unbelievable.”\n",
      "\n",
      "Now, the bankruptcy has set up a rush among investors and customers to salvage funds from what remains of FTX. A surge of customers tried to withdraw funds from the platform this week, and the company couldn’t meet the demand. The exchange owes as much as $8 billion, according to people familiar with its finances.\n",
      "\n",
      "FTX’s collapse has destabilized the crypto industry, which was already reeling from a crash in the spring that drained $1 trillion from the market. The prices of the leading cryptocurrencies, Bitcoin and Ether, have plummeted. The crypto lender BlockFi, which was closely entangled with FTX, announced on Thursday that it was suspending operations as a result of FTX’s collapse.\n",
      "\n",
      "Mr. Bankman-Fried was backed by some of the highest-profile venture capital investors in Silicon Valley, including Sequoia Capital and Lightspeed Venture Partners. Some of those investors, facing questions about how closely they scrutinized FTX before they put money into it, have said that their nine-figure investments in the crypto exchange are now essentially worthless.\n",
      "\n",
      "The company’s demise has also set off a reckoning over risky practices that have become pervasive in crypto, an industry that was founded partly as a corrective to the type of dangerous financial engineering that caused the 2008 economic crisis.\n",
      "\n",
      "“I’m really sorry, again, that we ended up here,” Mr. Bankman-Fried said on Twitter on Friday. “Hopefully this can bring some amount of transparency, trust, and governance.”\n",
      "\n",
      "The bankruptcy filing marks the start of what will probably be months or even years of legal fallout, as lawyers try to work out whether the exchange can ever continue to operate in some form and customers demand compensation. FTX is already the target of investigations by the Securities and Exchange Commission and the Justice Department, with investigators focused on whether the company improperly used customer funds to prop up Alameda Research, a trading firm that Mr. Bankman-Fried also founded.\n",
      "\n",
      "The bankruptcy filing included FTX, its U.S. arm and Alameda. According to a bare-bones legal filing in U.S. Bankruptcy Court in Delaware, FTX has assets valued between $10 billion and $50 billion, with the size of its liabilities in the same range. The company has more than 100,000 creditors, the filing said.\n",
      "\n",
      "The bankruptcy is a stunning fall from grace for the 30-year-old Mr. Bankman-Fried, who cultivated a reputation as a boy genius with a host of endearing quirks, including a habit of sleeping on a beanbag at the office. At one point, he was one of the richest people in the industry, with an estimated fortune of $24 billion. He hobnobbed with actors, professional athletes and former world leaders.\n",
      "\n",
      "Mr. Bankman-Fried’s crypto empire had an elaborate structure. The bankruptcy filing lists more than 130 corporate entities affiliated with FTX and Alameda. But as of June, FTX had only about 300 employees, a point of pride for Mr. Bankman-Fried, who said he had resisted calls from venture investors to hire more staff.\n",
      "\n",
      "“We told them additional employees added too quickly were net negative,” Mr. Bankman-Fried said on Twitter in June. “They could take it or leave it.”\n",
      "\n",
      "Unusually for a major start-up, none of FTX’s investors had seats on the board, which instead consisted of Mr. Bankman-Fried, another FTX executive and a lawyer in Antigua and Barbuda.\n",
      "\n",
      "FTX and Alameda were based in the Bahamas, where Mr. Bankman-Fried and a small circle of top executives called most of the shots and lived together in a luxury resort. Officially, Alameda was run by Caroline Ellison, a former trader for the hedge fund Jane Street, but Mr. Bankman-Fried was heavily involved, contributing to the decision-making on big trades, according to a person familiar with the matter.\n",
      "\n",
      "In addition to Mr. Bankman-Fried and Ms. Ellison, the circle of executives running FTX included Nishad Singh, FTX’s director of engineering, and Gary Wang, the chief technology officer. Few others had visibility into how the company was run: When the firm collapsed this week, lower-ranking employees were left confused and blindsided, according to people familiar with the matter. Mr. Singh and Ms. Ellison did not respond to requests for comment; Mr. Wang could not immediately be reached.\n",
      "\n",
      "As a crypto exchange, FTX provided a marketplace for customers to buy, sell and store a wide range of digital currencies. Most of its revenue stemmed from a risky type of trade — in which crypto investors borrowed money to make huge bets on the future prices of cryptocurrencies — that remains illegal in the United States. But Mr. Bankman-Fried also ran a smaller U.S. affiliate that offered more basic trading options.\n",
      "\n",
      "Mr. Bankman-Fried’s problems started over the weekend, when the chief executive of Binance, the largest crypto exchange, suggested publicly that FTX might be on shaky financial footing. A rush of customers tried to withdraw their crypto holdings from the platform, and FTX was unable to meet the demand.\n",
      "\n",
      "On Tuesday, Mr. Bankman-Fried said he had struck a deal to sell FTX to Binance. But after reviewing the company’s financial documents, Binance’s chief executive, Changpeng Zhao, pulled out of the agreement, leaving Mr. Bankman-Fried with limited options.\n",
      "\n",
      "In calls with investors and messages to employees this week, he apologized repeatedly and stressed that he was working hard to raise money and resolve the situation. But the hole was ultimately too big to fill.\n",
      "\n",
      "FTX’s bankruptcy is the latest — and by far the biggest — in a series of bankruptcies that have shaken the crypto world this year. After a market crash in the spring, two crypto lending companies, Celsius Network and Voyager Digital, filed for bankruptcy, kicking off months of legal maneuvering over how their remaining assets should be divided. In an ironic twist, FTX had recently won an auction to buy Voyager’s remaining assets.\n",
      "\n",
      "As it enters its own bankruptcy process, FTX will be led by Mr. Ray, who has ample experience managing distressed situations. He helped manage Enron after the collapse of its business in an accounting fraud scandal in 2001. And he helped liquidate the trust of the subprime mortgage company ResCap after its 2012 bankruptcy.\n",
      "\n",
      "The bankruptcy proceedings may be only the beginning of Mr. Bankman-Fried’s legal troubles. Federal investigators are examining the relationship between FTX and Alameda, and customers are likely to file lawsuits.\n",
      "\n",
      "Mr. Bankman-Fried’s old allies have quickly abandoned him. On Thursday night, the team running the FTX Future Fund, a charitable group that Mr. Bankman-Fried bankrolled, announced that they were resigning.\n",
      "\n",
      "“We were shocked and immensely saddened to learn of the recent events at FTX,” they wrote in a statement. “We have fundamental questions about the legitimacy and integrity of the business operations that were funding the FTX Foundation and the Future Fund.”\n",
      "\n",
      "Not long ago, Mr. Bankman-Fried was performing a comedy routine onstage at a conference with Anthony Scaramucci, the former White House communications director and a business partner of FTX.\n",
      "\n",
      "“I’m disappointed,” Mr. Scaramucci said in an interview on CNBC on Friday. “Duped, I guess, is the right word.”\n",
      "\n"
     ]
    }
   ],
   "source": [
    "from redis.commands.search.query import Query\n",
    "import numpy as np\n",
    "\n",
    "vec = np.array(openai.Embedding.create(input = [prompt], model = model)['data'][0]['embedding'], dtype=np.float32).tobytes()\n",
    "q = Query('*=>[KNN 1 @vector $query_vec AS vector_score]')\\\n",
    "    .sort_by('vector_score')\\\n",
    "    .return_fields('content')\\\n",
    "    .dialect(2)    \n",
    "params = {\"query_vec\": vec}\n",
    "\n",
    "context = client.ft('idx').search(q, query_params=params).docs[0].content\n",
    "print(context)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Repeat the Question to OpenAI with context\n",
    "Now that we have relevant context, add that to the prompt to OpenAI and get a very different response."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "No, Sam Bankman-Fried's company FTX is not considered a well-managed company as it has filed for bankruptcy and owes as much as $8 billion to its creditors. The collapse of FTX has destabilized the crypto industry, and the company is already the target of investigations by the Securities and Exchange Commission and the Justice Department. FTX was widely viewed as one of the most stable and responsible companies in the freewheeling, loosely regulated crypto industry, but its risky practices have become pervasive in crypto, leading to a reckoning.\n"
     ]
    }
   ],
   "source": [
    "prompt = f\"\"\"\n",
    "Using the information delimited by triple backticks, answer this question: Is Sam Bankman-Fried's company, FTX, considered a well-managed company?\n",
    "\n",
    "Context: ```{context}```\n",
    "\"\"\"\n",
    "\n",
    "response = get_completion(prompt)\n",
    "print(response)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.6"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}