openai-cookbook/examples/GPT_with_vision_for_video_understanding.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Processing and narrating a video with GPT's visual capabilities and the TTS API\n",
    "\n",
    "This notebook demonstrates how to use GPT's visual capabilities with a video. GPT-4 doesn't take videos as input directly, but we can use vision and the new 128K context window to describe the static frames of a whole video at once. We'll walk through two examples:\n",
    "\n",
    "1. Using GPT-4 to get a description of a video\n",
    "2. Generating a voiceover for a video with GPT-4 and the TTS API\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from IPython.display import display, Image, Audio\n",
    "\n",
    "import cv2  # We're using OpenCV to read video\n",
    "import base64\n",
    "import time\n",
    "import openai\n",
    "import os\n",
    "import requests"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Using GPT's visual capabilities to get a description of a video\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, we use OpenCV to extract frames from a nature [video](https://www.youtube.com/watch?v=kQ_7GtE529M) containing bisons and wolves:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "618 frames read.\n"
     ]
    }
   ],
   "source": [
    "video = cv2.VideoCapture(\"data/bison.mp4\")\n",
    "\n",
    "base64Frames = []\n",
    "while video.isOpened():\n",
    "    success, frame = video.read()\n",
    "    if not success:\n",
    "        break\n",
    "    _, buffer = cv2.imencode(\".jpg\", frame)\n",
    "    base64Frames.append(base64.b64encode(buffer).decode(\"utf-8\"))\n",
    "\n",
    "video.release()\n",
    "print(len(base64Frames), \"frames read.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Display frames to make sure we've read them in correctly:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/jpeg": "/9j/4AAQSkZJRgABAQAAAQABAAD/2wBDAAIBAQEBAQIBAQECAgICAgQDAgICAgUEBAMEBgUGBgYFBgYGBwkIBgcJBwYGCAsICQoKCgoKBggLDAsKDAkKCgr/2wBDAQICAgICAgUDAwUKBwYHCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgr/wAARCALQBQADASIAAhEBAxEB/8QAHwAAAQUBAQEBAQEAAAAAAAAAAAECAwQFBgcICQoL/8QAtRAAAgEDAwIEAwUFBAQAAAF9AQIDAAQRBRIhMUEGE1FhByJxFDKBkaEII0KxwRVS0fAkM2JyggkKFhcYGRolJicoKSo0NTY3ODk6Q0RFRkdISUpTVFVWV1hZWmNkZWZnaGlqc3R1dnd4eXqDhIWGh4iJipKTlJWWl5iZmqKjpKWmp6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uHi4+Tl5ufo6erx8vP09fb3+Pn6/8QAHwEAAwEBAQEBAQEBAQAAAAAAAAECAwQFBgcICQoL/8QAtREAAgECBAQDBAcFBAQAAQJ3AAECAxEEBSExBhJBUQdhcRMiMoEIFEKRobHBCSMzUvAVYnLRChYkNOEl8RcYGRomJygpKjU2Nzg5OkNERUZHSElKU1RVVldYWVpjZGVmZ2hpanN0dXZ3eHl6goOEhYaHiImKkpOUlZaXmJmaoqOkpaanqKmqsrO0tba3uLm6wsPExcbHyMnK0tPU1dbX2Nna4uPk5ebn6Onq8vP09fb3+Pn6/9oADAMBAAIRAxEAPwD6PV3B6fpVhOlMWNSfpUgGBig8sKjdhuqSmOmWzg0AJRRRg+lABRUlFAEdIUB5paKAEI44FSJyuaTy/emK27tTsAXQ4Bqt5fvVqgDJxVKdlYCh9mPcn8qu2L+RwOvqaeIVIzTBGB3q4y5gJrqcMA3fvVGSUuWTt3GKmYE5GfzpPJ3DJH5VTcbAUpYFjPmBzzyRjpV/RrqIt5Mh78HFJ9njK7WX86EhhUjArF3lIDUnKCAiN87uOO1ZxhxyenpUiEKMKelKeetF3FWQECKA3Ap9OKCgoPepAbyDxT1JI5prAL60BiBgUAPJA60DkZFQOHPVuO+KltsBDuHanZWuA6il+T3pKQBRQBk4p3l+9ADcE9BS4PoaeBgYooAjwR1FFSfjShSeQKAIsE9BS4Poak2N6UbWHagCKipNn+z+lGz/AGf0oAjop/lH3/KgRlTn+lADKKkooAjopShFG0+lACUUuD6GkoAKKkooAjoqSo6ACiilClulAEVFSUUAFFFFABRRRQAUUUUAFFFFADfL96bUlJ5OPWugBlFBBBwaKACiiigBAmDnNLRRQAUoGTikooAkooornAjoqSiugCOgjIxTnAA4FNoAKKKKACmqpByRTqKAG+X70eX706igBrRg8/0qJ1G7pU9IUBoAgwPQUVPsHqaawwcUARU4JkZzTqKAG+X70hGDin0hTJzmgCCipNo/u/pRgegoAZg+howfQ1Jg+hpKAGYPoaMH0NPooAZg+hpKkpjfeoAig71JTIQQpyMc0+gAooooArgZpSMdxT+DwfSkwnt+dZgQUUUjSIvDNiqinIT3FoPQ1DK6k8N+RqCa4jRSd3NHKLmIrnCgZGSe9Vmxg/Wn3GoIn31H/fNUrnWrGB9jSAk8jbXTGLsTZEso68VVYMrbgelVpvEtnnCqT+NVptf358qI9eMnJq+VickizNErHcE6j60xrTzBsxt471VXU52/dkYHUDNPW+mkYM7Yx6VQueI+TS1KZD4HXp60fZIEXIPOemOlV7jVHjG1XUgH1qjJrtxGS7EYquVmTt0NVYFUZ4/KkmiiUZ46d6wbnxLcP80ZwPbis6+1q9kUgSH65o5RHS3ptUQmZlA9zWDqn2WZW2XIbngKetY9/d311JtMuefyotbabh3ds+maaVhPYr6lY28rAQAZ/iz3qlBoZeZYxBnngk10EWnyyOP3Z+boSKv2nh9HG3YcjqcVruZqMmc9a+G3WTK24+9hq29O8IST/eshwemOta+maBAh+dWODzXX6BokUYBCsfl5DHpWMqqiaxhcyvDfgiNCpkhUKT83y12FhpVpYIVtowu7GSB1xT4bdYhgDAqZWJGBXLWquT902hTS1EVhGcseB6CpYryIHcgGQepFRsing0ogC9BWCbtqapWJ0dZJckDr2qZVAHJxVRYyuTk8mpkmdBgkn8aUXYZNhf736UocJ0NRfaT6H86Qzhh8yn25quYCczn1pPPz3/Wq7zAdE/WkWXdn5egz1qtwFadpe2B6VG/QGlVdvehx8tRpcCFn+bbmoqldQGyBSDjtVk8yCnR96TB9vzpQQnfNA3sOpCwXrTTMg5NRTXtuvHmDI7ZqVFthcn3j0NNklUDFUrjVUjOE5+hrMu9bky6q3bgCtY07bhqzbku1iUbnAFVZtbjjOBlvxrm7jWZpQAvGO2c1WaeeXlzyfetFGK6CZvXPiw5IQBSD6c1nT+IrmRSpz+dUPIlc8gmp4tHmkUt6jOD1pgiGW8mlb5pGPPGT0qMySucsST9a0rPQJCmZQfw7VYtfD+6UhySM/lQDbMiOGR+ufpirFvpM9zIoRDiult9JtEPI/MVN9mhiOUUD8KBXZzx8MSnr/OpLbw3KJRk/rW+ABwKdCRvAIpPYNRmjaTDBxKgOSK0/7Ptf+eYqKIAZ20+oLNSmOTnFJRXGAq9R9afUdOD4GCKAG09TkVCr5OCwp9NppgSUVHT0+6KQC0UVG7kHvQBJSJ0/GmrIcf405WGOTT6ALUdSVHV0+oElFAPGaM1mA7y/emkYOKd5ntQJMjOP1ppN7ANoopV6ikAxVIOSKdRRTbbYBRRRSARlyOlMRF3DipKKpSsrAJtX0prqR0b9KfQQD1FJNJ6gN8v3p8QwcUlPTp+NIBaKVV3d6Ch7UAJQMZ5FLsb0pQh7mgAXaT92n+QxGdn60kcRYnBqdRhQD2FAEIgdedp/Okqx161E8RXpz9BQAyil2P8A3T+VGx/7p/KgBKR/umnbW/un8qRlOMMCKAI6kwPQUgUCloAjoqQgHqKCM8GgCOipMD0FB5GKAI6KKKACo6n2L6Uzyx/dNAEdKq7u9PoxQBHRQQQcGigAop3l+9BTAzmgBkYPXFOwfQ0Q/dp9VP4gK9FSUEZGK2AjooooAkoPTio6KAHRoWIz61LsUcmmo7RnKHFIxODzQASAEYWmbW9Kcv3aWgBmD6GjB9DT6KAI6KKKACiiigCZVBGTR5fvSQysOoHX0qyIhIo+XH0oArFCBxzSbSeq/pVr7MPf86TyF7E0AVCgzwadTvL96PL96AGBQO1I49BUuweppmCOooAZg+hpCCOoqSggHqKAI6KfsXOaNgoAZRTih7UmxqAEpCmTnNOII6ikoAQJg5zS0UUAMb71NZRgnFSFMnOaTy/egCGnhVI6U/yz2NNP1oAjooooAKinJAJBqWmyRhlOV60AVgW6AnmjY/8AdP5VOIF4IH60/YPU0AVdj/3T+VGx/wC6fyq1sHqaNg9TQBVYYPU1Fv8A9r9atvApOB+tRSW6qcH68UkmiGyk8zK54/GoJZJicqx6+lXpowRgCoDGFGSOavTlDSxTZpQOpH1NQvvPGf1q5OOgxwaqyllJHoeKFBtXCzZBKvOGFUdRthKmSBwelX5CSOT3qnc52nkitoKwjKmsowcmMZ9xURt4lJKxgGrs6ORnnPvVSZZghKAE9s1siZFF9RhiJBHIPSq82sSEkRqfbAzVyLw7JKwMr5zzipDoNvHnLnI709EZcrMWaS7uDkZT0NN+xu5xK7deg6VvNoqDhJOSe4qSHRoWcgAt7elMLMwEsFAxhjz+tB0aMDJQYJ44rq4NGQjBXgfrUq6Iu0KVOfcVLkkUoNnIW+ho042qCMc8Vp2nhncNxiB5/hroLbQY2cDZk9qvw6PtGFBGOnFJzSKVM5y10BY5Qwt24PetRdEicDbGcjqB1rUGlTnkAZzViz06aOQMSCe1Q6yGopGbZ6L5Z3CI8f
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "display_handle = display(None, display_id=True)\n",
    "for img in base64Frames:\n",
    "    display_handle.update(Image(data=base64.b64decode(img.encode(\"utf-8\"))))\n",
    "    time.sleep(0.025)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once we have the video frames, we craft our prompt and send a request to GPT (Note that we don't need to send every frame for GPT to understand what's going on):\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Title: Survival Instincts: The Epic Standoff between Bison and Wolves in the Frigid Wild\n",
      "\n",
      "Description: \n",
      "Witness the raw and riveting drama unfold on the snowy plains, where the formidable bison faces off against a determined pack of wolves. Set against a dazzling winter backdrop, this video captures the harrowing and breathtaking interaction between predator and prey. With survival at stake, each move is a dance of life and death in nature's great theater. Watch as these majestic creatures engage in a timeless struggle, showcasing the power, resilience, and indomitable spirit that define the wild. Join us for an unforgettable journey into the heart of nature's resilience – the ultimate testament to the cycle of life in the animal kingdom. #Wildlife #Nature #Survival #BisonVsWolves\n"
     ]
    }
   ],
   "source": [
    "PROMPT_MESSAGES = [\n",
    "    {\n",
    "        \"role\": \"user\",\n",
    "        \"content\": [\n",
    "            \"These are frames from a video that I want to upload. Generate a compelling description that I can upload along with the video.\",\n",
    "            *map(lambda x: {\"image\": x, \"resize\": 768}, base64Frames[0::10]),\n",
    "        ],\n",
    "    },\n",
    "]\n",
    "params = {\n",
    "    \"model\": \"gpt-4-vision-preview\",\n",
    "    \"messages\": PROMPT_MESSAGES,\n",
    "    \"api_key\": os.environ[\"OPENAI_API_KEY\"],\n",
    "    \"headers\": {\"Openai-Version\": \"2020-11-07\"},\n",
    "    \"max_tokens\": 200,\n",
    "}\n",
    "\n",
    "result = openai.ChatCompletion.create(**params)\n",
    "print(result.choices[0].message.content)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Generating a voiceover for a video with GPT-4 and the TTS API\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's create a voiceover for this video in the style of David Attenborough. Using the same video frames we prompt GPT to give us a short script:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "In the vast, white expanse of the winter landscape, a drama unfolds that is as timeless as it is raw. Here, in the cradle of nature's harshest trials, a pack of grey wolves has singled out a bison from the herd—a desperate struggle for life and sustenance is about to begin.\n",
      "\n",
      "In a carefully orchestrated assault, the pack encircles their quarry, each wolf keenly aware of its role. Muscles tense and breaths visible in the frigid air, they inch closer, probing for a weakness. The bison, a formidable giant, stands its ground, backed by the survival instincts honed over millennia. Its hulking form casts a solitary shadow against the snow's blinding canvas.\n",
      "\n",
      "The dance of predator and prey plays out as a symphony of survival—each movement, each feint, holds the weight of life itself. The wolves take turns attacking, conserving strength while wearing down their target. The herd, once the bison's allies, scatter into the distance, a stark reminder that in these wild territories, the law of survival supersedes the bonds of kinship.\n",
      "\n",
      "A burst of activity—the wolves close in. The bison, though mighty, is tiring, its breaths labored, its movements sluggish. The wolves sense the turning tide. With relentless determination, they press their advantage, a testament to the brutal beauty of the natural order.\n",
      "\n",
      "As the struggle reaches its inevitable conclusion, we are reminded of the delicate balance that governs these wild spaces. Life, death, struggle, and survival—the cycle continues, each chapter written in the snow, for as long as the wolf roams and the bison roves these frozen plains.\n"
     ]
    }
   ],
   "source": [
    "PROMPT_MESSAGES = [\n",
    "    {\n",
    "        \"role\": \"user\",\n",
    "        \"content\": [\n",
    "            \"These are frames of a video. Create a short voiceover script in the style of David Attenborough. Only include the narration.\",\n",
    "            *map(lambda x: {\"image\": x, \"resize\": 768}, base64Frames[0::10]),\n",
    "        ],\n",
    "    },\n",
    "]\n",
    "params = {\n",
    "    \"model\": \"gpt-4-vision-preview\",\n",
    "    \"messages\": PROMPT_MESSAGES,\n",
    "    \"api_key\": os.environ[\"OPENAI_API_KEY\"],\n",
    "    \"headers\": {\"Openai-Version\": \"2020-11-07\"},\n",
    "    \"max_tokens\": 500,\n",
    "}\n",
    "\n",
    "result = openai.ChatCompletion.create(**params)\n",
    "print(result.choices[0].message.content)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can pass the script to the TTS API where it will generate an mp3 of the voiceover:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "                <audio  controls=\"controls\" >\n",
       "                    <source src=\"data:audio/wav;base64,//PkxABlXDn4A1rAABn22HWcCSUzbU4s058s4r027U2as3jI2Cw0yIzxwzRgyxYyxQyxYyxAzpcypUyZEx40xYsxYkxIkxIsxYc3KNSDMYyEMQjEIwAMQjEAzlMYzCEsmWnLTlpy05aMBALMFtC5BeBAIXcLkF4C0ZcsvGgHQDororoB0Hy8CARAIg4ioioioiokQg+iuiukWmOmOmOmOoOkQmImIoIqRUipFSKCJFpjpjqDqnVOqdU6x1BFSKkVIqRUipF2LEVOqdU6p1TrHXeu9d6pFSLEXYuxdi7F2MQWOu9d6713rvXeu9ia7F2LsXYuxdi7F2MQXeu9d6713rvZ2ztr7EGcM4ZwzhnDOGcNcZ2ztnbO2ds7Z2ztr7OGcM4ZwzhnDOHIchr7W2ds7Z2ztr7lu+5DkOQzhnDXHIchnDE13rvXeu9d7O2dtfZwzhhi7F2LsYgzhrjO2ds7Z2ztnbO3La+zhnDDF2MQZwzhrjuOW5bO2ds7a+5blu+5DWGcM4ZwzhrjkO45bW2ds7Z2ztnblw/Dj+OQ5DkOQ5DuQ4oEHNwHqY5wr5ugEYmB4BWbe+G5vUlmmFsE+YGIMBgxgWGIwIkZzxDRnRGiGDqEAYL4JxhcAnEAEZlWC+GN//PkxD1z/DnwCZ7gAgB7GFB6Y8JJjQUgwBPqyE8rBzOInAAQAIgMSiIxSEjAwBDgO8AKAy6WbmARiYdBJmUEmDA+YMCqjRhEPGCQIp0q9PSAWCGLigZIDhr9emvWCaRKaVJgoDgYKlgAGAQEJAIUA5gsCr2DgPJ2dGvmUckrhvQsGLBMYEC5WCW7l3Uzaxb5B5l8radOtQiCz1PgYbiEYmEw2FQuYlEIQNwcBwwAs0chlTL2MrCuBLXTaS5zuMsWbXeFwXsgd0BGA5UtlmC1UJb1SWAZtpVp7mE0kYS+WFusAfmfpdPU1y5XnW+Um2jjBwAQDiQMbHAyV7TkaEho0mi3aAS1TGX9Lqu8rIXObxTEcAamS6rz2RJyXajjfyhyHJjVyUQzK5SkAwZW1Vy5DCYHUviLeUr1QADAE0gDANuRbpVFwRABVjF7kTlSoogwCr6QWLbL5RBLUpJIZAEGNlRRAAHgV833gCGW6Q1EmFalULTlpI5HeO9KorKoedRnbpMAYYvtGswQAlBF0GGRCYpDIYF10KYIOMXZQuGpxncx4EuTOhPgMyiBw2x1ZzF9sUOUZNgxHivzB/AfNVhN4weB6zYBMpN1IKowmQeTAaAdMbsJgwDgnzKDBNMUwPAx//PkxEB0vDooAZ7gAFhwkFhhYHGZzAYdGBhgWmfygZNEJnQal9wuByQAGAhYY2GQBEZi0bgpBmGhyY6I5hVQGBhSABgYKAwsLgYDUmxxIk3cM4AkwopTARlNCvA6ktTIStMyB0yCwjSikMwAIVLYXB6Ea1Ay7HEF2CTkYMKRjMNF5zLTqPzsE04UzFAzMcCEudAzcGNF7AcHFnrPYeztfZWATDAPBAGQ4Jgu6KgQGAsyCBjGg3MNA+jpKkFOw7kTicESfUQd9VF3ZbADFGIOJIKW0l6wJyWIyR9FB4ebxs7KIEh+3FuMQUGgdazX1vN+6MnfuA40sCrpapVACYCZzlwO6EhZ0z3GU07T2rrnYm99Rw5exN+6r5uO2zIGYugldRtiWGVVRvTCjCCZYJylsPszeNpbtzXkIwC5whAcmarAkCo+jQTQmNPZw4EJvx5+HbhujVva+3yYEOWlcJeMFX87b7wpd1OpSm0xJY4sAk6lAlG1MXacHqCR6VpQXDixntlBd9aiNyKSnD/M6Z9FoS57mwELAZBVKpDnRwlYNkQJgkIEoYjaIeGazq6pgP4HIYa0l2GTlBPRiBBCqZU2FDGKkgUZhm4jeYZUH9mEYAmJhm4PoYJuBaGAWgm5gIAC//PkxEB0/DoYCd/gAKmBHgkxgI4H2YDSBSGBBASZgJQBUYDYAQmAwgBhgFoDIMAOpgLAAeIwDsOIQUBQKOqxTA5SMkgYxUCjYg1CpoNuEs1ylDc0hMGPA4DLjL4tMuHsiQBkEPA4MBcCDgEMmk4z+DQUNzUA2M0iYzSADEZCERtMhLEwYDDHwaMLDohDQWASdjLFYyyJhYJtIWfOO66tSbqt8yV1XUe+EF1TAADMCAUugYXEwOGQACZgwCoitkR+YM72ONLbl0Gu3DcCTb8wLTRGXvjKIi4r+xySu7lTyuPrlZLSt4wxE1WEsk0JF0iAisaVDvMtROwZixVnTV5esLI2srpa7zGW8kFI70KdmflLcWuyrNnTiuc3KJpbMftTLSgKBHTZgXOVGHAlgKeqOyRaEL4I7oDVyvLCYDZ0oqiaz5IZpyAlW9XDutu/DbLlbMrcmLB0akz/tEclVZPaHX+cJw20u1WgraazEH2eGCpUqq3r6yt4nCaexBocAQy9bJkTWCs5dxDZrsQ3ALTXajcVopQ7154lhpfWmGvUsAvZD5IqGFlgJRgZQGOY2QwCGD9Auxijop4aWwC2GCoh6phIw/QYEIAan8GifhABsJdnWJQZCAYXWoOGYgRpyEqG//PkxD9plDocFv82TCYaGb58NWUQjsVIjBwQx2QAQgY6wnHopgYIYmahw+CQAzsLMRFjuW8wERFiaZLtEwoYuTAYvBRgFjYxMYbOYKgmUktKY8CGzixkyiY+kmCohhpoYmaAMCNJAQUqGbnZgoIDh5NhuamCqkrfksgwVU1VVNebK1kotoBQwPXqkGnI5q5HraMumdfRfD9r3jbW2yszorUEMhTpUpXq1ZTJRJO5ZDlrncuG43LpHbjsofhlkUZQXUR4hpS6EzSvgIANAYA0OWuo1aNstZU7dPyBe0UOUcxS6lsEQxSy+khDcX2YJDDUHiW+pkutNqTNKQCXiEBQXfdBOyRVdWxibEIea2zZd7V27s6hyTtbizyWn/l96qsOno1h4pIuhOSHFhGYRtYOIRekizlwPFpJKICgblPDsplcGUsEW4xOwJAL9St/YvF3dae7rjW+sMjcOQJXdyD5DlIKsUcB1HYhMMNhrS+C3DeuGqZ5FRjMjzA2zAuREwwg/QHMK2BITMG4Ac6l4UJMRfDXDP+kz8wPwAZMRuMvDGIQKAwO0CbO3nlNMCmMUBzM6Q8MkilOflaMtgiM57PM/iNMFg+OiBcwUJzAyBN3CM0QDDr6gMTiQ1oejHJyMXPk//PkxGt5dDoECv94aPQjA0cwT2+SMfhQyydQKYzLhcM/AUoJphgLG/jKYcRRoggmLTAbAEJkElGQ2ae/WxtQdGgfcbbG5kYKmpmMY0ZhnRgnQy0ARGapXhnEboPJoF0gwBCABIzp6CoDWw+zdmmqPInukzRGh/VgUZgEA1zrlL+lpoqX7YclomAnAnWnsgFZvDbmpaJ+J+oJACBEUQuBlClUGdLYd1lTRCoDFvsUVfDiPabitqyU80EiZJgoAoDCAFLGf5BZajxrFGgIXlTSUWVlVO3JckOs7UATHhcCMofmGoxU1C3zfa+p933+d+Rs+UCZYwNdyby8E4S7SQQhBbFIuXpXOYAAbQAAA1AhkEAoHFmi6UoWU0l6oo3AmBqgCq5MCUxRoEr3DAErciA2N12iFvFZUe25MxceDUr2HLSbi9CabW1bm0V29zSlMmiPy2F209FqRiAqdia+H4edg8BOlXl8keNpbK33lvH1dCBIwwNdyajaPm0BxJW0hnC0IZd9l8pcd30QTBPHOMHQRsz6fuDBDDpMki2k9dhqTHnRsMwimwwrShTH9ljkPnTjc/ztNLTHsaTMdpR5vzExBDZANzBwTzApQzGENDBYhDMoHhAERgMXBgMDJg+B5mUJ//PkxFh1jDooCvd2WMDQPMYSIMFgZMoTjNEz2M4C0ONJeNIg5NQkSBSOGQ43hAfGZMBqAUZULGKNxpA8Y4BAUbBxwY4OHBCpiCGHRSCph4QYQSmHC5tJGYU0GzIhghYYYSsMgBp5gQUMjYcBlsggcXw2iYjCH4byB4xCSza6nkCAQwkXMVKzUVcwUsCg+SE
       "                    Your browser does not support the audio element.\n",
       "                </audio>\n",
       "              "
      ],
      "text/plain": [
       "<IPython.lib.display.Audio object>"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "response = requests.post(\n",
    "    \"https://api.openai.com/v1/audio/speech\",\n",
    "    headers={\n",
    "        \"Authorization\": f\"Bearer {os.environ['OPENAI_API_KEY']}\",\n",
    "    },\n",
    "    json={\n",
    "        \"model\": \"tts-1\",\n",
    "        \"input\": result.choices[0].message.content,\n",
    "        \"voice\": \"onyx\",\n",
    "    },\n",
    ")\n",
    "\n",
    "audio = b\"\"\n",
    "for chunk in response.iter_content(chunk_size=1024 * 1024):\n",
    "    audio += chunk\n",
    "Audio(audio)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "openai",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}