openai-cookbook/examples/GPT_with_vision_for_video_understanding.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Processing and narrating a video with GPT's visual capabilities and the TTS API\n",
    "\n",
    "This notebook demonstrates how to use GPT's visual capabilities with a video. GPT-4 doesn't take videos as input directly, but we can use vision and the new 128K context window to describe the static frames of a whole video at once. We'll walk through two examples:\n",
    "\n",
    "1. Using GPT-4 to get a description of a video\n",
    "2. Generating a voiceover for a video with GPT-4 and the TTS API\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from IPython.display import display, Image, Audio\n",
    "\n",
    "import cv2  # We're using OpenCV to read video, to install !pip install opencv-python\n",
    "import base64\n",
    "import time\n",
    "from openai import OpenAI\n",
    "import os\n",
    "import requests\n",
    "\n",
    "client = OpenAI(api_key=os.environ.get(\"OPENAI_API_KEY\", \"<your OpenAI API key if not set as env var>\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Using GPT's visual capabilities to get a description of a video\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, we use OpenCV to extract frames from a nature [video](https://www.youtube.com/watch?v=kQ_7GtE529M) containing bisons and wolves:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "618 frames read.\n"
     ]
    }
   ],
   "source": [
    "video = cv2.VideoCapture(\"data/bison.mp4\")\n",
    "\n",
    "base64Frames = []\n",
    "while video.isOpened():\n",
    "    success, frame = video.read()\n",
    "    if not success:\n",
    "        break\n",
    "    _, buffer = cv2.imencode(\".jpg\", frame)\n",
    "    base64Frames.append(base64.b64encode(buffer).decode(\"utf-8\"))\n",
    "\n",
    "video.release()\n",
    "print(len(base64Frames), \"frames read.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Display frames to make sure we've read them in correctly:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/jpeg": "/9j/4AAQSkZJRgABAQAAAQABAAD/2wBDAAIBAQEBAQIBAQECAgICAgQDAgICAgUEBAMEBgUGBgYFBgYGBwkIBgcJBwYGCAsICQoKCgoKBggLDAsKDAkKCgr/2wBDAQICAgICAgUDAwUKBwYHCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgr/wAARCALQBQADASIAAhEBAxEB/8QAHwAAAQUBAQEBAQEAAAAAAAAAAAECAwQFBgcICQoL/8QAtRAAAgEDAwIEAwUFBAQAAAF9AQIDAAQRBRIhMUEGE1FhByJxFDKBkaEII0KxwRVS0fAkM2JyggkKFhcYGRolJicoKSo0NTY3ODk6Q0RFRkdISUpTVFVWV1hZWmNkZWZnaGlqc3R1dnd4eXqDhIWGh4iJipKTlJWWl5iZmqKjpKWmp6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uHi4+Tl5ufo6erx8vP09fb3+Pn6/8QAHwEAAwEBAQEBAQEBAQAAAAAAAAECAwQFBgcICQoL/8QAtREAAgECBAQDBAcFBAQAAQJ3AAECAxEEBSExBhJBUQdhcRMiMoEIFEKRobHBCSMzUvAVYnLRChYkNOEl8RcYGRomJygpKjU2Nzg5OkNERUZHSElKU1RVVldYWVpjZGVmZ2hpanN0dXZ3eHl6goOEhYaHiImKkpOUlZaXmJmaoqOkpaanqKmqsrO0tba3uLm6wsPExcbHyMnK0tPU1dbX2Nna4uPk5ebn6Onq8vP09fb3+Pn6/9oADAMBAAIRAxEAPwD6PV3B6fpVhOlMWNSfpUgGBig8sKjdhuqSmOmWzg0AJRRRg+lABRUlFAEdIUB5paKAEI44FSJyuaTy/emK27tTsAXQ4Bqt5fvVqgDJxVKdlYCh9mPcn8qu2L+RwOvqaeIVIzTBGB3q4y5gJrqcMA3fvVGSUuWTt3GKmYE5GfzpPJ3DJH5VTcbAUpYFjPmBzzyRjpV/RrqIt5Mh78HFJ9njK7WX86EhhUjArF3lIDUnKCAiN87uOO1ZxhxyenpUiEKMKelKeetF3FWQECKA3Ap9OKCgoPepAbyDxT1JI5prAL60BiBgUAPJA60DkZFQOHPVuO+KltsBDuHanZWuA6il+T3pKQBRQBk4p3l+9ADcE9BS4PoaeBgYooAjwR1FFSfjShSeQKAIsE9BS4Poak2N6UbWHagCKipNn+z+lGz/AGf0oAjop/lH3/KgRlTn+lADKKkooAjopShFG0+lACUUuD6GkoAKKkooAjoqSo6ACiilClulAEVFSUUAFFFFABRRRQAUUUUAFFFFADfL96bUlJ5OPWugBlFBBBwaKACiiigBAmDnNLRRQAUoGTikooAkooornAjoqSiugCOgjIxTnAA4FNoAKKKKACmqpByRTqKAG+X70eX706igBrRg8/0qJ1G7pU9IUBoAgwPQUVPsHqaawwcUARU4JkZzTqKAG+X70hGDin0hTJzmgCCipNo/u/pRgegoAZg+howfQ1Jg+hpKAGYPoaMH0NPooAZg+hpKkpjfeoAig71JTIQQpyMc0+gAooooArgZpSMdxT+DwfSkwnt+dZgQUUUjSIvDNiqinIT3FoPQ1DK6k8N+RqCa4jRSd3NHKLmIrnCgZGSe9Vmxg/Wn3GoIn31H/fNUrnWrGB9jSAk8jbXTGLsTZEso68VVYMrbgelVpvEtnnCqT+NVptf358qI9eMnJq+VickizNErHcE6j60xrTzBsxt471VXU52/dkYHUDNPW+mkYM7Yx6VQueI+TS1KZD4HXp60fZIEXIPOemOlV7jVHjG1XUgH1qjJrtxGS7EYquVmTt0NVYFUZ4/KkmiiUZ46d6wbnxLcP80ZwPbis6+1q9kUgSH65o5RHS3ptUQmZlA9zWDqn2WZW2XIbngKetY9/d311JtMuefyotbabh3ds+maaVhPYr6lY28rAQAZ/iz3qlBoZeZYxBnngk10EWnyyOP3Z+boSKv2nh9HG3YcjqcVruZqMmc9a+G3WTK24+9hq29O8IST/eshwemOta+maBAh+dWODzXX6BokUYBCsfl5DHpWMqqiaxhcyvDfgiNCpkhUKT83y12FhpVpYIVtowu7GSB1xT4bdYhgDAqZWJGBXLWquT902hTS1EVhGcseB6CpYryIHcgGQepFRsing0ogC9BWCbtqapWJ0dZJckDr2qZVAHJxVRYyuTk8mpkmdBgkn8aUXYZNhf736UocJ0NRfaT6H86Qzhh8yn25quYCczn1pPPz3/Wq7zAdE/WkWXdn5egz1qtwFadpe2B6VG/QGlVdvehx8tRpcCFn+bbmoqldQGyBSDjtVk8yCnR96TB9vzpQQnfNA3sOpCwXrTTMg5NRTXtuvHmDI7ZqVFthcn3j0NNklUDFUrjVUjOE5+hrMu9bky6q3bgCtY07bhqzbku1iUbnAFVZtbjjOBlvxrm7jWZpQAvGO2c1WaeeXlzyfetFGK6CZvXPiw5IQBSD6c1nT+IrmRSpz+dUPIlc8gmp4tHmkUt6jOD1pgiGW8mlb5pGPPGT0qMySucsST9a0rPQJCmZQfw7VYtfD+6UhySM/lQDbMiOGR+ufpirFvpM9zIoRDiult9JtEPI/MVN9mhiOUUD8KBXZzx8MSnr/OpLbw3KJRk/rW+ABwKdCRvAIpPYNRmjaTDBxKgOSK0/7Ptf+eYqKIAZ20+oLNSmOTnFJRXGAq9R9afUdOD4GCKAG09TkVCr5OCwp9NppgSUVHT0+6KQC0UVG7kHvQBJSJ0/GmrIcf405WGOTT6ALUdSVHV0+oElFAPGaM1mA7y/emkYOKd5ntQJMjOP1ppN7ANoopV6ikAxVIOSKdRRTbbYBRRRSARlyOlMRF3DipKKpSsrAJtX0prqR0b9KfQQD1FJNJ6gN8v3p8QwcUlPTp+NIBaKVV3d6Ch7UAJQMZ5FLsb0pQh7mgAXaT92n+QxGdn60kcRYnBqdRhQD2FAEIgdedp/Okqx161E8RXpz9BQAyil2P8A3T+VGx/7p/KgBKR/umnbW/un8qRlOMMCKAI6kwPQUgUCloAjoqQgHqKCM8GgCOipMD0FB5GKAI6KKKACo6n2L6Uzyx/dNAEdKq7u9PoxQBHRQQQcGigAop3l+9BTAzmgBkYPXFOwfQ0Q/dp9VP4gK9FSUEZGK2AjooooAkoPTio6KAHRoWIz61LsUcmmo7RnKHFIxODzQASAEYWmbW9Kcv3aWgBmD6GjB9DT6KAI6KKKACiiigCZVBGTR5fvSQysOoHX0qyIhIo+XH0oArFCBxzSbSeq/pVr7MPf86TyF7E0AVCgzwadTvL96PL96AGBQO1I49BUuweppmCOooAZg+hpCCOoqSggHqKAI6KfsXOaNgoAZRTih7UmxqAEpCmTnNOII6ikoAQJg5zS0UUAMb71NZRgnFSFMnOaTy/egCGnhVI6U/yz2NNP1oAjooooAKinJAJBqWmyRhlOV60AVgW6AnmjY/8AdP5VOIF4IH60/YPU0AVdj/3T+VGx/wC6fyq1sHqaNg9TQBVYYPU1Fv8A9r9atvApOB+tRSW6qcH68UkmiGyk8zK54/GoJZJicqx6+lXpowRgCoDGFGSOavTlDSxTZpQOpH1NQvvPGf1q5OOgxwaqyllJHoeKFBtXCzZBKvOGFUdRthKmSBwelX5CSOT3qnc52nkitoKwjKmsowcmMZ9xURt4lJKxgGrs6ORnnPvVSZZghKAE9s1siZFF9RhiJBHIPSq82sSEkRqfbAzVyLw7JKwMr5zzipDoNvHnLnI709EZcrMWaS7uDkZT0NN+xu5xK7deg6VvNoqDhJOSe4qSHRoWcgAt7elMLMwEsFAxhjz+tB0aMDJQYJ44rq4NGQjBXgfrUq6Iu0KVOfcVLkkUoNnIW+ho042qCMc8Vp2nhncNxiB5/hroLbQY2cDZk9qvw6PtGFBGOnFJzSKVM5y10BY5Qwt24PetRdEicDbGcjqB1rUGlTnkAZzViz06aOQMSCe1Q6yGopGbZ6L5Z3CI8f
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "display_handle = display(None, display_id=True)\n",
    "for img in base64Frames:\n",
    "    display_handle.update(Image(data=base64.b64decode(img.encode(\"utf-8\"))))\n",
    "    time.sleep(0.025)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once we have the video frames, we craft our prompt and send a request to GPT (Note that we don't need to send every frame for GPT to understand what's going on):\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\"🐺 Survival of the Fittest: An Epic Tale in the Snow ❄️ - Witness the intense drama of nature as a pack of wolves face off against mighty bison in a harsh winter landscape. This raw footage captures the essence of the wild where every creature fights for survival. With each frame, experience the tension, the strategy, and the sheer force exerted in this life-or-death struggle. See nature's true colors in this gripping encounter on the snowy plains. 🦬\"\n",
      "\n",
      "Remember to respect wildlife and nature. This video may contain scenes that some viewers might find intense or distressing, but they depict natural animal behaviors important for ecological studies and understanding the reality of life in the wilderness.\n"
     ]
    }
   ],
   "source": [
    "PROMPT_MESSAGES = [\n",
    "    {\n",
    "        \"role\": \"user\",\n",
    "        \"content\": [\n",
    "            \"These are frames from a video that I want to upload. Generate a compelling description that I can upload along with the video.\",\n",
    "            *map(lambda x: {\"image\": x, \"resize\": 768}, base64Frames[0::50]),\n",
    "        ],\n",
    "    },\n",
    "]\n",
    "params = {\n",
    "    \"model\": \"gpt-4-vision-preview\",\n",
    "    \"messages\": PROMPT_MESSAGES,\n",
    "    \"max_tokens\": 200,\n",
    "}\n",
    "\n",
    "result = client.chat.completions.create(**params)\n",
    "print(result.choices[0].message.content)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Generating a voiceover for a video with GPT-4 and the TTS API\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's create a voiceover for this video in the style of David Attenborough. Using the same video frames we prompt GPT to give us a short script:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "In the vast, white expanse of the northern wilderness, a drama as old as time unfolds. Here, amidst the silence of the snow, the wolf pack circles, their breaths visible as they cautiously approach their formidable quarry, the bison. These wolves are practiced hunters, moving with strategic precision, yet the bison, a titan of strength, stands resolute, a force to be reckoned with.\n",
      "\n",
      "As tension crackles in the frozen air, the wolves close in, their eyes locked on their target. The bison, wary of every movement, prepares to defend its life. It's a perilous dance between predator and prey, where each step could be the difference between life and death.\n",
      "\n",
      "In an instant, the quiet of the icy landscape is shattered. The bison charges, a desperate bid for survival as the pack swarms. The wolves are relentless, each one aware that their success depends on the strength of the collective. The bison, though powerful, is outnumbered, its massive form stirring up clouds of snow as it struggles.\n",
      "\n",
      "It's an epic battle, a testament to the harsh realities of nature. In these moments, there is no room for error, for either side. The wolves, agile and tenacious, work in unison, their bites a chorus aiming to bring down the great beast. The bison, its every heaving breath a testament to its will to survive, fights fiercely, but the odds are not in its favor.\n",
      "\n",
      "With the setting sun casting long shadows over the snow, the outcome is inevitable. Nature, in all its raw beauty and brutality, does not show favor. The wolves, now victors, gather around their prize, their survival in this harsh climate secured for a moment longer. It's a poignant reminder of the circle of life that rules this pristine wilderness, a reminder that every creature plays its part in the enduring saga of the natural world.\n"
     ]
    }
   ],
   "source": [
    "PROMPT_MESSAGES = [\n",
    "    {\n",
    "        \"role\": \"user\",\n",
    "        \"content\": [\n",
    "            \"These are frames of a video. Create a short voiceover script in the style of David Attenborough. Only include the narration.\",\n",
    "            *map(lambda x: {\"image\": x, \"resize\": 768}, base64Frames[0::60]),\n",
    "        ],\n",
    "    },\n",
    "]\n",
    "params = {\n",
    "    \"model\": \"gpt-4-vision-preview\",\n",
    "    \"messages\": PROMPT_MESSAGES,\n",
    "    \"max_tokens\": 500,\n",
    "}\n",
    "\n",
    "result = client.chat.completions.create(**params)\n",
    "print(result.choices[0].message.content)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can pass the script to the TTS API where it will generate an mp3 of the voiceover:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "                <audio  controls=\"controls\" >\n",
       "                    <source src=\"data:audio/wav;base64,//PkxABhnDnsAOawCD18XMSm4zcRDkbj7WjePjlSDdNjcKDXpzVqzSoz/081OdTrU5xN7TtA1KNTDYo0INSjUg0INAjSUylMowEMsmAQmMICGAhmYRiAWYLSFpC5BaQuQg4W3LxoB0A6AdFdMdCWXjLaFyEAiARIhYiwigCgiD6K6D6Y6Y6EtItIdItIhIRIhMRMRYipFSJiF40i1TrvXeoGqdYdQdIhMRMRUi7F2LCKAKaKbqnVOu9d6w6p13sTWIuxdi7FhF2LsXYuxQdU6p13rvZ2u9U6x1iLsYgzhhi7F2Lsa419h672ds7Z25bO2JrochyHIZwwxnDkOQxNnbO2vs7Z2ztd7L2uOQzhnDOGcOQ5DkM7Z277/u21tnbO3fchnDOGcLsYg5DOHIa+5bvxt22ds7Z219rj+OQ5DkOQ5DuP45bO2dsPXe19y2dtfa4zhnDXHIch2HIdx33/jb/xt/4fp4bch3IxKH8dhyHIdx34fjbtuW/8Pw2/8OQw/kOQw7D+Ow/Evdtnbvxt/4fp42ox2hZDlbY8PfDjo27kcDKWJoMGoSgwhQEjD3ARMJUAEwkgxTCxFfMPUTMwmw3TB/DRMEUH8wNgITAmApMCwCEwbQHTApAyMDsDIwFw//PkxExxbDnYAV7IABQwKQHDA5AOMBsBQwEQGiUCwCvmVGX1No0KPm4+ADwIqYgom+NNExRpoNKCKgE008SSCTCJkDEgpVB0mFJjTWJBg7nFkgEmYoyIJeYtshUBph00LGr1TRDkQcqxAEDska0LLIAxoZCJFQaKQjARyGpMGvkCit0O5JB0CwmOaXEgBMNg6aYsWRBg5cHDrEVygka7DKWKRBfwiATcMQ6HS+y0IYhocEUwljP4qh4LAQwNAIWKHJBq9dhSxWR3kRImqGLl3C/Cv4Qr9FNxYZQVVtV01VNKSqaKcPcpguhHiWNlmWv8S+bV3Uw3bhlw1eObYjC5ERoGYooeyBt2TQQu9rLpUyGr5Oys1I6XpZJ9JWxx4G0X8jalw+CrXrfdJ2G2WrMhh4mApVI9O4vdL5ZLPVGyIhMFTVZrMRQNP5DJgTNkiliF5mvM4iZf1JV5Eh4xGVuoes6RWfprihy4kETsoSoylyu9IlYRrKtsUEQao3ZYuBRnmVWQyV8rNTIcxwMxO0d5MZqV2TJYA5gy2po0Mm6GZDFVQWMwxUI/MDmHyDEYxyQxOMC/MJvBizA/wL4wMQBoMPhBQDAXwakwGsDKMA9AZh4fzDAUDFoKDMk9jMknDC8M//PkxFl6zDoMAZ/oAMzXGswnI4yuBwx7DEAgAYKAEZIj2YFhMYZggYkjEYoDMYZhmAAhMAg0MoTnMGxoMpBIMPBSMdiAMWAQMQrwMF3JACDEphmXh/my5zGWhemVQRme40mZoFmUxTAo7DCkEzCUHTBcDRoEzL0rDCbBzFstDGoF2sgYJjDQXzFwlRkDxoTRYIkuodgFZIGCgwBBEwVAcwTAdQhPcKASFgPMPBJMEByAoDINQCmuQASCQTMBg5BwDK7tts4zhtmXvOIbgUAIhSogMHTJSYLtQWocyOA2ZvuqhB7FkhYmwxpytjUFAEwISkhDrntXSuV/BABAYLASQgQDQJL0pwBQBSUAkimer2jrjKqoI4wPA0YNASW3R8GgGqSCHnUiEr52TVL0un85qs/qejavNKo+/URhpgMw/s+1iMVHMScFgDdJXkPMpdeCZ1Zy12ptRdpflJ+qa/zKQV5XORuf1jVrzWpPT/eguHp16MaGIujBDuw490698TikSdymhmdgx5Xbd+Dqjos7kcqduFN7NQE80Uf1/5ht7aoAkk4ZUhDRrj+gHTE5qYWipR796TmoitEbpct5k5t+mXIS8ZmwwRjtEwmLgGKYhgVhhXgaGBSAcYPwa4wCqYVY//PkxEBtfDoQF97YACGYSoDhgIAsmCOB+YAoKJgEAJmAiESGAOFUFQDAGBxmYOLlAkW7RICB4wodMjCjSHwyg4M/YzllM16JNqgTdns4A3NOSyZ1MLBjJR4xofNBkgAFmROBjYkYCOoogoHHAIw0PMTBjY3k6xnMsASIUTshD3OG9L2Pbg6bBI7j7oPY1mA5MwWKJeuI7QFAExgKDF6ofQmolpOspgGXSKWOg7sDM+yl1m0kG0RlLIEjEuErEdlqDAM4S0Wov8vmVPBTe4zvv3LnYgR+HIkTL2eICXZYsmAtxgMVXarGrgLgTXi5qvFyKEpwITYYcpKmGXAkBb1aubBWZQ+0BTRwpQ16QzTSZDORROyQsQwbssiHWuLmZg4jHKOJUj+tcf9zIGocoKazACiyzoBZuvd0LSwjLncVsgt+GdxWddKgWs/0kfR63cfh9VyPOyxrzSZU4z6zlV+2ts7Z+rvecxIcl/OHDbU4q8DC3Bo5yPw7VjUuWHtq/azD7EZZMdqCiTDwQEk8HMoFMghERjWnuMwx9AWMMDEJYTUtwrswIwH9MhWD+zA/Qg8wrkNfMQBRFQKMdQlMfg/NHGmMJg9MhKIMkCLMZDXNrRHMaEMzWpAsRzMeHMPBIwWy//PkxF1stDoEAP94aM0yNCgQGQBMYMEpmuqmfxAZicxwoMBYWGDhQMi0xaX0hzDhoMsBcVGZpUkmBxOZjT4KWhnaOHDFuZaX5xe+mxheBEoZ7aRpsNmnEAFWQc8MJskjGjUmNEIQhQwqCV6loF3jgHYaq0vw8buO4u5YVKEsulIPAdFHsTik1HmqKUl2X/AwBWqYLACY7bQzMN9Mv46sAMDmqqWLc2as4LrNle1YVRRNZW6fYej69jSIovl1Ze/NFXvwPjK4U7DXW5OpArptnfZwoIXa8MJbqni7C85pfTJGRyyHVWwtpLhQA2BqMOQzYduBJiL1qkPv67kLZyuZ3VMHddpiMJht1H+d+LuDD7SYKhmWNDlTlNtbgKGI1A0bfR1oAemMt/jHqCln31f534akVLI4Mj9HGIPiOtwJGqGUzENfMW4nZk0/R4UFqirzskt01ePZWZqO2LkXrUE9S37sYTEvAiQxY1dPNwwirjBCzBM9HVdhOS6NgjKWgUQ348ZJMQhG+TQgQ+kwnkMxMVIDxTDawSkwFUA0MBxAUDAxgZQwfwH1MAVATjCPwXswOQBsMDAAEjC9wH0wB0ABMGaATzAHgEwwXICqMB1ANDA5TTAcAyQVDHsFDOxqj4Aq//PkxH13tDn8AV/oAA27IEzlNc12FMwxFgzAFcCAyYgEMYBhEYlhwZaAaYZDOYQjIYsBuZ6EAYXiWaekobFEQYzFgb1lEYbjAYzKyZpBSYMpyaOBKZQKsbkk+Y2gOZNF8YUgaYAgiJAuFwEBANxsCAENAGFQQCgBJ3toyGqvd/SIBIKeEiAVhcseCAVG2npaI+JzrCGB4ApnsEXMzaakzhM9ZI1uniiKqPCwLsyFIgiBNP1GdIqFRF0lzLxgnT1Qp2p9gjwt2lDKIqmmlRBEYbg77xvM/r9rdSZTZZKzqersIe1Y95RVuKxUuEunHbrJmXNKaRcXepZRMnbJHWdxF21qP440vr2Hesw5ALtMukrluI9C7GRzaZq000lywG6lplkjdBgq52BRRu7rUzasTcZkT0srYZOtqsd3HVf51n8kVW1MyWkgOhg7Knld2LU9uc7YhiXy+Dofb/luU2aWNRyC+UFqZuR2H4K9/86WmQIIKBADGezQ01nY33MLsOSTRtzg4yE4lGMRGS0TIbxhww88OeMsqEJDAzQTgxyMGOMvSIkDAegJAwRMDSMFmAazAZALQxLIN2MI6A6zAAqjLMXjEMCjJ42DFMezFwZzcJjjPw5jLoFAEFgUggxCFeCj//PkxHF8dDokrZ/oAAZGUwRXg0kJAx9A8x6OA1usI1KRI7zMEwlBswgDcwnBYyECcwKE4MH8WBAwgCowpE8wVEYw+IAzQIASWYxuKwyoCoxMF8YFQwlAlmSYy2xkIh0KQMIb2uZdMRg4MohsMwBgMdxVMJwLMDgYclhTxsPtKYN0ZfAUMu/GEJZEBocGTJi
       "                    Your browser does not support the audio element.\n",
       "                </audio>\n",
       "              "
      ],
      "text/plain": [
       "<IPython.lib.display.Audio object>"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "response = requests.post(\n",
    "    \"https://api.openai.com/v1/audio/speech\",\n",
    "    headers={\n",
    "        \"Authorization\": f\"Bearer {os.environ['OPENAI_API_KEY']}\",\n",
    "    },\n",
    "    json={\n",
    "        \"model\": \"tts-1-1106\",\n",
    "        \"input\": result.choices[0].message.content,\n",
    "        \"voice\": \"onyx\",\n",
    "    },\n",
    ")\n",
    "\n",
    "audio = b\"\"\n",
    "for chunk in response.iter_content(chunk_size=1024 * 1024):\n",
    "    audio += chunk\n",
    "Audio(audio)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "openai",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.16"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}