langchain/docs/modules/agents/toolkits/examples/openapi.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "85fb2c03-ab88-4c8c-97e3-a7f2954555ab",
   "metadata": {},
   "source": [
    "# OpenAPI agents\n",
    "\n",
    "We can construct agents to consume arbitrary APIs, here APIs conformant to the OpenAPI/Swagger specification."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a389367b",
   "metadata": {},
   "source": [
    "# 1st example: hierarchical planning agent\n",
    "\n",
    "In this example, we'll consider an approach called hierarchical planning, common in robotics and appearing in recent works for LLMs X robotics. We'll see it's a viable approach to start working with a massive API spec AND to assist with user queries that require multiple steps against the API.\n",
    "\n",
    "The idea is simple: to get coherent agent behavior over long sequences behavior & to save on tokens, we'll separate concerns: a \"planner\" will be responsible for what endpoints to call and a \"controller\" will be responsible for how to call them.\n",
    "\n",
    "In the initial implementation, the planner is an LLM chain that has the name and a short description for each endpoint in context. The controller is an LLM agent that is instantiated with documentation for only the endpoints for a particular plan. There's a lot left to get this working very robustly :)\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4b6ecf6e",
   "metadata": {},
   "source": [
    "## To start, let's collect some OpenAPI specs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "0adf3537",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os, yaml"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "eb15cea0",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--2023-03-31 15:45:56--  https://raw.githubusercontent.com/openai/openai-openapi/master/openapi.yaml\n",
      "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.110.133, 185.199.109.133, 185.199.111.133, ...\n",
      "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.110.133|:443... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: 122995 (120K) [text/plain]\n",
      "Saving to: ‘openapi.yaml’\n",
      "\n",
      "openapi.yaml        100%[===================>] 120.11K  --.-KB/s    in 0.01s   \n",
      "\n",
      "2023-03-31 15:45:56 (10.4 MB/s) - ‘openapi.yaml’ saved [122995/122995]\n",
      "\n",
      "--2023-03-31 15:45:57--  https://www.klarna.com/us/shopping/public/openai/v0/api-docs\n",
      "Resolving www.klarna.com (www.klarna.com)... 52.84.150.34, 52.84.150.46, 52.84.150.61, ...\n",
      "Connecting to www.klarna.com (www.klarna.com)|52.84.150.34|:443... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: unspecified [application/json]\n",
      "Saving to: ‘api-docs’\n",
      "\n",
      "api-docs                [ <=>                ]   1.87K  --.-KB/s    in 0s      \n",
      "\n",
      "2023-03-31 15:45:57 (261 MB/s) - ‘api-docs’ saved [1916]\n",
      "\n",
      "--2023-03-31 15:45:57--  https://raw.githubusercontent.com/APIs-guru/openapi-directory/main/APIs/spotify.com/1.0.0/openapi.yaml\n",
      "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.110.133, 185.199.109.133, 185.199.111.133, ...\n",
      "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.110.133|:443... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: 286747 (280K) [text/plain]\n",
      "Saving to: ‘openapi.yaml’\n",
      "\n",
      "openapi.yaml        100%[===================>] 280.03K  --.-KB/s    in 0.02s   \n",
      "\n",
      "2023-03-31 15:45:58 (13.3 MB/s) - ‘openapi.yaml’ saved [286747/286747]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "!wget https://raw.githubusercontent.com/openai/openai-openapi/master/openapi.yaml\n",
    "!mv openapi.yaml openai_openapi.yaml\n",
    "!wget https://www.klarna.com/us/shopping/public/openai/v0/api-docs\n",
    "!mv api-docs klarna_openapi.yaml\n",
    "!wget https://raw.githubusercontent.com/APIs-guru/openapi-directory/main/APIs/spotify.com/1.0.0/openapi.yaml\n",
    "!mv openapi.yaml spotify_openapi.yaml"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "690a35bf",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.agents.agent_toolkits.openapi.spec import reduce_openapi_spec"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "69a8e1b9",
   "metadata": {},
   "outputs": [],
   "source": [
    "with open(\"openai_openapi.yaml\") as f:\n",
    "    raw_openai_api_spec = yaml.load(f, Loader=yaml.Loader)\n",
    "openai_api_spec = reduce_openapi_spec(raw_openai_api_spec)\n",
    "    \n",
    "with open(\"klarna_openapi.yaml\") as f:\n",
    "    raw_klarna_api_spec = yaml.load(f, Loader=yaml.Loader)\n",
    "klarna_api_spec = reduce_openapi_spec(raw_klarna_api_spec)\n",
    "\n",
    "with open(\"spotify_openapi.yaml\") as f:\n",
    "    raw_spotify_api_spec = yaml.load(f, Loader=yaml.Loader)\n",
    "spotify_api_spec = reduce_openapi_spec(raw_spotify_api_spec)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ba833d49",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "We'll work with the Spotify API as one of the examples of a somewhat complex API. There's a bit of auth-related setup to do if you want to replicate this.\n",
    "\n",
    "- You'll have to set up an application in the Spotify developer console, documented [here](https://developer.spotify.com/documentation/general/guides/authorization/), to get credentials: `CLIENT_ID`, `CLIENT_SECRET`, and `REDIRECT_URI`.\n",
    "- To get an access tokens (and keep them fresh), you can implement the oauth flows, or you can use `spotipy`. If you've set your Spotify creedentials as environment variables `SPOTIPY_CLIENT_ID`, `SPOTIPY_CLIENT_SECRET`, and `SPOTIPY_REDIRECT_URI`, you can use the helper functions below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "a82c2cfa",
   "metadata": {},
   "outputs": [],
   "source": [
    "import spotipy.util as util\n",
    "from langchain.requests import RequestsWrapper\n",
    "\n",
    "def construct_spotify_auth_headers(raw_spec: dict):\n",
    "    scopes = list(raw_spec['components']['securitySchemes']['oauth_2_0']['flows']['authorizationCode']['scopes'].keys())\n",
    "    access_token = util.prompt_for_user_token(scope=','.join(scopes))\n",
    "    return {\n",
    "        'Authorization': f'Bearer {access_token}'\n",
    "    }\n",
    "\n",
    "# Get API credentials.\n",
    "headers = construct_spotify_auth_headers(raw_spotify_api_spec)\n",
    "requests_wrapper = RequestsWrapper(headers=headers)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "76349780",
   "metadata": {},
   "source": [
    "## How big is this spec?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "2a93271e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "63"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "endpoints = [\n",
    "    (route, operation)\n",
    "    for route, operations in raw_spotify_api_spec[\"paths\"].items()\n",
    "    for operation in operations\n",
    "    if operation in [\"get\", \"post\"]\n",
    "]\n",
    "len(endpoints)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "eb829190",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "80326"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import tiktoken\n",
    "enc = tiktoken.encoding_for_model('text-davinci-003')\n",
    "def count_tokens(s): return len(enc.encode(s))\n",
    "\n",
    "count_tokens(yaml.dump(raw_spotify_api_spec))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cbc4964e",
   "metadata": {},
   "source": [
    "## Let's see some examples!\n",
    "\n",
    "Starting with GPT-4. (Some robustness iterations under way for GPT-3 family.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "7f42ee84",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/jeremywelborn/src/langchain/langchain/llms/openai.py:169: UserWarning: You are trying to use a chat model. This way of initializing it is no longer supported. Instead, please use: `from langchain.chat_models import ChatOpenAI`\n",
      "  warnings.warn(\n",
      "/Users/jeremywelborn/src/langchain/langchain/llms/openai.py:608: UserWarning: You are trying to use a chat model. This way of initializing it is no longer supported. Instead, please use: `from langchain.chat_models import ChatOpenAI`\n",
      "  warnings.warn(\n"
     ]
    }
   ],
   "source": [
    "from langchain.llms.openai import OpenAI\n",
    "from langchain.agents.agent_toolkits.openapi import planner\n",
    "llm = OpenAI(model_name=\"gpt-4\", temperature=0.0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "38762cc0",
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
      "\u001b[32;1m\u001b[1;3mAction: api_planner\n",
      "Action Input: I need to find the right API calls to create a playlist with the first song from Kind of Blue and name it Machine Blues\u001b[0m\n",
      "Observation: \u001b[36;1m\u001b[1;3m1. GET /search to search for the album \"Kind of Blue\"\n",
      "2. GET /albums/{id}/tracks to get the tracks from the \"Kind of Blue\" album\n",
      "3. GET /me to get the current user's information\n",
      "4. POST /users/{user_id}/playlists to create a new playlist named \"Machine Blues\" for the current user\n",
      "5. POST /playlists/{playlist_id}/tracks to add the first song from \"Kind of Blue\" to the \"Machine Blues\" playlist\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3mI have the plan, now I need to execute the API calls.\n",
      "Action: api_controller\n",
      "Action Input: 1. GET /search to search for the album \"Kind of Blue\"\n",
      "2. GET /albums/{id}/tracks to get the tracks from the \"Kind of Blue\" album\n",
      "3. GET /me to get the current user's information\n",
      "4. POST /users/{user_id}/playlists to create a new playlist named \"Machine Blues\" for the current user\n",
      "5. POST /playlists/{playlist_id}/tracks to add the first song from \"Kind of Blue\" to the \"Machine Blues\" playlist\u001b[0m\n",
      "\n",
      "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
      "\u001b[32;1m\u001b[1;3mAction: requests_get\n",
      "Action Input: {\"url\": \"https://api.spotify.com/v1/search?q=Kind%20of%20Blue&type=album\", \"output_instructions\": \"Extract the id of the first album in the search results\"}\u001b[0m\n",
      "Observation: \u001b[36;1m\u001b[1;3m1weenld61qoidwYuZ1GESA\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3mAction: requests_get\n",
      "Action Input: {\"url\": \"https://api.spotify.com/v1/albums/1weenld61qoidwYuZ1GESA/tracks\", \"output_instructions\": \"Extract the id of the first track in the album\"}\u001b[0m\n",
      "Observation: \u001b[36;1m\u001b[1;3m7q3kkfAVpmcZ8g6JUThi3o\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3mAction: requests_get\n",
      "Action Input: {\"url\": \"https://api.spotify.com/v1/me\", \"output_instructions\": \"Extract the id of the current user\"}\u001b[0m\n",
      "Observation: \u001b[36;1m\u001b[1;3m22rhrz4m4kvpxlsb5hezokzwi\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3mAction: requests_post\n",
      "Action Input: {\"url\": \"https://api.spotify.com/v1/users/22rhrz4m4kvpxlsb5hezokzwi/playlists\", \"data\": {\"name\": \"Machine Blues\"}, \"output_instructions\": \"Extract the id of the created playlist\"}\u001b[0m\n",
      "Observation: \u001b[33;1m\u001b[1;3m7lzoEi44WOISnFYlrAIqyX\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3mAction: requests_post\n",
      "Action Input: {\"url\": \"https://api.spotify.com/v1/playlists/7lzoEi44WOISnFYlrAIqyX/tracks\", \"data\": {\"uris\": [\"spotify:track:7q3kkfAVpmcZ8g6JUThi3o\"]}, \"output_instructions\": \"Confirm that the track was added to the playlist\"}\n",
      "\u001b[0m\n",
      "Observation: \u001b[33;1m\u001b[1;3mThe track was added to the playlist, confirmed by the snapshot_id: MiwxODMxNTMxZTFlNzg3ZWFlZmMxYTlmYWQyMDFiYzUwNDEwMTAwZmE1.\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3mI am finished executing the plan.\n",
      "Final Answer: The first song from the \"Kind of Blue\" album has been added to the \"Machine Blues\" playlist.\u001b[0m\n",
      "\n",
      "\u001b[1m> Finished chain.\u001b[0m\n",
      "\n",
      "Observation: \u001b[33;1m\u001b[1;3mThe first song from the \"Kind of Blue\" album has been added to the \"Machine Blues\" playlist.\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3mI am finished executing the plan and have created the playlist with the first song from Kind of Blue.\n",
      "Final Answer: I have created a playlist called \"Machine Blues\" with the first song from the \"Kind of Blue\" album.\u001b[0m\n",
      "\n",
      "\u001b[1m> Finished chain.\u001b[0m\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'I have created a playlist called \"Machine Blues\" with the first song from the \"Kind of Blue\" album.'"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "spotify_agent = planner.create_openapi_agent(spotify_api_spec, requests_wrapper, llm)\n",
    "user_query = \"make me a playlist with the first song from kind of blue. call it machine blues.\"\n",
    "spotify_agent.run(user_query)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "96184181",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
      "\u001b[32;1m\u001b[1;3mAction: api_planner\n",
      "Action Input: I need to find the right API calls to get a blues song recommendation for the user\u001b[0m\n",
      "Observation: \u001b[36;1m\u001b[1;3m1. GET /me to get the current user's information\n",
      "2. GET /recommendations/available-genre-seeds to retrieve a list of available genres\n",
      "3. GET /recommendations with the seed_genre parameter set to \"blues\" to get a blues song recommendation for the user\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3mI have the plan, now I need to execute the API calls.\n",
      "Action: api_controller\n",
      "Action Input: 1. GET /me to get the current user's information\n",
      "2. GET /recommendations/available-genre-seeds to retrieve a list of available genres\n",
      "3. GET /recommendations with the seed_genre parameter set to \"blues\" to get a blues song recommendation for the user\u001b[0m\n",
      "\n",
      "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
      "\u001b[32;1m\u001b[1;3mAction: requests_get\n",
      "Action Input: {\"url\": \"https://api.spotify.com/v1/me\", \"output_instructions\": \"Extract the user's id and username\"}\u001b[0m\n",
      "Observation: \u001b[36;1m\u001b[1;3mID: 22rhrz4m4kvpxlsb5hezokzwi, Username: Jeremy Welborn\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3mAction: requests_get\n",
      "Action Input: {\"url\": \"https://api.spotify.com/v1/recommendations/available-genre-seeds\", \"output_instructions\": \"Extract the list of available genres\"}\u001b[0m\n",
      "Observation: \u001b[36;1m\u001b[1;3macoustic, afrobeat, alt-rock, alternative, ambient, anime, black-metal, bluegrass, blues, bossanova, brazil, breakbeat, british, cantopop, chicago-house, children, chill, classical, club, comedy, country, dance, dancehall, death-metal, deep-house, detroit-techno, disco, disney, drum-and-bass, dub, dubstep, edm, electro, electronic, emo, folk, forro, french, funk, garage, german, gospel, goth, grindcore, groove, grunge, guitar, happy, hard-rock, hardcore, hardstyle, heavy-metal, hip-hop, holidays, honky-tonk, house, idm, indian, indie, indie-pop, industrial, iranian, j-dance, j-idol, j-pop, j-rock, jazz, k-pop, kids, latin, latino, malay, mandopop, metal, metal-misc, metalcore, minimal-techno, movies, mpb, new-age, new-release, opera, pagode, party, philippines-\u001b[0m\n",
      "Thought:"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Retrying langchain.llms.openai.completion_with_retry.<locals>._completion_with_retry in 4.0 seconds as it raised RateLimitError: That model is currently overloaded with other requests. You can retry your request, or contact us through our help center at help.openai.com if the error persists. (Please include the request ID 2167437a0072228238f3c0c5b3882764 in your message.).\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[32;1m\u001b[1;3mAction: requests_get\n",
      "Action Input: {\"url\": \"https://api.spotify.com/v1/recommendations?seed_genres=blues\", \"output_instructions\": \"Extract the list of recommended tracks with their ids and names\"}\u001b[0m\n",
      "Observation: \u001b[36;1m\u001b[1;3m[\n",
      "  {\n",
      "    id: '03lXHmokj9qsXspNsPoirR',\n",
      "    name: 'Get Away Jordan'\n",
      "  }\n",
      "]\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3mI am finished executing the plan.\n",
      "Final Answer: The recommended blues song for user Jeremy Welborn (ID: 22rhrz4m4kvpxlsb5hezokzwi) is \"Get Away Jordan\" with the track ID: 03lXHmokj9qsXspNsPoirR.\u001b[0m\n",
      "\n",
      "\u001b[1m> Finished chain.\u001b[0m\n",
      "\n",
      "Observation: \u001b[33;1m\u001b[1;3mThe recommended blues song for user Jeremy Welborn (ID: 22rhrz4m4kvpxlsb5hezokzwi) is \"Get Away Jordan\" with the track ID: 03lXHmokj9qsXspNsPoirR.\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3mI am finished executing the plan and have the information the user asked for.\n",
      "Final Answer: The recommended blues song for you is \"Get Away Jordan\" with the track ID: 03lXHmokj9qsXspNsPoirR.\u001b[0m\n",
      "\n",
      "\u001b[1m> Finished chain.\u001b[0m\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'The recommended blues song for you is \"Get Away Jordan\" with the track ID: 03lXHmokj9qsXspNsPoirR.'"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "user_query = \"give me a song I'd like, make it blues-ey\"\n",
    "spotify_agent.run(user_query)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d5317926",
   "metadata": {},
   "source": [
    "#### Try another API.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "06c3d6a8",
   "metadata": {},
   "outputs": [],
   "source": [
    "headers = {\n",
    "    \"Authorization\": f\"Bearer {os.getenv('OPENAI_API_KEY')}\"\n",
    "}\n",
    "openai_requests_wrapper=RequestsWrapper(headers=headers)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "3a9cc939",
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
      "\u001b[32;1m\u001b[1;3mAction: api_planner\n",
      "Action Input: I need to find the right API calls to generate a short piece of advice\u001b[0m\n",
      "Observation: \u001b[36;1m\u001b[1;3m1. GET /engines to retrieve the list of available engines\n",
      "2. POST /completions with the selected engine and a prompt for generating a short piece of advice\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3mI have the plan, now I need to execute the API calls.\n",
      "Action: api_controller\n",
      "Action Input: 1. GET /engines to retrieve the list of available engines\n",
      "2. POST /completions with the selected engine and a prompt for generating a short piece of advice\u001b[0m\n",
      "\n",
      "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
      "\u001b[32;1m\u001b[1;3mAction: requests_get\n",
      "Action Input: {\"url\": \"https://api.openai.com/v1/engines\", \"output_instructions\": \"Extract the ids of the engines\"}\u001b[0m\n",
      "Observation: \u001b[36;1m\u001b[1;3mbabbage, davinci, text-davinci-edit-001, babbage-code-search-code, text-similarity-babbage-001, code-davinci-edit-001, text-davinci-001, ada, babbage-code-search-text, babbage-similarity, whisper-1, code-search-babbage-text-001, text-curie-001, code-search-babbage-code-001, text-ada-001, text-embedding-ada-002, text-similarity-ada-001, curie-instruct-beta, ada-code-search-code, ada-similarity, text-davinci-003, code-search-ada-text-001, text-search-ada-query-001, davinci-search-document, ada-code-search-text, text-search-ada-doc-001, davinci-instruct-beta, text-similarity-curie-001, code-search-ada-code-001\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3mI will use the \"davinci\" engine to generate a short piece of advice.\n",
      "Action: requests_post\n",
      "Action Input: {\"url\": \"https://api.openai.com/v1/completions\", \"data\": {\"engine\": \"davinci\", \"prompt\": \"Give me a short piece of advice on how to be more productive.\"}, \"output_instructions\": \"Extract the text from the first choice\"}\u001b[0m\n",
      "Observation: \u001b[33;1m\u001b[1;3m\"you must provide a model parameter\"\u001b[0m\n",
      "Thought:!! Could not _extract_tool_and_input from \"I cannot finish executing the plan without knowing how to provide the model parameter correctly.\" in _get_next_action\n",
      "\u001b[32;1m\u001b[1;3mI cannot finish executing the plan without knowing how to provide the model parameter correctly.\u001b[0m\n",
      "\n",
      "\u001b[1m> Finished chain.\u001b[0m\n",
      "\n",
      "Observation: \u001b[33;1m\u001b[1;3mI need more information on how to provide the model parameter correctly in the POST request to generate a short piece of advice.\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3mI need to adjust my plan to include the model parameter in the POST request.\n",
      "Action: api_planner\n",
      "Action Input: I need to find the right API calls to generate a short piece of advice, including the model parameter in the POST request\u001b[0m\n",
      "Observation: \u001b[36;1m\u001b[1;3m1. GET /models to retrieve the list of available models\n",
      "2. Choose a suitable model from the list\n",
      "3. POST /completions with the chosen model as a parameter to generate a short piece of advice\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3mI have an updated plan, now I need to execute the API calls.\n",
      "Action: api_controller\n",
      "Action Input: 1. GET /models to retrieve the list of available models\n",
      "2. Choose a suitable model from the list\n",
      "3. POST /completions with the chosen model as a parameter to generate a short piece of advice\u001b[0m\n",
      "\n",
      "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
      "\u001b[32;1m\u001b[1;3mAction: requests_get\n",
      "Action Input: {\"url\": \"https://api.openai.com/v1/models\", \"output_instructions\": \"Extract the ids of the available models\"}\u001b[0m\n",
      "Observation: \u001b[36;1m\u001b[1;3mbabbage, davinci, text-davinci-edit-001, babbage-code-search-code, text-similarity-babbage-001, code-davinci-edit-001, text-davinci-edit-001, ada\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3mAction: requests_post\n",
      "Action Input: {\"url\": \"https://api.openai.com/v1/completions\", \"data\": {\"model\": \"davinci\", \"prompt\": \"Give me a short piece of advice on how to improve communication skills.\"}, \"output_instructions\": \"Extract the text from the first choice\"}\u001b[0m\n",
      "Observation: \u001b[33;1m\u001b[1;3m\"I'd like to broaden my horizon.\\n\\nI was trying to\"\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3mI cannot finish executing the plan without knowing some other information.\n",
      "\n",
      "Final Answer: The generated text is not a piece of advice on improving communication skills. I would need to retry the API call with a different prompt or model to get a more relevant response.\u001b[0m\n",
      "\n",
      "\u001b[1m> Finished chain.\u001b[0m\n",
      "\n",
      "Observation: \u001b[33;1m\u001b[1;3mThe generated text is not a piece of advice on improving communication skills. I would need to retry the API call with a different prompt or model to get a more relevant response.\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3mI need to adjust my plan to include a more specific prompt for generating a short piece of advice on improving communication skills.\n",
      "Action: api_planner\n",
      "Action Input: I need to find the right API calls to generate a short piece of advice on improving communication skills, including the model parameter in the POST request\u001b[0m\n",
      "Observation: \u001b[36;1m\u001b[1;3m1. GET /models to retrieve the list of available models\n",
      "2. Choose a suitable model for generating text (e.g., text-davinci-002)\n",
      "3. POST /completions with the chosen model and a prompt related to improving communication skills to generate a short piece of advice\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3mI have an updated plan, now I need to execute the API calls.\n",
      "Action: api_controller\n",
      "Action Input: 1. GET /models to retrieve the list of available models\n",
      "2. Choose a suitable model for generating text (e.g., text-davinci-002)\n",
      "3. POST /completions with the chosen model and a prompt related to improving communication skills to generate a short piece of advice\u001b[0m\n",
      "\n",
      "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
      "\u001b[32;1m\u001b[1;3mAction: requests_get\n",
      "Action Input: {\"url\": \"https://api.openai.com/v1/models\", \"output_instructions\": \"Extract the names of the models\"}\u001b[0m\n",
      "Observation: \u001b[36;1m\u001b[1;3mbabbage, davinci, text-davinci-edit-001, babbage-code-search-code, text-similarity-babbage-001, code-davinci-edit-001, text-davinci-edit-001, ada\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3mAction: requests_post\n",
      "Action Input: {\"url\": \"https://api.openai.com/v1/completions\", \"data\": {\"model\": \"text-davinci-002\", \"prompt\": \"Give a short piece of advice on how to improve communication skills\"}, \"output_instructions\": \"Extract the text from the first choice\"}\u001b[0m\n",
      "Observation: \u001b[33;1m\u001b[1;3m\"Some basic advice for improving communication skills would be to make sure to listen\"\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3mI am finished executing the plan.\n",
      "\n",
      "Final Answer: Some basic advice for improving communication skills would be to make sure to listen.\u001b[0m\n",
      "\n",
      "\u001b[1m> Finished chain.\u001b[0m\n",
      "\n",
      "Observation: \u001b[33;1m\u001b[1;3mSome basic advice for improving communication skills would be to make sure to listen.\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3mI am finished executing the plan and have the information the user asked for.\n",
      "Final Answer: A short piece of advice for improving communication skills is to make sure to listen.\u001b[0m\n",
      "\n",
      "\u001b[1m> Finished chain.\u001b[0m\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'A short piece of advice for improving communication skills is to make sure to listen.'"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Meta!\n",
    "llm = OpenAI(model_name=\"gpt-4\", temperature=0.25)\n",
    "openai_agent = planner.create_openapi_agent(openai_api_spec, openai_requests_wrapper, llm)\n",
    "user_query = \"generate a short piece of advice\"\n",
    "openai_agent.run(user_query)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f32bc6ec",
   "metadata": {},
   "source": [
    "Takes awhile to get there!"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "461229e4",
   "metadata": {},
   "source": [
    "## 2nd example: \"json explorer\" agent\n",
    "\n",
    "Here's an agent that's not particularly practical, but neat! The agent has access to 2 toolkits. One comprises tools to interact with json: one tool to list the keys of a json object and another tool to get the value for a given key. The other toolkit comprises `requests` wrappers to send GET and POST requests. This agent consumes a lot calls to the language model, but does a surprisingly decent job.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "f8dfa1d3",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.agents import create_openapi_agent\n",
    "from langchain.agents.agent_toolkits import OpenAPIToolkit\n",
    "from langchain.llms.openai import OpenAI\n",
    "from langchain.requests import TextRequestsWrapper\n",
    "from langchain.tools.json.tool import JsonSpec"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "9ecd1ba0-3937-4359-a41e-68605f0596a1",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "with open(\"openai_openapi.yaml\") as f:\n",
    "    data = yaml.load(f, Loader=yaml.FullLoader)\n",
    "json_spec=JsonSpec(dict_=data, max_value_length=4000)\n",
    "\n",
    "\n",
    "openapi_toolkit = OpenAPIToolkit.from_llm(OpenAI(temperature=0), json_spec, openai_requests_wrapper, verbose=True)\n",
    "openapi_agent_executor = create_openapi_agent(\n",
    "    llm=OpenAI(temperature=0),\n",
    "    toolkit=openapi_toolkit,\n",
    "    verbose=True\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "548db7f7-337b-4ba8-905c-e7fd58c01799",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
      "\u001b[32;1m\u001b[1;3mAction: json_explorer\n",
      "Action Input: What is the base url for the API?\u001b[0m\n",
      "\n",
      "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
      "\u001b[32;1m\u001b[1;3mAction: json_spec_list_keys\n",
      "Action Input: data\u001b[0m\n",
      "Observation: \u001b[36;1m\u001b[1;3m['openapi', 'info', 'servers', 'tags', 'paths', 'components', 'x-oaiMeta']\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3m I should look at the servers key to see what the base url is\n",
      "Action: json_spec_list_keys\n",
      "Action Input: data[\"servers\"][0]\u001b[0m\n",
      "Observation: \u001b[36;1m\u001b[1;3mValueError('Value at path `data[\"servers\"][0]` is not a dict, get the value directly.')\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3m I should get the value of the servers key\n",
      "Action: json_spec_get_value\n",
      "Action Input: data[\"servers\"][0]\u001b[0m\n",
      "Observation: \u001b[33;1m\u001b[1;3m{'url': 'https://api.openai.com/v1'}\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3m I now know the base url for the API\n",
      "Final Answer: The base url for the API is https://api.openai.com/v1\u001b[0m\n",
      "\n",
      "\u001b[1m> Finished chain.\u001b[0m\n",
      "\n",
      "Observation: \u001b[33;1m\u001b[1;3mThe base url for the API is https://api.openai.com/v1\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3m I should find the path for the /completions endpoint.\n",
      "Action: json_explorer\n",
      "Action Input: What is the path for the /completions endpoint?\u001b[0m\n",
      "\n",
      "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
      "\u001b[32;1m\u001b[1;3mAction: json_spec_list_keys\n",
      "Action Input: data\u001b[0m\n",
      "Observation: \u001b[36;1m\u001b[1;3m['openapi', 'info', 'servers', 'tags', 'paths', 'components', 'x-oaiMeta']\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3m I should look at the paths key to see what endpoints exist\n",
      "Action: json_spec_list_keys\n",
      "Action Input: data[\"paths\"]\u001b[0m\n",
      "Observation: \u001b[36;1m\u001b[1;3m['/engines', '/engines/{engine_id}', '/completions', '/chat/completions', '/edits', '/images/generations', '/images/edits', '/images/variations', '/embeddings', '/audio/transcriptions', '/audio/translations', '/engines/{engine_id}/search', '/files', '/files/{file_id}', '/files/{file_id}/content', '/answers', '/classifications', '/fine-tunes', '/fine-tunes/{fine_tune_id}', '/fine-tunes/{fine_tune_id}/cancel', '/fine-tunes/{fine_tune_id}/events', '/models', '/models/{model}', '/moderations']\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3m I now know the path for the /completions endpoint\n",
      "Final Answer: The path for the /completions endpoint is data[\"paths\"][2]\u001b[0m\n",
      "\n",
      "\u001b[1m> Finished chain.\u001b[0m\n",
      "\n",
      "Observation: \u001b[33;1m\u001b[1;3mThe path for the /completions endpoint is data[\"paths\"][2]\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3m I should find the required parameters for the POST request.\n",
      "Action: json_explorer\n",
      "Action Input: What are the required parameters for a POST request to the /completions endpoint?\u001b[0m\n",
      "\n",
      "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
      "\u001b[32;1m\u001b[1;3mAction: json_spec_list_keys\n",
      "Action Input: data\u001b[0m\n",
      "Observation: \u001b[36;1m\u001b[1;3m['openapi', 'info', 'servers', 'tags', 'paths', 'components', 'x-oaiMeta']\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3m I should look at the paths key to see what endpoints exist\n",
      "Action: json_spec_list_keys\n",
      "Action Input: data[\"paths\"]\u001b[0m\n",
      "Observation: \u001b[36;1m\u001b[1;3m['/engines', '/engines/{engine_id}', '/completions', '/chat/completions', '/edits', '/images/generations', '/images/edits', '/images/variations', '/embeddings', '/audio/transcriptions', '/audio/translations', '/engines/{engine_id}/search', '/files', '/files/{file_id}', '/files/{file_id}/content', '/answers', '/classifications', '/fine-tunes', '/fine-tunes/{fine_tune_id}', '/fine-tunes/{fine_tune_id}/cancel', '/fine-tunes/{fine_tune_id}/events', '/models', '/models/{model}', '/moderations']\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3m I should look at the /completions endpoint to see what parameters are required\n",
      "Action: json_spec_list_keys\n",
      "Action Input: data[\"paths\"][\"/completions\"]\u001b[0m\n",
      "Observation: \u001b[36;1m\u001b[1;3m['post']\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3m I should look at the post key to see what parameters are required\n",
      "Action: json_spec_list_keys\n",
      "Action Input: data[\"paths\"][\"/completions\"][\"post\"]\u001b[0m\n",
      "Observation: \u001b[36;1m\u001b[1;3m['operationId', 'tags', 'summary', 'requestBody', 'responses', 'x-oaiMeta']\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3m I should look at the requestBody key to see what parameters are required\n",
      "Action: json_spec_list_keys\n",
      "Action Input: data[\"paths\"][\"/completions\"][\"post\"][\"requestBody\"]\u001b[0m\n",
      "Observation: \u001b[36;1m\u001b[1;3m['required', 'content']\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3m I should look at the content key to see what parameters are required\n",
      "Action: json_spec_list_keys\n",
      "Action Input: data[\"paths\"][\"/completions\"][\"post\"][\"requestBody\"][\"content\"]\u001b[0m\n",
      "Observation: \u001b[36;1m\u001b[1;3m['application/json']\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3m I should look at the application/json key to see what parameters are required\n",
      "Action: json_spec_list_keys\n",
      "Action Input: data[\"paths\"][\"/completions\"][\"post\"][\"requestBody\"][\"content\"][\"application/json\"]\u001b[0m\n",
      "Observation: \u001b[36;1m\u001b[1;3m['schema']\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3m I should look at the schema key to see what parameters are required\n",
      "Action: json_spec_list_keys\n",
      "Action Input: data[\"paths\"][\"/completions\"][\"post\"][\"requestBody\"][\"content\"][\"application/json\"][\"schema\"]\u001b[0m\n",
      "Observation: \u001b[36;1m\u001b[1;3m['$ref']\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3m I should look at the $ref key to see what parameters are required\n",
      "Action: json_spec_list_keys\n",
      "Action Input: data[\"paths\"][\"/completions\"][\"post\"][\"requestBody\"][\"content\"][\"application/json\"][\"schema\"][\"$ref\"]\u001b[0m\n",
      "Observation: \u001b[36;1m\u001b[1;3mValueError('Value at path `data[\"paths\"][\"/completions\"][\"post\"][\"requestBody\"][\"content\"][\"application/json\"][\"schema\"][\"$ref\"]` is not a dict, get the value directly.')\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3m I should look at the $ref key to get the value directly\n",
      "Action: json_spec_get_value\n",
      "Action Input: data[\"paths\"][\"/completions\"][\"post\"][\"requestBody\"][\"content\"][\"application/json\"][\"schema\"][\"$ref\"]\u001b[0m\n",
      "Observation: \u001b[33;1m\u001b[1;3m#/components/schemas/CreateCompletionRequest\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3m I should look at the CreateCompletionRequest schema to see what parameters are required\n",
      "Action: json_spec_list_keys\n",
      "Action Input: data[\"components\"][\"schemas\"][\"CreateCompletionRequest\"]\u001b[0m\n",
      "Observation: \u001b[36;1m\u001b[1;3m['type', 'properties', 'required']\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3m I should look at the required key to see what parameters are required\n",
      "Action: json_spec_get_value\n",
      "Action Input: data[\"components\"][\"schemas\"][\"CreateCompletionRequest\"][\"required\"]\u001b[0m\n",
      "Observation: \u001b[33;1m\u001b[1;3m['model']\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3m I now know the final answer\n",
      "Final Answer: The required parameters for a POST request to the /completions endpoint are 'model'.\u001b[0m\n",
      "\n",
      "\u001b[1m> Finished chain.\u001b[0m\n",
      "\n",
      "Observation: \u001b[33;1m\u001b[1;3mThe required parameters for a POST request to the /completions endpoint are 'model'.\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3m I now know the parameters needed to make the request.\n",
      "Action: requests_post\n",
      "Action Input: { \"url\": \"https://api.openai.com/v1/completions\", \"data\": { \"model\": \"davinci\", \"prompt\": \"tell me a joke\" } }\u001b[0m\n",
      "Observation: \u001b[33;1m\u001b[1;3m{\"id\":\"cmpl-70Ivzip3dazrIXU8DSVJGzFJj2rdv\",\"object\":\"text_completion\",\"created\":1680307139,\"model\":\"davinci\",\"choices\":[{\"text\":\" with mummy not there”\\n\\nYou dig deep and come up with,\",\"index\":0,\"logprobs\":null,\"finish_reason\":\"length\"}],\"usage\":{\"prompt_tokens\":4,\"completion_tokens\":16,\"total_tokens\":20}}\n",
      "\u001b[0m\n",
      "Thought:\u001b[32;1m\u001b[1;3m I now know the final answer.\n",
      "Final Answer: The response of the POST request is {\"id\":\"cmpl-70Ivzip3dazrIXU8DSVJGzFJj2rdv\",\"object\":\"text_completion\",\"created\":1680307139,\"model\":\"davinci\",\"choices\":[{\"text\":\" with mummy not there”\\n\\nYou dig deep and come up with,\",\"index\":0,\"logprobs\":null,\"finish_reason\":\"length\"}],\"usage\":{\"prompt_tokens\":4,\"completion_tokens\":16,\"total_tokens\":20}}\u001b[0m\n",
      "\n",
      "\u001b[1m> Finished chain.\u001b[0m\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'The response of the POST request is {\"id\":\"cmpl-70Ivzip3dazrIXU8DSVJGzFJj2rdv\",\"object\":\"text_completion\",\"created\":1680307139,\"model\":\"davinci\",\"choices\":[{\"text\":\" with mummy not there”\\\\n\\\\nYou dig deep and come up with,\",\"index\":0,\"logprobs\":null,\"finish_reason\":\"length\"}],\"usage\":{\"prompt_tokens\":4,\"completion_tokens\":16,\"total_tokens\":20}}'"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "openapi_agent_executor.run(\"Make a post request to openai /completions. The prompt should be 'tell me a joke.'\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}