From 7b426f21143904c2d1da63a9f97a190c1e999699 Mon Sep 17 00:00:00 2001 From: Your Name Date: Sun, 31 Mar 2024 22:03:07 -0700 Subject: [PATCH] adding streaming to assistants api cookbook --- examples/Assistants_API_overview_python.ipynb | 738 +++++++++--------- 1 file changed, 385 insertions(+), 353 deletions(-) diff --git a/examples/Assistants_API_overview_python.ipynb b/examples/Assistants_API_overview_python.ipynb index 6435e497..0ad84878 100644 --- a/examples/Assistants_API_overview_python.ipynb +++ b/examples/Assistants_API_overview_python.ipynb @@ -4,14 +4,14 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# Assistants API Overview (Python SDK)" + "# Assistants API Overview with Streaming (Python SDK)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "The new [Assistants API](https://platform.openai.com/docs/assistants/overview) is a stateful evolution of our [Chat Completions API](https://platform.openai.com/docs/guides/text-generation/chat-completions-api) meant to simplify the creation of assistant-like experiences, and enable developer access to powerful tools like Code Interpreter and Retrieval." + "The new [Assistants API](https://platform.openai.com/docs/assistants/overview) is a stateful evolution of our [Chat Completions API](https://platform.openai.com/docs/guides/text-generation/chat-completions-api) meant to simplify the creation of assistant-like experiences, and enable developer access to powerful tools like Code Interpreter, Retrieval and Function Calling. The API now also supports streaming, which means you can build real-time experiences for your UI. " ] }, { @@ -29,7 +29,7 @@ "\n", "The primitives of the **Chat Completions API** are `Messages`, on which you perform a `Completion` with a `Model` (`gpt-3.5-turbo`, `gpt-4`, etc). It is lightweight and powerful, but inherently stateless, which means you have to manage conversation state, tool definitions, retrieval documents, and code execution manually.\n", "\n", - "The primitives of the **Assistants API** are\n", + "In contrast, the primitives of the **Assistants API** are\n", "\n", "- `Assistants`, which encapsulate a base model, instructions, tools, and (context) documents,\n", "- `Threads`, which represent the state of a conversation, and\n", @@ -52,7 +52,7 @@ }, { "cell_type": "code", - "execution_count": 1, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -92,7 +92,7 @@ }, { "cell_type": "code", - "execution_count": 3, + "execution_count": 4, "metadata": {}, "outputs": [], "source": [ @@ -167,14 +167,14 @@ }, { "cell_type": "code", - "execution_count": 4, + "execution_count": 5, "metadata": {}, "outputs": [ { "data": { "text/plain": [ - "{'id': 'asst_9HAjl9y41ufsViNcThW1EXUS',\n", - " 'created_at': 1699828331,\n", + "{'id': 'asst_2Hfaqc71l3fu7pvbd1PkPrpf',\n", + " 'created_at': 1711920054,\n", " 'description': None,\n", " 'file_ids': [],\n", " 'instructions': 'You are a personal math tutor. Answer questions briefly, in a sentence or less.',\n", @@ -234,14 +234,14 @@ }, { "cell_type": "code", - "execution_count": 5, + "execution_count": 6, "metadata": {}, "outputs": [ { "data": { "text/plain": [ - "{'id': 'thread_bw42vPoQtYBMQE84WubNcJXG',\n", - " 'created_at': 1699828331,\n", + "{'id': 'thread_LFIvmULMH7dUmorXIIcCmNEb',\n", + " 'created_at': 1711920081,\n", " 'metadata': {},\n", " 'object': 'thread'}" ] @@ -264,24 +264,28 @@ }, { "cell_type": "code", - "execution_count": 6, + "execution_count": 7, "metadata": {}, "outputs": [ { "data": { "text/plain": [ - "{'id': 'msg_IBiZDAWHhWPewxzN0EfTYNew',\n", + "{'id': 'msg_88FLsE82A89fyYRAYRuuuKWA',\n", " 'assistant_id': None,\n", + " 'completed_at': None,\n", " 'content': [{'text': {'annotations': [],\n", " 'value': 'I need to solve the equation `3x + 11 = 14`. Can you help me?'},\n", " 'type': 'text'}],\n", - " 'created_at': 1699828332,\n", + " 'created_at': 1711920085,\n", " 'file_ids': [],\n", + " 'incomplete_at': None,\n", + " 'incomplete_details': None,\n", " 'metadata': {},\n", " 'object': 'thread.message',\n", " 'role': 'user',\n", " 'run_id': None,\n", - " 'thread_id': 'thread_bw42vPoQtYBMQE84WubNcJXG'}" + " 'status': None,\n", + " 'thread_id': 'thread_LFIvmULMH7dUmorXIIcCmNEb'}" ] }, "metadata": {}, @@ -302,127 +306,94 @@ "metadata": {}, "source": [ "> **Note**\n", - "> Even though you're no longer sending the entire history each time, you will still be charged for the tokens of the entire conversation history with each Run.\n" + "> Even though you're no longer sending the entire history each time, you will still be charged for the tokens of the entire conversation history with each Run.\n", + "\n", + "Notice how the Thread we created is **not** associated with the Assistant we created earlier! Threads exist independently from Assistants, which may be different from what you'd expect if you've used ChatGPT (where a thread is tied to a model/GPT)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "### Runs\n", + "### Streaming Runs\n", "\n", - "Notice how the Thread we created is **not** associated with the Assistant we created earlier! Threads exist independently from Assistants, which may be different from what you'd expect if you've used ChatGPT (where a thread is tied to a model/GPT).\n", - "\n", - "To get a completion from an Assistant for a given Thread, we must create a Run. Creating a Run will indicate to an Assistant it should look at the messages in the Thread and take action: either by adding a single response, or using tools.\n", + "To get a completion from an Assistant for a given Thread, we must create a Run. Creating a Run will indicate to an Assistant that it should look at the messages in the Thread and take action: either by adding a single response, or using tools.\n", "\n", "> **Note**\n", "> Runs are a key difference between the Assistants API and Chat Completions API. While in Chat Completions the model will only ever respond with a single message, in the Assistants API a Run may result in an Assistant using one or multiple tools, and potentially adding multiple messages to the Thread.\n", "\n", - "To get our Assistant to respond to the user, let's create the Run. As mentioned earlier, you must specify _both_ the Assistant and the Thread.\n" + "In order to get the Assistant's response as a stream, we have to do a couple of things when setting up the run. First, we define an EventHandler class that specifies how we want to handle events in the response stream. Second, we use the 'create and stream' helper in the Python SDK to actually create the Run and stream the response. You only need to define EventHandler once, but every time you add a message to a thread and want to run it, you'd use the create_and_stream helper function again. \n", + "\n" ] }, { "cell_type": "code", - "execution_count": 7, + "execution_count": 9, "metadata": {}, "outputs": [ { - "data": { - "text/plain": [ - "{'id': 'run_LA08RjouV3RemQ78UZXuyzv6',\n", - " 'assistant_id': 'asst_9HAjl9y41ufsViNcThW1EXUS',\n", - " 'cancelled_at': None,\n", - " 'completed_at': None,\n", - " 'created_at': 1699828332,\n", - " 'expires_at': 1699828932,\n", - " 'failed_at': None,\n", - " 'file_ids': [],\n", - " 'instructions': 'You are a personal math tutor. Answer questions briefly, in a sentence or less.',\n", - " 'last_error': None,\n", - " 'metadata': {},\n", - " 'model': 'gpt-4-1106-preview',\n", - " 'object': 'thread.run',\n", - " 'required_action': None,\n", - " 'started_at': None,\n", - " 'status': 'queued',\n", - " 'thread_id': 'thread_bw42vPoQtYBMQE84WubNcJXG',\n", - " 'tools': []}" - ] - }, - "metadata": {}, - "output_type": "display_data" + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "assistant > Absolutely, I'd be happy to help you solve the equation `3x + 11 = 14`. The goal here is to isolate x (get x by itself on one side of the equation). Here are the steps to do that:\n", + "\n", + "1. **Subtract 11 from both sides**: You want to move the constant term (11) from the left side to the right side to isolate the variable term (3x). To maintain the equality, whatever you do to one side, you must do to the other.\n", + "\n", + " 3x + 11 - 11 = 14 - 11\n", + "\n", + " This simplifies to:\n", + "\n", + " 3x = 3\n", + "\n", + "2. **Divide both sides by 3**: Now that we have 3x on one side, we need to get x alone. Since x is being multiplied by 3, we can do the opposite operation, which is division by 3, to both sides of the equation.\n", + "\n", + " 3x / 3 = 3 / 3\n", + "\n", + " This simplifies to:\n", + "\n", + " x = 1\n", + "\n", + "You now have the solution to the equation, which is x = 1." + ] } ], "source": [ - "run = client.beta.threads.runs.create(\n", - " thread_id=thread.id,\n", - " assistant_id=assistant.id,\n", - ")\n", - "show_json(run)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Unlike creating a completion in the Chat Completions API, **creating a Run is an asynchronous operation**. It will return immediately with the Run's metadata, which includes a `status` that will initially be set to `queued`. The `status` will be updated as the Assistant performs operations (like using tools and adding messages).\n", + "from typing_extensions import override\n", + "from openai import AssistantEventHandler\n", "\n", - "To know when the Assistant has completed processing, we can poll the Run in a loop. (Support for streaming is coming soon!) While here we are only checking for a `queued` or `in_progress` status, in practice a Run may undergo a [variety of status changes](https://platform.openai.com/docs/api-reference/runs/object#runs/object-status) which you can choose to surface to the user. (These are called Steps, and will be covered later.)\n" - ] - }, - { - "cell_type": "code", - "execution_count": 8, - "metadata": {}, - "outputs": [], - "source": [ - "import time\n", "\n", - "def wait_on_run(run, thread):\n", - " while run.status == \"queued\" or run.status == \"in_progress\":\n", - " run = client.beta.threads.runs.retrieve(\n", - " thread_id=thread.id,\n", - " run_id=run.id,\n", - " )\n", - " time.sleep(0.5)\n", - " return run" - ] - }, - { - "cell_type": "code", - "execution_count": 9, - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "{'id': 'run_LA08RjouV3RemQ78UZXuyzv6',\n", - " 'assistant_id': 'asst_9HAjl9y41ufsViNcThW1EXUS',\n", - " 'cancelled_at': None,\n", - " 'completed_at': 1699828333,\n", - " 'created_at': 1699828332,\n", - " 'expires_at': None,\n", - " 'failed_at': None,\n", - " 'file_ids': [],\n", - " 'instructions': 'You are a personal math tutor. Answer questions briefly, in a sentence or less.',\n", - " 'last_error': None,\n", - " 'metadata': {},\n", - " 'model': 'gpt-4-1106-preview',\n", - " 'object': 'thread.run',\n", - " 'required_action': None,\n", - " 'started_at': 1699828332,\n", - " 'status': 'completed',\n", - " 'thread_id': 'thread_bw42vPoQtYBMQE84WubNcJXG',\n", - " 'tools': []}" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], - "source": [ - "run = wait_on_run(run, thread)\n", - "show_json(run)" + "class EventHandler(AssistantEventHandler): \n", + " @override\n", + " def on_text_created(self, text) -> None:\n", + " print(f\"\\nassistant > \", end=\"\", flush=True)\n", + " \n", + " @override\n", + " def on_text_delta(self, delta, snapshot):\n", + " print(delta.value, end=\"\", flush=True)\n", + " \n", + " def on_tool_call_created(self, tool_call):\n", + " print(f\"\\nassistant > {tool_call.type}\\n\", flush=True)\n", + " \n", + " def on_tool_call_delta(self, delta, snapshot):\n", + " if delta.type == 'code_interpreter':\n", + " if delta.code_interpreter.input:\n", + " print(delta.code_interpreter.input, end=\"\", flush=True)\n", + " if delta.code_interpreter.outputs:\n", + " print(f\"\\n\\noutput >\", flush=True)\n", + " for output in delta.code_interpreter.outputs:\n", + " if output.type == \"logs\":\n", + " print(f\"\\n{output.logs}\", flush=True)\n", + "\n", + "\n", + "\n", + "with client.beta.threads.runs.create_and_stream(\n", + " thread_id=thread.id,\n", + " assistant_id=assistant.id,\n", + " instructions=\"Please be a friendly math tutor and explain all of your steps for solving a question.\",\n", + " event_handler=EventHandler(),\n", + ") as stream:\n", + " stream.until_done()\n" ] }, { @@ -447,33 +418,41 @@ { "data": { "text/plain": [ - "{'data': [{'id': 'msg_S0ZtKIWjyWtbIW9JNUocPdUS',\n", - " 'assistant_id': 'asst_9HAjl9y41ufsViNcThW1EXUS',\n", + "{'data': [{'id': 'msg_43Iu2C9eofqEesCULzGwjEyR',\n", + " 'assistant_id': 'asst_2Hfaqc71l3fu7pvbd1PkPrpf',\n", + " 'completed_at': None,\n", " 'content': [{'text': {'annotations': [],\n", - " 'value': 'Yes. Subtract 11 from both sides to get `3x = 3`, then divide by 3 to find `x = 1`.'},\n", + " 'value': \"Absolutely, I'd be happy to help you solve the equation `3x + 11 = 14`. The goal here is to isolate x (get x by itself on one side of the equation). Here are the steps to do that:\\n\\n1. **Subtract 11 from both sides**: You want to move the constant term (11) from the left side to the right side to isolate the variable term (3x). To maintain the equality, whatever you do to one side, you must do to the other.\\n\\n 3x + 11 - 11 = 14 - 11\\n\\n This simplifies to:\\n\\n 3x = 3\\n\\n2. **Divide both sides by 3**: Now that we have 3x on one side, we need to get x alone. Since x is being multiplied by 3, we can do the opposite operation, which is division by 3, to both sides of the equation.\\n\\n 3x / 3 = 3 / 3\\n\\n This simplifies to:\\n\\n x = 1\\n\\nYou now have the solution to the equation, which is x = 1.\"},\n", " 'type': 'text'}],\n", - " 'created_at': 1699828333,\n", + " 'created_at': 1711921887,\n", " 'file_ids': [],\n", + " 'incomplete_at': None,\n", + " 'incomplete_details': None,\n", " 'metadata': {},\n", " 'object': 'thread.message',\n", " 'role': 'assistant',\n", - " 'run_id': 'run_LA08RjouV3RemQ78UZXuyzv6',\n", - " 'thread_id': 'thread_bw42vPoQtYBMQE84WubNcJXG'},\n", - " {'id': 'msg_IBiZDAWHhWPewxzN0EfTYNew',\n", + " 'run_id': 'run_bEB2mvszofeXTZPJqqAFVLrr',\n", + " 'status': None,\n", + " 'thread_id': 'thread_LFIvmULMH7dUmorXIIcCmNEb'},\n", + " {'id': 'msg_88FLsE82A89fyYRAYRuuuKWA',\n", " 'assistant_id': None,\n", + " 'completed_at': None,\n", " 'content': [{'text': {'annotations': [],\n", " 'value': 'I need to solve the equation `3x + 11 = 14`. Can you help me?'},\n", " 'type': 'text'}],\n", - " 'created_at': 1699828332,\n", + " 'created_at': 1711920085,\n", " 'file_ids': [],\n", + " 'incomplete_at': None,\n", + " 'incomplete_details': None,\n", " 'metadata': {},\n", " 'object': 'thread.message',\n", " 'role': 'user',\n", " 'run_id': None,\n", - " 'thread_id': 'thread_bw42vPoQtYBMQE84WubNcJXG'}],\n", + " 'status': None,\n", + " 'thread_id': 'thread_LFIvmULMH7dUmorXIIcCmNEb'}],\n", " 'object': 'list',\n", - " 'first_id': 'msg_S0ZtKIWjyWtbIW9JNUocPdUS',\n", - " 'last_id': 'msg_IBiZDAWHhWPewxzN0EfTYNew',\n", + " 'first_id': 'msg_43Iu2C9eofqEesCULzGwjEyR',\n", + " 'last_id': 'msg_88FLsE82A89fyYRAYRuuuKWA',\n", " 'has_more': False}" ] }, @@ -505,24 +484,75 @@ "execution_count": 11, "metadata": {}, "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "assistant > Of course, I'll explain each step in more detail.\n", + "\n", + "You're given the equation:\n", + "\n", + "`3x + 11 = 14`\n", + "\n", + "The goal is to solve for x, which means we want to find the value of x that makes this statement true. We need to isolate x on one side of the equation, which means we want to end up with an equation that looks like `x = ` some number.\n", + "\n", + "Here's how we do it, step by step:\n", + "\n", + "### Step 1: Subtract 11 from both sides.\n", + "We start by looking at the `+11` on the left side of the equation. We want to get rid of it so that we only have the term with x (the 3x) on that side. To do this, we do the opposite operation, which is subtraction. We subtract 11 from both sides of the equation:\n", + "\n", + "`3x + 11 - 11 = 14 - 11`\n", + "\n", + "On the left side, by subtracting 11 from 11, we get 0, so they cancel each other out. This leaves us with just `3x` on the left side.\n", + "\n", + "On the right side, subtracting 11 from 14 gives us 3.\n", + "\n", + "After performing the subtraction, the equation looks like this:\n", + "\n", + "`3x = 3`\n", + "\n", + "### Step 2: Divide both sides by 3.\n", + "The next step is to isolate x completely. Right now, x is being multiplied by 3. To undo this multiplication, we divide both sides of the equation by 3, the number that's multiplying x.\n", + "\n", + "`3x / 3 = 3 / 3`\n", + "\n", + "Doing this on the left side, the 3s cancel out because any number divided by itself is 1. What we have left is 1x, which is the same as just x.\n", + "\n", + "`x = 1`\n", + "\n", + "On the right side, 3 divided by 3 also equals 1.\n", + "\n", + "So, the simplification of the division step gives us the final answer:\n", + "\n", + "`x = 1`\n", + "\n", + "### Conclusion\n", + "We've now solved for x by performing two operations: subtracting 11 from both sides to get rid of the constant term and then dividing both sides by 3 to get rid of the coefficient of x. This gives us the solution for the equation, which is `x = 1`." + ] + }, { "data": { "text/plain": [ - "{'data': [{'id': 'msg_9MAeOrGriHcImeQnAzvYyJbs',\n", - " 'assistant_id': 'asst_9HAjl9y41ufsViNcThW1EXUS',\n", + "{'data': [{'id': 'msg_gFvDVfgTgGvQeGcipEGr4AkE',\n", + " 'assistant_id': 'asst_2Hfaqc71l3fu7pvbd1PkPrpf',\n", + " 'completed_at': None,\n", " 'content': [{'text': {'annotations': [],\n", - " 'value': 'Certainly. To solve for x in the equation `3x + 11 = 14`:\\n\\n1. Subtract 11 from both sides: `3x + 11 - 11 = 14 - 11` simplifies to `3x = 3`.\\n2. Divide both sides by 3: `3x / 3 = 3 / 3` simplifies to `x = 1`.\\n\\nSo, the solution is `x = 1`.'},\n", + " 'value': \"Of course, I'll explain each step in more detail.\\n\\nYou're given the equation:\\n\\n`3x + 11 = 14`\\n\\nThe goal is to solve for x, which means we want to find the value of x that makes this statement true. We need to isolate x on one side of the equation, which means we want to end up with an equation that looks like `x = ` some number.\\n\\nHere's how we do it, step by step:\\n\\n### Step 1: Subtract 11 from both sides.\\nWe start by looking at the `+11` on the left side of the equation. We want to get rid of it so that we only have the term with x (the 3x) on that side. To do this, we do the opposite operation, which is subtraction. We subtract 11 from both sides of the equation:\\n\\n`3x + 11 - 11 = 14 - 11`\\n\\nOn the left side, by subtracting 11 from 11, we get 0, so they cancel each other out. This leaves us with just `3x` on the left side.\\n\\nOn the right side, subtracting 11 from 14 gives us 3.\\n\\nAfter performing the subtraction, the equation looks like this:\\n\\n`3x = 3`\\n\\n### Step 2: Divide both sides by 3.\\nThe next step is to isolate x completely. Right now, x is being multiplied by 3. To undo this multiplication, we divide both sides of the equation by 3, the number that's multiplying x.\\n\\n`3x / 3 = 3 / 3`\\n\\nDoing this on the left side, the 3s cancel out because any number divided by itself is 1. What we have left is 1x, which is the same as just x.\\n\\n`x = 1`\\n\\nOn the right side, 3 divided by 3 also equals 1.\\n\\nSo, the simplification of the division step gives us the final answer:\\n\\n`x = 1`\\n\\n### Conclusion\\nWe've now solved for x by performing two operations: subtracting 11 from both sides to get rid of the constant term and then dividing both sides by 3 to get rid of the coefficient of x. This gives us the solution for the equation, which is `x = 1`.\"},\n", " 'type': 'text'}],\n", - " 'created_at': 1699828335,\n", + " 'created_at': 1711922159,\n", " 'file_ids': [],\n", + " 'incomplete_at': None,\n", + " 'incomplete_details': None,\n", " 'metadata': {},\n", " 'object': 'thread.message',\n", " 'role': 'assistant',\n", - " 'run_id': 'run_IFHfsubkJv7RSUbDZpNVs4PG',\n", - " 'thread_id': 'thread_bw42vPoQtYBMQE84WubNcJXG'}],\n", + " 'run_id': 'run_c6XJAi9f0so3Tz3eHFpZsAkV',\n", + " 'status': None,\n", + " 'thread_id': 'thread_LFIvmULMH7dUmorXIIcCmNEb'}],\n", " 'object': 'list',\n", - " 'first_id': 'msg_9MAeOrGriHcImeQnAzvYyJbs',\n", - " 'last_id': 'msg_9MAeOrGriHcImeQnAzvYyJbs',\n", + " 'first_id': 'msg_gFvDVfgTgGvQeGcipEGr4AkE',\n", + " 'last_id': 'msg_gFvDVfgTgGvQeGcipEGr4AkE',\n", " 'has_more': False}" ] }, @@ -536,14 +566,15 @@ " thread_id=thread.id, role=\"user\", content=\"Could you explain this to me?\"\n", ")\n", "\n", - "# Execute our run\n", - "run = client.beta.threads.runs.create(\n", - " thread_id=thread.id,\n", - " assistant_id=assistant.id,\n", - ")\n", + "# Execute our run again to process the new message\n", + "with client.beta.threads.runs.create_and_stream(\n", + " thread_id=thread.id,\n", + " assistant_id=assistant.id,\n", + " instructions=\"Please be a friendly math tutor and explain all of your steps for solving a question.\",\n", + " event_handler=EventHandler(),\n", + ") as stream:\n", + " stream.until_done()\n", "\n", - "# Wait for completion\n", - "wait_on_run(run, thread)\n", "\n", "# Retrieve all the messages added after our last user message\n", "messages = client.beta.threads.messages.list(\n", @@ -574,30 +605,61 @@ "\n", "Since we've already created our Math Assistant, I've saved its ID in `MATH_ASSISTANT_ID`. I then defined two functions:\n", "\n", - "- `submit_message`: create a Message on a Thread, then start (and return) a new Run\n", + "- `submit_message`: create a Message on a Thread, then start and return a new Run\n", "- `get_response`: returns the list of Messages in a Thread\n" ] }, { "cell_type": "code", - "execution_count": 12, + "execution_count": 53, "metadata": {}, "outputs": [], "source": [ "from openai import OpenAI\n", + "from openai import AssistantEventHandler\n", + "from typing_extensions import override\n", + "import os \n", "\n", "MATH_ASSISTANT_ID = assistant.id # or a hard-coded ID like \"asst-...\"\n", "\n", "client = OpenAI(api_key=os.environ.get(\"OPENAI_API_KEY\", \"\"))\n", "\n", + "class EventHandler(AssistantEventHandler): \n", + " @override\n", + " def on_text_created(self, text) -> None:\n", + " print(f\"\\nassistant > \", end=\"\", flush=True)\n", + " \n", + " @override\n", + " def on_text_delta(self, delta, snapshot):\n", + " print(delta.value, end=\"\", flush=True)\n", + " \n", + " def on_tool_call_created(self, tool_call):\n", + " print(f\"\\nassistant > {tool_call.type}\\n\", flush=True)\n", + " \n", + " def on_tool_call_delta(self, delta, snapshot):\n", + " if delta.type == 'code_interpreter':\n", + " if delta.code_interpreter.input:\n", + " print(delta.code_interpreter.input, end=\"\", flush=True)\n", + " if delta.code_interpreter.outputs:\n", + " print(f\"\\n\\noutput >\", flush=True)\n", + " for output in delta.code_interpreter.outputs:\n", + " if output.type == \"logs\":\n", + " print(f\"\\n{output.logs}\", flush=True)\n", + "\n", "def submit_message(assistant_id, thread, user_message):\n", + " # Submit the initial message to the thread\n", " client.beta.threads.messages.create(\n", " thread_id=thread.id, role=\"user\", content=user_message\n", " )\n", - " return client.beta.threads.runs.create(\n", + "\n", + " # Create and stream the run with the given instructions\n", + " with client.beta.threads.runs.create_and_stream(\n", " thread_id=thread.id,\n", " assistant_id=assistant_id,\n", - " )\n", + " instructions=\"Please be a friendly math tutor.\",\n", + " event_handler=EventHandler(),\n", + " ) as stream:\n", + " stream.until_done()\n", "\n", "\n", "def get_response(thread):\n", @@ -615,101 +677,66 @@ }, { "cell_type": "code", - "execution_count": 13, - "metadata": {}, - "outputs": [], - "source": [ - "def create_thread_and_run(user_input):\n", - " thread = client.beta.threads.create()\n", - " run = submit_message(MATH_ASSISTANT_ID, thread, user_input)\n", - " return thread, run\n", - "\n", - "\n", - "# Emulating concurrent user requests\n", - "thread1, run1 = create_thread_and_run(\n", - " \"I need to solve the equation `3x + 11 = 14`. Can you help me?\"\n", - ")\n", - "thread2, run2 = create_thread_and_run(\"Could you explain linear algebra to me?\")\n", - "thread3, run3 = create_thread_and_run(\"I don't like math. What can I do?\")\n", - "\n", - "# Now all Runs are executing..." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Once all Runs are going, we can wait on each and get the responses.\n" - ] - }, - { - "cell_type": "code", - "execution_count": 14, + "execution_count": 54, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ - "# Messages\n", - "user: I need to solve the equation `3x + 11 = 14`. Can you help me?\n", - "assistant: Yes, subtract 11 from both sides to get `3x = 3`, then divide both sides by 3 to find `x = 1`.\n", "\n", - "# Messages\n", - "user: Could you explain linear algebra to me?\n", - "assistant: Linear algebra is the branch of mathematics that deals with vector spaces, linear equations, and matrices, focusing on the properties and operations that can be applied to vectors and linear transformations.\n", + "assistant > code_interpreter\n", "\n", - "# Messages\n", - "user: I don't like math. What can I do?\n", - "assistant: Try finding aspects of math that relate to your interests or daily life, and consider a tutor or interactive resources to make learning more enjoyable.\n", + "from sympy import symbols, Eq, solve\n", "\n", - "# Messages\n", - "user: I don't like math. What can I do?\n", - "assistant: Try finding aspects of math that relate to your interests or daily life, and consider a tutor or interactive resources to make learning more enjoyable.\n", - "user: Thank you!\n", - "assistant: You're welcome! If you have any more questions, feel free to ask.\n", - "\n" + "# Define the variable and equation\n", + "x = symbols('x')\n", + "equation = Eq(3*x + 11, 14)\n", + "\n", + "# Solving the equation\n", + "solution = solve(equation, x)\n", + "solution\n", + "\n", + "output >\n", + "\n", + "[1]\n", + "\n", + "assistant > Certainly! To solve the equation \\(3x + 11 = 14\\), you'll need to isolate the variable \\(x\\) on one side of the equation.\n", + "\n", + "Here's how we can do it step by step:\n", + "1. Subtract 11 from both sides to start.\n", + "\\[ 3x + 11 - 11 = 14 - 11 \\]\n", + "\\[ 3x = 3 \\]\n", + "\n", + "2. Now, divide both sides by 3 to solve for \\(x\\).\n", + "\\[ \\frac{3x}{3} = \\frac{3}{3} \\]\n", + "\\[ x = 1 \\]\n", + "\n", + "So the solution to the equation is \\(x = 1\\).\n", + "assistant > Sure, let's create a more challenging problem that involves multiple steps and possibly fractions or decimals. Here is an example:\n", + "\n", + "Solve for \\( x \\) in the equation:\n", + "\n", + "\\[ 4x - \\frac{5}{2} = \\frac{3x}{4} + 9.5 \\]\n", + "\n", + "This problem involves fractions and requires multiple steps to solve, including finding a common denominator and moving terms with \\( x \\) to one side of the equation. Would you like to try solving this problem, or do you need further assistance with it?" ] } ], "source": [ - "import time\n", - "\n", - "# Pretty printing helper\n", - "def pretty_print(messages):\n", - " print(\"# Messages\")\n", - " for m in messages:\n", - " print(f\"{m.role}: {m.content[0].text.value}\")\n", - " print()\n", - "\n", - "\n", - "# Waiting in a loop\n", - "def wait_on_run(run, thread):\n", - " while run.status == \"queued\" or run.status == \"in_progress\":\n", - " run = client.beta.threads.runs.retrieve(\n", - " thread_id=thread.id,\n", - " run_id=run.id,\n", - " )\n", - " time.sleep(0.5)\n", - " return run\n", - "\n", - "\n", - "# Wait for Run 1\n", - "run1 = wait_on_run(run1, thread1)\n", - "pretty_print(get_response(thread1))\n", + "def create_thread_and_run(user_input):\n", + " thread = client.beta.threads.create()\n", + " run = submit_message(MATH_ASSISTANT_ID, thread, user_input)\n", + " return thread, run\n", "\n", - "# Wait for Run 2\n", - "run2 = wait_on_run(run2, thread2)\n", - "pretty_print(get_response(thread2))\n", "\n", - "# Wait for Run 3\n", - "run3 = wait_on_run(run3, thread3)\n", - "pretty_print(get_response(thread3))\n", + "# Emulating concurrent user requests\n", + "thread1, run1 = create_thread_and_run(\n", + " \"I need to solve the equation `3x + 11 = 14`. Can you help me?\"\n", + ")\n", + "thread2, run2 = create_thread_and_run(\"Can you generate another math problem like `3x + 11 = 14` for me? Make it difficult.\")\n", "\n", - "# Thank our assistant on Thread 3 :)\n", - "run4 = submit_message(MATH_ASSISTANT_ID, thread3, \"Thank you!\")\n", - "run4 = wait_on_run(run4, thread3)\n", - "pretty_print(get_response(thread3))" + "# Now all Runs are executing..." ] }, { @@ -750,14 +777,14 @@ }, { "cell_type": "code", - "execution_count": 15, + "execution_count": 36, "metadata": {}, "outputs": [ { "data": { "text/plain": [ - "{'id': 'asst_9HAjl9y41ufsViNcThW1EXUS',\n", - " 'created_at': 1699828331,\n", + "{'id': 'asst_2Hfaqc71l3fu7pvbd1PkPrpf',\n", + " 'created_at': 1711920054,\n", " 'description': None,\n", " 'file_ids': [],\n", " 'instructions': 'You are a personal math tutor. Answer questions briefly, in a sentence or less.',\n", @@ -789,26 +816,60 @@ }, { "cell_type": "code", - "execution_count": 16, + "execution_count": 40, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ - "# Messages\n", - "user: Generate the first 20 fibbonaci numbers with code.\n", - "assistant: The first 20 Fibonacci numbers are: 0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, 233, 377, 610, 987, 1597, 2584, and 4181.\n", - "\n" + "\n", + "assistant > code_interpreter\n", + "\n", + "# Function to generate the first n Fibonacci numbers\n", + "def generate_fibonacci(n):\n", + " fib_sequence = [0, 1]\n", + " while len(fib_sequence) < n:\n", + " fib_sequence.append(fib_sequence[-1] + fib_sequence[-2])\n", + " return fib_sequence[:n]\n", + "\n", + "# Generate the first 20 Fibonacci numbers\n", + "first_20_fib_numbers = generate_fibonacci(20)\n", + "first_20_fib_numbers\n", + "\n", + "output >\n", + "\n", + "[0,\n", + " 1,\n", + " 1,\n", + " 2,\n", + " 3,\n", + " 5,\n", + " 8,\n", + " 13,\n", + " 21,\n", + " 34,\n", + " 55,\n", + " 89,\n", + " 144,\n", + " 233,\n", + " 377,\n", + " 610,\n", + " 987,\n", + " 1597,\n", + " 2584,\n", + " 4181]\n", + "\n", + "assistant > The first 20 numbers in the Fibonacci sequence are:\n", + "\n", + "0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, 233, 377, 610, 987, 1597, 2584, 4181." ] } ], "source": [ "thread, run = create_thread_and_run(\n", " \"Generate the first 20 fibbonaci numbers with code.\"\n", - ")\n", - "run = wait_on_run(run, thread)\n", - "pretty_print(get_response(thread))" + ")\n" ] }, { @@ -831,13 +892,24 @@ }, { "cell_type": "code", - "execution_count": 17, + "execution_count": 42, "metadata": {}, - "outputs": [], + "outputs": [ + { + "ename": "AttributeError", + "evalue": "'NoneType' object has no attribute 'id'", + "output_type": "error", + "traceback": [ + "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", + "\u001b[0;31mAttributeError\u001b[0m Traceback (most recent call last)", + "Cell \u001b[0;32mIn[42], line 2\u001b[0m\n\u001b[1;32m 1\u001b[0m run_steps \u001b[38;5;241m=\u001b[39m client\u001b[38;5;241m.\u001b[39mbeta\u001b[38;5;241m.\u001b[39mthreads\u001b[38;5;241m.\u001b[39mruns\u001b[38;5;241m.\u001b[39msteps\u001b[38;5;241m.\u001b[39mlist(thread_id\u001b[38;5;241m=\u001b[39mthread\u001b[38;5;241m.\u001b[39mid, \n\u001b[0;32m----> 2\u001b[0m run_id\u001b[38;5;241m=\u001b[39m\u001b[43mrun\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mid\u001b[49m, order\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124masc\u001b[39m\u001b[38;5;124m\"\u001b[39m)\n", + "\u001b[0;31mAttributeError\u001b[0m: 'NoneType' object has no attribute 'id'" + ] + } + ], "source": [ - "run_steps = client.beta.threads.runs.steps.list(\n", - " thread_id=thread.id, run_id=run.id, order=\"asc\"\n", - ")" + "run_steps = client.beta.threads.runs.steps.list(thread_id=thread.id, \n", + "run_id=run.id, order=\"asc\")" ] }, { @@ -932,16 +1004,16 @@ }, { "cell_type": "code", - "execution_count": 19, + "execution_count": 43, "metadata": {}, "outputs": [ { "data": { "text/plain": [ - "{'id': 'asst_9HAjl9y41ufsViNcThW1EXUS',\n", - " 'created_at': 1699828331,\n", + "{'id': 'asst_2Hfaqc71l3fu7pvbd1PkPrpf',\n", + " 'created_at': 1711920054,\n", " 'description': None,\n", - " 'file_ids': ['file-MdXcQI8OdPp76wukWI4dpLwW'],\n", + " 'file_ids': ['file-wGSa82c3EhDR2onbkT8GxVSR'],\n", " 'instructions': 'You are a personal math tutor. Answer questions briefly, in a sentence or less.',\n", " 'metadata': {},\n", " 'model': 'gpt-4-1106-preview',\n", @@ -974,27 +1046,25 @@ }, { "cell_type": "code", - "execution_count": 20, + "execution_count": 44, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ - "# Messages\n", - "user: What are some cool math concepts behind this ML paper pdf? Explain in two sentences.\n", - "assistant: I am unable to find specific sections referring to \"cool math concepts\" directly in the paper using the available tools. I will now read the beginning of the paper to identify any mathematical concepts that are fundamental to the paper.\n", - "assistant: The paper discusses leveraging large language models as a framework for unsupervised multitask learning, where tasks are implicitly defined by the context within sequences of text. It explores the zero-shot learning capabilities of such models by showing that when a language model is trained on a vast dataset, it begins to generalize and perform tasks without explicit supervision, achieving competitive results across various natural language processing tasks using a probabilistic framework based on sequential modeling and conditional probabilities.\n", - "\n" + "\n", + "assistant > retrieval\n", + "\n", + "\n", + "assistant > The ML paper discusses the use of a Transformer-based architecture leveraging a predictive model capable of generalizing from unordered tokens to structured patterns in data . It also mentions the Children's Book Test to evaluate the performance of language models on different categories of words, which provides insight into the mathematical modeling of language understanding and prediction of contextually relevant tokens ." ] } ], "source": [ "thread, run = create_thread_and_run(\n", " \"What are some cool math concepts behind this ML paper pdf? Explain in two sentences.\"\n", - ")\n", - "run = wait_on_run(run, thread)\n", - "pretty_print(get_response(thread))" + ")\n" ] }, { @@ -1028,7 +1098,7 @@ }, { "cell_type": "code", - "execution_count": 21, + "execution_count": 45, "metadata": {}, "outputs": [], "source": [ @@ -1074,7 +1144,7 @@ }, { "cell_type": "code", - "execution_count": 22, + "execution_count": 46, "metadata": {}, "outputs": [ { @@ -1119,7 +1189,7 @@ }, { "cell_type": "code", - "execution_count": 23, + "execution_count": 47, "metadata": {}, "outputs": [], "source": [ @@ -1176,16 +1246,16 @@ }, { "cell_type": "code", - "execution_count": 24, + "execution_count": 48, "metadata": {}, "outputs": [ { "data": { "text/plain": [ - "{'id': 'asst_9HAjl9y41ufsViNcThW1EXUS',\n", - " 'created_at': 1699828331,\n", + "{'id': 'asst_2Hfaqc71l3fu7pvbd1PkPrpf',\n", + " 'created_at': 1711920054,\n", " 'description': None,\n", - " 'file_ids': ['file-MdXcQI8OdPp76wukWI4dpLwW'],\n", + " 'file_ids': ['file-wGSa82c3EhDR2onbkT8GxVSR'],\n", " 'instructions': 'You are a personal math tutor. Answer questions briefly, in a sentence or less.',\n", " 'metadata': {},\n", " 'model': 'gpt-4-1106-preview',\n", @@ -1194,6 +1264,7 @@ " 'tools': [{'type': 'code_interpreter'},\n", " {'type': 'retrieval'},\n", " {'function': {'name': 'display_quiz',\n", + " 'description': \"Displays a quiz to the student, and returns the student's response. A single quiz can have multiple questions.\",\n", " 'parameters': {'type': 'object',\n", " 'properties': {'title': {'type': 'string'},\n", " 'questions': {'type': 'array',\n", @@ -1204,8 +1275,7 @@ " 'enum': ['MULTIPLE_CHOICE', 'FREE_RESPONSE']},\n", " 'choices': {'type': 'array', 'items': {'type': 'string'}}},\n", " 'required': ['question_text']}}},\n", - " 'required': ['title', 'questions']},\n", - " 'description': \"Displays a quiz to the student, and returns the student's response. A single quiz can have multiple questions.\"},\n", + " 'required': ['title', 'questions']}},\n", " 'type': 'function'}]}" ] }, @@ -1234,26 +1304,39 @@ }, { "cell_type": "code", - "execution_count": 25, + "execution_count": 49, "metadata": {}, "outputs": [ { - "data": { - "text/plain": [ - "'requires_action'" - ] - }, - "execution_count": 25, - "metadata": {}, - "output_type": "execute_result" + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "assistant > retrieval\n", + "\n", + "\n", + "assistant > From the content of the provided document 'Language Models are Unsupervised Multitask Learners', we can formulate a quiz with one open-ended question and one multiple-choice question. \n", + "\n", + "Here are the drafted quiz questions:\n", + "\n", + "**Open-Ended Question:**\n", + "What is the primary goal of advancing toward more general systems in the field of machine learning, according to the article?\n", + "\n", + "**Multiple-Choice Question:**\n", + "Which one of the following tasks does the language model GPT-2 NOT achieve state of the art results in, according to a zero-shot setting in the document?\n", + "- A) Reading Comprehension\n", + "- B) Translation\n", + "- C) Summarization\n", + "- D) Language Modeling\n", + "\n", + "Let's use the quiz tool to present these questions to the student." + ] } ], "source": [ "thread, run = create_thread_and_run(\n", " \"Make a quiz with 2 questions: One open ended, one multiple choice. Then, give me feedback for the responses.\"\n", - ")\n", - "run = wait_on_run(run, thread)\n", - "run.status" + ")\n" ] }, { @@ -1265,53 +1348,20 @@ }, { "cell_type": "code", - "execution_count": 26, + "execution_count": 50, "metadata": {}, "outputs": [ { - "data": { - "text/plain": [ - "{'id': 'run_98PGE3qGtHoaWaCLoytyRUBf',\n", - " 'assistant_id': 'asst_9HAjl9y41ufsViNcThW1EXUS',\n", - " 'cancelled_at': None,\n", - " 'completed_at': None,\n", - " 'created_at': 1699828370,\n", - " 'expires_at': 1699828970,\n", - " 'failed_at': None,\n", - " 'file_ids': ['file-MdXcQI8OdPp76wukWI4dpLwW'],\n", - " 'instructions': 'You are a personal math tutor. Answer questions briefly, in a sentence or less.',\n", - " 'last_error': None,\n", - " 'metadata': {},\n", - " 'model': 'gpt-4-1106-preview',\n", - " 'object': 'thread.run',\n", - " 'required_action': {'submit_tool_outputs': {'tool_calls': [{'id': 'call_Zf650sWT1wW4Uwbf5YeDS0VG',\n", - " 'function': {'arguments': '{\\n \"title\": \"Mathematics Quiz\",\\n \"questions\": [\\n {\\n \"question_text\": \"Explain why the square root of a negative number is not a real number.\",\\n \"question_type\": \"FREE_RESPONSE\"\\n },\\n {\\n \"question_text\": \"What is the value of an angle in a regular pentagon?\",\\n \"choices\": [\\n \"72 degrees\",\\n \"90 degrees\",\\n \"108 degrees\",\\n \"120 degrees\"\\n ],\\n \"question_type\": \"MULTIPLE_CHOICE\"\\n }\\n ]\\n}',\n", - " 'name': 'display_quiz'},\n", - " 'type': 'function'}]},\n", - " 'type': 'submit_tool_outputs'},\n", - " 'started_at': 1699828370,\n", - " 'status': 'requires_action',\n", - " 'thread_id': 'thread_bICTESFvWoRdj0O0SzsosLCS',\n", - " 'tools': [{'type': 'code_interpreter'},\n", - " {'type': 'retrieval'},\n", - " {'function': {'name': 'display_quiz',\n", - " 'parameters': {'type': 'object',\n", - " 'properties': {'title': {'type': 'string'},\n", - " 'questions': {'type': 'array',\n", - " 'description': 'An array of questions, each with a title and potentially options (if multiple choice).',\n", - " 'items': {'type': 'object',\n", - " 'properties': {'question_text': {'type': 'string'},\n", - " 'question_type': {'type': 'string',\n", - " 'enum': ['MULTIPLE_CHOICE', 'FREE_RESPONSE']},\n", - " 'choices': {'type': 'array', 'items': {'type': 'string'}}},\n", - " 'required': ['question_text']}}},\n", - " 'required': ['title', 'questions']},\n", - " 'description': \"Displays a quiz to the student, and returns the student's response. A single quiz can have multiple questions.\"},\n", - " 'type': 'function'}]}" - ] - }, - "metadata": {}, - "output_type": "display_data" + "ename": "AttributeError", + "evalue": "'NoneType' object has no attribute 'model_dump_json'", + "output_type": "error", + "traceback": [ + "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", + "\u001b[0;31mAttributeError\u001b[0m Traceback (most recent call last)", + "Cell \u001b[0;32mIn[50], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m \u001b[43mshow_json\u001b[49m\u001b[43m(\u001b[49m\u001b[43mrun\u001b[49m\u001b[43m)\u001b[49m\n", + "Cell \u001b[0;32mIn[4], line 4\u001b[0m, in \u001b[0;36mshow_json\u001b[0;34m(obj)\u001b[0m\n\u001b[1;32m 3\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21mshow_json\u001b[39m(obj):\n\u001b[0;32m----> 4\u001b[0m display(json\u001b[38;5;241m.\u001b[39mloads(\u001b[43mobj\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mmodel_dump_json\u001b[49m()))\n", + "\u001b[0;31mAttributeError\u001b[0m: 'NoneType' object has no attribute 'model_dump_json'" + ] } ], "source": [ @@ -1330,31 +1380,19 @@ }, { "cell_type": "code", - "execution_count": 27, + "execution_count": 51, "metadata": {}, "outputs": [ { - "name": "stdout", - "output_type": "stream", - "text": [ - "Function Name: display_quiz\n", - "Function Arguments:\n" + "ename": "AttributeError", + "evalue": "'NoneType' object has no attribute 'required_action'", + "output_type": "error", + "traceback": [ + "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", + "\u001b[0;31mAttributeError\u001b[0m Traceback (most recent call last)", + "Cell \u001b[0;32mIn[51], line 2\u001b[0m\n\u001b[1;32m 1\u001b[0m \u001b[38;5;66;03m# Extract single tool call\u001b[39;00m\n\u001b[0;32m----> 2\u001b[0m tool_call \u001b[38;5;241m=\u001b[39m \u001b[43mrun\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mrequired_action\u001b[49m\u001b[38;5;241m.\u001b[39msubmit_tool_outputs\u001b[38;5;241m.\u001b[39mtool_calls[\u001b[38;5;241m0\u001b[39m]\n\u001b[1;32m 3\u001b[0m name \u001b[38;5;241m=\u001b[39m tool_call\u001b[38;5;241m.\u001b[39mfunction\u001b[38;5;241m.\u001b[39mname\n\u001b[1;32m 4\u001b[0m arguments \u001b[38;5;241m=\u001b[39m json\u001b[38;5;241m.\u001b[39mloads(tool_call\u001b[38;5;241m.\u001b[39mfunction\u001b[38;5;241m.\u001b[39marguments)\n", + "\u001b[0;31mAttributeError\u001b[0m: 'NoneType' object has no attribute 'required_action'" ] - }, - { - "data": { - "text/plain": [ - "{'title': 'Mathematics Quiz',\n", - " 'questions': [{'question_text': 'Explain why the square root of a negative number is not a real number.',\n", - " 'question_type': 'FREE_RESPONSE'},\n", - " {'question_text': 'What is the value of an angle in a regular pentagon?',\n", - " 'choices': ['72 degrees', '90 degrees', '108 degrees', '120 degrees'],\n", - " 'question_type': 'MULTIPLE_CHOICE'}]}" - ] - }, - "execution_count": 27, - "metadata": {}, - "output_type": "execute_result" } ], "source": [ @@ -1377,24 +1415,18 @@ }, { "cell_type": "code", - "execution_count": 28, + "execution_count": 52, "metadata": {}, "outputs": [ { - "name": "stdout", - "output_type": "stream", - "text": [ - "Quiz: Mathematics Quiz\n", - "\n", - "Explain why the square root of a negative number is not a real number.\n", - "\n", - "What is the value of an angle in a regular pentagon?\n", - "0. 72 degrees\n", - "1. 90 degrees\n", - "2. 108 degrees\n", - "3. 120 degrees\n", - "\n", - "Responses: [\"I don't know.\", 'a']\n" + "ename": "NameError", + "evalue": "name 'arguments' is not defined", + "output_type": "error", + "traceback": [ + "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", + "\u001b[0;31mNameError\u001b[0m Traceback (most recent call last)", + "Cell \u001b[0;32mIn[52], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m responses \u001b[38;5;241m=\u001b[39m display_quiz(\u001b[43marguments\u001b[49m[\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mtitle\u001b[39m\u001b[38;5;124m\"\u001b[39m], arguments[\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mquestions\u001b[39m\u001b[38;5;124m\"\u001b[39m])\n\u001b[1;32m 2\u001b[0m \u001b[38;5;28mprint\u001b[39m(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mResponses:\u001b[39m\u001b[38;5;124m\"\u001b[39m, responses)\n", + "\u001b[0;31mNameError\u001b[0m: name 'arguments' is not defined" ] } ], @@ -1548,7 +1580,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.9.13" + "version": "3.11.8" } }, "nbformat": 4,