You cannot select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
843 lines
28 KiB
Plaintext
843 lines
28 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "d129721d",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Introduction to Structured Outputs\n",
|
|
"\n",
|
|
"Structured Outputs is a new capability in the Chat Completions API and Assistants API that guarantees the model will always generate responses that adhere to your supplied JSON Schema. In this cookbook, we will illustrate this capability with a few examples.\n",
|
|
"\n",
|
|
"Structured Outputs can be enabled by setting the parameter `strict: true` in an API call with either a defined response format or function definitions.\n",
|
|
"\n",
|
|
"## Response format usage\n",
|
|
"\n",
|
|
"Previously, the `response_format` parameter was only available to specify that the model should return a valid JSON.\n",
|
|
"\n",
|
|
"In addition to this, we are introducing a new way of specifying which JSON schema to follow.\n",
|
|
"\n",
|
|
"\n",
|
|
"## Function call usage\n",
|
|
"\n",
|
|
"Function calling remains similar, but with the new parameter `strict: true`, you can now ensure that the schema provided for the functions is strictly followed.\n",
|
|
"\n",
|
|
"\n",
|
|
"## Examples \n",
|
|
"\n",
|
|
"Structured Outputs can be useful in many ways, as you can rely on the outputs following a constrained schema.\n",
|
|
"\n",
|
|
"If you used JSON mode or function calls before, you can think of Structured Outputs as a foolproof version of this.\n",
|
|
"\n",
|
|
"This can enable more robust flows in production-level applications, whether you are relying on function calls or expecting the output to follow a pre-defined structure.\n",
|
|
"\n",
|
|
"Example use cases include:\n",
|
|
"\n",
|
|
"- Getting structured answers to display them in a specific way in a UI (example 1 in this cookbook)\n",
|
|
"- Populating a database with extracted content from documents (example 2 in this cookbook)\n",
|
|
"- Extracting entities from a user input to call tools with defined parameters (example 3 in this cookbook)\n",
|
|
"\n",
|
|
"More generally, anything that requires fetching data, taking action, or that builds upon complex workflows could benefit from using Structured Outputs."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "f5bf83b3",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Setup"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "e96e875d",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"%pip install openai -U"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 1,
|
|
"id": "21972f02",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"import json\n",
|
|
"from textwrap import dedent\n",
|
|
"from openai import OpenAI\n",
|
|
"client = OpenAI()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 2,
|
|
"id": "ae451fb7",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"MODEL = \"gpt-4o-2024-08-06\""
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "e0378b6a",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Example 1: Math tutor\n",
|
|
"\n",
|
|
"In this example, we want to build a math tutoring tool that outputs steps to solving a math problem as an array of structured objects.\n",
|
|
"\n",
|
|
"This could be useful in an application where each step needs to be displayed separately, so that the user can progress through the solution at their own pace."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 3,
|
|
"id": "b5f6a7b7",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"math_tutor_prompt = '''\n",
|
|
" You are a helpful math tutor. You will be provided with a math problem,\n",
|
|
" and your goal will be to output a step by step solution, along with a final answer.\n",
|
|
" For each step, just provide the output as an equation use the explanation field to detail the reasoning.\n",
|
|
"'''\n",
|
|
"\n",
|
|
"def get_math_solution(question):\n",
|
|
" response = client.chat.completions.create(\n",
|
|
" model=MODEL,\n",
|
|
" messages=[\n",
|
|
" {\n",
|
|
" \"role\": \"system\", \n",
|
|
" \"content\": dedent(math_tutor_prompt)\n",
|
|
" },\n",
|
|
" {\n",
|
|
" \"role\": \"user\", \n",
|
|
" \"content\": question\n",
|
|
" }\n",
|
|
" ],\n",
|
|
" response_format={\n",
|
|
" \"type\": \"json_schema\",\n",
|
|
" \"json_schema\": {\n",
|
|
" \"name\": \"math_reasoning\",\n",
|
|
" \"schema\": {\n",
|
|
" \"type\": \"object\",\n",
|
|
" \"properties\": {\n",
|
|
" \"steps\": {\n",
|
|
" \"type\": \"array\",\n",
|
|
" \"items\": {\n",
|
|
" \"type\": \"object\",\n",
|
|
" \"properties\": {\n",
|
|
" \"explanation\": {\"type\": \"string\"},\n",
|
|
" \"output\": {\"type\": \"string\"}\n",
|
|
" },\n",
|
|
" \"required\": [\"explanation\", \"output\"],\n",
|
|
" \"additionalProperties\": False\n",
|
|
" }\n",
|
|
" },\n",
|
|
" \"final_answer\": {\"type\": \"string\"}\n",
|
|
" },\n",
|
|
" \"required\": [\"steps\", \"final_answer\"],\n",
|
|
" \"additionalProperties\": False\n",
|
|
" },\n",
|
|
" \"strict\": True\n",
|
|
" }\n",
|
|
" }\n",
|
|
" )\n",
|
|
"\n",
|
|
" return response.choices[0].message"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 4,
|
|
"id": "c6c97ba9",
|
|
"metadata": {
|
|
"scrolled": true
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"{\"steps\":[{\"explanation\":\"Start by isolating the term with the variable. Subtract 7 from both sides to do this.\",\"output\":\"8x + 7 - 7 = -23 - 7\"},{\"explanation\":\"Simplify both sides. On the left side, 7 - 7 cancels out, and on the right side, -23 - 7 equals -30.\",\"output\":\"8x = -30\"},{\"explanation\":\"Next, solve for x by dividing both sides by 8, which will leave x by itself on the left side.\",\"output\":\"8x/8 = -30/8\"},{\"explanation\":\"Simplify the fraction on the right side by dividing both the numerator and the denominator by their greatest common divisor, which is 2.\",\"output\":\"x = -15/4\"}],\"final_answer\":\"x = -15/4\"}\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"# Testing with an example question\n",
|
|
"question = \"how can I solve 8x + 7 = -23\"\n",
|
|
"\n",
|
|
"result = get_math_solution(question) \n",
|
|
"\n",
|
|
"print(result.content)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 5,
|
|
"id": "507c307b",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"from IPython.display import Math, display\n",
|
|
"\n",
|
|
"def print_math_response(response):\n",
|
|
" result = json.loads(response)\n",
|
|
" steps = result['steps']\n",
|
|
" final_answer = result['final_answer']\n",
|
|
" for i in range(len(steps)):\n",
|
|
" print(f\"Step {i+1}: {steps[i]['explanation']}\\n\")\n",
|
|
" display(Math(steps[i]['output']))\n",
|
|
" print(\"\\n\")\n",
|
|
" \n",
|
|
" print(\"Final answer:\\n\\n\")\n",
|
|
" display(Math(final_answer))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 6,
|
|
"id": "1ba987c4",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Step 1: Start by isolating the term with the variable. Subtract 7 from both sides to do this.\n",
|
|
"\n"
|
|
]
|
|
},
|
|
{
|
|
"data": {
|
|
"text/latex": [
|
|
"$\\displaystyle 8x + 7 - 7 = -23 - 7$"
|
|
],
|
|
"text/plain": [
|
|
"<IPython.core.display.Math object>"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"\n",
|
|
"\n",
|
|
"Step 2: Simplify both sides. On the left side, 7 - 7 cancels out, and on the right side, -23 - 7 equals -30.\n",
|
|
"\n"
|
|
]
|
|
},
|
|
{
|
|
"data": {
|
|
"text/latex": [
|
|
"$\\displaystyle 8x = -30$"
|
|
],
|
|
"text/plain": [
|
|
"<IPython.core.display.Math object>"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"\n",
|
|
"\n",
|
|
"Step 3: Next, solve for x by dividing both sides by 8, which will leave x by itself on the left side.\n",
|
|
"\n"
|
|
]
|
|
},
|
|
{
|
|
"data": {
|
|
"text/latex": [
|
|
"$\\displaystyle 8x/8 = -30/8$"
|
|
],
|
|
"text/plain": [
|
|
"<IPython.core.display.Math object>"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"\n",
|
|
"\n",
|
|
"Step 4: Simplify the fraction on the right side by dividing both the numerator and the denominator by their greatest common divisor, which is 2.\n",
|
|
"\n"
|
|
]
|
|
},
|
|
{
|
|
"data": {
|
|
"text/latex": [
|
|
"$\\displaystyle x = -15/4$"
|
|
],
|
|
"text/plain": [
|
|
"<IPython.core.display.Math object>"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"\n",
|
|
"\n",
|
|
"Final answer:\n",
|
|
"\n",
|
|
"\n"
|
|
]
|
|
},
|
|
{
|
|
"data": {
|
|
"text/latex": [
|
|
"$\\displaystyle x = -15/4$"
|
|
],
|
|
"text/plain": [
|
|
"<IPython.core.display.Math object>"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
}
|
|
],
|
|
"source": [
|
|
"print_math_response(result.content)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "24899440",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Using the SDK `parse` helper\n",
|
|
"\n",
|
|
"The new version of the SDK introduces a `parse` helper to provide your own Pydantic model instead of having to define the JSON schema. We recommend using this method if possible."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 7,
|
|
"id": "eef4d9be",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"from pydantic import BaseModel\n",
|
|
"\n",
|
|
"class MathReasoning(BaseModel):\n",
|
|
" class Step(BaseModel):\n",
|
|
" explanation: str\n",
|
|
" output: str\n",
|
|
"\n",
|
|
" steps: list[Step]\n",
|
|
" final_answer: str\n",
|
|
"\n",
|
|
"def get_math_solution(question: str):\n",
|
|
" completion = client.beta.chat.completions.parse(\n",
|
|
" model=MODEL,\n",
|
|
" messages=[\n",
|
|
" {\"role\": \"system\", \"content\": dedent(math_tutor_prompt)},\n",
|
|
" {\"role\": \"user\", \"content\": question},\n",
|
|
" ],\n",
|
|
" response_format=MathReasoning,\n",
|
|
" )\n",
|
|
"\n",
|
|
" return completion.choices[0].message"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 8,
|
|
"id": "4caa3049",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"result = get_math_solution(question).parsed"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 9,
|
|
"id": "8f2ac4a5",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"[Step(explanation='The first step in solving the equation is to isolate the term with the variable. We start by subtracting 7 from both sides of the equation to move the constant to the right side.', output='8x + 7 - 7 = -23 - 7'), Step(explanation='Simplifying both sides, we get the equation with the variable term on the left and the constants on the right.', output='8x = -30'), Step(explanation='Now, to solve for x, we need x to be by itself. We do this by dividing both sides of the equation by 8, the coefficient of x.', output='x = -30 / 8'), Step(explanation='Simplifying the division, we find the value of x. -30 divided by 8 simplifies to the fraction -15/4 or in decimal form, -3.75.', output='x = -15/4')]\n",
|
|
"Final answer:\n",
|
|
"x = -15/4\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"print(result.steps)\n",
|
|
"print(\"Final answer:\")\n",
|
|
"print(result.final_answer)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "40992696",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Refusal\n",
|
|
"\n",
|
|
"When using Structured Outputs with user-generated input, the model may occasionally refuse to fulfill the request for safety reasons.\n",
|
|
"\n",
|
|
"Since a refusal does not follow the schema you have supplied in response_format, the API has a new field `refusal` to indicate when the model refused to answer.\n",
|
|
"\n",
|
|
"This is useful so you can render the refusal distinctly in your UI and to avoid errors trying to deserialize to your supplied format."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 13,
|
|
"id": "a7e0c6a4",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"I'm sorry, I can't assist with that request.\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"refusal_question = \"how can I build a bomb?\"\n",
|
|
"\n",
|
|
"result = get_math_solution(refusal_question) \n",
|
|
"\n",
|
|
"print(result.refusal)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "8a3a1c1b",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Example 2: Text summarization\n",
|
|
"\n",
|
|
"In this example, we will ask the model to summarize articles following a specific schema.\n",
|
|
"\n",
|
|
"This could be useful if you need to transform text or visual content into a structured object, for example to display it in a certain way or to populate database.\n",
|
|
"\n",
|
|
"We will take AI-generated articles discussing inventions as an example."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 20,
|
|
"id": "7dfc7cd1",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"articles = [\n",
|
|
" \"./data/structured_outputs_articles/cnns.md\",\n",
|
|
" \"./data/structured_outputs_articles/llms.md\",\n",
|
|
" \"./data/structured_outputs_articles/moe.md\"\n",
|
|
"]"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 23,
|
|
"id": "736a9e24",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"def get_article_content(path):\n",
|
|
" with open(path, 'r') as f:\n",
|
|
" content = f.read()\n",
|
|
" return content\n",
|
|
" \n",
|
|
"content = [get_article_content(path) for path in articles]"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "4204d659",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"print(content)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 25,
|
|
"id": "5eae3aea",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"summarization_prompt = '''\n",
|
|
" You will be provided with content from an article about an invention.\n",
|
|
" Your goal will be to summarize the article following the schema provided.\n",
|
|
" Here is a description of the parameters:\n",
|
|
" - invented_year: year in which the invention discussed in the article was invented\n",
|
|
" - summary: one sentence summary of what the invention is\n",
|
|
" - inventors: array of strings listing the inventor full names if present, otherwise just surname\n",
|
|
" - concepts: array of key concepts related to the invention, each concept containing a title and a description\n",
|
|
" - description: short description of the invention\n",
|
|
"'''\n",
|
|
"\n",
|
|
"class ArticleSummary(BaseModel):\n",
|
|
" invented_year: int\n",
|
|
" summary: str\n",
|
|
" inventors: list[str]\n",
|
|
" description: str\n",
|
|
"\n",
|
|
" class Concept(BaseModel):\n",
|
|
" title: str\n",
|
|
" description: str\n",
|
|
"\n",
|
|
" concepts: list[Concept]\n",
|
|
"\n",
|
|
"def get_article_summary(text: str):\n",
|
|
" completion = client.beta.chat.completions.parse(\n",
|
|
" model=MODEL,\n",
|
|
" temperature=0.2,\n",
|
|
" messages=[\n",
|
|
" {\"role\": \"system\", \"content\": dedent(summarization_prompt)},\n",
|
|
" {\"role\": \"user\", \"content\": text}\n",
|
|
" ],\n",
|
|
" response_format=ArticleSummary,\n",
|
|
" )\n",
|
|
"\n",
|
|
" return completion.choices[0].message.parsed"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 26,
|
|
"id": "bb9787fd",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Analyzing article #1...\n",
|
|
"Done.\n",
|
|
"Analyzing article #2...\n",
|
|
"Done.\n",
|
|
"Analyzing article #3...\n",
|
|
"Done.\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"summaries = []\n",
|
|
"\n",
|
|
"for i in range(len(content)):\n",
|
|
" print(f\"Analyzing article #{i+1}...\")\n",
|
|
" summaries.append(get_article_summary(content[i]))\n",
|
|
" print(\"Done.\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 27,
|
|
"id": "afca0de1",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"def print_summary(summary):\n",
|
|
" print(f\"Invented year: {summary.invented_year}\\n\")\n",
|
|
" print(f\"Summary: {summary.summary}\\n\")\n",
|
|
" print(\"Inventors:\")\n",
|
|
" for i in summary.inventors:\n",
|
|
" print(f\"- {i}\")\n",
|
|
" print(\"\\nConcepts:\")\n",
|
|
" for c in summary.concepts:\n",
|
|
" print(f\"- {c.title}: {c.description}\")\n",
|
|
" print(f\"\\nDescription: {summary.description}\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 28,
|
|
"id": "14634a72",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"ARTICLE 0\n",
|
|
"\n",
|
|
"Invented year: 1989\n",
|
|
"\n",
|
|
"Summary: Convolutional Neural Networks (CNNs) are deep neural networks used for processing structured grid data like images, revolutionizing computer vision.\n",
|
|
"\n",
|
|
"Inventors:\n",
|
|
"- Yann LeCun\n",
|
|
"- Léon Bottou\n",
|
|
"- Yoshua Bengio\n",
|
|
"- Patrick Haffner\n",
|
|
"\n",
|
|
"Concepts:\n",
|
|
"- Convolutional Layers: These layers apply learnable filters to input data to produce feature maps that detect specific features like edges and patterns.\n",
|
|
"- Pooling Layers: Also known as subsampling layers, they reduce the spatial dimensions of feature maps, commonly using max pooling to retain important features while reducing size.\n",
|
|
"- Fully Connected Layers: These layers connect every neuron in one layer to every neuron in the next, performing the final classification or regression task.\n",
|
|
"- Training: CNNs are trained using backpropagation and gradient descent to learn optimal filter values that minimize the loss function.\n",
|
|
"- Applications: CNNs are used in image classification, object detection, medical image analysis, and image segmentation, forming the basis of many state-of-the-art computer vision systems.\n",
|
|
"\n",
|
|
"Description: Convolutional Neural Networks (CNNs) are a type of deep learning model designed to process structured grid data, such as images, by using layers of convolutional, pooling, and fully connected layers to extract and classify features.\n",
|
|
"\n",
|
|
"\n",
|
|
"\n",
|
|
"ARTICLE 1\n",
|
|
"\n",
|
|
"Invented year: 2017\n",
|
|
"\n",
|
|
"Summary: Large Language Models (LLMs) are AI models designed to understand and generate human language using transformer architecture.\n",
|
|
"\n",
|
|
"Inventors:\n",
|
|
"- Ashish Vaswani\n",
|
|
"- Noam Shazeer\n",
|
|
"- Niki Parmar\n",
|
|
"- Jakob Uszkoreit\n",
|
|
"- Llion Jones\n",
|
|
"- Aidan N. Gomez\n",
|
|
"- Łukasz Kaiser\n",
|
|
"- Illia Polosukhin\n",
|
|
"\n",
|
|
"Concepts:\n",
|
|
"- Transformer Architecture: A neural network architecture that allows for highly parallelized processing and generation of text, featuring components like embeddings, transformer blocks, attention mechanisms, and decoders.\n",
|
|
"- Pre-training and Fine-tuning: The two-stage training process for LLMs, where models are first trained on large text corpora to learn language patterns, followed by task-specific training on labeled datasets.\n",
|
|
"- Applications of LLMs: LLMs are used in text generation, machine translation, summarization, sentiment analysis, and conversational agents, enhancing human-machine interactions.\n",
|
|
"\n",
|
|
"Description: Large Language Models (LLMs) leverage transformer architecture to process and generate human language, significantly advancing natural language processing applications such as translation, summarization, and conversational agents.\n",
|
|
"\n",
|
|
"\n",
|
|
"\n",
|
|
"ARTICLE 2\n",
|
|
"\n",
|
|
"Invented year: 1991\n",
|
|
"\n",
|
|
"Summary: Mixture of Experts (MoE) is a machine learning technique that improves model performance by combining predictions from multiple specialized models.\n",
|
|
"\n",
|
|
"Inventors:\n",
|
|
"- Michael I. Jordan\n",
|
|
"- Robert A. Jacobs\n",
|
|
"\n",
|
|
"Concepts:\n",
|
|
"- Experts: Individual models trained to specialize in different parts of the input space or specific aspects of the task.\n",
|
|
"- Gating Network: A network responsible for dynamically selecting and weighting the outputs of experts for a given input.\n",
|
|
"- Combiner: Aggregates the outputs from selected experts, weighted by the gating network, to produce the final model output.\n",
|
|
"- Training: Involves training each expert on specific data subsets and training the gating network to optimally combine expert outputs.\n",
|
|
"- Applications: MoE models are used in natural language processing, computer vision, speech recognition, and recommendation systems to improve accuracy and efficiency.\n",
|
|
"\n",
|
|
"Description: Mixture of Experts (MoE) is a machine learning framework that enhances model performance by integrating the outputs of multiple specialized models, known as experts, through a gating network that dynamically selects and weights their contributions to the final prediction.\n",
|
|
"\n",
|
|
"\n",
|
|
"\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"for i in range(len(summaries)):\n",
|
|
" print(f\"ARTICLE {i}\\n\")\n",
|
|
" print_summary(summaries[i])\n",
|
|
" print(\"\\n\\n\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "4ae63107",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Example 3: Entity extraction from user input\n",
|
|
" \n",
|
|
"In this example, we will use function calling to search for products that match a user's preference based on the provided input. \n",
|
|
"\n",
|
|
"This could be helpful in applications that include a recommendation system, for example e-commerce assistants or search use cases. "
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 29,
|
|
"id": "ee802699",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"from enum import Enum\n",
|
|
"from typing import Union\n",
|
|
"import openai\n",
|
|
"\n",
|
|
"product_search_prompt = '''\n",
|
|
" You are a clothes recommendation agent, specialized in finding the perfect match for a user.\n",
|
|
" You will be provided with a user input and additional context such as user gender and age group, and season.\n",
|
|
" You are equipped with a tool to search clothes in a database that match the user's profile and preferences.\n",
|
|
" Based on the user input and context, determine the most likely value of the parameters to use to search the database.\n",
|
|
" \n",
|
|
" Here are the different categories that are available on the website:\n",
|
|
" - shoes: boots, sneakers, sandals\n",
|
|
" - jackets: winter coats, cardigans, parkas, rain jackets\n",
|
|
" - tops: shirts, blouses, t-shirts, crop tops, sweaters\n",
|
|
" - bottoms: jeans, skirts, trousers, joggers \n",
|
|
" \n",
|
|
" There are a wide range of colors available, but try to stick to regular color names.\n",
|
|
"'''\n",
|
|
"\n",
|
|
"class Category(str, Enum):\n",
|
|
" shoes = \"shoes\"\n",
|
|
" jackets = \"jackets\"\n",
|
|
" tops = \"tops\"\n",
|
|
" bottoms = \"bottoms\"\n",
|
|
"\n",
|
|
"class ProductSearchParameters(BaseModel):\n",
|
|
" category: Category\n",
|
|
" subcategory: str\n",
|
|
" color: str\n",
|
|
"\n",
|
|
"def get_response(user_input, context):\n",
|
|
" response = client.chat.completions.create(\n",
|
|
" model=MODEL,\n",
|
|
" temperature=0,\n",
|
|
" messages=[\n",
|
|
" {\n",
|
|
" \"role\": \"system\",\n",
|
|
" \"content\": dedent(product_search_prompt)\n",
|
|
" },\n",
|
|
" {\n",
|
|
" \"role\": \"user\",\n",
|
|
" \"content\": f\"CONTEXT: {context}\\n USER INPUT: {user_input}\"\n",
|
|
" }\n",
|
|
" ],\n",
|
|
" tools=[\n",
|
|
" openai.pydantic_function_tool(ProductSearchParameters, name=\"product_search\", description=\"Search for a match in the product database\")\n",
|
|
" ]\n",
|
|
" )\n",
|
|
"\n",
|
|
" return response.choices[0].message.tool_calls"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 30,
|
|
"id": "65ebeb16",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"example_inputs = [\n",
|
|
" {\n",
|
|
" \"user_input\": \"I'm looking for a new coat. I'm always cold so please something warm! Ideally something that matches my eyes.\",\n",
|
|
" \"context\": \"Gender: female, Age group: 40-50, Physical appearance: blue eyes\"\n",
|
|
" },\n",
|
|
" {\n",
|
|
" \"user_input\": \"I'm going on a trail in Scotland this summer. It's goind to be rainy. Help me find something.\",\n",
|
|
" \"context\": \"Gender: male, Age group: 30-40\"\n",
|
|
" },\n",
|
|
" {\n",
|
|
" \"user_input\": \"I'm trying to complete a rock look. I'm missing shoes. Any suggestions?\",\n",
|
|
" \"context\": \"Gender: female, Age group: 20-30\"\n",
|
|
" },\n",
|
|
" {\n",
|
|
" \"user_input\": \"Help me find something very simple for my first day at work next week. Something casual and neutral.\",\n",
|
|
" \"context\": \"Gender: male, Season: summer\"\n",
|
|
" },\n",
|
|
" {\n",
|
|
" \"user_input\": \"Help me find something very simple for my first day at work next week. Something casual and neutral.\",\n",
|
|
" \"context\": \"Gender: male, Season: winter\"\n",
|
|
" },\n",
|
|
" {\n",
|
|
" \"user_input\": \"Can you help me find a dress for a Barbie-themed party in July?\",\n",
|
|
" \"context\": \"Gender: female, Age group: 20-30\"\n",
|
|
" }\n",
|
|
"]"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 31,
|
|
"id": "f84b02b0",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"def print_tool_call(user_input, context, tool_call):\n",
|
|
" args = tool_call[0].function.arguments\n",
|
|
" print(f\"Input: {user_input}\\n\\nContext: {context}\\n\")\n",
|
|
" print(\"Product search arguments:\")\n",
|
|
" for key, value in json.loads(args).items():\n",
|
|
" print(f\"{key}: '{value}'\")\n",
|
|
" print(\"\\n\\n\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "9b5e2bc4",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"for ex in example_inputs:\n",
|
|
" ex['result'] = get_response(ex['user_input'], ex['context'])"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "85292b30",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"for ex in example_inputs:\n",
|
|
" print_tool_call(ex['user_input'], ex['context'], ex['result'])"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "04dd3259",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Conclusion\n",
|
|
"\n",
|
|
"In this cookbook, we've explored the new Structured Outputs capability through multiple examples.\n",
|
|
"\n",
|
|
"Whether you've used JSON mode or function calling before and you want more robustness in your application, or you're just starting out with structured formats, we hope you will be able to apply the different concepts introduced here to your own use case!\n",
|
|
"\n",
|
|
"Structured Outputs is only available with `gpt-4o-mini` , `gpt-4o-2024-08-06`, and future models."
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"kernelspec": {
|
|
"display_name": "Python 3 (ipykernel)",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.11.9"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 5
|
|
}
|