mirror of
https://github.com/openai/openai-cookbook
synced 2024-11-04 06:00:33 +00:00
705 lines
31 KiB
Plaintext
705 lines
31 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "7bcdad0e-b67c-4927-b00e-3a4d950cd8ce",
|
|
"metadata": {},
|
|
"source": [
|
|
"# How to fine-tune chat models\n",
|
|
"\n",
|
|
"This notebook provides a step-by-step guide for our new `gpt-3.5-turbo` fine-tuning. We'll perform entity extraction using the [RecipeNLG dataset](https://github.com/Glorf/recipenlg), which provides various recipes and a list of extracted generic ingredients for each. This is a common dataset for named entity recognition (NER) tasks.\n",
|
|
"\n",
|
|
"We will go through the following steps:\n",
|
|
"\n",
|
|
"1. **Setup:** Loading our dataset and filtering down to one domain to fine-tune on.\n",
|
|
"2. **Data preparation:** Preparing your data for fine-tuning by creating training and validation examples, and uploading them to the `Files` endpoint.\n",
|
|
"3. **Fine-tuning:** Creating your fine-tuned model.\n",
|
|
"4. **Inference:** Using your fine-tuned model for inference on new inputs.\n",
|
|
"\n",
|
|
"By the end of this you should be able to train, evaluate and deploy a fine-tuned `gpt-3.5-turbo` model.\n",
|
|
"\n",
|
|
"For more information on fine-tuning, you can refer to our [documentation guide](https://platform.openai.com/docs/guides/fine-tuning), [API reference](https://platform.openai.com/docs/api-reference/fine-tuning) or [blog post](https://openai.com/blog/gpt-3-5-turbo-fine-tuning-and-api-updates)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "6f49cb10-f895-41f4-aa97-da606d0084d4",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Setup"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "32036e70",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"# make sure to use the latest version of the openai python package\n",
|
|
"!pip install --upgrade openai "
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 12,
|
|
"id": "6e1f4403-37e1-4115-a215-12fd7daa1eb6",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"import json\n",
|
|
"import openai\n",
|
|
"import os\n",
|
|
"import pandas as pd\n",
|
|
"from pprint import pprint\n",
|
|
"\n",
|
|
"client = openai.OpenAI(api_key=os.environ.get(\"OPENAI_API_KEY\", \"<your OpenAI API key if not set as env var>\"))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "a468d660",
|
|
"metadata": {},
|
|
"source": [
|
|
"\n",
|
|
"Fine-tuning works best when focused on a particular domain. It's important to make sure your dataset is both focused enough for the model to learn, but general enough that unseen examples won't be missed. Having this in mind, we have extracted a subset from the RecipesNLG dataset to only contain documents from www.cookbooks.com."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 3,
|
|
"id": "f57ebc23-14b7-47f9-90b8-1d791ccfc9bc",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/html": [
|
|
"<div>\n",
|
|
"<style scoped>\n",
|
|
" .dataframe tbody tr th:only-of-type {\n",
|
|
" vertical-align: middle;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .dataframe tbody tr th {\n",
|
|
" vertical-align: top;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .dataframe thead th {\n",
|
|
" text-align: right;\n",
|
|
" }\n",
|
|
"</style>\n",
|
|
"<table border=\"1\" class=\"dataframe\">\n",
|
|
" <thead>\n",
|
|
" <tr style=\"text-align: right;\">\n",
|
|
" <th></th>\n",
|
|
" <th>title</th>\n",
|
|
" <th>ingredients</th>\n",
|
|
" <th>directions</th>\n",
|
|
" <th>link</th>\n",
|
|
" <th>source</th>\n",
|
|
" <th>NER</th>\n",
|
|
" </tr>\n",
|
|
" </thead>\n",
|
|
" <tbody>\n",
|
|
" <tr>\n",
|
|
" <th>0</th>\n",
|
|
" <td>No-Bake Nut Cookies</td>\n",
|
|
" <td>[\"1 c. firmly packed brown sugar\", \"1/2 c. eva...</td>\n",
|
|
" <td>[\"In a heavy 2-quart saucepan, mix brown sugar...</td>\n",
|
|
" <td>www.cookbooks.com/Recipe-Details.aspx?id=44874</td>\n",
|
|
" <td>www.cookbooks.com</td>\n",
|
|
" <td>[\"brown sugar\", \"milk\", \"vanilla\", \"nuts\", \"bu...</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>1</th>\n",
|
|
" <td>Jewell Ball'S Chicken</td>\n",
|
|
" <td>[\"1 small jar chipped beef, cut up\", \"4 boned ...</td>\n",
|
|
" <td>[\"Place chipped beef on bottom of baking dish....</td>\n",
|
|
" <td>www.cookbooks.com/Recipe-Details.aspx?id=699419</td>\n",
|
|
" <td>www.cookbooks.com</td>\n",
|
|
" <td>[\"beef\", \"chicken breasts\", \"cream of mushroom...</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>2</th>\n",
|
|
" <td>Creamy Corn</td>\n",
|
|
" <td>[\"2 (16 oz.) pkg. frozen corn\", \"1 (8 oz.) pkg...</td>\n",
|
|
" <td>[\"In a slow cooker, combine all ingredients. C...</td>\n",
|
|
" <td>www.cookbooks.com/Recipe-Details.aspx?id=10570</td>\n",
|
|
" <td>www.cookbooks.com</td>\n",
|
|
" <td>[\"frozen corn\", \"cream cheese\", \"butter\", \"gar...</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>3</th>\n",
|
|
" <td>Chicken Funny</td>\n",
|
|
" <td>[\"1 large whole chicken\", \"2 (10 1/2 oz.) cans...</td>\n",
|
|
" <td>[\"Boil and debone chicken.\", \"Put bite size pi...</td>\n",
|
|
" <td>www.cookbooks.com/Recipe-Details.aspx?id=897570</td>\n",
|
|
" <td>www.cookbooks.com</td>\n",
|
|
" <td>[\"chicken\", \"chicken gravy\", \"cream of mushroo...</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>4</th>\n",
|
|
" <td>Reeses Cups(Candy)</td>\n",
|
|
" <td>[\"1 c. peanut butter\", \"3/4 c. graham cracker ...</td>\n",
|
|
" <td>[\"Combine first four ingredients and press in ...</td>\n",
|
|
" <td>www.cookbooks.com/Recipe-Details.aspx?id=659239</td>\n",
|
|
" <td>www.cookbooks.com</td>\n",
|
|
" <td>[\"peanut butter\", \"graham cracker crumbs\", \"bu...</td>\n",
|
|
" </tr>\n",
|
|
" </tbody>\n",
|
|
"</table>\n",
|
|
"</div>"
|
|
],
|
|
"text/plain": [
|
|
" title ingredients \n",
|
|
"0 No-Bake Nut Cookies [\"1 c. firmly packed brown sugar\", \"1/2 c. eva... \\\n",
|
|
"1 Jewell Ball'S Chicken [\"1 small jar chipped beef, cut up\", \"4 boned ... \n",
|
|
"2 Creamy Corn [\"2 (16 oz.) pkg. frozen corn\", \"1 (8 oz.) pkg... \n",
|
|
"3 Chicken Funny [\"1 large whole chicken\", \"2 (10 1/2 oz.) cans... \n",
|
|
"4 Reeses Cups(Candy) [\"1 c. peanut butter\", \"3/4 c. graham cracker ... \n",
|
|
"\n",
|
|
" directions \n",
|
|
"0 [\"In a heavy 2-quart saucepan, mix brown sugar... \\\n",
|
|
"1 [\"Place chipped beef on bottom of baking dish.... \n",
|
|
"2 [\"In a slow cooker, combine all ingredients. C... \n",
|
|
"3 [\"Boil and debone chicken.\", \"Put bite size pi... \n",
|
|
"4 [\"Combine first four ingredients and press in ... \n",
|
|
"\n",
|
|
" link source \n",
|
|
"0 www.cookbooks.com/Recipe-Details.aspx?id=44874 www.cookbooks.com \\\n",
|
|
"1 www.cookbooks.com/Recipe-Details.aspx?id=699419 www.cookbooks.com \n",
|
|
"2 www.cookbooks.com/Recipe-Details.aspx?id=10570 www.cookbooks.com \n",
|
|
"3 www.cookbooks.com/Recipe-Details.aspx?id=897570 www.cookbooks.com \n",
|
|
"4 www.cookbooks.com/Recipe-Details.aspx?id=659239 www.cookbooks.com \n",
|
|
"\n",
|
|
" NER \n",
|
|
"0 [\"brown sugar\", \"milk\", \"vanilla\", \"nuts\", \"bu... \n",
|
|
"1 [\"beef\", \"chicken breasts\", \"cream of mushroom... \n",
|
|
"2 [\"frozen corn\", \"cream cheese\", \"butter\", \"gar... \n",
|
|
"3 [\"chicken\", \"chicken gravy\", \"cream of mushroo... \n",
|
|
"4 [\"peanut butter\", \"graham cracker crumbs\", \"bu... "
|
|
]
|
|
},
|
|
"execution_count": 3,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"# Read in the dataset we'll use for this task.\n",
|
|
"# This will be the RecipesNLG dataset, which we've cleaned to only contain documents from www.cookbooks.com\n",
|
|
"recipe_df = pd.read_csv(\"data/cookbook_recipes_nlg_10k.csv\")\n",
|
|
"\n",
|
|
"recipe_df.head()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "2b3151e9-8715-47bd-a153-195d6a0d0a70",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Data preparation\n",
|
|
"\n",
|
|
"We'll begin by preparing our data. When fine-tuning with the `ChatCompletion` format, each training example is a simple list of `messages`. For example, an entry could look like:\n",
|
|
"\n",
|
|
"```\n",
|
|
"[{'role': 'system',\n",
|
|
" 'content': 'You are a helpful recipe assistant. You are to extract the generic ingredients from each of the recipes provided.'},\n",
|
|
"\n",
|
|
" {'role': 'user',\n",
|
|
" 'content': 'Title: No-Bake Nut Cookies\\n\\nIngredients: [\"1 c. firmly packed brown sugar\", \"1/2 c. evaporated milk\", \"1/2 tsp. vanilla\", \"1/2 c. broken nuts (pecans)\", \"2 Tbsp. butter or margarine\", \"3 1/2 c. bite size shredded rice biscuits\"]\\n\\nGeneric ingredients: '},\n",
|
|
"\n",
|
|
" {'role': 'assistant',\n",
|
|
" 'content': '[\"brown sugar\", \"milk\", \"vanilla\", \"nuts\", \"butter\", \"bite size shredded rice biscuits\"]'}]\n",
|
|
"```\n",
|
|
"\n",
|
|
"During the training process this conversation will be split, with the final entry being the `completion` that the model will produce, and the remainder of the `messages` acting as the prompt. Consider this when building your training examples - if your model will act on multi-turn conversations, then please provide representative examples so it doesn't perform poorly when the conversation starts to expand.\n",
|
|
"\n",
|
|
"Please note that currently there is a 4096 token limit for each training example. Anything longer than this will be truncated at 4096 tokens.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 4,
|
|
"id": "9a8216b0-d1dc-472d-b07d-1be03acd70a5",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"{'messages': [{'content': 'You are a helpful recipe assistant. You are to '\n",
|
|
" 'extract the generic ingredients from each of the '\n",
|
|
" 'recipes provided.',\n",
|
|
" 'role': 'system'},\n",
|
|
" {'content': 'Title: No-Bake Nut Cookies\\n'\n",
|
|
" '\\n'\n",
|
|
" 'Ingredients: [\"1 c. firmly packed brown sugar\", '\n",
|
|
" '\"1/2 c. evaporated milk\", \"1/2 tsp. vanilla\", \"1/2 '\n",
|
|
" 'c. broken nuts (pecans)\", \"2 Tbsp. butter or '\n",
|
|
" 'margarine\", \"3 1/2 c. bite size shredded rice '\n",
|
|
" 'biscuits\"]\\n'\n",
|
|
" '\\n'\n",
|
|
" 'Generic ingredients: ',\n",
|
|
" 'role': 'user'},\n",
|
|
" {'content': '[\"brown sugar\", \"milk\", \"vanilla\", \"nuts\", '\n",
|
|
" '\"butter\", \"bite size shredded rice biscuits\"]',\n",
|
|
" 'role': 'assistant'}]}\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"training_data = []\n",
|
|
"\n",
|
|
"system_message = \"You are a helpful recipe assistant. You are to extract the generic ingredients from each of the recipes provided.\"\n",
|
|
"\n",
|
|
"def create_user_message(row):\n",
|
|
" return f\"\"\"Title: {row['title']}\\n\\nIngredients: {row['ingredients']}\\n\\nGeneric ingredients: \"\"\"\n",
|
|
"\n",
|
|
"def prepare_example_conversation(row):\n",
|
|
" messages = []\n",
|
|
" messages.append({\"role\": \"system\", \"content\": system_message})\n",
|
|
"\n",
|
|
" user_message = create_user_message(row)\n",
|
|
" messages.append({\"role\": \"user\", \"content\": user_message})\n",
|
|
"\n",
|
|
" messages.append({\"role\": \"assistant\", \"content\": row[\"NER\"]})\n",
|
|
"\n",
|
|
" return {\"messages\": messages}\n",
|
|
"\n",
|
|
"pprint(prepare_example_conversation(recipe_df.iloc[0]))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "82fa4fae",
|
|
"metadata": {},
|
|
"source": [
|
|
"Let's now do this for a subset of the dataset to use as our training data. You can begin with even 30-50 well-pruned examples. You should see performance continue to scale linearly as you increase the size of the training set, but your jobs will also take longer."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 5,
|
|
"id": "8f37aff9",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"{'messages': [{'role': 'system', 'content': 'You are a helpful recipe assistant. You are to extract the generic ingredients from each of the recipes provided.'}, {'role': 'user', 'content': 'Title: No-Bake Nut Cookies\\n\\nIngredients: [\"1 c. firmly packed brown sugar\", \"1/2 c. evaporated milk\", \"1/2 tsp. vanilla\", \"1/2 c. broken nuts (pecans)\", \"2 Tbsp. butter or margarine\", \"3 1/2 c. bite size shredded rice biscuits\"]\\n\\nGeneric ingredients: '}, {'role': 'assistant', 'content': '[\"brown sugar\", \"milk\", \"vanilla\", \"nuts\", \"butter\", \"bite size shredded rice biscuits\"]'}]}\n",
|
|
"{'messages': [{'role': 'system', 'content': 'You are a helpful recipe assistant. You are to extract the generic ingredients from each of the recipes provided.'}, {'role': 'user', 'content': 'Title: Jewell Ball\\'S Chicken\\n\\nIngredients: [\"1 small jar chipped beef, cut up\", \"4 boned chicken breasts\", \"1 can cream of mushroom soup\", \"1 carton sour cream\"]\\n\\nGeneric ingredients: '}, {'role': 'assistant', 'content': '[\"beef\", \"chicken breasts\", \"cream of mushroom soup\", \"sour cream\"]'}]}\n",
|
|
"{'messages': [{'role': 'system', 'content': 'You are a helpful recipe assistant. You are to extract the generic ingredients from each of the recipes provided.'}, {'role': 'user', 'content': 'Title: Creamy Corn\\n\\nIngredients: [\"2 (16 oz.) pkg. frozen corn\", \"1 (8 oz.) pkg. cream cheese, cubed\", \"1/3 c. butter, cubed\", \"1/2 tsp. garlic powder\", \"1/2 tsp. salt\", \"1/4 tsp. pepper\"]\\n\\nGeneric ingredients: '}, {'role': 'assistant', 'content': '[\"frozen corn\", \"cream cheese\", \"butter\", \"garlic powder\", \"salt\", \"pepper\"]'}]}\n",
|
|
"{'messages': [{'role': 'system', 'content': 'You are a helpful recipe assistant. You are to extract the generic ingredients from each of the recipes provided.'}, {'role': 'user', 'content': 'Title: Chicken Funny\\n\\nIngredients: [\"1 large whole chicken\", \"2 (10 1/2 oz.) cans chicken gravy\", \"1 (10 1/2 oz.) can cream of mushroom soup\", \"1 (6 oz.) box Stove Top stuffing\", \"4 oz. shredded cheese\"]\\n\\nGeneric ingredients: '}, {'role': 'assistant', 'content': '[\"chicken\", \"chicken gravy\", \"cream of mushroom soup\", \"shredded cheese\"]'}]}\n",
|
|
"{'messages': [{'role': 'system', 'content': 'You are a helpful recipe assistant. You are to extract the generic ingredients from each of the recipes provided.'}, {'role': 'user', 'content': 'Title: Reeses Cups(Candy) \\n\\nIngredients: [\"1 c. peanut butter\", \"3/4 c. graham cracker crumbs\", \"1 c. melted butter\", \"1 lb. (3 1/2 c.) powdered sugar\", \"1 large pkg. chocolate chips\"]\\n\\nGeneric ingredients: '}, {'role': 'assistant', 'content': '[\"peanut butter\", \"graham cracker crumbs\", \"butter\", \"powdered sugar\", \"chocolate chips\"]'}]}\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"# use the first 100 rows of the dataset for training\n",
|
|
"training_df = recipe_df.loc[0:100]\n",
|
|
"\n",
|
|
"# apply the prepare_example_conversation function to each row of the training_df\n",
|
|
"training_data = training_df.apply(prepare_example_conversation, axis=1).tolist()\n",
|
|
"\n",
|
|
"for example in training_data[:5]:\n",
|
|
" print(example)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "6fb0f6a7",
|
|
"metadata": {},
|
|
"source": [
|
|
"In addition to training data, we can also **optionally** provide validation data, which will be used to make sure that the model does not overfit your training set."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 6,
|
|
"id": "5b853efa-dfea-4770-ab88-9b7e17794421",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"validation_df = recipe_df.loc[101:200]\n",
|
|
"validation_data = validation_df.apply(prepare_example_conversation, axis=1).tolist()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "1d5e7bfe-f6c8-4a23-a951-3df3f3791d7f",
|
|
"metadata": {},
|
|
"source": [
|
|
"We then need to save our data as `.jsonl` files, with each line being one training example conversation.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 7,
|
|
"id": "8d2eb207-2c2b-43f6-a613-64a7e92d494d",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"def write_jsonl(data_list: list, filename: str) -> None:\n",
|
|
" with open(filename, \"w\") as out:\n",
|
|
" for ddict in data_list:\n",
|
|
" jout = json.dumps(ddict) + \"\\n\"\n",
|
|
" out.write(jout)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 8,
|
|
"id": "8b53e7a2-1cac-4c5f-8ba4-3292ba2a0770",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"training_file_name = \"tmp_recipe_finetune_training.jsonl\"\n",
|
|
"write_jsonl(training_data, training_file_name)\n",
|
|
"\n",
|
|
"validation_file_name = \"tmp_recipe_finetune_validation.jsonl\"\n",
|
|
"write_jsonl(validation_data, validation_file_name)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "80b274a3",
|
|
"metadata": {},
|
|
"source": [
|
|
"This is what the first 5 lines of our training `.jsonl` file look like:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 9,
|
|
"id": "283c4ec2",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"{\"messages\": [{\"role\": \"system\", \"content\": \"You are a helpful recipe assistant. You are to extract the generic ingredients from each of the recipes provided.\"}, {\"role\": \"user\", \"content\": \"Title: No-Bake Nut Cookies\\n\\nIngredients: [\\\"1 c. firmly packed brown sugar\\\", \\\"1/2 c. evaporated milk\\\", \\\"1/2 tsp. vanilla\\\", \\\"1/2 c. broken nuts (pecans)\\\", \\\"2 Tbsp. butter or margarine\\\", \\\"3 1/2 c. bite size shredded rice biscuits\\\"]\\n\\nGeneric ingredients: \"}, {\"role\": \"assistant\", \"content\": \"[\\\"brown sugar\\\", \\\"milk\\\", \\\"vanilla\\\", \\\"nuts\\\", \\\"butter\\\", \\\"bite size shredded rice biscuits\\\"]\"}]}\n",
|
|
"{\"messages\": [{\"role\": \"system\", \"content\": \"You are a helpful recipe assistant. You are to extract the generic ingredients from each of the recipes provided.\"}, {\"role\": \"user\", \"content\": \"Title: Jewell Ball'S Chicken\\n\\nIngredients: [\\\"1 small jar chipped beef, cut up\\\", \\\"4 boned chicken breasts\\\", \\\"1 can cream of mushroom soup\\\", \\\"1 carton sour cream\\\"]\\n\\nGeneric ingredients: \"}, {\"role\": \"assistant\", \"content\": \"[\\\"beef\\\", \\\"chicken breasts\\\", \\\"cream of mushroom soup\\\", \\\"sour cream\\\"]\"}]}\n",
|
|
"{\"messages\": [{\"role\": \"system\", \"content\": \"You are a helpful recipe assistant. You are to extract the generic ingredients from each of the recipes provided.\"}, {\"role\": \"user\", \"content\": \"Title: Creamy Corn\\n\\nIngredients: [\\\"2 (16 oz.) pkg. frozen corn\\\", \\\"1 (8 oz.) pkg. cream cheese, cubed\\\", \\\"1/3 c. butter, cubed\\\", \\\"1/2 tsp. garlic powder\\\", \\\"1/2 tsp. salt\\\", \\\"1/4 tsp. pepper\\\"]\\n\\nGeneric ingredients: \"}, {\"role\": \"assistant\", \"content\": \"[\\\"frozen corn\\\", \\\"cream cheese\\\", \\\"butter\\\", \\\"garlic powder\\\", \\\"salt\\\", \\\"pepper\\\"]\"}]}\n",
|
|
"{\"messages\": [{\"role\": \"system\", \"content\": \"You are a helpful recipe assistant. You are to extract the generic ingredients from each of the recipes provided.\"}, {\"role\": \"user\", \"content\": \"Title: Chicken Funny\\n\\nIngredients: [\\\"1 large whole chicken\\\", \\\"2 (10 1/2 oz.) cans chicken gravy\\\", \\\"1 (10 1/2 oz.) can cream of mushroom soup\\\", \\\"1 (6 oz.) box Stove Top stuffing\\\", \\\"4 oz. shredded cheese\\\"]\\n\\nGeneric ingredients: \"}, {\"role\": \"assistant\", \"content\": \"[\\\"chicken\\\", \\\"chicken gravy\\\", \\\"cream of mushroom soup\\\", \\\"shredded cheese\\\"]\"}]}\n",
|
|
"{\"messages\": [{\"role\": \"system\", \"content\": \"You are a helpful recipe assistant. You are to extract the generic ingredients from each of the recipes provided.\"}, {\"role\": \"user\", \"content\": \"Title: Reeses Cups(Candy) \\n\\nIngredients: [\\\"1 c. peanut butter\\\", \\\"3/4 c. graham cracker crumbs\\\", \\\"1 c. melted butter\\\", \\\"1 lb. (3 1/2 c.) powdered sugar\\\", \\\"1 large pkg. chocolate chips\\\"]\\n\\nGeneric ingredients: \"}, {\"role\": \"assistant\", \"content\": \"[\\\"peanut butter\\\", \\\"graham cracker crumbs\\\", \\\"butter\\\", \\\"powdered sugar\\\", \\\"chocolate chips\\\"]\"}]}\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"# print the first 5 lines of the training file\n",
|
|
"!head -n 5 tmp_recipe_finetune_training.jsonl"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "0d149e2e-50dd-45c1-bd8d-1291975670b4",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Upload files\n",
|
|
"\n",
|
|
"You can now upload the files to our `Files` endpoint to be used by the fine-tuned model.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 13,
|
|
"id": "69462d9e-e6bd-49b9-a064-9eae4ea5b7a8",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Training file ID: file-PVkEstNM2WWd1OQe3Hp3tC5E\n",
|
|
"Validation file ID: file-WSdTwLYrKxNhKi1WWGjxXi87\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"with open(training_file_name, \"rb\") as training_fd:\n",
|
|
" training_response = client.files.create(\n",
|
|
" file=training_fd, purpose=\"fine-tune\"\n",
|
|
" )\n",
|
|
"\n",
|
|
"training_file_id = training_response.id\n",
|
|
"\n",
|
|
"with open(validation_file_name, \"rb\") as validation_fd:\n",
|
|
" validation_response = client.files.create(\n",
|
|
" file=validation_fd, purpose=\"fine-tune\"\n",
|
|
" )\n",
|
|
"validation_file_id = validation_response.id\n",
|
|
"\n",
|
|
"print(\"Training file ID:\", training_file_id)\n",
|
|
"print(\"Validation file ID:\", validation_file_id)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "d61cd381-63ad-4ed9-b0be-47a438891028",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Fine-tuning\n",
|
|
"\n",
|
|
"Now we can create our fine-tuning job with the generated files and an optional suffix to identify the model. The response will contain an `id` which you can use to retrieve updates on the job.\n",
|
|
"\n",
|
|
"Note: The files have to first be processed by our system, so you might get a `File not ready` error. In that case, simply retry a few minutes later.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 14,
|
|
"id": "05541ceb-5628-447e-962d-7e57c112439c",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Job ID: ftjob-bIVrnhnZEEizSP7rqWsRwv2R\n",
|
|
"Status: validating_files\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"response = client.fine_tuning.jobs.create(\n",
|
|
" training_file=training_file_id,\n",
|
|
" validation_file=validation_file_id,\n",
|
|
" model=\"gpt-3.5-turbo\",\n",
|
|
" suffix=\"recipe-ner\",\n",
|
|
")\n",
|
|
"\n",
|
|
"job_id = response.id\n",
|
|
"\n",
|
|
"print(\"Job ID:\", response.id)\n",
|
|
"print(\"Status:\", response.status)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "1de3ed71-f2d4-4138-95a3-70da187a007e",
|
|
"metadata": {},
|
|
"source": [
|
|
"#### Check job status\n",
|
|
"\n",
|
|
"You can make a `GET` request to the `https://api.openai.com/v1/alpha/fine-tunes` endpoint to list your alpha fine-tune jobs. In this instance you'll want to check that the ID you got from the previous step ends up as `status: succeeded`.\n",
|
|
"\n",
|
|
"Once it is completed, you can use the `result_files` to sample the results from the validation set (if you uploaded one), and use the ID from the `fine_tuned_model` parameter to invoke your trained model.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 21,
|
|
"id": "d7392f48",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Job ID: ftjob-bIVrnhnZEEizSP7rqWsRwv2R\n",
|
|
"Status: running\n",
|
|
"Trained Tokens: None\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"response = client.fine_tuning.jobs.retrieve(job_id)\n",
|
|
"\n",
|
|
"print(\"Job ID:\", response.id)\n",
|
|
"print(\"Status:\", response.status)\n",
|
|
"print(\"Trained Tokens:\", response.trained_tokens)\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "30a57fbb",
|
|
"metadata": {},
|
|
"source": [
|
|
"We can track the progress of the fine-tune with the events endpoint. You can rerun the cell below a few times until the fine-tune is ready.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 47,
|
|
"id": "08cace28",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Step 131/303: training loss=0.25, validation loss=0.37\n",
|
|
"Step 141/303: training loss=0.00, validation loss=0.19\n",
|
|
"Step 151/303: training loss=0.00, validation loss=0.11\n",
|
|
"Step 161/303: training loss=0.00, validation loss=0.06\n",
|
|
"Step 171/303: training loss=0.10, validation loss=0.00\n",
|
|
"Step 181/303: training loss=0.00, validation loss=0.38\n",
|
|
"Step 191/303: training loss=0.00, validation loss=0.15\n",
|
|
"Step 201/303: training loss=0.06, validation loss=0.64\n",
|
|
"Step 211/303: training loss=0.00, validation loss=0.04\n",
|
|
"Step 221/303: training loss=0.59, validation loss=0.85\n",
|
|
"Step 231/303: training loss=0.00, validation loss=0.00\n",
|
|
"Step 241/303: training loss=0.04, validation loss=0.42\n",
|
|
"Step 251/303: training loss=0.00, validation loss=0.14\n",
|
|
"Step 261/303: training loss=0.00, validation loss=0.00\n",
|
|
"Step 271/303: training loss=0.15, validation loss=0.50\n",
|
|
"Step 281/303: training loss=0.00, validation loss=0.72\n",
|
|
"Step 291/303: training loss=0.08, validation loss=0.16\n",
|
|
"Step 301/303: training loss=0.00, validation loss=1.76\n",
|
|
"New fine-tuned model created: ft:gpt-3.5-turbo-0613:personal:recipe-ner:8PjmcwDH\n",
|
|
"The job has successfully completed\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"response = client.fine_tuning.jobs.list_events(job_id)\n",
|
|
"\n",
|
|
"events = response.data\n",
|
|
"events.reverse()\n",
|
|
"\n",
|
|
"for event in events:\n",
|
|
" print(event.message)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "d0da4e32",
|
|
"metadata": {},
|
|
"source": [
|
|
"Now that it's done, we can get a fine-tuned model ID from the job:\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 48,
|
|
"id": "40b28c26",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Fine-tuned model ID: ft:gpt-3.5-turbo-0613:personal:recipe-ner:8PjmcwDH\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"response = client.fine_tuning.jobs.retrieve(job_id)\n",
|
|
"fine_tuned_model_id = response.fine_tuned_model\n",
|
|
"\n",
|
|
"if fine_tuned_model_id is None: \n",
|
|
" raise RuntimeError(\"Fine-tuned model ID not found. Your job has likely not been completed yet.\")\n",
|
|
"\n",
|
|
"print(\"Fine-tuned model ID:\", fine_tuned_model_id)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "0025e392-84cd-4566-a384-ea31ca43e567",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Inference"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "0ab9ac11",
|
|
"metadata": {},
|
|
"source": [
|
|
"The last step is to use your fine-tuned model for inference. Similar to the classic `FineTuning`, you simply call `ChatCompletions` with your new fine-tuned model name filling the `model` parameter.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 49,
|
|
"id": "1c7de631-b68f-4eff-9ae7-051641579c2b",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"[{'content': 'You are a helpful recipe assistant. You are to extract the '\n",
|
|
" 'generic ingredients from each of the recipes provided.',\n",
|
|
" 'role': 'system'},\n",
|
|
" {'content': 'Title: Beef Brisket\\n'\n",
|
|
" '\\n'\n",
|
|
" 'Ingredients: [\"4 lb. beef brisket\", \"1 c. catsup\", \"1 c. water\", '\n",
|
|
" '\"1/2 onion, minced\", \"2 Tbsp. cider vinegar\", \"1 Tbsp. prepared '\n",
|
|
" 'horseradish\", \"1 Tbsp. prepared mustard\", \"1 tsp. salt\", \"1/2 '\n",
|
|
" 'tsp. pepper\"]\\n'\n",
|
|
" '\\n'\n",
|
|
" 'Generic ingredients: ',\n",
|
|
" 'role': 'user'}]\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"test_df = recipe_df.loc[201:300]\n",
|
|
"test_row = test_df.iloc[0]\n",
|
|
"test_messages = []\n",
|
|
"test_messages.append({\"role\": \"system\", \"content\": system_message})\n",
|
|
"user_message = create_user_message(test_row)\n",
|
|
"test_messages.append({\"role\": \"user\", \"content\": create_user_message(test_row)})\n",
|
|
"\n",
|
|
"pprint(test_messages)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 50,
|
|
"id": "1a1d2589",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"[\"beef brisket\", \"catsup\", \"water\", \"onion\", \"cider vinegar\", \"horseradish\", \"mustard\", \"salt\", \"pepper\"]\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"response = client.chat.completions.create(\n",
|
|
" model=fine_tuned_model_id, messages=test_messages, temperature=0, max_tokens=500\n",
|
|
")\n",
|
|
"print(response.choices[0].message.content)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "07799909-3f2a-4274-b81e-dabc048be28f",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Conclusion\n",
|
|
"\n",
|
|
"Congratulations, you are now ready to fine-tune your own models using the `ChatCompletion` format! We look forward to seeing what you build\n"
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"kernelspec": {
|
|
"display_name": "Python 3",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.11.3"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 5
|
|
}
|