mirror of
https://github.com/openai/openai-cookbook
synced 2024-11-11 13:11:02 +00:00
Updates how to fine-tune chat models cookbook with 4o-mini (#1304)
This commit is contained in:
Shyamal H Anadkat
GitHub
parent
8a288eb83a
commit
1a028ba713
@ -7,7 +7,9 @@
|
||||
"source": [
|
||||
"# How to fine-tune chat models\n",
|
||||
"\n",
|
||||
"This notebook provides a step-by-step guide for our new `gpt-3.5-turbo` fine-tuning. We'll perform entity extraction using the [RecipeNLG dataset](https://github.com/Glorf/recipenlg), which provides various recipes and a list of extracted generic ingredients for each. This is a common dataset for named entity recognition (NER) tasks.\n",
|
||||
"Fine-tuning improves the model by training on many more examples than can fit in a prompt, letting you achieve better results on a wide number of tasks. This notebook provides a step-by-step guide for our new GPT-4o mini fine-tuning. We'll perform entity extraction using the [RecipeNLG dataset](https://github.com/Glorf/recipenlg), which provides various recipes and a list of extracted generic ingredients for each. This is a common dataset for named entity recognition (NER) tasks.\n",
|
||||
"\n",
|
||||
"Note: **GPT-4o mini fine-tuning is available to developers in our [Tier 4 and 5 usage tiers](https://platform.openai.com/docs/guides/rate-limits/usage-tiers).** You can start fine-tuning GPT-4o mini by visiting your fine-tuning dashboard, clicking \"create\", and selecting “gpt-4o-mini-2024-07-18” from the base model drop-down.\n",
|
||||
"\n",
|
||||
"We will go through the following steps:\n",
|
||||
"\n",
|
||||
@ -16,9 +18,9 @@
|
||||
"3. **Fine-tuning:** Creating your fine-tuned model.\n",
|
||||
"4. **Inference:** Using your fine-tuned model for inference on new inputs.\n",
|
||||
"\n",
|
||||
"By the end of this you should be able to train, evaluate and deploy a fine-tuned `gpt-3.5-turbo` model.\n",
|
||||
"By the end of this you should be able to train, evaluate and deploy a fine-tuned `gpt-4o-mini-2024-07-18` model.\n",
|
||||
"\n",
|
||||
"For more information on fine-tuning, you can refer to our [documentation guide](https://platform.openai.com/docs/guides/fine-tuning), [API reference](https://platform.openai.com/docs/api-reference/fine-tuning) or [blog post](https://openai.com/blog/gpt-3-5-turbo-fine-tuning-and-api-updates)"
|
||||
"For more information on fine-tuning, you can refer to our [documentation guide](https://platform.openai.com/docs/guides/fine-tuning) or [API reference](https://platform.openai.com/docs/api-reference/fine-tuning).\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
@ -26,23 +28,23 @@
|
||||
"id": "6f49cb10-f895-41f4-aa97-da606d0084d4",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Setup"
|
||||
"## Setup\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"execution_count": 6,
|
||||
"id": "32036e70",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# make sure to use the latest version of the openai python package\n",
|
||||
"!pip install --upgrade openai "
|
||||
"!pip install --upgrade --quiet openai"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 12,
|
||||
"execution_count": 69,
|
||||
"id": "6e1f4403-37e1-4115-a215-12fd7daa1eb6",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
@ -53,7 +55,11 @@
|
||||
"import pandas as pd\n",
|
||||
"from pprint import pprint\n",
|
||||
"\n",
|
||||
"client = openai.OpenAI(api_key=os.environ.get(\"OPENAI_API_KEY\", \"<your OpenAI API key if not set as env var>\"))"
|
||||
"client = openai.OpenAI(\n",
|
||||
" api_key=os.environ.get(\"OPENAI_API_KEY\"),\n",
|
||||
" organization=\"<org id>\",\n",
|
||||
" project=\"<project id>\",\n",
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
@ -61,13 +67,12 @@
|
||||
"id": "a468d660",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"\n",
|
||||
"Fine-tuning works best when focused on a particular domain. It's important to make sure your dataset is both focused enough for the model to learn, but general enough that unseen examples won't be missed. Having this in mind, we have extracted a subset from the RecipesNLG dataset to only contain documents from www.cookbooks.com."
|
||||
"Fine-tuning works best when focused on a particular domain. It's important to make sure your dataset is both focused enough for the model to learn, but general enough that unseen examples won't be missed. Having this in mind, we have extracted a subset from the RecipesNLG dataset to only contain documents from [cookbooks.com](https://cookbooks.com/).\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 3,
|
||||
"execution_count": 70,
|
||||
"id": "f57ebc23-14b7-47f9-90b8-1d791ccfc9bc",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
@ -151,22 +156,22 @@
|
||||
"</div>"
|
||||
],
|
||||
"text/plain": [
|
||||
" title ingredients \n",
|
||||
"0 No-Bake Nut Cookies [\"1 c. firmly packed brown sugar\", \"1/2 c. eva... \\\n",
|
||||
" title ingredients \\\n",
|
||||
"0 No-Bake Nut Cookies [\"1 c. firmly packed brown sugar\", \"1/2 c. eva... \n",
|
||||
"1 Jewell Ball'S Chicken [\"1 small jar chipped beef, cut up\", \"4 boned ... \n",
|
||||
"2 Creamy Corn [\"2 (16 oz.) pkg. frozen corn\", \"1 (8 oz.) pkg... \n",
|
||||
"3 Chicken Funny [\"1 large whole chicken\", \"2 (10 1/2 oz.) cans... \n",
|
||||
"4 Reeses Cups(Candy) [\"1 c. peanut butter\", \"3/4 c. graham cracker ... \n",
|
||||
"\n",
|
||||
" directions \n",
|
||||
"0 [\"In a heavy 2-quart saucepan, mix brown sugar... \\\n",
|
||||
" directions \\\n",
|
||||
"0 [\"In a heavy 2-quart saucepan, mix brown sugar... \n",
|
||||
"1 [\"Place chipped beef on bottom of baking dish.... \n",
|
||||
"2 [\"In a slow cooker, combine all ingredients. C... \n",
|
||||
"3 [\"Boil and debone chicken.\", \"Put bite size pi... \n",
|
||||
"4 [\"Combine first four ingredients and press in ... \n",
|
||||
"\n",
|
||||
" link source \n",
|
||||
"0 www.cookbooks.com/Recipe-Details.aspx?id=44874 www.cookbooks.com \\\n",
|
||||
" link source \\\n",
|
||||
"0 www.cookbooks.com/Recipe-Details.aspx?id=44874 www.cookbooks.com \n",
|
||||
"1 www.cookbooks.com/Recipe-Details.aspx?id=699419 www.cookbooks.com \n",
|
||||
"2 www.cookbooks.com/Recipe-Details.aspx?id=10570 www.cookbooks.com \n",
|
||||
"3 www.cookbooks.com/Recipe-Details.aspx?id=897570 www.cookbooks.com \n",
|
||||
@ -180,7 +185,7 @@
|
||||
"4 [\"peanut butter\", \"graham cracker crumbs\", \"bu... "
|
||||
]
|
||||
},
|
||||
"execution_count": 3,
|
||||
"execution_count": 70,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
@ -220,7 +225,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 4,
|
||||
"execution_count": 71,
|
||||
"id": "9a8216b0-d1dc-472d-b07d-1be03acd70a5",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
@ -249,23 +254,22 @@
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"training_data = []\n",
|
||||
"\n",
|
||||
"system_message = \"You are a helpful recipe assistant. You are to extract the generic ingredients from each of the recipes provided.\"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"def create_user_message(row):\n",
|
||||
" return f\"\"\"Title: {row['title']}\\n\\nIngredients: {row['ingredients']}\\n\\nGeneric ingredients: \"\"\"\n",
|
||||
" return f\"Title: {row['title']}\\n\\nIngredients: {row['ingredients']}\\n\\nGeneric ingredients: \"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"def prepare_example_conversation(row):\n",
|
||||
" messages = []\n",
|
||||
" messages.append({\"role\": \"system\", \"content\": system_message})\n",
|
||||
" return {\n",
|
||||
" \"messages\": [\n",
|
||||
" {\"role\": \"system\", \"content\": system_message},\n",
|
||||
" {\"role\": \"user\", \"content\": create_user_message(row)},\n",
|
||||
" {\"role\": \"assistant\", \"content\": row[\"NER\"]},\n",
|
||||
" ]\n",
|
||||
" }\n",
|
||||
"\n",
|
||||
" user_message = create_user_message(row)\n",
|
||||
" messages.append({\"role\": \"user\", \"content\": user_message})\n",
|
||||
"\n",
|
||||
" messages.append({\"role\": \"assistant\", \"content\": row[\"NER\"]})\n",
|
||||
"\n",
|
||||
" return {\"messages\": messages}\n",
|
||||
"\n",
|
||||
"pprint(prepare_example_conversation(recipe_df.iloc[0]))"
|
||||
]
|
||||
@ -275,12 +279,12 @@
|
||||
"id": "82fa4fae",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Let's now do this for a subset of the dataset to use as our training data. You can begin with even 30-50 well-pruned examples. You should see performance continue to scale linearly as you increase the size of the training set, but your jobs will also take longer."
|
||||
"Let's now do this for a subset of the dataset to use as our training data. You can begin with even 30-50 well-pruned examples. You should see performance continue to scale linearly as you increase the size of the training set, but your jobs will also take longer.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 5,
|
||||
"execution_count": 72,
|
||||
"id": "8f37aff9",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
@ -312,18 +316,19 @@
|
||||
"id": "6fb0f6a7",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"In addition to training data, we can also **optionally** provide validation data, which will be used to make sure that the model does not overfit your training set."
|
||||
"In addition to training data, we can also **optionally** provide validation data, which will be used to make sure that the model does not overfit your training set.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 6,
|
||||
"execution_count": 73,
|
||||
"id": "5b853efa-dfea-4770-ab88-9b7e17794421",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"validation_df = recipe_df.loc[101:200]\n",
|
||||
"validation_data = validation_df.apply(prepare_example_conversation, axis=1).tolist()"
|
||||
"validation_data = validation_df.apply(\n",
|
||||
" prepare_example_conversation, axis=1).tolist()"
|
||||
]
|
||||
},
|
||||
{
|
||||
@ -336,7 +341,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 7,
|
||||
"execution_count": 74,
|
||||
"id": "8d2eb207-2c2b-43f6-a613-64a7e92d494d",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
@ -350,7 +355,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 8,
|
||||
"execution_count": 75,
|
||||
"id": "8b53e7a2-1cac-4c5f-8ba4-3292ba2a0770",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
@ -367,12 +372,12 @@
|
||||
"id": "80b274a3",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"This is what the first 5 lines of our training `.jsonl` file look like:"
|
||||
"This is what the first 5 lines of our training `.jsonl` file look like:\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 9,
|
||||
"execution_count": 76,
|
||||
"id": "283c4ec2",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
@ -405,7 +410,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 13,
|
||||
"execution_count": 77,
|
||||
"id": "69462d9e-e6bd-49b9-a064-9eae4ea5b7a8",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
@ -413,24 +418,20 @@
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Training file ID: file-PVkEstNM2WWd1OQe3Hp3tC5E\n",
|
||||
"Validation file ID: file-WSdTwLYrKxNhKi1WWGjxXi87\n"
|
||||
"Training file ID: file-3wfAfDoYcGrSpaE17qK0vXT0\n",
|
||||
"Validation file ID: file-HhFhnyGJhazYdPcd3wrtvIoX\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"with open(training_file_name, \"rb\") as training_fd:\n",
|
||||
" training_response = client.files.create(\n",
|
||||
" file=training_fd, purpose=\"fine-tune\"\n",
|
||||
" )\n",
|
||||
"def upload_file(file_name: str, purpose: str) -> str:\n",
|
||||
" with open(file_name, \"rb\") as file_fd:\n",
|
||||
" response = client.files.create(file=file_fd, purpose=purpose)\n",
|
||||
" return response.id\n",
|
||||
"\n",
|
||||
"training_file_id = training_response.id\n",
|
||||
"\n",
|
||||
"with open(validation_file_name, \"rb\") as validation_fd:\n",
|
||||
" validation_response = client.files.create(\n",
|
||||
" file=validation_fd, purpose=\"fine-tune\"\n",
|
||||
" )\n",
|
||||
"validation_file_id = validation_response.id\n",
|
||||
"training_file_id = upload_file(training_file_name, \"fine-tune\")\n",
|
||||
"validation_file_id = upload_file(validation_file_name, \"fine-tune\")\n",
|
||||
"\n",
|
||||
"print(\"Training file ID:\", training_file_id)\n",
|
||||
"print(\"Validation file ID:\", validation_file_id)"
|
||||
@ -450,7 +451,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 14,
|
||||
"execution_count": 81,
|
||||
"id": "05541ceb-5628-447e-962d-7e57c112439c",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
@ -458,16 +459,18 @@
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Job ID: ftjob-bIVrnhnZEEizSP7rqWsRwv2R\n",
|
||||
"Job ID: ftjob-UiaiLwGdGBfdLQDBAoQheufN\n",
|
||||
"Status: validating_files\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"MODEL = \"gpt-4o-mini-2024-07-18\"\n",
|
||||
"\n",
|
||||
"response = client.fine_tuning.jobs.create(\n",
|
||||
" training_file=training_file_id,\n",
|
||||
" validation_file=validation_file_id,\n",
|
||||
" model=\"gpt-3.5-turbo\",\n",
|
||||
" model=MODEL,\n",
|
||||
" suffix=\"recipe-ner\",\n",
|
||||
")\n",
|
||||
"\n",
|
||||
@ -491,7 +494,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 21,
|
||||
"execution_count": 91,
|
||||
"id": "d7392f48",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
@ -499,7 +502,7 @@
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Job ID: ftjob-bIVrnhnZEEizSP7rqWsRwv2R\n",
|
||||
"Job ID: ftjob-UiaiLwGdGBfdLQDBAoQheufN\n",
|
||||
"Status: running\n",
|
||||
"Trained Tokens: None\n"
|
||||
]
|
||||
@ -510,7 +513,7 @@
|
||||
"\n",
|
||||
"print(\"Job ID:\", response.id)\n",
|
||||
"print(\"Status:\", response.status)\n",
|
||||
"print(\"Trained Tokens:\", response.trained_tokens)\n"
|
||||
"print(\"Trained Tokens:\", response.trained_tokens)"
|
||||
]
|
||||
},
|
||||
{
|
||||
@ -523,7 +526,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 47,
|
||||
"execution_count": 94,
|
||||
"id": "08cace28",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
@ -531,25 +534,25 @@
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Step 131/303: training loss=0.25, validation loss=0.37\n",
|
||||
"Step 141/303: training loss=0.00, validation loss=0.19\n",
|
||||
"Step 151/303: training loss=0.00, validation loss=0.11\n",
|
||||
"Step 161/303: training loss=0.00, validation loss=0.06\n",
|
||||
"Step 171/303: training loss=0.10, validation loss=0.00\n",
|
||||
"Step 181/303: training loss=0.00, validation loss=0.38\n",
|
||||
"Step 191/303: training loss=0.00, validation loss=0.15\n",
|
||||
"Step 201/303: training loss=0.06, validation loss=0.64\n",
|
||||
"Step 211/303: training loss=0.00, validation loss=0.04\n",
|
||||
"Step 221/303: training loss=0.59, validation loss=0.85\n",
|
||||
"Step 231/303: training loss=0.00, validation loss=0.00\n",
|
||||
"Step 241/303: training loss=0.04, validation loss=0.42\n",
|
||||
"Step 251/303: training loss=0.00, validation loss=0.14\n",
|
||||
"Step 261/303: training loss=0.00, validation loss=0.00\n",
|
||||
"Step 271/303: training loss=0.15, validation loss=0.50\n",
|
||||
"Step 281/303: training loss=0.00, validation loss=0.72\n",
|
||||
"Step 291/303: training loss=0.08, validation loss=0.16\n",
|
||||
"Step 301/303: training loss=0.00, validation loss=1.76\n",
|
||||
"New fine-tuned model created: ft:gpt-3.5-turbo-0613:personal:recipe-ner:8PjmcwDH\n",
|
||||
"Step 288/303: training loss=0.00\n",
|
||||
"Step 289/303: training loss=0.01\n",
|
||||
"Step 290/303: training loss=0.00, validation loss=0.31\n",
|
||||
"Step 291/303: training loss=0.00\n",
|
||||
"Step 292/303: training loss=0.00\n",
|
||||
"Step 293/303: training loss=0.00\n",
|
||||
"Step 294/303: training loss=0.00\n",
|
||||
"Step 295/303: training loss=0.00\n",
|
||||
"Step 296/303: training loss=0.00\n",
|
||||
"Step 297/303: training loss=0.00\n",
|
||||
"Step 298/303: training loss=0.01\n",
|
||||
"Step 299/303: training loss=0.00\n",
|
||||
"Step 300/303: training loss=0.00, validation loss=0.04\n",
|
||||
"Step 301/303: training loss=0.16\n",
|
||||
"Step 302/303: training loss=0.00\n",
|
||||
"Step 303/303: training loss=0.00, full validation loss=0.33\n",
|
||||
"Checkpoint created at step 101 with Snapshot ID: ft:gpt-4o-mini-2024-07-18:openai-gtm:recipe-ner:9o1eNlSa:ckpt-step-101\n",
|
||||
"Checkpoint created at step 202 with Snapshot ID: ft:gpt-4o-mini-2024-07-18:openai-gtm:recipe-ner:9o1eNFnj:ckpt-step-202\n",
|
||||
"New fine-tuned model created: ft:gpt-4o-mini-2024-07-18:openai-gtm:recipe-ner:9o1eNNKO\n",
|
||||
"The job has successfully completed\n"
|
||||
]
|
||||
}
|
||||
@ -574,7 +577,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 48,
|
||||
"execution_count": 95,
|
||||
"id": "40b28c26",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
@ -582,7 +585,7 @@
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Fine-tuned model ID: ft:gpt-3.5-turbo-0613:personal:recipe-ner:8PjmcwDH\n"
|
||||
"Fine-tuned model ID: ft:gpt-4o-mini-2024-07-18:openai-gtm:recipe-ner:9o1eNNKO\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
@ -590,8 +593,10 @@
|
||||
"response = client.fine_tuning.jobs.retrieve(job_id)\n",
|
||||
"fine_tuned_model_id = response.fine_tuned_model\n",
|
||||
"\n",
|
||||
"if fine_tuned_model_id is None: \n",
|
||||
" raise RuntimeError(\"Fine-tuned model ID not found. Your job has likely not been completed yet.\")\n",
|
||||
"if fine_tuned_model_id is None:\n",
|
||||
" raise RuntimeError(\n",
|
||||
" \"Fine-tuned model ID not found. Your job has likely not been completed yet.\"\n",
|
||||
" )\n",
|
||||
"\n",
|
||||
"print(\"Fine-tuned model ID:\", fine_tuned_model_id)"
|
||||
]
|
||||
@ -601,7 +606,7 @@
|
||||
"id": "0025e392-84cd-4566-a384-ea31ca43e567",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Inference"
|
||||
"## Inference\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
@ -614,7 +619,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 49,
|
||||
"execution_count": 96,
|
||||
"id": "1c7de631-b68f-4eff-9ae7-051641579c2b",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
@ -650,7 +655,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 50,
|
||||
"execution_count": 97,
|
||||
"id": "1a1d2589",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
@ -696,7 +701,7 @@
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.11.3"
|
||||
"version": "3.11.8"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
|
||||
@ -176,9 +176,10 @@
|
||||
|
||||
- title: How to fine-tune chat models
|
||||
path: examples/How_to_finetune_chat_models.ipynb
|
||||
date: 2023-08-22
|
||||
date: 2024-07-23
|
||||
authors:
|
||||
- simonpfish
|
||||
- shyamal-anadkat
|
||||
tags:
|
||||
- completions
|
||||
|
||||
|
||||
Loading…
Reference in New Issue
Block a user