openai-cookbook/examples/Deterministic_outputs_with_the_seed_parameter.ipynb

{"cells":[{"cell_type":"markdown","metadata":{"cell_id":"67bb097e130b41099c9d257dc06a4054","deepnote_cell_type":"markdown"},"source":["# How to make your completions outputs consistent with the new seed parameter\n","\n","**TLDR**: Developers can now specify `seed` parameter in the Chat Completion request for consistent completions. We always include a `system_fingerprint` in the response that helps developers understand changes in our system that will affect determinism.\n","\n","### Context\n","\n","Determinism has always been a big request from user communities when using our APIs. For instance, when granted the capability of getting deterministic numerical result, users can unlock quite a bit of use cases that’s sensitive to numerical changes.\n","\n","#### Model level features for consistent outputs\n","\n","The Chat Completions and Completions APIs are non-deterministic by default (which means model outputs may differ from request to request), but now offer some control towards deterministic outputs using a few model level controls.\n","\n","This can unlock consistent completions which enables full control on the model behaviors for anything built on top of the APIs, and quite useful for reproducing results and testing so you know get peace of mind from knowing exactly what you’d get.\n","\n","#### Implementing consistent outputs\n","\n","To receive _mostly_ deterministic outputs across API calls:\n","\n","- Set the `seed` parameter to any integer of your choice, but use the same value across requests. For example, `12345`.\n","- Set all other parameters (prompt, temperature, top_p, etc.) to the same values across requests.\n","- In the response, check the `system_fingerprint` field. The system fingerprint is an identifier for the current combination of model weights, infrastructure, and other configuration options used by OpenAI servers to generate the completion. It changes whenever you change request parameters, or OpenAI updates numerical configuration of the infrastructure serving our models (which may happen a few times a year).\n","\n","If the `seed`, request parameters, and `system_fingerprint` all match across your requests, then model outputs will mostly be identical. There is a small chance that responses differ even when request parameters and `system_fingerprint` match, due to the inherent non-determinism of computers.\n"]},{"cell_type":"markdown","metadata":{"cell_id":"f49611fa59af4303883d76c491095fea","deepnote_cell_type":"markdown"},"source":["### Model level controls for consistent outputs - `seed` and `system_fingerprint`\n","\n","##### `seed`\n","\n","If specified, our system will make a best effort to sample deterministically, such that repeated requests with the same seed and parameters should return the same result. Determinism is not guaranteed, and you should refer to the `system_fingerprint` response parameter to monitor changes in the backend.\n","\n","##### `system_fingerprint`\n","\n","This fingerprint represents the backend configuration that the model runs with. It can be used in conjunction with the seed request parameter to understand when backend changes have been made that might impact determinism.This is the indicator on whether users should expect \"almost always the same result\".\n"]},{"cell_type":"markdown","metadata":{"cell_id":"cc6cd37b9a2243aaa4688ef8832512eb","deepnote_cell_type":"markdown"},"source":["## Example: Generating a consistent short story with a fixed seed\n","\n","In this example, we will demonstrate how to generate a consistent short story using a fixed seed. This can be particularly useful in scenarios where you need to reproduce the same results for testing, debugging, or for applications that require consistent outputs.\n"]},{"cell_type":"code","execution_count":null,"metadata":{"cell_id":"48fd2d4c95ad465090ef97254a4a10d2","deepnote_cell_type":"code"},"outputs":[],"source":["import asyncio\n","import openai\n","import pprint\n","import difflib\n","from IPython.display import display, HTML\n","\n","GPT_MODEL = \"gpt-3.5-turbo-1106\""]},{"cell_type":"cod