openai-cookbook/examples/Reproducible_outputs_with_the_seed_parameter.ipynb

{"cells":[{"cell_type":"markdown","metadata":{"cell_id":"67bb097e130b41099c9d257dc06a4054","deepnote_cell_type":"markdown"},"source":["# How to make your completions outputs reproducible with the new seed parameter\n","\n","**TLDR**: Developers can now specify `seed` parameter in the Chat Completion request to receive (mostly) consistent outputs. To help you keep track of these changes, we expose the `system_fingerprint` field. If this value is different, you may see different outputs due to changes we've made on our systems. Please note that this feature is in beta and only currently supported for `gpt-4-1106-preview` and `gpt-3.5-turbo-1106`.\n","\n","### Context\n","\n","Reproducibility has always been a big request from user communities when using our APIs. For instance, when granted the capability of getting reproducible numerical result, users can unlock quite a bit of use cases that’s sensitive to numerical changes.\n","\n","#### Model level features for consistent outputs\n","\n","The Chat Completions and Completions APIs are non-deterministic by default (which means model outputs may differ from request to request), but now offer some control towards deterministic outputs using a few model level controls.\n","\n","This can unlock consistent completions which enables full control on the model behaviors for anything built on top of the APIs, and quite useful for reproducing results and testing so you know get peace of mind from knowing exactly what you’d get.\n","\n","#### Implementing consistent outputs\n","\n","To receive _mostly_ deterministic outputs across API calls:\n","\n","- Set the `seed` parameter to any integer of your choice, but use the same value across requests. For example, `12345`.\n","- Set all other parameters (prompt, temperature, top_p, etc.) to the same values across requests.\n","- In the response, check the `system_fingerprint` field. The system fingerprint is an identifier for the current combination of model weights, infrastructure, and other configuration options used by OpenAI servers to generate the completion. It changes whenever you change request parameters, or OpenAI updates numerical configuration of the infrastructure serving our models (which may happen a few times a year).\n","\n","If the `seed`, request parameters, and `system_fingerprint` all match across your requests, then model outputs will mostly be identical. There is a small chance that responses differ even when request parameters and `system_fingerprint` match, due to the inherent non-determinism of our models.\n"]},{"cell_type":"markdown","metadata":{"cell_id":"f49611fa59af4303883d76c491095fea","deepnote_cell_type":"markdown"},"source":["### Model level controls for consistent outputs - `seed` and `system_fingerprint`\n","\n","##### `seed`\n","\n","If specified, our system will make a best effort to sample deterministically, such that repeated requests with the same seed and parameters should return the same result. Determinism is not guaranteed, and you should refer to the `system_fingerprint` response parameter to monitor changes in the backend.\n","\n","##### `system_fingerprint`\n","\n","This fingerprint represents the backend configuration that the model runs with. It can be used in conjunction with the seed request parameter to understand when backend changes have been made that might impact determinism.This is the indicator on whether users should expect \"almost always the same result\".\n"]},{"cell_type":"markdown","metadata":{"cell_id":"cc6cd37b9a2243aaa4688ef8832512eb","deepnote_cell_type":"markdown"},"source":["## Example: Generating a short excerpt with a fixed seed\n","\n","In this example, we will demonstrate how to generate a short excerpt using a fixed seed. This can be particularly useful in scenarios where you need to generate consistent results for testing, debugging, or for applications that require consistent outputs."]},{"cell_type":"markdown","metadata":{},"source":["### Python SDK\n","\n","> **Note**\n","> Switch to latest version of the SDK (`1.3.3` at time of writing)."]},{"cell_type":