updates streaming guide with note about usage field

This commit is contained in:
Ted Sanders 2023-01-20 14:38:12 -08:00
parent 5c757fe90d
commit 6e346ac850

View File

@ -1,6 +1,7 @@
{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
@ -14,8 +15,14 @@
"\n",
"To stream completions, set `stream=True` when calling the Completions endpoint. This will return an object that streams back text as [data-only server-sent events](https://app.mode.com/openai/reports/4fce5ba22b5b/runs/f518a0be4495).\n",
"\n",
"## Downsides\n",
"\n",
"Note that using `stream=True` in a production application makes it more difficult to moderate the content of the completions, which has implications for [approved usage](https://beta.openai.com/docs/usage-guidelines).\n",
"\n",
"Another small drawback of streaming responses is that the response no longer includes the `usage` field to tell you how many tokens were consumed. After receiving and combining all of the responses, you can calculate this yourself using [`tiktoken`](How_to_count_tokens_with_tiktoken.ipynb).\n",
"\n",
"## Example code\n",
"\n",
"Below is a Python code example of how to receive streaming completions."
]
},
@ -31,10 +38,11 @@
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## A typical completion request\n",
"### A typical completion request\n",
"\n",
"With a typical Completions API call, the text is first computed and then returned all at once."
]
@ -80,10 +88,11 @@
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## A streaming completion request\n",
"### A streaming completion request\n",
"\n",
"With a streaming Completions API call, the text is sent back via a series of events. In Python, you can iterate over these events with a `for` loop."
]
@ -328,10 +337,11 @@
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## Time comparison\n",
"### Time comparison\n",
"\n",
"In the example above, both requests took about 7 seconds to fully complete.\n",
"\n",
@ -355,7 +365,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.9"
"version": "3.9.9 (main, Dec 7 2021, 18:04:56) \n[Clang 13.0.0 (clang-1300.0.29.3)]"
},
"orig_nbformat": 4,
"vscode": {