mirror of
https://github.com/openai/openai-cookbook
synced 2024-11-04 06:00:33 +00:00
updates streaming guide with note about usage field
This commit is contained in:
parent
5c757fe90d
commit
6e346ac850
@ -1,6 +1,7 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"attachments": {},
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
@ -14,8 +15,14 @@
|
||||
"\n",
|
||||
"To stream completions, set `stream=True` when calling the Completions endpoint. This will return an object that streams back text as [data-only server-sent events](https://app.mode.com/openai/reports/4fce5ba22b5b/runs/f518a0be4495).\n",
|
||||
"\n",
|
||||
"## Downsides\n",
|
||||
"\n",
|
||||
"Note that using `stream=True` in a production application makes it more difficult to moderate the content of the completions, which has implications for [approved usage](https://beta.openai.com/docs/usage-guidelines).\n",
|
||||
"\n",
|
||||
"Another small drawback of streaming responses is that the response no longer includes the `usage` field to tell you how many tokens were consumed. After receiving and combining all of the responses, you can calculate this yourself using [`tiktoken`](How_to_count_tokens_with_tiktoken.ipynb).\n",
|
||||
"\n",
|
||||
"## Example code\n",
|
||||
"\n",
|
||||
"Below is a Python code example of how to receive streaming completions."
|
||||
]
|
||||
},
|
||||
@ -31,10 +38,11 @@
|
||||
]
|
||||
},
|
||||
{
|
||||
"attachments": {},
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## A typical completion request\n",
|
||||
"### A typical completion request\n",
|
||||
"\n",
|
||||
"With a typical Completions API call, the text is first computed and then returned all at once."
|
||||
]
|
||||
@ -80,10 +88,11 @@
|
||||
]
|
||||
},
|
||||
{
|
||||
"attachments": {},
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## A streaming completion request\n",
|
||||
"### A streaming completion request\n",
|
||||
"\n",
|
||||
"With a streaming Completions API call, the text is sent back via a series of events. In Python, you can iterate over these events with a `for` loop."
|
||||
]
|
||||
@ -328,10 +337,11 @@
|
||||
]
|
||||
},
|
||||
{
|
||||
"attachments": {},
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Time comparison\n",
|
||||
"### Time comparison\n",
|
||||
"\n",
|
||||
"In the example above, both requests took about 7 seconds to fully complete.\n",
|
||||
"\n",
|
||||
@ -355,7 +365,7 @@
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.9.9"
|
||||
"version": "3.9.9 (main, Dec 7 2021, 18:04:56) \n[Clang 13.0.0 (clang-1300.0.29.3)]"
|
||||
},
|
||||
"orig_nbformat": 4,
|
||||
"vscode": {
|
||||
|
Loading…
Reference in New Issue
Block a user