Mention choices behavior

pull/1172/head
Brian Zhang 3 weeks ago
parent 115af68e5b
commit 67655a61eb

@ -574,7 +574,10 @@
"source": [
"### 4. How to get token usage data for streamed chat completion response\n",
"\n",
"You can get token usage data for your streamed response by setting `stream_options={\"include_usage\": True}`. When you do so, an extra chunk will be returned as the final chunk. You can access the usage data for the whole response via the `usage` field on this chunk. Note that all other chunks will also have a `usage` field, but they all contain a null value.\n",
"You can get token usage statistics for your streamed response by setting `stream_options={\"include_usage\": True}`. When you do so, an extra chunk will be streamed as the final chunk. You can access the usage data for the entire request via the `usage` field on this chunk. A few important notes when you set `stream_options={\"include_usage\": True}`:\n",
"* The value for the `usage` field on all chunks except for the last one will be null.\n",
"* The `usage` field on the last chunk contains token usage statistics for the entire request.\n",
"* The `choices` field on the last chunk will always be an empty array `[]`.\n",
"\n",
"Let's see how it works using the example in 2."
]

Loading…
Cancel
Save