From 67655a61ebbb1f536527943793495ba23bba76ba Mon Sep 17 00:00:00 2001 From: Brian Zhang Date: Wed, 1 May 2024 21:59:00 -0700 Subject: [PATCH] Mention choices behavior --- examples/How_to_stream_completions.ipynb | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/examples/How_to_stream_completions.ipynb b/examples/How_to_stream_completions.ipynb index 61d2133a..a1d9098c 100644 --- a/examples/How_to_stream_completions.ipynb +++ b/examples/How_to_stream_completions.ipynb @@ -574,7 +574,10 @@ "source": [ "### 4. How to get token usage data for streamed chat completion response\n", "\n", - "You can get token usage data for your streamed response by setting `stream_options={\"include_usage\": True}`. When you do so, an extra chunk will be returned as the final chunk. You can access the usage data for the whole response via the `usage` field on this chunk. Note that all other chunks will also have a `usage` field, but they all contain a null value.\n", + "You can get token usage statistics for your streamed response by setting `stream_options={\"include_usage\": True}`. When you do so, an extra chunk will be streamed as the final chunk. You can access the usage data for the entire request via the `usage` field on this chunk. A few important notes when you set `stream_options={\"include_usage\": True}`:\n", + "* The value for the `usage` field on all chunks except for the last one will be null.\n", + "* The `usage` field on the last chunk contains token usage statistics for the entire request.\n", + "* The `choices` field on the last chunk will always be an empty array `[]`.\n", "\n", "Let's see how it works using the example in 2." ]