Archives/openai-cookbook
2
0
Fork 0
mirror of https://github.com/openai/openai-cookbook synced 2024-11-11 13:11:02 +00:00
Code Issues Packages Projects Releases Wiki Activity

Logprobs suggestions (#948)

Browse Source
This commit is contained in:
Enoch Cheung 2023-12-20 21:57:47 -08:00 committed by GitHub
parent 7517075115
commit a00ccf8bad
1 changed files with 5 additions and 4 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

9
examples/Using_logprobs.ipynb
View File

@ -8,11 +8,12 @@
"\n",
"This notebook demonstrates the use of the `logprobs` parameter in the Chat Completions API. When `logprobs` is enabled, the API returns the log probabilities of each output token, along with a limited number of the most likely tokens at each token position and their log probabilities. The relevant request parameters are:\n",
"* `logprobs`: Whether to return log probabilities of the output tokens or not. If true, returns the log probabilities of each output token returned in the content of message. This option is currently not available on the `gpt-4-vision-preview` model.\n",
"* `top_logprobs`: An integer between 0 and 5 specifying the number of most likely tokens to return at each token position, each with an associated log probability. logprobs must be set to true if this parameter is used.\n",
"* `top_logprobs`: An integer between 0 and 5 specifying the number of most likely tokens to return at each token position, each with an associated log probability. `logprobs` must be set to true if this parameter is used.\n",
"\n",
"Log probabilities of output tokens indicate the likelihood of each token occurring in the sequence given the context. To simplify, a logprob is `log(p)`, where `p` = probability of a token occurring at a specific position based on the other tokens in the sentence. Some key points about `logprobs`:\n",
"* Higher log probabilities suggest a higher likelihood of the token in that context. This allows users to gauge the model's confidence in its output or explore alternative responses the model considered. \n",
"* It also allow us to compute the overall probability of a sequence as the sum of the log probs of the individual tokens. This is useful for scoring and ranking model outputs. It's pretty common to take the average logprob of a sentence to choose the best generation.\n",
"Log probabilities of output tokens indicate the likelihood of each token occurring in the sequence given the context. To simplify, a logprob is `log(p)`, where `p` = probability of a token occurring at a specific position based on the previous tokens in the context. Some key points about `logprobs`:\n",
"* Higher log probabilities suggest a higher likelihood of the token in that context. This allows users to gauge the model's confidence in its output or explore alternative responses the model considered.\n",
"* Logprob can be any negative number or `0.0`. `0.0` corresponds to 100% probability.\n",
"* Logprobs allow us to compute the joint probability of a sequence as the sum of the logprobs of the individual tokens. This is useful for scoring and ranking model outputs. Another common approach is to take the average per-token logprob of a sentence to choose the best generation.\n",
"* We can examine the `logprobs` assigned to different candidate tokens to understand what options the model considered plausible or implausible.\n",
"\n",
"While there are a wide array of use cases for `logprobs`, this notebook will focus on its use for:\n",