Improved Readability

Having the "recommended way of usage" as a separate line feels like more clear making user identify at a glance that this is something different, as we have at the below.

Initially I skimmed thru, I didn't observe the recommendation. At the end, it says, "similar to temperate & top_p...", then I went on top to observe the line. So, proposing the change.
pull/445/head
Harsha 2 months ago committed by GitHub
parent f046ef3893
commit cc1e82ca4e
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194

@ -4,7 +4,9 @@ When designing and testing prompts, you typically interact with the LLM via an A
**Temperature** - In short, the lower the `temperature`, the more deterministic the results in the sense that the highest probable next token is always picked. Increasing temperature could lead to more randomness, which encourages more diverse or creative outputs. You are essentially increasing the weights of the other possible tokens. In terms of application, you might want to use a lower temperature value for tasks like fact-based QA to encourage more factual and concise responses. For poem generation or other creative tasks, it might be beneficial to increase the temperature value.
**Top P** - A sampling technique with temperature, called nucleus sampling, where you can control how deterministic the model is. If you are looking for exact and factual answers keep this low. If you are looking for more diverse responses, increase to a higher value. If you use Top P it means that only the tokens comprising the `top_p` probability mass are considered for responses, so a low `top_p` value selects the most confident responses. This means that a high `top_p` value will enable the model to look at more possible words, including less likely ones, leading to more diverse outputs. The general recommendation is to alter temperature or Top P but not both.
**Top P** - A sampling technique with temperature, called nucleus sampling, where you can control how deterministic the model is. If you are looking for exact and factual answers keep this low. If you are looking for more diverse responses, increase to a higher value. If you use Top P it means that only the tokens comprising the `top_p` probability mass are considered for responses, so a low `top_p` value selects the most confident responses. This means that a high `top_p` value will enable the model to look at more possible words, including less likely ones, leading to more diverse outputs.
The general recommendation is to alter temperature or Top P but not both.
**Max Length** - You can manage the number of tokens the model generates by adjusting the `max length`. Specifying a max length helps you prevent long or irrelevant responses and control costs.

Loading…
Cancel
Save