Archives/Prompt-Engineering-Guide
2
0
Fork 0
mirror of https://github.com/dair-ai/Prompt-Engineering-Guide synced 2024-11-02 15:40:13 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #148 from omerbensaadon/zero-shot-edits

Browse Source 
Adding details to zero shot prompting
This commit is contained in:
Elvis Saravia 2023-04-23 20:13:06 -06:00 committed by GitHub
commit 5b059f20ee
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 4 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
pages/techniques/zeroshot.en.mdx
View File

@ -1,5 +1,7 @@
# Zero-Shot Prompting # Zero-Shot Prompting
LLMs today trained on large amounts of data and tuned to follow instructions, are capable of performing tasks zero-shot. We tried a few zero-shot examples in the previous section. Here is one of the examples we used: Large LLMs today, such as GPT-3, are tuned to follow instructions and are trained on large amounts of data; so they are capable of performing some tasks "zero-shot."
We tried a few zero-shot examples in the previous section. Here is one of the examples we used:
*Prompt:* *Prompt:*
``` ```
@ -14,7 +16,7 @@ Sentiment:
Neutral Neutral
``` ```
Note that in the prompt above we didn't provide the model with any examples -- that's the zero-shot capabilities at work. Note that in the prompt above we didn't provide the model with any examples of text alongside their classifications, the LLM already understands "sentiment" -- that's the zero-shot capabilities at work.
Instruction tuning has shown to improve zero-shot learning [Wei et al. (2022)](https://arxiv.org/pdf/2109.01652.pdf). Instruction tuning is essentially the concept of finetuning models on datasets described via instructions. Furthermore, [RLHF](https://arxiv.org/abs/1706.03741) (reinforcement learning from human feedback) has been adopted to scale instruction tuning wherein the model is aligned to better fit human preferences. This recent development powers models like ChatGPT. We will discuss all these approaches and methods in upcoming sections. Instruction tuning has shown to improve zero-shot learning [Wei et al. (2022)](https://arxiv.org/pdf/2109.01652.pdf). Instruction tuning is essentially the concept of finetuning models on datasets described via instructions. Furthermore, [RLHF](https://arxiv.org/abs/1706.03741) (reinforcement learning from human feedback) has been adopted to scale instruction tuning wherein the model is aligned to better fit human preferences. This recent development powers models like ChatGPT. We will discuss all these approaches and methods in upcoming sections.