Prompt-Engineering-Guide/README.md

# Prompt Engineering Guide

This guide contains a non-exhaustive set of learning guides and tools about prompt engineering. It includes several materials, guides, examples, papers, and much more. The repo is intented to be used as a research and educational reference for practitioners and developers.

Table of Contents

- [Papers](#papers)
- [Tools & Libraries](#tools--libraries)
- [Datasets](#datasets)
- [Blog, Guides, Tutorials and Other Readings](#blog-guides-tutorials-and-other-readings)

## Papers

- Surveys / Overviews:
  - [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://arxiv.org/abs/2107.13586) 
  - [A Taxonomy of Prompt Modifiers for Text-To-Image Generation](https://arxiv.org/abs/2204.13988)
  - [Emergent Abilities of Large Language Models](https://arxiv.org/abs/2206.07682)
- Applications:
  - [Legal Prompt Engineering for Multilingual Legal Judgement Prediction](https://arxiv.org/abs/2212.02199)
  - [Investigating Prompt Engineering in Diffusion Models](https://arxiv.org/abs/2211.15462)
  - [Conversing with Copilot: Exploring Prompt Engineering for Solving CS1 Problems Using Natural Language](https://arxiv.org/abs/2210.15157)
  - [Piloting Copilot and Codex: Hot Temperature, Cold Prompts, or Black Magic?](https://arxiv.org/abs/2210.14699)
- Approaches/Techniques:
  - [Ask Me Anything: A simple strategy for prompting language models](https://paperswithcode.com/paper/ask-me-anything-a-simple-strategy-for)
  - [Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity](https://arxiv.org/abs/2104.08786)
  - [AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts](https://arxiv.org/abs/2010.15980)
  - [Large Language Models Are Human-Level Prompt Engineers](https://sites.google.com/view/automatic-prompt-engineer?pli=1)
  - [Structured Prompting: Scaling In-Context Learning to 1,000 Examples](https://arxiv.org/abs/2212.06713)
  - [Reframing Instructional Prompts to GPTk's Language](https://arxiv.org/abs/2109.07830)
  - [Promptagator: Few-shot Dense Retrieval From 8 Examples](https://arxiv.org/abs/2209.11755)
  - [Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm](https://www.arxiv-vanity.com/papers/2102.07350/)
  - [PromptChainer: Chaining Large Language Model Prompts through Visual Programming](https://arxiv.org/abs/2203.06566)
- Collections:
  - [Papers with Code](https://paperswithcode.com/task/prompt-engineering)
  - [Prompt Papers](https://github.com/thunlp/PromptPapers#papers)


## Tools & Libraries

- [OpenAI Playground](https://beta.openai.com/playground)
- [GPTTools](https://gpttools.com/comparisontool)
- [Lexica](https://lexica.art/)
- [Prompt Base](https://promptbase.com/)
- [Playground](https://playgroundai.com/)
- [OpenPrompt](https://github.com/thunlp/OpenPrompt)
- [Visual Prompt Builder](https://tools.saxifrage.xyz/prompt)
- [Prompt Generator for OpenAI's DALL-E 2](http://dalle2-prompt-generator.s3-website-us-west-2.amazonaws.com/)
- [AI Test Kitchen](https://aitestkitchen.withgoogle.com/)
- [Prompt Engine](https://github.com/microsoft/prompt-engine)
- [PromptSource](https://github.com/bigscience-workshop/promptsource)
- [sharegpt](https://sharegpt.com/)

## Datasets
- [PartiPrompts](https://parti.research.google/)
- [Real Toxicity Prompts](https://allenai.org/data/real-toxicity-prompts)
- [P3 - Public Pool of Prompts](https://huggingface.co/datasets/bigscience/P3)
- [WritingPrompts](WritingPrompts)
- [Midjourney Prompts](https://huggingface.co/datasets/succinctly/midjourney-prompts)
- [Awesome ChatGPT Prompts](https://huggingface.co/datasets/fka/awesome-chatgpt-prompts)
- [Stable Diffusion Dataset](https://huggingface.co/datasets/Gustavosta/Stable-Diffusion-Prompts)

## Blog, Guides, Tutorials and Other Readings
- [Prompt Engineering by co:here](https://docs.cohere.ai/docs/prompt-engineering)
- [Prompt Engineering by Microsoft](https://microsoft.github.io/prompt-engineering/)
- [Best practices for prompt engineering with OpenAI API](https://help.openai.com/en/articles/6654000-best-practices-for-prompt-engineering-with-openai-api)
- [Start with an Instruction](https://beta.openai.com/docs/quickstart/start-with-an-instruction)
- [DALLE Prompt Book](https://dallery.gallery/the-dalle-2-prompt-book/)
- [DALL·E 2 Prompt Engineering Guide](https://docs.google.com/document/d/11WlzjBT0xRpQhP9tFMtxzd0q6ANIdHPUBkMV-YB043U/edit#)
- [Prompt injection attacks against GPT-3](https://simonwillison.net/2022/Sep/12/prompt-injection/)
- [Language Models and Prompt Engineering: Systematic Survey of Prompting Methods in NLP](https://youtube.com/watch?v=OsbUfL8w-mo&feature=shares)
- [Learn Prompting](https://learnprompting.org/)
- [3 Principles for prompt engineering with GPT-3](https://www.linkedin.com/pulse/3-principles-prompt-engineering-gpt-3-ben-whately/)
- [Extrapolating to Unnatural Language Processing with GPT-3's In-context Learning: The Good, the Bad, and the Mysterious](http://ai.stanford.edu/blog/in-context-learning/)
- [Prompt Engineering Topic by GitHub](https://github.com/topics/prompt-engineering)
- [Prompt Engineering Template](https://docs.google.com/spreadsheets/d/1-snKDn38-KypoYCk9XLPg799bHcNFSBAVu2HVvFEAkA/edit#gid=0)
- [Awesome ChatGPT Prompts](https://github.com/f/awesome-chatgpt-prompts)
- [Prompt Engineering: From Words to Art](https://www.saxifrage.xyz/post/prompt-engineering)
- [NLP for Text-to-Image Generators: Prompt Analysis](https://heartbeat.comet.ml/nlp-for-text-to-image-generators-prompt-analysis-part-1-5076a44d8365)
- [GPT3 and Prompts: A quick primer](https://buildspace.so/notes/intro-to-gpt3-prompts)
- [Prompt Engineering in GPT-3](https://www.analyticsvidhya.com/blog/2022/05/prompt-engineering-in-gpt-3/)
- [Talking to machines: prompt engineering & injection](https://artifact-research.com/artificial-intelligence/talking-to-machines-prompt-engineering-injection/)
- [A beginner-friendly guide to generative language models - LaMBDA guide](https://aitestkitchen.withgoogle.com/how-lamda-works)
- [Giving GPT-3 a Turing Test](https://lacker.io/ai/2020/07/06/giving-gpt-3-a-turing-test.html)
- [Prompts as Programming by Gwern](https://www.gwern.net/GPT-3#prompts-as-programming)
- [AI Content Generation](https://www.jonstokes.com/p/ai-content-generation-part-1-machine)
- [How to Draw Anything](https://andys.page/posts/how-to-draw/)
- [How to write good prompts](https://andymatuschak.org/prompts/)
- [Prompting Methods with Language Models and Their Applications to Weak Supervision](https://snorkel.ai/prompting-methods-with-language-models-nlp/)
- [How to get images that don't suck](https://www.reddit.com/r/StableDiffusion/comments/x41n87/how_to_get_images_that_dont_suck_a/)
- [Best 100+ Stable Diffusion Prompts](https://mpost.io/best-100-stable-diffusion-prompts-the-most-beautiful-ai-text-to-image-prompts/)
- [Notes for Prompt Engineering by sw-yx](https://github.com/sw-yx/ai-notes)

# Lecture
Full tutorial and lecture coming soon!

---
Feel free to open a PR if you think something is missing here. Always welcome feedback and suggestions.
Update README.md 2022-12-16 16:07:07 +00:00			`# Prompt Engineering Guide`
Create README.md 2022-12-16 16:05:47 +00:00
Update README.md 2022-12-17 20:13:41 +00:00			`This guide contains a non-exhaustive set of learning guides and tools about prompt engineering. It includes several materials, guides, examples, papers, and much more. The repo is intented to be used as a research and educational reference for practitioners and developers.`
Update README.md 2022-12-16 16:07:07 +00:00
Update README.md 2022-12-17 15:55:13 +00:00			`Table of Contents`

			`- [Papers](#papers)`
			`- [Tools & Libraries](#tools--libraries)`
			`- [Datasets](#datasets)`
			`- [Blog, Guides, Tutorials and Other Readings](#blog-guides-tutorials-and-other-readings)`

Update README.md 2022-12-16 16:07:16 +00:00			`## Papers`
Update README.md 2022-12-16 16:07:07 +00:00
Update README.md 2022-12-17 20:05:44 +00:00			`- Surveys / Overviews:`
Update README.md 2022-12-16 22:23:01 +00:00			`- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://arxiv.org/abs/2107.13586)`
			`- [A Taxonomy of Prompt Modifiers for Text-To-Image Generation](https://arxiv.org/abs/2204.13988)`
Update README.md 2022-12-17 20:05:44 +00:00			`- [Emergent Abilities of Large Language Models](https://arxiv.org/abs/2206.07682)`
Update README.md 2022-12-16 22:23:01 +00:00			`- Applications:`
			`- [Legal Prompt Engineering for Multilingual Legal Judgement Prediction](https://arxiv.org/abs/2212.02199)`
			`- [Investigating Prompt Engineering in Diffusion Models](https://arxiv.org/abs/2211.15462)`
			`- [Conversing with Copilot: Exploring Prompt Engineering for Solving CS1 Problems Using Natural Language](https://arxiv.org/abs/2210.15157)`
			`- [Piloting Copilot and Codex: Hot Temperature, Cold Prompts, or Black Magic?](https://arxiv.org/abs/2210.14699)`
			`- Approaches/Techniques:`
Update README.md 2022-12-17 15:48:02 +00:00			`- [Ask Me Anything: A simple strategy for prompting language models](https://paperswithcode.com/paper/ask-me-anything-a-simple-strategy-for)`
Update README.md 2022-12-17 20:05:44 +00:00			`- [Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity](https://arxiv.org/abs/2104.08786)`
Update README.md 2022-12-17 20:10:14 +00:00			`- [AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts](https://arxiv.org/abs/2010.15980)`
Update README.md 2022-12-16 22:23:01 +00:00			`- [Large Language Models Are Human-Level Prompt Engineers](https://sites.google.com/view/automatic-prompt-engineer?pli=1)`
Update README.md 2022-12-19 17:19:53 +00:00			`- [Structured Prompting: Scaling In-Context Learning to 1,000 Examples](https://arxiv.org/abs/2212.06713)`
Update README.md 2022-12-17 20:05:44 +00:00			`- [Reframing Instructional Prompts to GPTk's Language](https://arxiv.org/abs/2109.07830)`
Update README.md 2022-12-16 22:23:01 +00:00			`- [Promptagator: Few-shot Dense Retrieval From 8 Examples](https://arxiv.org/abs/2209.11755)`
Update README.md 2022-12-17 15:48:02 +00:00			`- [Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm](https://www.arxiv-vanity.com/papers/2102.07350/)`
Update README.md 2022-12-17 20:10:14 +00:00			`- [PromptChainer: Chaining Large Language Model Prompts through Visual Programming](https://arxiv.org/abs/2203.06566)`
Update README.md 2022-12-17 15:48:02 +00:00			`- Collections:`
			`- [Papers with Code](https://paperswithcode.com/task/prompt-engineering)`
			`- [Prompt Papers](https://github.com/thunlp/PromptPapers#papers)`
Update README.md 2022-12-16 22:23:01 +00:00
Update README.md 2022-12-16 16:07:38 +00:00
Update README.md 2022-12-17 15:48:02 +00:00			`## Tools & Libraries`
Update README.md 2022-12-16 20:22:23 +00:00
Update README.md 2022-12-16 20:35:27 +00:00			`- [OpenAI Playground](https://beta.openai.com/playground)`
			`- [GPTTools](https://gpttools.com/comparisontool)`
			`- [Lexica](https://lexica.art/)`
Update README.md 2022-12-17 17:16:08 +00:00			`- [Prompt Base](https://promptbase.com/)`
Update README.md 2022-12-16 20:35:27 +00:00			`- [Playground](https://playgroundai.com/)`
Update README.md 2022-12-17 15:48:02 +00:00			`- [OpenPrompt](https://github.com/thunlp/OpenPrompt)`
Update README.md 2022-12-16 20:35:27 +00:00			`- [Visual Prompt Builder](https://tools.saxifrage.xyz/prompt)`
			`- [Prompt Generator for OpenAI's DALL-E 2](http://dalle2-prompt-generator.s3-website-us-west-2.amazonaws.com/)`
			`- [AI Test Kitchen](https://aitestkitchen.withgoogle.com/)`
			`- [Prompt Engine](https://github.com/microsoft/prompt-engine)`
Update README.md 2022-12-17 15:48:02 +00:00			`- [PromptSource](https://github.com/bigscience-workshop/promptsource)`
Update README.md 2022-12-20 12:12:57 +00:00			`- [sharegpt](https://sharegpt.com/)`
Update README.md 2022-12-16 20:22:23 +00:00
Update README.md 2022-12-16 20:35:27 +00:00			`## Datasets`
			`- [PartiPrompts](https://parti.research.google/)`
			`- [Real Toxicity Prompts](https://allenai.org/data/real-toxicity-prompts)`
Update README.md 2022-12-16 20:56:04 +00:00			`- [P3 - Public Pool of Prompts](https://huggingface.co/datasets/bigscience/P3)`
Update README.md 2022-12-16 20:49:57 +00:00			`- [WritingPrompts](WritingPrompts)`
Update README.md 2022-12-16 20:54:42 +00:00			`- [Midjourney Prompts](https://huggingface.co/datasets/succinctly/midjourney-prompts)`
Update README.md 2022-12-16 21:00:33 +00:00			`- [Awesome ChatGPT Prompts](https://huggingface.co/datasets/fka/awesome-chatgpt-prompts)`
			`- [Stable Diffusion Dataset](https://huggingface.co/datasets/Gustavosta/Stable-Diffusion-Prompts)`
Update README.md 2022-12-16 17:38:17 +00:00
Update README.md 2022-12-16 20:54:42 +00:00			`## Blog, Guides, Tutorials and Other Readings`
Update README.md 2022-12-16 17:43:59 +00:00			`- [Prompt Engineering by co:here](https://docs.cohere.ai/docs/prompt-engineering)`
Update README.md 2022-12-16 18:03:44 +00:00			`- [Prompt Engineering by Microsoft](https://microsoft.github.io/prompt-engineering/)`
Update README.md 2022-12-17 19:51:29 +00:00			`- [Best practices for prompt engineering with OpenAI API](https://help.openai.com/en/articles/6654000-best-practices-for-prompt-engineering-with-openai-api)`
Update README.md 2022-12-16 20:22:23 +00:00			`- [Start with an Instruction](https://beta.openai.com/docs/quickstart/start-with-an-instruction)`
Update README.md 2022-12-16 17:46:50 +00:00			`- [DALLE Prompt Book](https://dallery.gallery/the-dalle-2-prompt-book/)`
Update README.md 2022-12-16 17:53:36 +00:00			`- [DALL·E 2 Prompt Engineering Guide](https://docs.google.com/document/d/11WlzjBT0xRpQhP9tFMtxzd0q6ANIdHPUBkMV-YB043U/edit#)`
Update README.md 2022-12-17 20:10:14 +00:00			`- [Prompt injection attacks against GPT-3](https://simonwillison.net/2022/Sep/12/prompt-injection/)`
Update README.md 2022-12-17 15:48:02 +00:00			`- [Language Models and Prompt Engineering: Systematic Survey of Prompting Methods in NLP](https://youtube.com/watch?v=OsbUfL8w-mo&feature=shares)`
Update README.md 2022-12-16 19:56:47 +00:00			`- [Learn Prompting](https://learnprompting.org/)`
Update README.md 2022-12-17 19:51:29 +00:00			`- [3 Principles for prompt engineering with GPT-3](https://www.linkedin.com/pulse/3-principles-prompt-engineering-gpt-3-ben-whately/)`
			`- [Extrapolating to Unnatural Language Processing with GPT-3's In-context Learning: The Good, the Bad, and the Mysterious](http://ai.stanford.edu/blog/in-context-learning/)`
Update README.md 2022-12-16 18:22:49 +00:00			`- [Prompt Engineering Topic by GitHub](https://github.com/topics/prompt-engineering)`
Update README.md 2022-12-16 17:53:36 +00:00			`- [Prompt Engineering Template](https://docs.google.com/spreadsheets/d/1-snKDn38-KypoYCk9XLPg799bHcNFSBAVu2HVvFEAkA/edit#gid=0)`
Update README.md 2022-12-16 18:21:32 +00:00			`- [Awesome ChatGPT Prompts](https://github.com/f/awesome-chatgpt-prompts)`
Update README.md 2022-12-16 17:58:20 +00:00			`- [Prompt Engineering: From Words to Art](https://www.saxifrage.xyz/post/prompt-engineering)`
Update README.md 2022-12-16 20:54:42 +00:00			`- [NLP for Text-to-Image Generators: Prompt Analysis](https://heartbeat.comet.ml/nlp-for-text-to-image-generators-prompt-analysis-part-1-5076a44d8365)`
Update README.md 2022-12-17 19:51:29 +00:00			`- [GPT3 and Prompts: A quick primer](https://buildspace.so/notes/intro-to-gpt3-prompts)`
			`- [Prompt Engineering in GPT-3](https://www.analyticsvidhya.com/blog/2022/05/prompt-engineering-in-gpt-3/)`
Update README.md 2022-12-17 16:58:37 +00:00			`- [Talking to machines: prompt engineering & injection](https://artifact-research.com/artificial-intelligence/talking-to-machines-prompt-engineering-injection/)`
Update README.md 2022-12-16 19:48:23 +00:00			`- [A beginner-friendly guide to generative language models - LaMBDA guide](https://aitestkitchen.withgoogle.com/how-lamda-works)`
			`- [Giving GPT-3 a Turing Test](https://lacker.io/ai/2020/07/06/giving-gpt-3-a-turing-test.html)`
Update README.md 2022-12-16 20:22:23 +00:00			`- [Prompts as Programming by Gwern](https://www.gwern.net/GPT-3#prompts-as-programming)`
			`- [AI Content Generation](https://www.jonstokes.com/p/ai-content-generation-part-1-machine)`
			`- [How to Draw Anything](https://andys.page/posts/how-to-draw/)`
			`- [How to write good prompts](https://andymatuschak.org/prompts/)`
Update README.md 2022-12-16 20:49:57 +00:00			`- [Prompting Methods with Language Models and Their Applications to Weak Supervision](https://snorkel.ai/prompting-methods-with-language-models-nlp/)`
Update README.md 2022-12-16 20:22:23 +00:00			`- [How to get images that don't suck](https://www.reddit.com/r/StableDiffusion/comments/x41n87/how_to_get_images_that_dont_suck_a/)`
			`- [Best 100+ Stable Diffusion Prompts](https://mpost.io/best-100-stable-diffusion-prompts-the-most-beautiful-ai-text-to-image-prompts/)`
			`- [Notes for Prompt Engineering by sw-yx](https://github.com/sw-yx/ai-notes)`
Update README.md 2022-12-17 16:09:53 +00:00
			`# Lecture`
			`Full tutorial and lecture coming soon!`

			`---`
			`Feel free to open a PR if you think something is missing here. Always welcome feedback and suggestions.`