2022-12-16 16:07:07 +00:00
# Prompt Engineering Guide
2022-12-16 16:05:47 +00:00
2023-01-22 22:13:46 +00:00
This guide contains a set of learning guides and tools related to prompt engineering. It includes several materials, guides, examples, papers, and much more. The repo is intended to be used as a research and educational reference for practitioners and developers.
2022-12-16 16:07:07 +00:00
2023-01-11 16:03:40 +00:00
**📣 Full lecture + notebook + exercises on the ~30th of January** (announcement will happen on [Twitter ](https://twitter.com/dair_ai ))
2023-01-09 00:13:20 +00:00
2023-01-07 00:06:30 +00:00
## Table of Contents
2022-12-17 15:55:13 +00:00
- [Papers ](#papers )
- [Tools & Libraries ](#tools--libraries )
- [Datasets ](#datasets )
- [Blog, Guides, Tutorials and Other Readings ](#blog-guides-tutorials-and-other-readings )
2022-12-16 16:07:16 +00:00
## Papers
2022-12-16 16:07:07 +00:00
2022-12-17 20:05:44 +00:00
- Surveys / Overviews:
2022-12-16 22:23:01 +00:00
- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing ](https://arxiv.org/abs/2107.13586 )
- [A Taxonomy of Prompt Modifiers for Text-To-Image Generation ](https://arxiv.org/abs/2204.13988 )
2022-12-17 20:05:44 +00:00
- [Emergent Abilities of Large Language Models ](https://arxiv.org/abs/2206.07682 )
2023-01-15 05:18:12 +00:00
- [A Survey for In-context Learning ](https://arxiv.org/pdf/2301.00234.pdf )
2023-01-22 23:12:42 +00:00
2022-12-16 22:23:01 +00:00
- Applications:
- [Legal Prompt Engineering for Multilingual Legal Judgement Prediction ](https://arxiv.org/abs/2212.02199 )
- [Investigating Prompt Engineering in Diffusion Models ](https://arxiv.org/abs/2211.15462 )
- [Conversing with Copilot: Exploring Prompt Engineering for Solving CS1 Problems Using Natural Language ](https://arxiv.org/abs/2210.15157 )
- [Piloting Copilot and Codex: Hot Temperature, Cold Prompts, or Black Magic? ](https://arxiv.org/abs/2210.14699 )
2023-01-22 23:12:42 +00:00
2022-12-16 22:23:01 +00:00
- Approaches/Techniques:
2022-12-17 15:48:02 +00:00
- [Ask Me Anything: A simple strategy for prompting language models ](https://paperswithcode.com/paper/ask-me-anything-a-simple-strategy-for )
2022-12-17 20:05:44 +00:00
- [Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity ](https://arxiv.org/abs/2104.08786 )
2022-12-17 20:10:14 +00:00
- [AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts ](https://arxiv.org/abs/2010.15980 )
2022-12-16 22:23:01 +00:00
- [Large Language Models Are Human-Level Prompt Engineers ](https://sites.google.com/view/automatic-prompt-engineer?pli=1 )
2023-01-12 03:22:44 +00:00
- [BERTese: Learning to Speak to BERT ](https://aclanthology.org/2021.eacl-main.316/ )
2023-01-21 05:03:12 +00:00
- [Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity ](https://arxiv.org/abs/2104.08786 )
2023-01-05 03:48:37 +00:00
- [Large Language Models are Zero-Shot Reasoners ](https://arxiv.org/abs/2205.11916 )
2023-01-22 23:12:42 +00:00
- [Language Models are Few-Shot Learners ](https://arxiv.org/abs/2005.14165 )
2022-12-19 17:19:53 +00:00
- [Structured Prompting: Scaling In-Context Learning to 1,000 Examples ](https://arxiv.org/abs/2212.06713 )
2023-01-05 03:48:37 +00:00
- [Chain of Thought Prompting Elicits Reasoning in Large Language Models ](https://arxiv.org/abs/2201.11903 )
2023-01-22 23:14:28 +00:00
- [Calibrate Before Use: Improving Few-Shot Performance of Language Models ](https://arxiv.org/abs/2102.09690 )
2022-12-17 20:05:44 +00:00
- [Reframing Instructional Prompts to GPTk's Language ](https://arxiv.org/abs/2109.07830 )
2022-12-16 22:23:01 +00:00
- [Promptagator: Few-shot Dense Retrieval From 8 Examples ](https://arxiv.org/abs/2209.11755 )
2023-01-12 04:14:51 +00:00
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation ](https://arxiv.org/abs/2101.00190 )
2023-01-12 03:42:08 +00:00
- [Making Pre-trained Language Models Better Few-shot Learners ](https://aclanthology.org/2021.acl-long.295/ )
2022-12-17 15:48:02 +00:00
- [Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm ](https://www.arxiv-vanity.com/papers/2102.07350/ )
2023-01-07 00:06:08 +00:00
- [A Taxonomy of Prompt Modifiers for Text-To-Image Generation ](https://arxiv.org/abs/2204.13988 )
2022-12-17 20:10:14 +00:00
- [PromptChainer: Chaining Large Language Model Prompts through Visual Programming ](https://arxiv.org/abs/2203.06566 )
2023-01-12 01:47:14 +00:00
- [How Can We Know What Language Models Know? ](https://direct.mit.edu/tacl/article/doi/10.1162/tacl_a_00324/96460/How-Can-We-Know-What-Language-Models-Know )
2022-12-17 15:48:02 +00:00
- Collections:
- [Papers with Code ](https://paperswithcode.com/task/prompt-engineering )
- [Prompt Papers ](https://github.com/thunlp/PromptPapers#papers )
2022-12-16 22:23:01 +00:00
2022-12-16 16:07:38 +00:00
2022-12-17 15:48:02 +00:00
## Tools & Libraries
2022-12-16 20:22:23 +00:00
2022-12-16 20:35:27 +00:00
- [OpenAI Playground ](https://beta.openai.com/playground )
- [GPTTools ](https://gpttools.com/comparisontool )
2023-01-20 00:15:08 +00:00
- [LangChain ](https://github.com/hwchase17/langchain )
2023-01-05 18:15:14 +00:00
- [EveryPrompt ](https://www.everyprompt.com/ )
2023-01-05 18:17:43 +00:00
- [DUST ](https://dust.tt/ )
2023-01-22 22:45:39 +00:00
- [Dyno ](https://trydyno.com/ )
2023-01-05 18:22:42 +00:00
- [Prompts.ai ](https://github.com/sevazhidkov/prompts-ai )
2022-12-16 20:35:27 +00:00
- [Lexica ](https://lexica.art/ )
2023-01-10 04:13:22 +00:00
- [Scale SpellBook ](https://scale.com/spellbook )
2023-01-05 03:50:47 +00:00
- [Interactive Composition Explorer ](https://github.com/oughtinc/ice )
2023-01-17 18:32:37 +00:00
- [LearnGPT ](https://www.learngpt.com/ )
2023-01-07 02:19:46 +00:00
- [Promptable ](https://promptable.ai/ )
2023-01-05 18:09:37 +00:00
- [GPT Index ](https://github.com/jerryjliu/gpt_index )
2022-12-17 17:16:08 +00:00
- [Prompt Base ](https://promptbase.com/ )
2022-12-16 20:35:27 +00:00
- [Playground ](https://playgroundai.com/ )
2022-12-17 15:48:02 +00:00
- [OpenPrompt ](https://github.com/thunlp/OpenPrompt )
2022-12-16 20:35:27 +00:00
- [Visual Prompt Builder ](https://tools.saxifrage.xyz/prompt )
- [Prompt Generator for OpenAI's DALL-E 2 ](http://dalle2-prompt-generator.s3-website-us-west-2.amazonaws.com/ )
- [AI Test Kitchen ](https://aitestkitchen.withgoogle.com/ )
2023-01-05 18:03:55 +00:00
- [betterprompt ](https://github.com/krrishdholakia/betterprompt )
2022-12-16 20:35:27 +00:00
- [Prompt Engine ](https://github.com/microsoft/prompt-engine )
2022-12-17 15:48:02 +00:00
- [PromptSource ](https://github.com/bigscience-workshop/promptsource )
2022-12-20 12:12:57 +00:00
- [sharegpt ](https://sharegpt.com/ )
2023-01-05 17:32:06 +00:00
- [DreamStudio ](https://beta.dreamstudio.ai/ )
2022-12-16 20:22:23 +00:00
2022-12-16 20:35:27 +00:00
## Datasets
- [PartiPrompts ](https://parti.research.google/ )
- [Real Toxicity Prompts ](https://allenai.org/data/real-toxicity-prompts )
2022-12-21 19:44:18 +00:00
- [DiffusionDB ](https://github.com/poloclub/diffusiondb )
2022-12-16 20:56:04 +00:00
- [P3 - Public Pool of Prompts ](https://huggingface.co/datasets/bigscience/P3 )
2022-12-16 20:49:57 +00:00
- [WritingPrompts ](WritingPrompts )
2022-12-16 20:54:42 +00:00
- [Midjourney Prompts ](https://huggingface.co/datasets/succinctly/midjourney-prompts )
2022-12-16 21:00:33 +00:00
- [Awesome ChatGPT Prompts ](https://huggingface.co/datasets/fka/awesome-chatgpt-prompts )
- [Stable Diffusion Dataset ](https://huggingface.co/datasets/Gustavosta/Stable-Diffusion-Prompts )
2022-12-16 17:38:17 +00:00
2022-12-16 20:54:42 +00:00
## Blog, Guides, Tutorials and Other Readings
2023-01-07 04:15:30 +00:00
- [Pretrain, Prompt, Predict - A New Paradigm for NLP ](http://pretrain.nlpedia.ai/ )
2023-01-05 03:36:09 +00:00
- [Prompt Engineering 101 - Introduction and resources ](https://www.linkedin.com/pulse/prompt-engineering-101-introduction-resources-amatriain/ )
2022-12-16 17:43:59 +00:00
- [Prompt Engineering by co:here ](https://docs.cohere.ai/docs/prompt-engineering )
2022-12-16 18:03:44 +00:00
- [Prompt Engineering by Microsoft ](https://microsoft.github.io/prompt-engineering/ )
2022-12-17 19:51:29 +00:00
- [Best practices for prompt engineering with OpenAI API ](https://help.openai.com/en/articles/6654000-best-practices-for-prompt-engineering-with-openai-api )
2022-12-16 20:22:23 +00:00
- [Start with an Instruction ](https://beta.openai.com/docs/quickstart/start-with-an-instruction )
2023-01-05 03:44:33 +00:00
- [CMU Advanced NLP 2022: Prompting ](https://youtube.com/watch?v=5ef83Wljm-M&feature=shares )
- [Prompt Engineering 101: Autocomplete, Zero-shot, One-shot, and Few-shot prompting ](https://youtube.com/watch?v=v2gD8BHOaX4&feature=shares )
2022-12-23 13:35:22 +00:00
- [Prompt engineering davinci-003 on our own docs for automated support (Part I) ](https://www.patterns.app/blog/2022/12/21/finetune-llm-tech-support/ )
2022-12-16 17:46:50 +00:00
- [DALLE Prompt Book ](https://dallery.gallery/the-dalle-2-prompt-book/ )
2022-12-16 17:53:36 +00:00
- [DALL·E 2 Prompt Engineering Guide ](https://docs.google.com/document/d/11WlzjBT0xRpQhP9tFMtxzd0q6ANIdHPUBkMV-YB043U/edit# )
2022-12-17 20:10:14 +00:00
- [Prompt injection attacks against GPT-3 ](https://simonwillison.net/2022/Sep/12/prompt-injection/ )
2022-12-17 15:48:02 +00:00
- [Language Models and Prompt Engineering: Systematic Survey of Prompting Methods in NLP ](https://youtube.com/watch?v=OsbUfL8w-mo&feature=shares )
2022-12-22 02:12:45 +00:00
- [A Complete Introduction to Prompt Engineering for Large Language Models ](https://www.mihaileric.com/posts/a-complete-introduction-to-prompt-engineering/ )
2022-12-16 19:56:47 +00:00
- [Learn Prompting ](https://learnprompting.org/ )
2022-12-17 19:51:29 +00:00
- [3 Principles for prompt engineering with GPT-3 ](https://www.linkedin.com/pulse/3-principles-prompt-engineering-gpt-3-ben-whately/ )
- [Extrapolating to Unnatural Language Processing with GPT-3's In-context Learning: The Good, the Bad, and the Mysterious ](http://ai.stanford.edu/blog/in-context-learning/ )
2022-12-16 18:22:49 +00:00
- [Prompt Engineering Topic by GitHub ](https://github.com/topics/prompt-engineering )
2022-12-16 17:53:36 +00:00
- [Prompt Engineering Template ](https://docs.google.com/spreadsheets/d/1-snKDn38-KypoYCk9XLPg799bHcNFSBAVu2HVvFEAkA/edit#gid=0 )
2022-12-16 18:21:32 +00:00
- [Awesome ChatGPT Prompts ](https://github.com/f/awesome-chatgpt-prompts )
2022-12-16 17:58:20 +00:00
- [Prompt Engineering: From Words to Art ](https://www.saxifrage.xyz/post/prompt-engineering )
2022-12-16 20:54:42 +00:00
- [NLP for Text-to-Image Generators: Prompt Analysis ](https://heartbeat.comet.ml/nlp-for-text-to-image-generators-prompt-analysis-part-1-5076a44d8365 )
2023-01-20 18:23:20 +00:00
- [Mysteries of mode collapse ](https://www.lesswrong.com/posts/t9svvNPNmFf5Qa3TA/mysteries-of-mode-collapse )
2022-12-17 19:51:29 +00:00
- [GPT3 and Prompts: A quick primer ](https://buildspace.so/notes/intro-to-gpt3-prompts )
- [Prompt Engineering in GPT-3 ](https://www.analyticsvidhya.com/blog/2022/05/prompt-engineering-in-gpt-3/ )
2022-12-17 16:58:37 +00:00
- [Talking to machines: prompt engineering & injection ](https://artifact-research.com/artificial-intelligence/talking-to-machines-prompt-engineering-injection/ )
2022-12-16 19:48:23 +00:00
- [A beginner-friendly guide to generative language models - LaMBDA guide ](https://aitestkitchen.withgoogle.com/how-lamda-works )
- [Giving GPT-3 a Turing Test ](https://lacker.io/ai/2020/07/06/giving-gpt-3-a-turing-test.html )
2022-12-16 20:22:23 +00:00
- [Prompts as Programming by Gwern ](https://www.gwern.net/GPT-3#prompts-as-programming )
- [AI Content Generation ](https://www.jonstokes.com/p/ai-content-generation-part-1-machine )
- [How to Draw Anything ](https://andys.page/posts/how-to-draw/ )
- [How to write good prompts ](https://andymatuschak.org/prompts/ )
2022-12-16 20:49:57 +00:00
- [Prompting Methods with Language Models and Their Applications to Weak Supervision ](https://snorkel.ai/prompting-methods-with-language-models-nlp/ )
2022-12-16 20:22:23 +00:00
- [How to get images that don't suck ](https://www.reddit.com/r/StableDiffusion/comments/x41n87/how_to_get_images_that_dont_suck_a/ )
- [Best 100+ Stable Diffusion Prompts ](https://mpost.io/best-100-stable-diffusion-prompts-the-most-beautiful-ai-text-to-image-prompts/ )
- [Notes for Prompt Engineering by sw-yx ](https://github.com/sw-yx/ai-notes )
2023-01-23 05:54:21 +00:00
- [A Generic Framework for ChatGPT Prompt Engineering ](https://medium.com/@thorbjoern.heise/a-generic-framework-for-chatgpt-prompt-engineering-7097f6513a0b )
2022-12-17 16:09:53 +00:00
2022-12-30 05:17:52 +00:00
# Lecture + Tutorial
2023-01-12 03:23:18 +00:00
Full tutorial and lecture coming soon!
2022-12-17 16:09:53 +00:00
---
Feel free to open a PR if you think something is missing here. Always welcome feedback and suggestions.
2023-01-17 23:29:47 +00:00
Join our [Discord ](https://discord.gg/SKgkVT8BGJ )