Prompt Engineering Guide

This guide contains a set of recent papers, learning guides, and tools related to prompt engineering. The repo is intended as a research and educational reference for practitioners and developers.

Announcements:

📣 Full lecture + notebook + exercises on the ~15th of Feb (announcement will happen on Twitter)
📣 Examples and explainers coming soon! (reach out if you want to contribute)
Join our Discord to discuss more about prompt engineering

Guides
Papers
Tools & Libraries
Datasets
Blog, Guides, Tutorials and Other Readings

Guides

The following are a set of guides on prompt engineering developed by us. Guides are work in progress.

Papers

(Sorted by Release Date)

Surveys / Overviews:
- A Survey for In-context Learning (Dec 2022)
- Towards Reasoning in Large Language Models: A Survey (Dec 2022)
- Emergent Abilities of Large Language Models (Jun 2022)
- A Taxonomy of Prompt Modifiers for Text-To-Image Generation (Apr 2022)
- Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing (Jul 2021)
Approaches/Techniques:
- Multimodal Chain-of-Thought Reasoning in Language Models (Feb 2023)
- Large Language Models Can Be Easily Distracted by Irrelevant Context (Feb 2023)
- Synthetic Prompting: Generating Chain-of-Thought Demonstrations for Large Language Models (Feb 2023)
- Progressive Prompts: Continual Learning for Language Models (Jan 2023)
- Batch Prompting: Efficient Inference with LLM APIs (Jan 2023)
- On Second Thought, Let's Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoning (Dec 2022)
- Constitutional AI: Harmlessness from AI Feedback (Dec 2022)
- Successive Prompting for Decomposing Complex Questions (Dec 2022)
- Discovering Language Model Behaviors with Model-Written Evaluations (Dec 2022)
- Structured Prompting: Scaling In-Context Learning to 1,000 Examples (Dec 2022)
- PAL: Program-aided Language Models (Nov 2022)
- Large Language Models Are Human-Level Prompt Engineers (Nov 2022)
- Ignore Previous Prompt: Attack Techniques For Language Models (Nov 2022)
- Machine Generated Text: A Comprehensive Survey of Threat Models and Detection Methods (Nov 2022)
- Teaching Algorithmic Reasoning via In-context Learning (Nov 2022)
- Enhancing Self-Consistency and Performance of Pre-Trained Language Models through Natural Language Inference (Nov 2022)
- Ask Me Anything: A simple strategy for prompting language models (Oct 2022)
- ReAct: Synergizing Reasoning and Acting in Language Models (Oct 2022)
- Prompting GPT-3 To Be Reliable (Oct 2022)
- Decomposed Prompting: A Modular Approach for Solving Complex Tasks (Oct 2022)
- Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-Thought (Oct 2022)
- Evaluating the Susceptibility of Pre-Trained Language Models via Handcrafted Adversarial Examples (Sep 2022)
- Promptagator: Few-shot Dense Retrieval From 8 Examples (Sep 2022)
- On the Advance of Making Language Models Better Reasoners (June 2022)
- Large Language Models are Zero-Shot Reasoners (May 2022)
- MRKL Systems: A modular, neuro-symbolic architecture that combines large language models, external knowledge sources and discrete reasoning (May 2022)
- Toxicity Detection with Generative Prompt-based Inference (May 2022)
- The Unreliability of Explanations in Few-shot Prompting for Textual Reasoning (May 2022)
- A Taxonomy of Prompt Modifiers for Text-To-Image Generation (Apr 2022)
- PromptChainer: Chaining Large Language Model Prompts through Visual Programming (Mar 2022)
- Self-Consistency Improves Chain of Thought Reasoning in Language Models (March 2022)
- Rethinking the Role of Demonstrations: What Makes In-Context Learning Work? (Feb 2022)
- Chain of Thought Prompting Elicits Reasoning in Large Language Models (Jan 2022)
- Show Your Work: Scratchpads for Intermediate Computation with Language Models (Nov 2021)
- Generated Knowledge Prompting for Commonsense Reasoning (Oct 2021)
- Reframing Instructional Prompts to GPTk's Language (Sep 2021)
- Design Guidelines for Prompt Engineering Text-to-Image Generative Models (Sep 2021)
- Making Pre-trained Language Models Better Few-shot Learners (Aug 2021)
- Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity (April 2021)
- BERTese: Learning to Speak to BERT (April 2021)
- The Power of Scale for Parameter-Efficient Prompt Tuning (April 2021)
- Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm (Feb 2021)
- Calibrate Before Use: Improving Few-Shot Performance of Language Models (Feb 2021)
- Prefix-Tuning: Optimizing Continuous Prompts for Generation (Jan 2021)
- AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts (Oct 2020)
- Language Models are Few-Shot Learners (May 2020)
- How Can We Know What Language Models Know? (July 2020)
Applications:
- Commonsense-Aware Prompting for Controllable Empathetic Dialogue Generation (Feb 2023)
- Legal Prompt Engineering for Multilingual Legal Judgement Prediction (Dec 2022)
- Investigating Prompt Engineering in Diffusion Models (Nov 2022)
- Conversing with Copilot: Exploring Prompt Engineering for Solving CS1 Problems Using Natural Language (Oct 2022)
- Piloting Copilot and Codex: Hot Temperature, Cold Prompts, or Black Magic? (Oct 2022)
- Plot Writing From Scratch Pre-Trained Language Models (July 2022)
Collections:

Tools & Libraries

(Sorted by Name)

Datasets

(Sorted by Name)

Blog, Guides, Tutorials and Other Readings

(Sorted by Name)

Lecture + Tutorial

Full tutorial and lecture coming soon!

Feel free to open a PR if you think something is missing here. Always welcome feedback and suggestions.

Join our Discord

16 KiB Raw Blame History

Prompt Engineering Guide

Table of Contents

Guides

Papers

(Sorted by Release Date)

Tools & Libraries

(Sorted by Name)

Datasets

(Sorted by Name)

Blog, Guides, Tutorials and Other Readings

(Sorted by Name)

Lecture + Tutorial

16 KiB

Raw Blame History