Prompt Engineering

Reframing Instructional Prompts to GPTk's Language

ACL 2022 Findings paper showing that manually reframing instructional prompts — decomposing tasks, itemizing steps, adding positive examples — yields 6–12% performance gains on GPT-2 and GPT-3. The key insight is that models respond better to concrete, step-by-step instructions than to long abstract descriptions.

Autoresearch Skill: autonomous skill optimization via eval loops

A Claude Code skill that autonomously optimizes other skills by running eval loops — binary scoring, targeted mutation, keep/discard decisions. Andrej Karpathy's training loop methodology applied to prompt engineering.

VS Code PR: LSP linting for AI prompt files

VS Code PR that adds LSP-based real-time linting for AI prompt files — flags weak instructions, ambiguity, XML structure issues, redundancy, template variable errors, and token budget problems. Like ESLint for prompts.

Cohere Prompt Tuner: Automated Prompt Optimization

Cohere's Prompt Tuner automatically improves prompts for their models by testing variations and selecting what performs best — prompt optimization as a first-class product feature rather than a manual art.

Making Large Language Models Work For You

Simon Willison's WordCamp 2023 keynote on making LLMs work for you — a practical, grounded introduction covering what LLMs actually are, their real limitations, and the specific use cases where they're genuinely useful today. One of the better practitioner introductions from 2023.

Experimental Prompting DSL: Origin of Instructor

Jason Liu's experimental prompting DSL for structured LLM outputs via OpenAI function calling — an early version of what became the Instructor library. Shows the origin of the pydantic-based approach to reliable structured extraction from LLMs.

Beware of Unreliable Data in Model Evaluation

Cleanlab's case study showing that noisy test data leads to suboptimal prompt selection for LLMs — you can choose the wrong prompt because your evaluation data contains labeling errors. A practical warning about data quality in LLM evaluation pipelines.

Prompt Engineering: Lilian Weng's Comprehensive Survey

Lilian Weng's canonical reference on prompt engineering techniques — zero-shot, few-shot, chain-of-thought, self-consistency, tree-of-thoughts, and more — grounded in research papers. Still the most comprehensive single-author survey of the space.

From Prompt Alchemy to Prompt Engineering: Analytic Augmentation

An essay arguing for 'analytic augmentation' — using structured logical reasoning and philosophical method to improve LLM prompts, moving from intuitive prompt tweaking (alchemy) to principled prompt design (engineering). Connects formal reasoning frameworks to practical prompt construction.

Promptable.js: TypeScript Library for LLM Apps

Promptable.js is an early TypeScript library for building LLM apps with Prompt, Search, Chain, and Trace primitives — a TypeScript analogue to LangChain that appeared in February 2023 before LangChain's own TypeScript support matured. Historically significant as one of the first LLM frameworks for TypeScript developers.

Prompt Parrot — Replicate

Prompt Parrot is a Replicate-hosted model that generates creative variations of text-to-image prompts. A simple tool for exploring the prompt space around a seed idea when using Stable Diffusion or similar generators.

Automatic Prompt Engineer (APE)

Automatic Prompt Engineer (APE) is a research implementation that automatically generates and selects optimal prompts for LLMs using a generate-then-score approach. An early automated prompt optimization method predating DSPy.

Stable Diffusion Parameters Guide

A practical overview of the key parameters for controlling AI image generation in Stable Diffusion — steps, CFG scale, sampler, seed, and dimensions. A useful reference from the early days when these knobs were being collectively figured out.

A Complete Introduction to Prompt Engineering

Mihail Eric's comprehensive introduction to prompt engineering for LLMs — covering few-shot prompting, chain-of-thought, instruction tuning, and evaluation. Published in late 2022 when prompt engineering was emerging as a recognized discipline.

Manifest: Prompt Programming with Foundation Models

Manifest is a Python library from Stanford's HazyResearch lab for prompt programming with foundation models — a unified interface across providers with caching, batching, and structured output support. An early formalization of LLM programming patterns before LangChain dominated.

Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity

Interactive and Visual Prompt Engineering for Ad-hoc Task Adaptation with Large Language Models

A research paper presenting interactive and visual tools for prompt engineering that let users iteratively adapt large language models to ad-hoc tasks without fine-tuning. It addresses the core usability gap in prompt engineering: the feedback loop between prompt edits and output quality is opaque without tooling that makes the connection visible.

Large Language Models Are Human-Level Prompt Engineers

Large Language Models Are Human-Level Prompt Engineers (APE) introduces an automated method for generating and selecting optimal prompts using LLMs themselves, matching or beating human-crafted instructions on 19 of 24 NLP tasks. It reframes prompt engineering as a program search problem, making manual iteration unnecessary.

MagicPrompt-Stable-Diffusion

MagicPrompt-Stable-Diffusion is a GPT-2-based model fine-tuned to generate effective prompts for Stable Diffusion. It solves the prompt engineering problem for image generation: given a simple idea, it produces elaborate prompt text that reliably produces better images.

CLIP Interrogator

CLIP Interrogator by pharmapsychotic is a Google Colab tool that reverse-engineers what prompt would produce a given image — using CLIP to describe an image in terms that Stable Diffusion understands. The go-to tool in 2022 for figuring out how to replicate an image style.

Krea.ai: AI Prompt Explorer and Gallery

Krea.ai is a prompt exploration and gallery tool for Stable Diffusion — browse millions of AI-generated images with their prompts, and use those prompts as starting points for your own generations. The community discovery layer that was missing from Stable Diffusion's launch.

Lexica: Stable Diffusion Search Engine

Lexica is a search engine for Stable Diffusion images and the prompts that generated them. It became the go-to reference for empirical prompt knowledge — what prompts produce what visual aesthetics — and later added its own image generation feature.

GPT-3 Complete-Program Synthesis (Riley Goodside)

Riley Goodside demonstrates complete-program synthesis using GPT-3 via the 'format trick' — combining instruction prompts with contextually informative templates to generate entire programs deterministically. An early exploration of LLMs for structured code generation beyond autocomplete.

GPT-3 Long Instruction Following (Riley Goodside)

Riley Goodside demonstrates GPT-3 following a nearly 2,000-character instruction prompt precisely. An early illustration that LLMs can be reliable instruction followers with detailed context, countering the intuition that prompts should be kept short.

The DALL-E 2 Prompt Book

Unofficial visual reference guide by Guy Parsons (dallery.gallery) covering how to prompt DALL-E 2 across photography, illustration styles, art history movements, 3D artwork, and editing techniques. A practical taxonomy of the prompt space for the first widely-accessible diffusion image model.

Prompts.ai: Advanced GPT-3 Playground

Prompts.ai was a community-built advanced GPT-3 playground that extended OpenAI's official interface with chaining, templates, and variable injection — an early tool for the prompt engineering community before ChatGPT made LLMs mainstream.

Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm

Reynolds and McDonell (2021) argue that prompting large language models is better understood as programming than as few-shot learning — the few examples in a prompt aren't training data but rather code that specifies the desired computation. This reframing opens up principled prompt design strategies that beat naive templating.

PromptSource: An Integrated Development Environment and Repository for Natural Language Prompts

PromptSource is an IDE and community repository for creating, sharing, and iterating on natural language prompts that map dataset examples to input-output pairs for language model training and evaluation. With over 2,000 prompts for ~170 datasets, it provided the infrastructure behind the T0 family of models and multitask prompted training research.