How we pick prompts

The bar is set by Arnie936/llm-prompting-tests, the best collection of test prompts we found. What makes those prompts good became our checklist. A prompt gets in only if it meets most of these:

Hard to fake. The result has to work, not just look plausible: the simulation runs, the JSON loads, the count is exact. A model can't talk its way through.
Concrete, checkable requirements. Numbers, timings, file names, named classes, explicit acceptance criteria. Vague prompts ('make a cool game') can't be graded.
Several systems that must fit together. Resources, timing and agents in the bee hive; physics, AI and HUD in a racing game. Small models usually get one part right and break the rest.
Constraints that force real decisions. One file, no external assets, a banned library, a dry-run switch that must be on by default, no downloads.
Room for honesty. Self-checks of invariants instead of 'it exists', a note on what was left out, a summary of assumptions. Strong models report gaps; weak ones hide them.
It separates models. Strong models pass, weaker ones fail in recognisable ways, and we write those failure modes down.
Real work. ComfyUI and n8n folders, apps people would actually use, not only puzzles.

Short quick checks follow the same idea: an unambiguous answer, a known trap (an altered classic puzzle, a distracting detail, a character-level constraint) and an answer we verified ourselves.

How answers are verified

We don't invent expected answers. Arithmetic and probabilities were recomputed with exact fractions, logic puzzles brute-forced, the SimpleBench and BFCL answers come from their official keys, and IFEval constraints are machine-checkable (there's a checker on those pages). For build tests the grading criteria come from the prompt's own requirements or the source's published rubric.

Licences

Every entry names its author, links the exact upstream file pinned to a commit, and records the licence. Full text only where the licence allows it; otherwise a short excerpt, our summary and a link. Details on the sources page. Each prompt keeps its own licence (see source.license). Our own text (summaries, grading notes) is CC BY 4.0; our translations of CC BY-NC-SA prompts are CC BY-NC-SA 4.0.

Running tests fairly

Fresh chat, no system prompt, the same settings for every model (temperature, context length, quantisation).
Judge the first answer. KCORES runs each prompt 3 times and keeps the best run; that's fair too if you do it for every model.
Give build tests enough output tokens and context: long specs get cut off on small context settings in Ollama or LM Studio.
Public prompts may be in training data. Change a number or a name and see whether the answer changes with it.
Check whether a model fits your GPU or Mac first with Can I run this LLM?

Data

All 62 entries as JSON: /api/prompts.json (excerpt-only entries contain the excerpt and the link, not the full text). Also /llms.txt. This site sets no cookies and uses cookieless Vercel Web Analytics; the answer checker runs entirely in your browser.