LLM Test Prompts

All prompts

Reasoning traps

Altered classic puzzles that catch memorised answers. 16 prompts. Search and filter these.

PromptDifficultyLangWhat it testsSource
Trolley problem with five dead peopleeasyENReading versus pattern-matching: the five people on the main track are already dead.Misguided Attention
Inverse Monty Hall: two cars, one donkeymediumENWhether the model recomputes the probabilities when the classic setup is inverted.Misguided Attention
Schrödinger's cat that is already deadeasyENReading the first sentence: the cat is dead when it goes into the box.Misguided Attention
River crossing where the boat fits botheasyENResisting a memorised multi-trip puzzle solution when one trip is enough.Misguided Attention
Measure 4 litres with a 6 l and a 12 l jugmediumENChecking feasibility before searching: every measurable amount is a multiple of gcd(6, 12) = 6.Misguided Attention
Measure 3 litres with a 1 l and a 2 l jugeasyENAvoiding list-writing autopilot on a trivial task.Misguided Attention
1 kilogram of feathers or 1 pound of steeleasyENUnits over the memorised 'they weigh the same'.Misguided Attention
13 coins, one fake, and a digital scalemediumENAdapting to the tool actually given: a digital scale shows weights; it isn't a two-pan balance.Misguided Attention
Measure 60 minutes with two 60-minute ropeseasyENSpotting that the target equals one rope's burn time.Misguided Attention
Measure 20 minutes with two 60-minute ropeshardENAdmitting when the standard tricks can't reach the target.Misguided Attention
Birthday problem: at least two do NOT sharemediumENReading the negation instead of solving the famous version.Misguided Attention
50 machines, 5 minutes, 5 widgetsmediumENRate reasoning on a variant of the cognitive reflection test.Misguided Attention
Stolen bat and ball: what did I pay?easyENNoticing the word 'stole'.Misguided Attention
Lotus flowers growing by two per daymediumENLinear versus exponential: the classic lily-pad answer (day 39) assumes doubling.Misguided Attention
Knights and knaves with no consistent answerhardENDetecting a contradiction instead of forcing an assignment.Misguided Attention
A 7-litre bucket with no bottom and a sealed topeasyENSpatial insight: turn it over.Misguided Attention

Other categories: Simulations & physics · Games · Apps · Creative coding & SVG · Agent tasks · Common sense · Instruction following · Tool use & JSON