5-minute test pack
10 prompts for a first look at a new model in Ollama, LM Studio or any chat app. Send each one in a fresh chat, without a system prompt, and compare the answer with the expected result. Most take seconds; the last two produce code you run.
| Prompt | Tests | Expected | |
|---|---|---|---|
| 1. 1 kilogram of feathers or 1 pound of steel Reasoning traps | Units over the memorised 'they weigh the same'. | The kilogram of feathers: 1 kg ≈ 2.2 lb, so it's more than twice as heavy as 1 lb of steel. | |
| 2. Inverse Monty Hall: two cars, one donkey Reasoning traps | Whether the model recomputes the probabilities when the classic setup is inverted. | Keep your original door. Your first pick is a car with probability 2/3; if you picked a car, the other unopened door hides the donkey, so switching wins only 1/3 of the time (standard assumption: the host knowingly reveals a car). | |
| 3. 50 machines, 5 minutes, 5 widgets Reasoning traps | Rate reasoning on a variant of the cognitive reflection test. | 50 minutes. Each machine makes 5 / (50 × 5) = 0.02 widgets per minute, so 100 machines make 2 per minute and need 50 minutes for 100 widgets. | |
| 4. Measure 4 litres with a 6 l and a 12 l jug Reasoning traps | Checking feasibility before searching: every measurable amount is a multiple of gcd(6, 12) = 6. | It can't be done with these two jugs alone, because every measurable amount is a multiple of 6 litres. Partial credit if the model says so after some trial and error; none for a fake sequence of pours. | |
| 5. Duct-taped sandwich stack Common sense | Physical intuition: tape on the top surface lifts only the top slice of bread. | A (4 whole sandwiches in Room A, 0 in Room B). The tape lifts only the top slice of bread off the uppermost sandwich; the rest of the stack stays in Room A, and no whole sandwich reaches Room B. | |
| 6. No commas, no letter c, at least 250 words Instruction following | Holding a character-level constraint over a long answer; 'c' is everywhere in research vocabulary. | Machine-checkable: zero commas, zero letters 'c' (either case), at least 250 words. Use the checker on this page. | |
| 7. Parallel tool calls with a distractor function Tool use & JSON | Choosing the right two of three tools, mapping 'perimeter of a circle' to circumference, passing radius 5 and diameter 10 correctly, and emitting valid JSON only. | Exactly two calls in valid JSON: circle.calculate_area with {"radius": 5} and circle.calculate_circumference with {"diameter": 10}. BFCL ground truth: [{"circle.calculate_area": {"radius": [5]}}, {"circle.calculate_circumference": {"diameter": [10]}}]. The rectangle function must not be called. (For reference, area ≈ 78.54 and circumference ≈ 31.42.) | |
| 8. Don't call the only tool when it doesn't fit Tool use & JSON | Tool-use restraint: the only available function computes BMI, but the user wants a triangle's area. | No function call: [] plus a short explanation that the BMI tool can't compute a triangle's area. Mentioning the area itself (0.5 × 10 × 5 = 25 m²) is fine. BFCL scores any call here as a failure. | |
| 9. SVG of a pelican riding a bicycle Creative coding & SVG | Spatial reasoning without seeing the output: the model must place a recognisable pelican on a bicycle with plausible contact points (seat, pedals, handlebars) in raw SVG coordinates. | Render the SVG. Good: a recognisable pelican (long beak, throat pouch), a bicycle with two similar wheels, a frame joining them and handlebars, and the bird actually sitting on it with its feet near the pedals. It's an informal visual check, so compare runs side by side. | |
| 10. Ball bouncing inside a spinning hexagon Simulations & physics | The short original of the heptagon family: collisions with moving walls (the wall's velocity at the contact point must enter the bounce), plus gravity and friction. | Runs (the model should state the library, usually pygame); the hexagon rotates; the ball falls under gravity, never leaves the hexagon, and gains or loses speed realistically when a moving wall hits it; friction slows it along the walls. |
Scoring tip: count clear passes out of 10 and keep the same settings (temperature, context length, quantisation) when you compare models. Run the pack twice: a model that passes only sometimes is guessing.