50 machines, 5 minutes, 5 widgets
What it tests: Rate reasoning on a variant of the cognitive reflection test.
Prompt length: 109 characters, roughly 30 tokens (estimate). · Evaluation results for many models ↗
Prompt
If it takes 50 machines 5 minutes to make 5 widgets, how long would it take 100 machines to make 100 widgets?
How to grade it
50 minutes. Each machine makes 5 / (50 × 5) = 0.02 widgets per minute, so 100 machines make 2 per minute and need 50 minutes for 100 widgets.
Answer check: Rate arithmetic with exact fractions.
Scorecard tick what the model got right: 0/4
Runs in your browser; nothing is saved or sent.
0/4
Where models usually fail
Answering 5 minutes, the answer to the original '5 machines, 5 widgets' version.
Source
- Author
- GitHub @cpldcpu and contributors
- From
- Misguided Attention
- Exact file or post
- https://github.com/cpldcpu/MisguidedAttention/blob/bbae011e1fb5ce239373cb2f381787baec0dd236/README.md
- Licence
- CC0 1.0
- How we use it
- Full text (licence allows reuse)
Testing a local model? Check first whether it fits your hardware with Can I run this LLM?