Ice cubes in a frying pan
What it tests: Common-sense physics hidden in an arithmetic word problem.
Prompt length: 427 characters, roughly 110 tokens (estimate).
Prompt
Beth places four whole ice cubes in a frying pan at the start of the first minute, then five at the start of the second minute and some more at the start of the third minute, but none in the fourth minute. If the average number of ice cubes per minute placed in the pan while it was frying a crispy egg was five, how many whole ice cubes can be found in the pan at the end of the third minute? A. 30 B. 0 C. 20 D. 10 E. 11 F. 5
How to grade it
B (0). The pan is frying a crispy egg, so the ice melts: no whole ice cubes are left at the end of the third minute.
Answer check: Official answer key in simple_bench_public.json: B.
Scorecard tick what the model got right: 0/4
Runs in your browser; nothing is saved or sent.
0/4
Where models usually fail
Solving for the cubes added in minute 3 (11) and answering 20 or 11.
Source
- Author
- SimpleBench team
- From
- SimpleBench (public sample)
- Exact file or post
- https://github.com/simple-bench/SimpleBench/blob/fbc2e429085bdedad7d1a236d2bc9bc18c95f16e/simple_bench_public.json
- Licence
- MIT
- How we use it
- Full text (licence allows reuse)
Testing a local model? Check first whether it fits your hardware with Can I run this LLM?