Measure 3 litres with a 1 l and a 2 l jug
What it tests: Avoiding list-writing autopilot on a trivial task.
Prompt length: 66 characters, roughly 20 tokens (estimate). · Evaluation results for many models ↗
Prompt
I have a 1- and a 2-liter jug. I want to measure exactly 3 liters.
How to grade it
Fill both jugs: 1 + 2 = 3 litres.
Answer check: 1 + 2 = 3.
Scorecard tick what the model got right: 0/4
Runs in your browser; nothing is saved or sent.
0/4
Where models usually fail
Multi-step pouring procedures, or claiming it's impossible because neither jug holds 3 litres.
Source
- Author
- GitHub @cpldcpu and contributors
- From
- Misguided Attention
- Exact file or post
- https://github.com/cpldcpu/MisguidedAttention/blob/bbae011e1fb5ce239373cb2f381787baec0dd236/README.md
- Licence
- CC0 1.0
- How we use it
- Full text (licence allows reuse)
Testing a local model? Check first whether it fits your hardware with Can I run this LLM?