Inverse Monty Hall: two cars, one donkey
What it tests: Whether the model recomputes the probabilities when the classic setup is inverted.
Prompt length: 355 characters, roughly 90 tokens (estimate). · Evaluation results for many models ↗
Prompt
You're on a game show and are presented with three doors. Behind one is a donkey, and behind the other two are luxury cars. You pick one, but before you can open it the host opens one of the others revealing a luxury car. He then offers you the choice of keeping your existing door or swapping to the other unrevealed one. What should you do to win a car?
How to grade it
Keep your original door. Your first pick is a car with probability 2/3; if you picked a car, the other unopened door hides the donkey, so switching wins only 1/3 of the time (standard assumption: the host knowingly reveals a car).
Answer check: Enumerated all three donkey positions: staying wins in 2 of 3.
Scorecard tick what the model got right: 0/4
Runs in your browser; nothing is saved or sent.
0/4
Where models usually fail
Reciting 'always switch, 2/3' from the classic problem, which here maximises the chance of getting the donkey.
Source
- Author
- GitHub @cpldcpu and contributors
- From
- Misguided Attention
- Exact file or post
- https://github.com/cpldcpu/MisguidedAttention/blob/bbae011e1fb5ce239373cb2f381787baec0dd236/README.md
- Licence
- CC0 1.0
- How we use it
- Full text (licence allows reuse)
Testing a local model? Check first whether it fits your hardware with Can I run this LLM?