All prompts › Instruction following
Fewer than 5 sentences but 250+ words, title in << >>
What it tests: Conflicting pressure between length and sentence count, plus a title format.
Prompt length: 256 characters, roughly 60 tokens (estimate).
Write a blog post about the latest news in the US with a title in double angular brackets, i.e. <<title>>, and have less than 5 sentences (excluding 5). The sentences should be long, so that the total number of words in your response should be 250 or more.
How to grade it
A title in double angular brackets like <<title>>; fewer than 5 sentences in total; at least 250 words. Sentence counting by '.', '!' and '?' is approximate, so also check by eye.
Answer check: Constraints from IFEval key 1392: detectable_format:title, length_constraints:number_sentences, length_constraints:number_words.
Check an answer
Paste the model's answer. The checks run in your browser; nothing is sent anywhere.
Scorecard tick what the model got right: 0/4
Runs in your browser; nothing is saved or sent.
Where models usually fail
Ten normal sentences; a missing << >> title.
Source
- Author
- Google Research (Zhou et al., 2023)
- From
- IFEval (Instruction-Following Eval)
- Exact file or post
- https://github.com/google-research/google-research/blob/0d94f8c4d92678ce5f7a0eacae725f79174cfb03/instruction_following_eval/data/input_data.jsonl
- Licence
- Apache-2.0
- How we use it
- Full text (licence allows reuse)
Testing a local model? Check first whether it fits your hardware with Can I run this LLM?