Don't call the only tool when it doesn't fit
What it tests: Tool-use restraint: the only available function computes BMI, but the user wants a triangle's area.
Prompt length: 844 characters, roughly 210 tokens (estimate).
Prompt
You can call these functions. Reply ONLY with a JSON array of calls in the form [{"name": "<function name>", "arguments": {...}}]. If none of the functions fits the request, reply with [] followed by one sentence explaining why.
Functions:
[
{
"name": "determine_body_mass_index",
"description": "Calculate body mass index given weight and height.",
"parameters": {
"type": "dict",
"properties": {
"weight": {
"type": "float",
"description": "Weight of the individual in kilograms."
},
"height": {
"type": "float",
"description": "Height of the individual in meters."
}
},
"required": [
"weight",
"height"
]
}
}
]
User: Calculate the area of a triangle given the base is 10 meters and height is 5 meters.How to grade it
No function call: [] plus a short explanation that the BMI tool can't compute a triangle's area. Mentioning the area itself (0.5 × 10 × 5 = 25 m²) is fine. BFCL scores any call here as a failure.
Answer check: BFCL irrelevance category: the correct behaviour is no call (id irrelevance_0).
Scorecard tick what the model got right: 0/4
Runs in your browser; nothing is saved or sent.
0/4
Where models usually fail
Calling determine_body_mass_index with weight 10 and height 5; inventing a calculate_triangle_area function that isn't in the list.
Source
- Author
- Gorilla team, UC Berkeley
- From
- Berkeley Function Calling Leaderboard (BFCL)
- Exact file or post
- https://github.com/ShishirPatil/gorilla/blob/6ea57973c7a6097fd7c5915698c54c17c5b1b6c8/berkeley-function-call-leaderboard/bfcl_eval/data/BFCL_v4_irrelevance.json
- Licence
- Apache-2.0
- How we use it
- Full text (licence allows reuse)
Testing a local model? Check first whether it fits your hardware with Can I run this LLM?