Parallel tool calls with a distractor function
What it tests: Choosing the right two of three tools, mapping 'perimeter of a circle' to circumference, passing radius 5 and diameter 10 correctly, and emitting valid JSON only.
Prompt length: 1,636 characters, roughly 410 tokens (estimate).
You can call these functions. Reply ONLY with a JSON array of calls in the form [{"name": "<function name>", "arguments": {...}}]. If none of the functions fits the request, reply with [] followed by one sentence explaining why.
Functions:
[
{
"name": "circle.calculate_circumference",
"description": "Calculate the circumference of a circle based on the diameter.",
"parameters": {
"type": "dict",
"properties": {
"diameter": {
"type": "integer",
"description": "The diameter of the circle."
}
},
"required": [
"diameter"
]
}
},
{
"name": "circle.calculate_area",
"description": "Calculate the area of a circle based on the radius.",
"parameters": {
"type": "dict",
"properties": {
"radius": {
"type": "integer",
"description": "The radius of the circle."
}
},
"required": [
"radius"
]
}
},
{
"name": "rectangle.calculate_perimeter",
"description": "Calculate the perimeter of a rectangle based on the length and breadth.",
"parameters": {
"type": "dict",
"properties": {
"length": {
"type": "integer",
"description": "The length of the rectangle."
},
"breadth": {
"type": "integer",
"description": "The breadth of the rectangle."
}
},
"required": [
"length",
"breadth"
]
}
}
]
User: Find the area and perimeter of a circle with a radius of 5 and also find the circumference of a circle with diameter of 10.How to grade it
Exactly two calls in valid JSON: circle.calculate_area with {"radius": 5} and circle.calculate_circumference with {"diameter": 10}. BFCL ground truth: [{"circle.calculate_area": {"radius": [5]}}, {"circle.calculate_circumference": {"diameter": [10]}}]. The rectangle function must not be called. (For reference, area ≈ 78.54 and circumference ≈ 31.42.)
Answer check: Ground truth from BFCL possible_answer/BFCL_v4_parallel_multiple.json (id parallel_multiple_2).
Scorecard tick what the model got right: 0/4
Runs in your browser; nothing is saved or sent.
Where models usually fail
Calling rectangle.calculate_perimeter for 'perimeter'; passing radius 5 as the diameter; prose or markdown fences around the JSON; computing the answers instead of calling the tools.
Source
- Author
- Gorilla team, UC Berkeley
- From
- Berkeley Function Calling Leaderboard (BFCL)
- Exact file or post
- https://github.com/ShishirPatil/gorilla/blob/6ea57973c7a6097fd7c5915698c54c17c5b1b6c8/berkeley-function-call-leaderboard/bfcl_eval/data/BFCL_v4_parallel_multiple.json
- Licence
- Apache-2.0
- How we use it
- Full text (licence allows reuse)
Testing a local model? Check first whether it fits your hardware with Can I run this LLM?