An AI agent is asked to produce a report of open support cases. It creates a file, and the benchmark marks the task as passed. But the report leaves out half the cases. The grader checked that the file existed, not that it contained the right information.
This is a hypothetical example. It illustrates a problem worth checking in any evaluation: the grader can accept an incomplete result or reject a valid one.
If a team uses a score to compare agents or support a release decision, it needs to know what the evaluation actually measured.
A score depends on more than the agent
An AI agent benchmark gives an agent a set of tasks and scores its results. Each task needs instructions, a way to determine what counts as success, and a way to check the agent's work.
The result can also depend on available tools, installed software, and limits such as time and memory. If one of those conditions prevents a correct solution from running, the resulting failure may say more about the evaluation than the agent.
The site's guide to testing AI agent tool calls, permissions, and results explains how to inspect what an agent actually did. When that work is scored by a benchmark, there is another question: does the grader recognize the result correctly?
What researchers found in a coding benchmark
In a November 2026 preprint, researchers reported an audit and maintenance effort for Terminal-Bench 2.0, a benchmark used to evaluate coding agents. The resulting update revised 28 of 89 tasks. That total includes problems reported in earlier audits and public discussions, along with 14 additional tasks the researchers identified as needing repair.
The issues included mismatches between task instructions and grading rules, changes in external dependencies, and resource limits that could interfere with valid solutions. In one task, the verifier required a database file to remain identical byte for byte. The task instructions had not stated that requirement, so the researchers made it explicit.
The researchers then compared results for 19 agents on the original and revised benchmark. Individual scores shifted by up to 12 percentage points, and leaderboard positions moved by up to three places. The overall ranking remained strongly correlated. Read Benchmarks as Software: Terminal-Bench and the Case for Continuous Benchmarks for the audit and its limits.
This was an audit and update of a particular benchmark. The 28 tasks are not an estimate of how often other AI benchmarks contain defects. The study shows how task wording, grading, dependencies, and resource limits can affect a published score.
Start with the decision the evaluation will support
Before inspecting a grader, ask what the team wants to learn.
Is the agent supposed to repair code, answer customer questions, research a topic, or complete a workflow using tools? Which mistakes would matter most in that work?
A score across short coding tasks can tell you how an agent performed on those tasks under the benchmark's conditions. It does not, by itself, establish that the agent can safely make changes in your production environment.
The site's guide to testing LLM applications starts with real user tasks for a reason. The work in an evaluation should resemble the work people will ask the system to do. The next step is checking whether the evaluation judges that work correctly.
Compare the instructions with the grader
Take a sample of tasks and read both the instructions given to the agent and the rules used to score its result.
Look for requirements that appear only in the grader. If a task asks for a report but the grader accepts only one unstated filename, decide whether that filename is genuinely required. If it is, add it to the task instructions. If it is not, change the grader so it accepts other valid filenames.
Then look for requirements the grader fails to check. If the task asks for every open support case, checking only that a file exists is too weak.
One practical test is to give the grader two prepared results. The first should satisfy the written task, even if it takes a different valid route from the expected solution. The second should look plausible but miss an important requirement. Check whether the grader accepts the first and rejects the second.
The same question arises when reviewing AI-generated tests: would the assertion catch a meaningful failure? In an agent benchmark, the grader must also account for valid differences in how an agent completes a task.
Check the environment with a known-correct solution
A task can be clear and its grader can be sound, yet the setup can still interfere with the result. A dependency may change, a service may become unavailable, or a resource limit may prevent a correct solution and the grader from running reliably.
Run a known-correct solution under the benchmark's actual conditions. Record the software versions, tools, permissions, and resource limits. If the solution fails, investigate the setup before counting that result against an agent.
The Terminal-Bench researchers used this kind of reference-solution check as part of their maintenance workflow. They also found that time budgets could affect conclusions about agent performance. That makes the benchmark version and its conditions part of the result.
If your team changes its own prompts or evaluation setup, record the change before comparing scores. The guide to prompt regression testing shows how to make the effects of prompt changes visible.
Inspect surprising passes and failures
An overall pass rate can hide the cases most worth investigating. Review tasks where an apparently good result failed, an incomplete result passed, or a small change to the evaluation caused a large score shift.
Check the agent's actions and the final state, not just its last message. Did it create the requested artifact? Did it change the right records? Did it use an allowed tool? Did a setup error occur before it could finish?
The site's review of AI agent testing gaps discusses a study of 157 open-source projects. Researchers found conventional tests in most of the repositories, while repository-visible checks for complete, multi-step agent workflows were less common. They could not assess private testing absent from those repositories. That finding is about project testing practices, not benchmark defect rates, but it reinforces the value of checking an agent's full path from decision to effect.
Keep the score and its limits together
An evaluation report should identify the tasks, benchmark version, grader, and conditions used. It should explain investigated failures and state what the evaluation did not cover.
That is where this work connects to AI assurance. A score is evidence for a claim about an AI system. Testers help determine whether that evidence supports the claim and where another kind of test is needed.
Judging an evaluation draws on requirements analysis, test design, expected-result checks, investigation, and an understanding of AI systems. The ASTQB AI Assurance Pro™ designation brings together software testing foundations, testing with generative AI, and testing AI-based systems. It does not validate a benchmark or replace a review of one. This example shows why teams need people who can examine the evidence behind an AI result.
The next time an agent receives an impressive score, ask what the evaluation checked and whether it would have caught the failures that matter.