How do you evaluate AI agents?

An agent evaluation is a repeatable task with defined inputs, an operating environment, success criteria, graders, and a record of what the agent actually did. Evaluate the combined model, instructions, tools, permissions, memory, and runtime rather than the model in isolation.

For a lead-follow-up agent, do not grade only the email draft. Verify that it selected the correct lead, used approved CRM fields, avoided contacting opted-out records, saved the activity once, stayed within the permitted channel, escalated uncertainty, and produced the expected database state.

Build a first suite from 20 to 30 representative tasks: normal cases, boundary cases, tool failures, missing data, conflicting instructions, permission denials, and adversarial inputs. Define the required outcome and forbidden actions before running the agent several times per case.

Track task completion, outcome correctness, tool-selection accuracy, policy violations, unnecessary actions, escalation quality, latency, cost, and variance across repeated trials. Use deterministic graders where the truth is machine-checkable, model graders for bounded qualitative rubrics, and human review to calibrate judgment.

There is no universal test-count threshold. Start with the smallest suite that represents material failure modes, add every confirmed production failure as a regression case, rerun the core suite before changes, and run broader evaluations on a regular operating cadence.