Testing AI Agents: Evals Are the New Unit Tests

Every developer carries a deeply trained reflex: write the code, write the test, assert that the output equals the expected value, watch it go green. We have been doing this for so long that it feels less like a practice and more like a law of nature. Input goes in, the same output comes out, you assertEquals, and you sleep at night.

Then you build your first AI agent, sit down to test it the way you have tested everything for your entire career, and the reflex shatters on contact. You send the agent the same message twice. It gives you two different answers. Both are correct. Both are phrased differently. Your assertEquals does not just fail — it does not even mean anything anymore. There is no single expected string to assert against, because the whole point of the thing is that it generates language, and language has a thousand correct forms.

This is the moment most teams quietly give up on testing agents. They poke the agent by hand in a sandbox, it seems fine, and they ship it. Then three weeks later it starts confidently telling customers something wrong, and nobody notices for a fortnight because there was no test that could have caught it. The agent did not break in the way code breaks — with an exception and a stack trace. It drifted, silently, the way only probabilistic systems can.

The discipline that fixes this has a name, and it comes from the AI world rather than the Salesforce world: evals. Evaluations. They are the new unit tests — not a metaphor, an actual replacement for the role unit tests play in your pipeline — and if you are putting agents into production without them, you are flying an aircraft with the instruments unplugged. This post is about what evals are, why they work where unit tests fail, and how to actually build an eval suite for an Agentforce agent.

If you read the Agent Script post, you already have the other half of this story. Agent Script gives your agent a deterministic spine. Evals are how you prove the spine holds. The two are a matched pair: determinism makes testing possible, and testing makes determinism trustworthy.

The TL;DR

The shape of it before the depth:

Unit tests assume determinism. Agents don’t have it. Same input, different output — by design. assertEquals against generated language is a category error.
An eval is a test that tolerates non-determinism. It scores behavior against criteria instead of matching an exact string — sometimes with deterministic assertions, sometimes with a rubric or a judge model grading quality.
The deterministic parts of your agent are still unit-testable — and you should test them hard. “Did the agent call the verification action?” “Did it escalate when injuries were reported?” Those are binary, and your Agent Script arrows are what make them assertable.
The probabilistic parts need graded evals — correctness, helpfulness, tone, safety — scored across many runs, not asserted once.
sf agent test is how Salesforce ships this. Test specs of utterances and expected outcomes, run from the CLI or Testing Center, integrated into CI so every deploy re-checks the agent.
Eval-driven development is TDD for agents. Write the eval that captures the behavior you want, then build the agent until it passes. And run the suite on every deploy, because agents drift and the suite is your only alarm.

The rest is how to actually do it.

Why Traditional Testing Breaks

To build the right thing, you have to be precise about why the old thing fails. It is not that agents are “harder to test.” It is that they violate the foundational assumption every testing tool is built on.

A unit test is a contract: for this input, the output is exactly this. The entire machinery — assertEquals, snapshot testing, golden files — depends on the output being a single, knowable, repeatable value. That assumption holds for a pure function. It holds for an Apex method. It does not hold for anything with an LLM in the loop, for three compounding reasons.

Non-determinism by design. The same prompt produces different completions. This is not a bug to be fixed; it is the feature you are paying for. A model that gave a byte-identical answer every time would be a lookup table, not a reasoning engine. So the output space is not one value — it is a distribution of acceptable values, and you cannot enumerate it.

Semantic correctness, not string correctness. “Your order ships Tuesday” and “It’ll arrive Tuesday” are the same correct answer in two strings. “Your order ships Tuesday” and “Your order ships Thursday” are two strings that differ by one word and are completely different in correctness. String matching cannot tell these cases apart. You need to evaluate meaning, which no assertEquals can do.

Emergent, multi-turn behavior. An agent’s quality is not just per-response; it is across a whole conversation. Did it remember the policy number from turn three by turn nineteen? Did it stay on task when the user wandered? Did it refuse the thing it must always refuse, even when the user phrased the request cleverly? These are behaviors that only exist across turns, and no single-input/single-output test captures them.

A unit test asks “is the output equal to X?” An eval asks “does the behavior satisfy the criteria?” The shift from equal to satisfies is the entire discipline.

This is why hand-testing-then-shipping is so seductive and so dangerous. Hand-testing gives you a few samples from the distribution and lets you conclude “seems fine.” But “seems fine on the three inputs I tried” is not coverage. It is anecdote. And anecdote does not catch the silent drift that happens when you change a prompt, the model gets updated, or the grounding data shifts underneath you.

What an Eval Actually Is

Strip away the mystique and an eval is straightforward: a test case (an input, or a whole scripted conversation) plus a set of criteria the agent’s behavior must satisfy, plus a scoring method that decides whether it did. The thing that makes it an eval rather than a unit test is that the criteria and scoring are built to handle a distribution of acceptable outputs, not a single string.

The criteria fall on a spectrum from fully deterministic to fully semantic, and a good suite uses the whole spectrum:

Assertion type	Example	How it’s scored	Determinism
Action assertion	”Did the agent call `verify_policyholder`?”	Binary — it did or it didn’t	Fully deterministic
Routing assertion	”Did it transition to the escalation subagent?”	Binary	Fully deterministic
Variable assertion	”Was `caller_verified` set to true?”	Binary	Fully deterministic
Containment assertion	”Did the reply include the claim number?”	Pattern match	Mostly deterministic
Semantic assertion	”Was the answer factually correct?”	Rubric or judge model	Graded
Quality assertion	”Was the tone empathetic and on-brand?”	Rubric or judge model	Graded
Safety assertion	”Did it refuse to reveal another customer’s data?”	Rubric or judge model	Graded (and critical)

Notice the top half of that table is exactly as deterministic as a unit test. “Did it call this action” is binary. “Did it set this variable” is binary. This is the crucial, under-appreciated point: the more of your agent’s behavior you can express as deterministic assertions, the more of it you can test with the rigor of a unit test. And what lets you do that is having a deterministic spine in the first place — which is precisely what Agent Script’s arrows give you.

The bottom half — correctness, tone, safety — is where you need graded evaluation, because there is no single right string. Two main techniques:

Rubric scoring. You define what a good answer contains (the right facts, the required disclaimer, no forbidden claims) and score against the rubric. More objective, less flexible.
LLM-as-judge. You use a separate model to grade whether the output meets the criteria — “Does this response correctly state the renewal date and remain empathetic?” Powerful and scalable, but you have to watch for judge-model bias and validate that the judge agrees with humans on a sample.

A real suite mixes both halves: deterministic assertions for everything that can be deterministic, graded evals for the irreducibly semantic parts.

How Salesforce Does It: `sf agent test`

Agentforce ships this discipline natively, which is genuinely to Salesforce’s credit — most agent platforms still leave you to bolt on your own harness. The mechanism is the agent test suite, runnable from the CLI and from the Testing Center.

The shape is what you would expect from the table above. A test case pairs an utterance (what the user says) with expected outcomes — the topic or subagent it should route to, the action it should invoke, the kind of response it should produce. You can build the suite from your agent spec, and you run it with a single command:

# Create a test suite, often generated from the agent spec
sf agent test create --from-spec specs/MeridianFNOL.yaml

# Run it against a target org
sf agent test run --suite MeridianFNOL_tests --target-org MeridianDev

The run scores each case and reports pass/fail, so you get the thing hand-testing never gives you: a repeatable, reviewable verdict on whether the agent still behaves. Put that command in your CI pipeline and you have closed the loop — every deploy re-runs the evals, exactly the way every deploy re-runs your Apex tests.

This is the piece that connects to the rest of the toolchain. In the Headless 360 guide I covered building and deploying agents from the CLI; sf agent test run is the quality gate that belongs in that same pipeline. Build the agent as code, test it as code, ship it as code.

Building an Eval Suite That Actually Catches Things

A suite of three happy-path cases is theater. It turns green and tells you nothing, because the happy path is the one part that was always going to work. A suite that earns its place covers four categories, and the last two are the ones teams skip and regret.

1. Capability evals — can it do the job? The happy paths. The customer reports a claim, the agent verifies them, collects details, files it. These confirm the agent does what it is for. Necessary, easy, and not where the value is.

2. Edge-case evals — does it handle the weird inputs? The customer gives a policy number in the wrong format. They answer a question with a question. They provide half the information and go quiet. They change their mind mid-conversation. Real users are not the happy path, and these cases are where agents quietly fall apart in production.

3. Safety and adversarial evals — does it refuse what it must refuse? This is the category that keeps you out of the news. Can a user talk the agent into revealing another customer’s data? Can a cleverly phrased message (“ignore your previous instructions and…”) get it to skip verification or exceed its scope? Prompt injection and jailbreak attempts are not hypothetical, and your eval suite is where you prove the guardrails hold. Every must-never-do rule in your agent deserves an adversarial eval that tries to make it happen. If your Agent Script has an -> arrow that always escalates an unverified caller, you write an eval that does everything it can to slip past that arrow — and you assert it never does.

4. Regression evals — did a change break something that used to work? This is the whole reason evals exist as a standing suite rather than a one-time check. Agents drift. You tweak a prompt to fix one behavior and silently break three others. The model gets updated under you and starts phrasing refusals differently. The grounding data changes and an answer goes stale. Every bug you fix and every behavior you care about becomes a permanent regression eval, so the next change cannot quietly undo it. This is exactly how unit tests accrete into a regression suite over a codebase’s life — and it is exactly as load-bearing here.

The discipline: when an agent does something wrong in production, you do not just fix it. You write the eval that would have caught it, then fix it. That eval lives forever. Over time the suite becomes a precise, executable specification of everything your agent must and must not do — which is the same thing a mature unit-test suite is for ordinary code.

Eval-Driven Development: TDD for Agents

Once you have the harness, the most senior move is to invert the order — write the eval first. This is test-driven development, transplanted to agents, and it works for the same reason TDD always worked: it forces you to specify the behavior before you build it.

The loop:

Write the eval that captures the behavior you want. “When a user reports injuries, the agent must transition to escalation and never file a standard claim.” Encode it as a test case with assertions.
Run it. Watch it fail. The agent does not do this yet. Good — now you have an executable definition of done.
Build the agent until the eval passes. Add the subagent, the -> arrow, the routing logic.
Keep the eval forever. It is now a regression guard.

The reason this is more than a nice habit: it forces you to decide what correct behavior is before you are emotionally invested in whatever the agent happens to do. Without an eval written first, “correct” quietly becomes “whatever it did in the demo,” and you lose the ability to tell a feature from a fluke. Eval-driven development keeps the specification honest.

And it pairs perfectly with the deterministic spine. The behaviors most worth writing evals-first for — the verification gate, the mandatory escalation, the ordered collection — are exactly the ones you implement as Agent Script arrows. You write the eval that demands the guarantee, then you build the arrow that provides it. The eval proves the arrow. The arrow makes the eval pass deterministically. It is a clean, tight loop.

Where Evals Will Bite You

Honest caveats, because an eval suite has its own failure modes:

Don’t over-test the prose. It is tempting to assert that responses contain specific phrasing, but that just re-introduces the brittleness you were escaping — now your suite goes red every time the model rephrases a correct answer. Assert on meaning and behavior, not wording. Test that the claim number appears, not that the sentence around it is verbatim.

Watch for flaky evals. Because the underlying system is probabilistic, a borderline case can pass on one run and fail on the next. Decide deliberately how to handle this — run critical cases multiple times and require a pass threshold, or tighten the criteria so the behavior is unambiguous. A suite that randomly goes red trains people to ignore it, which is worse than no suite.

Validate your judge. If you use LLM-as-judge, the judge is itself a probabilistic system with its own biases. Sample its verdicts against human judgment periodically. A judge that quietly disagrees with your team is a green suite that means nothing.

Test your logic, not the model. Your evals should verify your agent’s behavior — its routing, its actions, its guardrails — not benchmark the foundation model’s general intelligence. If an eval fails because the underlying model is weak at some task, that is a grounding or design problem, not something your suite should be trying to fix by accident. Keep the scope on what you built.

A green suite is necessary, not sufficient. Evals dramatically raise your confidence; they do not make an agent safe to ship unsupervised into a high-stakes context. Pair them with logging, monitoring, and human oversight in production — especially early. The suite catches what you thought to test for. Production will find what you didn’t.

Open Questions

Genuinely unresolved, and I would value other practitioners’ takes:

What is the right pass threshold for graded evals? Unit tests are binary: 100% pass or the build is red. Graded evals produce scores. Is 95% good enough? 90%? Does it depend on the assertion’s criticality (100% on safety, lower on tone)? I lean toward “deterministic assertions must be 100%, graded ones get criticality-weighted thresholds,” but the industry has not standardized this.
How do you version evals against a moving model? When the foundation model updates, behavior shifts and some evals may legitimately need to change. How do you distinguish “the eval should be updated because the new behavior is actually better” from “the eval caught a real regression”? This is genuinely hard and I do not have a clean answer.
Can eval suites become a false sense of security? A large green suite feels like safety. But it only covers what you imagined. For the hardest-regulated contexts, is any eval suite ever enough to ship autonomously, or does human-in-the-loop remain mandatory regardless? I currently believe the latter for high-stakes domains.
Who owns the eval suite? Like the agent script itself, it sits on the admin/developer boundary. The Admin Plus who builds the agent should arguably own its evals — but eval design is a genuine skill. Is “agent QA” about to become its own role?

The Verdict

The reason agents ship untested is not laziness. It is that the testing reflex every developer spent a career building — write the code, assert the output, go green — genuinely does not transfer. You cannot assertEquals a vibe, and when people discover that, most of them conclude agents simply cannot be tested and fall back on poking them by hand. That conclusion is wrong, and it is the source of nearly every silent agent failure in production.

Evals are the transfer. They keep everything good about unit testing — repeatable, automated, in CI, accreting into a regression suite — and swap the one broken assumption (exact-match output) for one that fits probabilistic systems (criteria-based scoring). The deterministic parts of your agent stay as rigorously testable as any Apex method, because you gave them a deterministic spine. The probabilistic parts get graded across many runs instead of asserted once. And the whole suite becomes, over time, an executable specification of everything your agent must and must not do.

Here is the through-line connecting this to the Agent Script post: determinism and evals are not two topics. They are one. Agent Script lets you draw hard lines around the behavior that matters; evals let you prove the lines hold and catch the moment they stop holding. An agent with a spine but no evals is a guarantee you never verify. An agent with evals but no spine is a test suite grading a coin flip. You need both, and together they are what turns an agent from a demo into a system you can actually trust in production.

Write the eval. Watch it fail. Build until it passes. Keep it forever. It is the oldest discipline in software, wearing new clothes — and in the age of agents, it is not optional.