When Green Checks Lie

A reliable AI workflow has to prove that useful work happened.

Software teams are good at monitoring machinery. We know how to check whether a service responds, a database accepts writes, or a scheduled job finishes on time.

An LLM-powered agent complicates that picture. It can complete every step we instrumented and still miss the user's goal. A successful tool call might produce an unusable artifact. A healthy data connection might deliver stale information. A completed deployment might leave the environment in a shape nobody approved.

The dashboard stays green because it is answering the wrong question.

For agentic software, I want monitoring to answer two things:

1. Did the intended result actually happen?
2. Did it happen within the permissions and limits we set?

I call this outcome observability. It connects system health to purpose instead of activity.

Activity is not evidence

Most operational signals describe an attempt:

• a job started and exited successfully;
• an API returned a success response;
• an agent invoked the expected tool;
• a pipeline consumed its queue; or
• a deployment workflow reached its final step.

These are useful facts, but none proves that the work was accepted, correct, timely, or authorized.

Imagine a document-generation agent that runs every morning. The scheduler fires, the model responds, the file is written, and the job exits with status zero. Yet the document fails a downstream validation rule and is never delivered. The automation ran perfectly. The product failed completely.

The missing signal is not another process check. It is proof that a valid document reached its destination.

Three controls that make the green check meaningful

Prove the result from outside the agent

Start with the state a user or downstream system can observe. A file should pass validation and arrive at its destination. A data workflow should deliver recent, usable records. A deployment should leave the expected application reachable.

This proof should not come exclusively from the component declaring success. Whenever possible, let the receiver verify what happened.

Constrain action before execution

Define the operations, resources, environments, time window, and spending limit available to the agent. Then enforce those limits at the tool boundary.

Attribution matters here. A useful record connects the requested action, the identity that authorized it, the policy that permitted it, and the effect that followed. That chain should be easy to reconstruct without relying on the agent's own explanation.

Compare reality with the reviewed design

Automation can drift even when each individual action looks reasonable. New dependencies appear. Configuration changes. Resources accumulate. The live environment gradually stops resembling the system people thought they approved.

Check the resulting state against an explicit specification. The deployment command's exit code is evidence about the command, not the environment it produced.

Pair attempts with proof

For every operational event, name the independent evidence that would make it meaningful.

This pairing exposes false confidence early. It also creates a place to enforce consequences.

When proof is missing, the safest behavior depends on the stakes: pause the workflow, request review, reject the output, reduce permissions, or stop further spending. Monitoring that cannot affect behavior is only reporting.

Build the proof into the workflow

Outcome observability works best as part of the design, not as a dashboard added later.

Start with the user-visible result and work backward. Decide what would convince a skeptical reviewer that the result occurred, then place that evidence beside the action that claims to produce it. At each boundary, require the proof that matters before more work is allowed.

This reframes agent review as a systems question. Instead of asking only whether the model produced a plausible response, ask:

What changed, who allowed it, and what proves the change was useful?

That question survives model upgrades, prompt rewrites, and new orchestration frameworks. The implementation can change while the evidence contract stays stable.

The new bottleneck

LLMs have made it inexpensive to generate code, research, documents, plans, and tool calls. More activity is no longer the scarce resource.

Trust is.

Trustworthy automation can show its work in terms the surrounding domain understands. It knows the difference between trying and completing. It stays inside explicit limits. When its evidence breaks, it stops before the failure compounds.

A green dashboard should mean more than “the software is still running.”

It should mean the software did the job people asked it to do.