All Glossaries
1
min read
Table of Contents

Agent Drift and Drift Rate

What is agent drift?

Agent drift is a gradual change in how an AI agent behaves or how accurate it is, compared with when it was evaluated and approved. Drift rate is how quickly that change happens, usually measured as the change in a key metric, such as verdict accuracy, over a set period.

Drift is dangerous because it is quiet. An agent that was 97 percent accurate at launch might be noticeably worse six months later without any single event anyone would notice.

What causes agent drift?

  • Model changes: The provider updates the underlying model version.
  • Prompt or configuration changes: Someone edits the agent's instructions to fix one problem and creates another.
  • Tool changes: An EDR or identity API changes its fields or behavior.
  • Data drift: A new log source, a changed schema or a new business application changes what "normal" looks like.
  • Threat drift: Attackers change techniques, so patterns the agent learned to trust stop being reliable.

How do you measure drift rate?

You measure drift rate by tracking the same metrics over time against a fixed baseline.

  • Re-run a frozen test set on a schedule. Because the cases do not change, any change in score is the agent's.
  • Track live metrics, such as agreement with sampled analyst reviews, escalation rate and action reversals, week by week.
  • Watch behavior, not just accuracy. A sudden change in how often an agent escalates or closes alerts can signal drift before accuracy visibly drops.
  • Set thresholds. Decide in advance how much change triggers a review and how much triggers rolling back autonomy.

What should you do when you detect drift?

  1. Check what changed recently: model, prompts, tools or data sources.
  2. Reduce the agent's autonomy for affected alert types until the cause is found. See governed autonomy.
  3. Re-evaluate against the test set and recent live cases.
  4. Fix the cause, re-test and restore autonomy only when the agent clears its pass bar again.
  5. Add the cases that exposed the drift to the test set.

Frequently asked questions

How often should we check for drift?

Continuously for live metrics, and at least monthly for frozen test-set runs. Always re-check after any change to the model, prompts, tools or key data sources.

Is drift always a bad thing?

Not always. An agent's behavior should change when the environment genuinely changes. The goal is to notice the change and confirm it is correct, not to freeze the agent.

Anomali and Agent Drift and Drift Rate

Anomali records agent decisions and their evidence over time, which supports tracking accuracy and spotting drift.

Request a demo to see how Anomali supports agent monitoring.

Read next

‍