Anomaly detection
Automatic flagging of unusual agent behaviour.
Anomaly detection compares what each agent is doing now with what that same agent normally does, and records an anomaly when the difference is too large to be chance. It also flags an agent that has gone quiet.
It works with no rule setup. Every anomaly is saved, linked to the run that caused it, and sent to your team as an alert.
Alerts only, no automatic stop
In beta, an anomaly alerts people and nothing else. The agent keeps running until someone acts. An admin can pause it manually with the Pause button on the agent's card or page (see Rules and circuit breaker).
How it decides what is normal
Each agent is judged against its own last 7 days, never against a global number. Guard rebuilds these baselines every night at 00:00 UTC for tokens per call, cost per call, latency and error rate. An agent needs about 10 samples before it is judged at all, so a brand-new agent has weaker detection on its first day.
Real anomalies are left out of the baseline, so a problem never teaches Guard that it is normal. Anomalies you mark as false positives stay in, so normal behaviour is learned.
The detector runs every 5 minutes over the last 5 minutes, so an anomaly appears within about 5 minutes of the trace arriving.
What it checks
| Check | Catches |
|---|---|
| Window average | The agent's overall tokens, cost or latency in the last 5 minutes has shifted. Flagged in either direction. |
| Single-call outlier | One call is far from typical, even if the rest are normal. A burst of bad calls is one row, shown with an OUTLIER tag. |
| Error rate | Many more runs failing than usual. Needs at least 5 runs and 2 errors in the window, so 1 failure out of 3 runs is not a crisis. A lower error rate is never an anomaly. |
| Loop | The same function called 20 or more times within 5 minutes. |
| Silence | An agent that normally sends steady traffic has sent nothing. |
Silence is judged on the agent's own pattern. It needs at least 30 traces and about a week of history, and an agent that is quiet every night or weekend does not alert. Paused agents are never flagged, and the anomaly closes by itself when traces resume.
Severity
| Severity | When |
|---|---|
medium | Clearly outside normal |
high | Far outside normal. Every loop is high. |
critical | Extremely far outside normal |
For silence, severity rises the longer the agent stays quiet.
One problem, one anomaly
A problem that lasts an hour is one open anomaly, not twelve. It keeps its worst reading, and Guard alerts again only if it gets worse, for example from medium to high. Once you resolve it, a new problem starts a new anomaly.
Which run caused it
Each anomaly records the trace that caused it, and for per-call anomalies the exact step. It also keeps evidence such as the function name and call count for a loop, or the failed trace IDs for an error spike. Use this to go from "error rate is up" to the run that failed.
Who is told
A new anomaly, or one that gets worse, is sent to the email and Slack channels your organisation has set up. See Alerts for how to set up channels and the follow-up message when an incident is resolved.
What you can do
| Action | Who | Effect |
|---|---|---|
| Resolve | Analyst or admin | Marks it handled |
| False positive | Analyst or admin | Marks it as not a real problem |
| Pause agent | Admin | Guard blocks every call from the agent until it is resumed |
Where to look first:
| Anomaly | First look | Usual action |
|---|---|---|
| Loop | The function name in the evidence | Pause if it keeps repeating, then fix the retry logic |
| Cost or tokens | The linked trace and step | Pause if cost is climbing. Check for a much longer prompt or document. |
| Latency | The linked slow trace | Usually the model provider. No pause needed unless it blocks users. |
| Error rate | The failed traces in the evidence | Fix the failing step. Pause if every run fails and keeps retrying. |
| Silence | When the last trace arrived | Check the agent process, deploy and API keys. Pausing will not help. |
Known limits in beta
- There is no automatic pause.
- Busy and quiet hours are not modelled for tokens, cost and latency, only for silence.
- Thresholds are the same for every agent and cannot be changed per agent yet.
- An agent whose calls vary hugely cannot flag a 10x call. For that agent, a 10x call is normal.
- Alert messages do not include a trace link or a pause link yet.