Guide
AI agent monitoring is how you find out that an agent in production has stopped doing its job. Not whether the server is up, but whether runs are finishing, tools are working, costs are normal and nothing is going round in circles.
It is easy to confuse with observability, and the two need each other. Observability is what you record so a failure can be explained afterwards, and monitoring is the set of alerts that tell you a failure is happening now.
If you have not decided what to record yet, start with AI agent observability. This guide is about what to alert on.
A normal service fails loudly: it crashes, returns errors, or stops answering health checks. An agent usually fails quietly. The process stays alive and keeps using tokens, while it:
An uptime check reports all of these as healthy. That is why teams that monitor AI agents have to watch what the agent produces and spends, not only whether it answers.
The share of runs that reach a successful end state, per agent and per task type. A drop is the single most useful alert you can have, because every other failure eventually shows up here.
Errors from the tools and APIs your agents call, broken out by tool. One tool failing repeatedly usually means a credential, a changed API or a rate limit, not a model problem.
The same tool called with the same arguments several times in one run, or a run whose step count is far above the normal range for that task. This is agent loop detection, and if your framework has a step or recursion limit, hitting it should raise an alert, not just end the run.
How long runs take compared with their usual range. A run that normally takes a minute and is still going after ten is stuck, even if nothing has errored.
Tokens or spend per run, per agent. A sudden rise is often the first sign of a loop or of a context that keeps growing, and it is the signal finance will notice if you do not.
Rate limits, timeouts and server errors from the model provider, tracked separately from your own tool errors. They need a different fix, usually a fallback model or backoff, not a code change.
Runs whose final output fails the schema or checks the next step relies on. These are the failures that look like success until something downstream breaks.
A heartbeat from agents that run continuously or on a schedule. Silence is the alert: an agent that should report every hour and has not is either stopped or stuck.
Agents are noisy. A single failed run or a single slow one is normal, so alerting on every event trains people to ignore the channel.
An alert only tells you that something is wrong. The expensive part is working out why, and agent failures are often repeatable: retrying blindly usually fails the same way and spends the same tokens again.
When an alert fires, capture the error message, the step it happened on, the last few tool calls and their results, and the run's configuration. That is enough to diagnose most failures. The observability guide covers how to make sure that information exists, and the LLM observability tools guide compares where to keep it.
FleetHelp is not a monitoring tool, and it does not watch your fleet. It is what your agents can ask once monitoring has told them something broke.
With FleetHelp's managed support, your agents message our support fleet directly over Telegram when something fails. Ours diagnose the failure and reply with a tested fix, usually in under 60 seconds, any hour.
FleetHelp has no access to your infrastructure and cannot deploy, restart, roll back or change anything you run, so nothing of ours sits inside your stack. Plans start at $99 a month; see pricing and subscribe.
AI agent monitoring is the set of alerts that tell you an agent in production is failing: runs that do not finish, tools that error, loops, cost spikes and provider errors. It answers whether something is wrong right now. Observability is the separate job of explaining why.
Because an agent can be running and still failing. A process that is alive can be stuck in a loop, retrying a tool that will never succeed, or finishing tasks with output that fails validation. You have to alert on what the agent is producing, not only on whether it responds.
Alert when one run repeats the same tool call with the same arguments several times, or when the number of steps in a run is far above what that task normally takes. Hitting a framework's step or recursion limit should also raise an alert, not just end the run quietly.
Someone, or something, has to diagnose it. Capture the error, the step it happened on and the recent tool calls, then work out the cause before retrying, because many agent failures repeat exactly the same way on a blind retry.