Decision support
AI agents versus chatbots: the one difference that changes your risk
A chatbot returns text. An agent takes actions. Everything expensive about the second one follows from that.
The question
Everything is called an agent now. Is this actually different from the chatbot we already tried?
The short answer
One difference matters and the rest is marketing: an agent takes actions with side effects in your systems, a chatbot returns text to a human. When output is text, a wrong answer costs you a confused user and a support ticket. When output is a tool call, a wrong answer costs you a refund issued, a record overwritten, an email sent to a customer. That moves the whole system from the category of “content quality problem” to “distributed system with an unreliable component in the write path” — which changes how you test it, how you budget it, how you roll it back, and what you have to be able to prove afterwards. If a vendor calls something an agent and it only produces text, it is a chatbot with a better model.
Written for
- Leaders being pitched “agents” after rejecting a chatbot
- Managers deciding whether an existing chatbot should be given tools
- Anyone writing the requirements for one
Why the answer is that
The useful test is: can it change anything without a human pressing a button? If yes, it is an agent and the engineering bar is much higher. If no, it is a chatbot, and most of what is written about agent reliability does not apply to you — which is good news, because chatbots are far cheaper to run safely.
The reason the distinction gets blurred is that the two are built from the same parts. Same model, same prompt, same retrieval. The difference is one function call, and it is genuinely tempting to add it: the chatbot already knows the order number, so why make the user click. But that one addition is what converts a quality problem into a correctness problem. A chatbot that is wrong 5% of the time is a mildly annoying product. An agent that is wrong 5% of the time and can issue refunds is an incident.
This also explains why the cost model changes shape. A chatbot turn is roughly one model call. An agent turn is a loop — think, call a tool, read the result, think again — so a single user request might be five or fifteen model calls, each carrying the full conversation plus every tool schema you attached. Teams that budget an agent using chatbot arithmetic are routinely out by an order of magnitude, and the retry path is where it compounds.
And the testing strategy inverts. For a chatbot you can review a sample of transcripts and form a judgement. For an agent you cannot, because the failure you care about is not “the wording was poor” but “it called the tool with the wrong argument in a case nobody tried.” That needs assertions on tool calls against real traffic, which is a test suite, not a review.
Side by side, on the dimensions that cost money
Same model underneath. The columns diverge because one of them can change your database.
| Dimension | Chatbot | Agent |
|---|---|---|
| Output | Text for a human to read and judge | Tool calls that change state |
| Cost of being wrong | A confused user, a support ticket | A wrong refund, a bad write, an email you cannot unsend |
| Calls per user request | Roughly one | Five to fifteen, each carrying full context and every tool schema |
| How you test it | Read a sample of transcripts | Assertions on tool calls, run against real traffic |
| Latency profile | One round trip, predictable | Serial tool calls; p95 is several times p50 |
| Failure mode | Unhelpful answer | Loop that never terminates, or a confident wrong action |
| Rollback | Nothing to roll back | You need a compensating action for every side effect |
| Audit requirement | Usually none | Reconstruct any decision: inputs, prompt version, model version, tool calls |
| Who signs off | Product | Product, security, and whoever owns the systems it writes to |
Every row on the right is a real engineering cost. If a proposal describes agent capabilities but budgets like the left column, that is the gap to raise.
What you are actually being sold
Four things get marketed as “agents”. Only two of them carry agent-level risk, and knowing which you are buying tells you what to ask for.
| What it does | Actually an agent? | What to require |
|---|---|---|
| Answers from your documents (RAG) | No — it is retrieval plus a chatbot | Citation presence, permission-filtered retrieval |
| Answers, then drafts an action for a human to approve | Partly — human is the actuator | Make the diff reviewable; log what was approved |
| Calls read-only tools to compose an answer | Yes, low blast radius | Bounded loops, cost ceiling, tracing on tool calls |
| Calls tools that write, autonomously | Yes, and this is the expensive one | Eval suite on tool arguments, approval gates, rollback plan |
| Coordinates several sub-agents that write | Yes, hardest to reason about | All of the above plus a termination guarantee |
The first two rows are where most teams should start, and where most successful deployments actually sit. Autonomy is a cost you take on deliberately, not a feature you get for free by using a newer framework.
Get the answer for your own system in four minutes
The free scorecard produces a score out of 100, a breakdown across reliability, cost and observability, and your gaps ranked by production risk with a fix and effort estimate for each. It prints to a PDF you can take into a planning meeting — which is usually more persuasive than an argument.
Run the scorecardIf you are the engineer who found this page: Written to be forwarded. If you are the engineer who found this, send the relevant one up — it is framed for the person who has to approve the work.
If you decide to do the work
The engagements this decision leads to, each with the full method and effort estimates — whether your team runs it or I do.
Questions
What is the difference between an AI agent and a chatbot?
An agent takes actions that change state in your systems. A chatbot returns text for a person to read.
They are built from the same parts — same model, same prompt, same retrieval — and the difference is whether tool calls with side effects are wired in. That one change converts a content-quality problem into a correctness problem, because a wrong output is now a wrong action rather than a confusing sentence.
How can I tell whether a vendor is selling a real agent?
Ask whether it can change anything without a human pressing a button.
If the answer is no, it is a chatbot with a good model, and it is cheaper and safer than an agent — that is not a criticism. If the answer is yes, ask what happens when it calls a tool with a wrong argument: what validates the call, what bounds the loop, and what the compensating action is.
Why do AI agents cost so much more to run than chatbots?
Because a chatbot turn is roughly one model call and an agent turn is a loop.
Think, call a tool, read the result, think again — one user request can be five to fifteen model calls, and each one resends the accumulated conversation plus every tool schema attached to the request. Teams that budget agents using chatbot arithmetic are commonly out by an order of magnitude, and retries compound it.
Should we upgrade our existing chatbot into an agent?
Only if the actions it would take are worth the engineering that makes them safe.
Adding a tool call is a few hours of work; the eval suite, tool-argument validation, approval gates, tracing, bounded loops and rollback path that make it responsible are the actual project. A good middle step is to have it draft the action and let a human approve — you get most of the time saving and the human remains the actuator.
Want a second opinion on the decision?
Describe your situation and I will tell you what I would do — including when that is "nothing" or "handle it internally". No charge for that, and it is genuinely how a lot of these conversations end.
Other decisions
- What an agent audit should cover
- Questions to ask before launch
- Build or buy: observability
- Metrics worth reporting upward
- Do you need outside help?
- Can an agent touch our internal data?
- Governance, audit logs and compliance
- Do we need our own model?
- Where to put a human in the loop
- Agent cost versus hiring
- Working with the systems you already have