Notes on making agents survive production
Roughly monthly. Each one aims to contain at least one number you can check and one thing you can do on Monday. If a post is just an opinion, I have not finished it.
- 8 min read
Pick the least agentic architecture that solves your problem
Almost every production agent is one of four shapes, and they are ordered by how much autonomy you give away. Most teams start at the wrong end of that list, because the interesting end is also the expensive one.
AI agent architecture best practicesagent architecture patternsAI orchestration platformmulti-agent vs single agentagentic workflow design
- 8 min read
Agent latency is a serial-hop problem, not a token problem
Teams optimise tokens and wonder why p95 barely moves. An agent request is a chain of round trips, and the tail is dominated by how many hops happen in sequence — plus the retries you cannot see in an average.
AI agent latency optimizationLLM latency p95reduce agent response timeagent performance tuningstreaming LLM latency
- 8 min read
Agent memory is four different problems wearing one word
Conversation history, running state, retrieved knowledge and learned preferences get called memory interchangeably. They have different storage, different lifetimes and different failure modes — and the fourth one is where agents start confidently repeating things that stopped being true.
long-term memory for AI agentsagent memory architectureconversation history LLMagent state managementvector memory
- 9 min read
Kubernetes defaults are wrong for agents, and the reason is request duration
Every default in a standard Deployment assumes requests finish in milliseconds. An agent request runs for thirty seconds to several minutes and spends almost all of it waiting on someone else, which breaks autoscaling, health checks and rolling deploys in that order.
AI agent deployment on Kubernetesdeploy LLM agent kubernetesautoscaling long running requestskubernetes graceful shutdown agentHPA queue depth
- 8 min read
You cannot prompt your way out of hallucination
Every team tries "only answer from the provided context" first, and every team finds it leaks. Hallucination is not a wording problem — it is what happens when nothing between the model and the user checks whether a claim is true.
AI agent hallucination preventionreduce LLM hallucinationgrounded generation citationsconstrained output LLMhallucination detection
- 7 min read
RAG is not an alternative to agents
They get compared as if you had to choose. RAG is a retrieval technique; an agent is a control-flow pattern. The real decision is whether retrieval happens once before the model runs, or repeatedly because the model asked.
RAG vs AI agentsagentic RAGretrieval augmented generationwhen to use RAGagentic retrieval
- 7 min read
What to look for in an agent design review
Reviewing an agent feature before it ships is the cheapest reliability work available, and most teams review the prompt rather than the architecture. Nine questions that catch the expensive problems while they are still free.
agent design reviewAI code review checklistLLM architecture reviewagent feature review
- 8 min read
Where your context window actually goes
A million-token window does not mean you should use it. Decompose one real conversation and the proportions are usually a surprise — history dominates, tool definitions are larger than expected, and the useful content is a minority of what you pay for.
LLM context windowcontext window budgetconversation history costLLM token usage
- 9 min read
Event-driven agents: what Kafka taught me about agent architecture
A 20% input spike once turned a healthy pipeline into an outage because it had no backpressure — it had optimism. Every concept that prevents that has a direct agent equivalent, and almost no agent codebase implements any of them.
event-driven agentsKafka LLM architectureasync agent designagent backpressureLLM streaming data
- 7 min read
LLM-as-judge is overused
It is slow, it costs money on every CI run, and it is non-deterministic — which means your test suite has flaky tests by construction. About 70% of the checks teams reach for a judge to make can be written as assertions instead.
LLM as judgeLLM evaluation methodsAI eval assertionsLLM testing CI
- 9 min read
MCP in production: what the protocol gives you and what it does not
Model Context Protocol moves your tool boundary into a place where validation and authorisation naturally live. That is the real benefit, and it is not the one usually advertised. What it does not give you: security, cheaper tokens, or a reason to trust tool arguments.
Model Context Protocol productionMCP best practicesMCP securityMCP tool designMCP vs function calling
- 8 min read
Model routing is the biggest cost lever, and it needs evals first
Routing tasks by difficulty typically recovers 40–70% of spend. It is also the only cost lever that can silently degrade quality — which is why the teams that most need it are usually the ones least equipped to do it safely.
LLM model routingreduce LLM costsmodel selection by taskLLM cost optimisation
- 8 min read
Prompt caching and semantic caching are not the same thing
They get discussed interchangeably and they solve different problems with different risks. One is a billing mechanism with a write premium that can lose you money; the other returns a previous answer to a different question and can leak across tenants.
prompt cachingsemantic caching LLMLLM cache hit rateLLM cost caching
- 7 min read
Prompt versioning: your prompts are deployable artefacts
If a customer reports a bad answer from last Tuesday, can you reconstruct the exact prompt that produced it? Most teams cannot, and it makes every subsequent investigation archaeology.
prompt versioningLLM prompt managementprompt rollbackLLM reproducibility
- 7 min read
The tool-definition tax: what your agent pays before it does anything
Every tool schema you attach is billed on every single call, whether the agent uses it or not — plus a per-model tool-use system prompt of 286 to 804 tokens. Here is how to measure that fixed cost and what it does to your bill at scale.
LLM cost optimisationtool definitions tokensagent costprompt cachingMCP
- 8 min read
Your agent doesn't have a model problem
Almost every failing agent I have looked at is failing on engineering discipline, not model capability. The evidence is that upgrading the model rarely fixes it — and the four things that do are unglamorous.
why AI agents fail in productionAI agent reliabilityLLM engineering disciplineproduction AI systems