How AI Agents Actually Work
How AI Agents Actually Work
A chatbot answers you once and stops. An agent keeps going until the job is done — reading files, running tools, checking its own work. Here's the loop that makes that possible.
Imagine telling a coding assistant to fix a flaky checkout test. A chatbot would explain how to fix it, maybe paste some code, and stop. An agent does something else: it opens your codebase, finds the test, runs it, watches it fail, edits the code, runs the test again, reads the new error, fixes that too, and only then reports back.
Nobody wrote it a step-by-step script for your repo. It figured out the steps as it went, because each result told it what to do next. That loop — act, look, decide, act again — is the whole trick.
A chatbot is one call. An agent is a loop.
When you chat with ChatGPT or Claude, the model gets your message, generates one reply, and stops. It can't check anything, change anything, or try again. That isn't a limitation of the model — it's a limitation of the setup. The model is built to respond, not to act.
An agent flips the setup. The model sits inside a while loop — a repeating cycle in code — and each time around it gets to pick an action. Developer Simon Willison summed it up in one line: an agent is something that runs tools in a loop to achieve a goal. Anthropic's own guide says much the same: agents are systems where models "dynamically direct their own processes and tool usage."
In plain terms, the loop has five stages, and it repeats them until the job is done or a limit is hit:
- Perceive — take in something new: your message, a file, an error, the result of the last action.
- Reason — the model looks at everything so far and decides what to do next.
- Plan — for harder jobs, break the goal into smaller steps first. Simple jobs skip this.
- Act — call a tool: search the web, read a file, run code, send a message.
- Observe — read what came back. Did it work? Is the job done? Does the plan need changing?
Then it goes around again. Written as code, the whole pattern is about six lines: while not done, ask the model; if it wants tools, run them and feed the results back; otherwise, return the answer. As Oracle's engineering blog puts it, "agents can handle sophisticated tasks, but their implementation is often straightforward."
The three ingredients: a model, tools, and a harness
Three things turn a plain language model — an AI trained to predict text — into an agent. The first is the model itself, doing the reasoning. The second is tools: the actions the model is allowed to ask for. A tool can read a file, query a database, call an API (a way for one program to talk to another), or run code. Without tools, a model can only write words about doing things.
The third ingredient is the least glamorous and the most important: the harness. That's the surrounding software that receives the model's tool requests, checks they're allowed, runs them, and hands the results back. The model proposes; the harness disposes. If the model asks to delete a production database, the harness is the thing that says no.
Anthropic calls the combination an "augmented LLM": a model extended with retrieval (looking things up), tools, and memory. The idea goes back to 2022, when researchers at Princeton and Google published the ReAct framework — reasoning plus acting — which interleaved a model's thinking with tool calls in one loop. It beat answering in one shot by 34% on the ALFWorld household-task benchmark and 10% on the WebShop shopping benchmark, according to Oracle's write-up.
A worked example: booking the trip a chatbot can't
Here's the difference in a job you'd actually give one: "Find me the three cheapest flights to Tokyo next month, check if my loyalty points cover any of them, and book the best option."
A chatbot can explain how loyalty points work. It can't do the job, because the job has steps that depend on each other. An agent calls a flight-search tool, then reads the prices. The prices tell it which airlines to check, so it calls the loyalty tool next. The points balance tells it which option is actually cheapest after points, so it books that one, checks the confirmation, and reports back. Nobody told it the order in advance. Each result decided the next step.
That's why the loop matters more than the model. Swap in a smarter model and the same loop gets better; remove the loop and even the smartest model is just a chatbot again.
Not everything called an "agent" is one
This is where the word gets abused. Anthropic's guide draws a line the whole industry now uses: a workflow follows code paths you wrote in advance, while an agent directs its own process. A support bot that always runs "classify the ticket, then route it" is a workflow, even if a model does the classifying. A coding agent that picks its own tools, order, and stopping point is an agent.
Most real systems land between the two. Anthropic's guide names five workflow patterns teams actually use: prompt chaining, routing, parallelization, orchestrator-workers, and evaluator-optimizer (one agent generates, another critiques, and they loop until it passes). A fixed workflow is cheaper, more predictable, and far easier to debug than a free-roaming agent — and most jobs don't need the roaming part. One piece of plumbing makes the "tools" side practical: MCP, Anthropic's Model Context Protocol, which lets an agent discover and call outside tools through one shared format instead of a custom integration per app. We covered how MCP works in detail earlier.
What trips people up
The loop is simple. Making it behave is not. Four problems show up again and again.
Runaway loops. Without hard stopping conditions, an agent can spin forever. Oracle gives the example of a scraping agent whose target site changed its layout: the tool kept returning empty results, the agent kept retrying, and it made 400 calls in five minutes before a rate limit stopped it. Every production agent needs three guardrails: a max number of steps, a token or money budget, and a rule to quit when repeated steps produce no new information.
Cost. Every loop iteration is another model call, and the bill compounds fast. Anthropic's internal data, cited by Oracle, puts a single agent at roughly 4x the tokens of a normal chat interaction — and multi-agent setups at around 15x. A lead agent spawning parallel sub-agents beat a single agent by 90.2% on Anthropic's internal research evaluations, but at that token multiple. Better answers, much bigger bills.
Context bloat. Every tool result gets pasted back into the model's working memory, and that memory is finite. Long runs drown in their own history: old outputs crowd out the actual goal, and the agent starts making worse decisions. The fixes are unglamorous — summarise old steps, trim irrelevant results, load details only when needed.
Debugging a moving target. A chatbot failure is one bad reply. An agent failure is a 20-step trace where step 14 called the wrong tool with the wrong argument, and everything after it inherited the mistake. Production teams log every stage, because without that trace, you're guessing at which step went wrong.
Worth knowing: the restraint is the point. A 2026 review of Anthropic's agent report found the dominant pattern in production isn't sprawling autonomous agents — it's a single agent calling tools with a human reviewing the output. And 88% of agent pilots never reach production. Start with the simplest loop that works. Add autonomy only when you can measure the improvement.
Try this first
If you want to understand agents rather than just read about them, give one a bounded job with a checkable result — "summarise this document and list the three numbers in it" — and watch the trace, not just the answer. Count the tool calls. Notice where it checked its own work and where it didn't. Then try the same job with a broken tool, and see whether it stops cleanly or spins.
The one thing to remember: an agent isn't a smarter chatbot. It's the same model, wrapped in a loop that lets it act, look, and change its mind. The loop is the product — everything else is just how well you guard it.
Questions people actually ask
Do AI agents remember things between conversations?
Only if someone builds that in. Inside one job, the agent remembers everything through its working context — its short-term memory. Across sessions, it remembers nothing by itself; a team has to store notes or past results somewhere the agent can look up later. If an agent seems to "remember" you, that's a feature its builders added, not something the model does on its own.
Is it safe to connect an agent to my email or bank account?
Only with strict limits. The safe pattern is least privilege: give the agent the narrowest access the job needs, require your approval before anything irreversible (sending money, deleting data), and run its code in a sandbox it can't escape. Before you connect anything real, check what it can do without asking you — that's the number that matters.
