Chatbot, RAG, or Agent? Choosing the Right Shape for an AI Feature

"We want to add AI" isn't a spec — it's three genuinely different architectures wearing the same marketing label, and picking the wrong one is the most common mistake we see in early planning conversations.

A chatbot is a conversation layer

The simplest shape is a straightforward chat interface over a model, optionally with some conversation memory. This is the right choice when the value is genuinely in open-ended conversation — answering varied questions, brainstorming, drafting — and there's no strict requirement that answers be grounded in a specific, verifiable source of truth.

RAG is for grounding answers in your own data

Retrieval-augmented generation adds a lookup step: relevant documents or records are retrieved from your own data first, then handed to the model as context for its answer. This is the right shape the moment accuracy against a specific knowledge base matters more than open-ended conversational ability — a support tool that must only answer from real documentation, or a search feature over an internal wiki. The engineering effort here is mostly in retrieval quality (chunking, embeddings, hybrid search), not in the generation step.

An agent is for taking multi-step action

An agent adds the ability to plan a sequence of steps and call tools or functions along the way — not just answering a question, but doing something as a result. This is the right shape when the task genuinely requires multiple steps with decisions in between, not just a single well-formed answer. It's also the shape that requires the most evaluation and guardrail work before it's safe to ship, because it can take real actions, not just say things.

The mistake is picking the impressive one, not the right one

Agents get the most attention, but a huge share of "AI feature" requests are actually well-served by a chatbot or a RAG system — simpler to build, easier to evaluate, and much easier to reason about when something goes wrong. The right question isn't "which is most advanced" — it's "does this task need grounding in specific data, or does it need to take multi-step action," and most of the time the honest answer points to the simpler option.