Reference architecture: AI customer support assistant
A support assistant classifies the customer's intent with a confidence score, answers policy questions with retrieval and citations, handles account questions through authenticated tools (order status, refund eligibility), hands off to a person when confidence is low or the customer asks, and logs every conversation for evaluation. The router and the handoff matter more than the model, because most support failures are the assistant trying to handle something it shouldn't. Measure it by conversations resolved without escalation, correct handoffs, and policy accuracy on sampled answers.
What it has to do
Customers ask about products, troubleshooting, orders, refunds and complaints, in their own words, at any hour. A static help centre makes them search, and a support team can't answer everything instantly. The assistant answers in a conversational way, takes simple actions, and knows when to pass the conversation to a person.
Architecture
channel (web, email, WhatsApp)
│
▼
auth ──> semantic cache ──hit──> answer
│ miss
▼
intent router (category + confidence)
├── policy question ──> RAG over help centre ──> answer with citation
├── account question ──> tools (order status, refund eligibility) ──> answer
├── action ──> write tools with rules or approval
└── low confidence, angry, or asks for a person ──> handoff with summary
│
▼
output guardrails ──> answer ──> trace, feedback, evals
Components
| Component | Role |
|---|---|
| Channel adapter | Web chat, email, WhatsApp, in-app |
| Auth | Links the conversation to a verified customer before account data is used |
| Semantic cache | Common policy questions, scoped by language and policy version |
| Intent router | Category plus a confidence score: FAQ, account, action, complaint, handoff |
| Policy RAG | Help centre and policy documents, versioned, with citations |
| Tools | Order status, refund eligibility, ticket creation; write tools need rules or approval |
| Guardrails | PII masking, scope limits, output schema, no promises outside policy |
| Handoff | Transfers the conversation to a person with a summary and the trace |
| Tracing and feedback | Resolution, escalation and satisfaction per conversation |
Key decisions
- Which actions the assistant can take alone, and which need a person. Start with read-only tools and a handoff for every write.
- How policy answers stay current when documents change: version the documents, scope the cache by version, and re-run policy evals on every update.
- What the assistant says when it doesn't know. It should say so and offer a person, not guess. See test abstention.
- Where the confidence threshold for handoff sits, chosen from data, not from a guess.
Failure modes to test
- Promising something the policy doesn't allow (the Air Canada bereavement fare case).
- Following a customer's instruction to change its behaviour (the $1 Tahoe case).
- Wrong order data because the model wrote it instead of the tool. See exact values.
- Not handing off when the customer is angry or asks for a person.
- Answering from an outdated policy version.
Metrics
- Resolution without escalation.
- Correct handoff rate (handed off when it should, not when it shouldn't).
- Policy accuracy on sampled answers, checked by your support lead.
- Cost per resolved conversation.
- Customer satisfaction after AI-handled conversations compared with human-handled ones.
From the floor
The customer support agent was the first project of the AI agents bootcamp, using Stripe's support as the example: the goal was an agent with routing, handoffs and RAG that can handle real queries like product questions, troubleshooting and refunds in a conversational way. The router returned a category and a confidence score, and if the category was unknown or the confidence was below 0.8, the query was handed off to a person through a Discord webhook with the escalated query, and otherwise it went to retrieval and an empathetic response, because human in the loop should always be part of the sequence.
In the next session, we took the same project through deployment: containerisation with Docker, CI/CD, and monitoring, following the same phases I use for every production project.