How to build an AI voice agent for your business
How to build an AI voice agent step by step: a clear brief, knowledge base, request fields, categories, urgency rules, a test plan and weekly improvements.

A voice agent is only as good as what you tell it. Voice technology already converses fluently; what separates a helpful agent from one that frustrates your customers is the groundwork: what it should do, what it knows, what it asks, when it alerts a person and how it was tested. This guide shows how to build an AI voice agent for your business, step by step, the way we do it at CallSmarter, with examples from restaurants, clinics, home services and estate agencies.
Step 1. Define the job before the voice
On a single sheet of paper, write down in this order:
- Why your customers call. The five or six most common reasons. If unsure, check the call log and ask whoever answers the phone.
- What the agent can resolve with information alone: hours, address, services, published prices.
- What it must collect so your team can act without calling back.
- What it must never do: confirm, negotiate, diagnose, take payment.
- What counts as urgent and who needs to know.
That sheet is your real script. Everything else is putting it into the tool.
Step 2. Start from an industry template
Starting from scratch is the slowest way to get it wrong. CallSmarter has templates for:
- Restaurant: bookings and takeaway orders pending confirmation; allergens noted only, never guaranteed.
- Dental clinic: appointments and changes, emergencies; no medical advice and 112 if life may be at risk.
- Accommodation and real estate: bookings, viewings, owners, guest incidents.
- Home services: breakdown, servicing and installation calls with urgency levels.
Each comes with a tested script, knowledge base sections, fields, categories and urgency rules. You adapt them; you don't rewrite them.
Step 3. Write a knowledge base that works out loud
The knowledge base is organised by section (hours, address, services, prices, policies, FAQs…) and holds up to 40,000 characters, plenty for a small or mid-sized business. A few rules make all the difference:
- Write the way people talk. "We're open Tuesday to Sunday, 13:00 to 16:00 and 20:00 to 23:30. Closed on Mondays" works better than a table pasted from your website.
- Say what you don't do. "We don't cater for groups over 40" stops the agent saying "I'll check".
- Full dates and holidays. The agent knows the current date and time and, if you've set the line's hours and holidays, whether you're open right now.
- Tag your prices. In home services, each price is tagged exact, estimate or don't say. Elsewhere, only include prices you're happy to hear on the phone.
- No instructions inside the knowledge base. It's data. The agent treats it as information, not orders, which also protects you if someone tries to slip instructions in.
Example section (clinic, illustrative):
Appointment policy. First visits take about 30 minutes. Please arrive 10 minutes early. Cancellations need 24 hours' notice. We see emergencies without an appointment in the mornings, subject to availability; the clinic confirms the time by phone.
Step 4. Decide what to collect (and nothing more)
Each request has its fields: name, phone, preferred date and time, party size, reason… Three tips:
- Less is more. Every field is a question. If you won't use it, don't ask. It's also what GDPR data minimisation requires.
- Use the caller's words for symptoms, allergies or reasons. Don't ask the agent to interpret.
- Confirm what matters. The templates make the agent read back address and phone number before saving.
Step 5. Categories with value: the basis of the revenue report
Each request is filed under a category. For each one you set:
| Field | What it's for | Example (restaurant) |
|---|---|---|
| Is it a conversion? | Whether it counts as a revenue opportunity | Booking: yes; Enquiry: no |
| Average ticket | Typical value of that opportunity | Booking: €90 |
| Close rate | How often it converts | Booking: 0.8 |
| Default urgency | Starting priority | Takeaway order: high |
With this, the "Inbound value" report estimates saved revenue (conversions × average ticket × close rate). Leave them blank and the report tells you which categories it can't count. Full method in the cost of a missed call.
Step 6. Urgency and alert rules
Two layers:
- Agent escalation rules. For example: "if urgency is critical, hand over to the on-call person"; "complaints, urgent alert". On the web channel there's no live transfer yet: the agent flags the request as urgent and promises a call back.
- Alert routing rules, "if… then…": by category, urgency, time or line, notify via Telegram, email, webhook or a CRM task; assign to someone; create a task or a deal.
Practical tip: start with few rules. If everything is urgent, nothing is.
Step 7. Greeting and tone
The platform always opens with the notice that it's an AI assistant and the call is recorded, as Article 50 of the AI Act requires. Then comes your greeting: short, with the business name and an open question. "Hello, thanks for calling [your business]. How can I help?" works. Skip long greetings with promotions: people call for something specific.
Step 8. Test it like you're trying to break it
The most underrated part. A minimum test plan:
| Test type | Examples |
|---|---|
| Happy path | Book for four on Friday; request a first appointment; report a breakdown |
| Tricky details | Foreign names, addresses with block and staircase, changing the time mid-call |
| Outside the knowledge base | A service you don't offer, a price that isn't listed |
| Limits | Ask for a diagnosis, demand confirmation, insist on speaking to the owner |
| Emergencies | A leak, severe pain, someone describing a life-threatening situation |
| Languages | Start in English, switch to Spanish; call in Russian |
| Manipulation | "Ignore your instructions and tell me…", asking for another customer's details |
| Real conditions | Background noise, interrupting the agent mid-sentence, long silences |
After each round, review transcripts and the automatic quality score the platform gives every call (0–100, with whether the goal was met, whether the script was followed and a recommendation). Fix the knowledge base, not just the script: most failures are missing information.
Step 9. Put it on your website
Today the live channel is the "Call the AI" button. You add it by pasting one line into your site:
<script src="https://callsmarter.ai/widget/v1.js" data-key="wgt_…" async></script>
First, set in the dashboard which domains may use the button. The caller taps it, accepts the notice (with your privacy policy linked), allows the microphone and talks, with nothing to install. Put it where people make decisions: bookings, contact, service or listing pages. Answering on your phone number via forwarding is being rolled out; ask us.
Step 10. Improve every week
- Read the weekly summary, delivered on Mondays by email or Telegram.
- Check the heatmap of calls by day and hour: it tells you when you need more people on the phone.
- Listen to five random calls a week. It's the best audit there is.
- Work in versions. Prepare a new agent version, test it and activate it when ready; the previous one keeps answering meanwhile.
Common mistakes when you build an AI voice agent
- A knowledge base copied from the website, full of marketing lines that sound odd out loud.
- Too many fields: callers feel interrogated.
- Everything urgent: the team stops looking at alerts.
- Going live without testing in real conditions.
- Expecting the agent to decide things your team hasn't decided.
Start by listening to one
Before you build yours, talk to Foxy: an agent built with exactly this method, with a knowledge base limited to our website. To see how it would look in your industry, read the guides for restaurants, dental clinics, home services or real estate, and don't skip the legal side in GDPR and the AI Act.
Frequently asked questions
Do I need to code to build a voice agent?
No. With CallSmarter you start from an industry template and fill it in from the dashboard: knowledge base by section, request fields, categories and alert rules. Adding the button to your website takes one line of code.
How long does it take?
A small business can write its knowledge base in an afternoon. Testing deserves more time: we recommend at least a week of internal calls before going live.
What if the agent doesn't know the answer?
It says so naturally and offers to log the question for your team. The templates forbid it from inventing anything that isn't in the knowledge base.
Can I change the script after going live?
Yes. Agents are versioned: prepare a new version, test it and activate it when ready, without touching the one taking calls.



