Guides

How to build an AI voice agent for your business

How to build an AI voice agent step by step: a clear brief, knowledge base, request fields, categories, urgency rules, a test plan and weekly improvements.

Foxy dressed as a wizard preparing a voice agent's script and knowledge base

A voice agent is only as good as what you tell it. Voice technology already converses fluently; what separates a helpful agent from one that frustrates your customers is the groundwork: what it should do, what it knows, what it asks, when it alerts a person and how it was tested. This guide shows how to build an AI voice agent for your business, step by step, the way we do it at CallSmarter, with examples from restaurants, clinics, home services and estate agencies.

Step 1. Define the job before the voice

On a single sheet of paper, write down in this order:

  1. Why your customers call. The five or six most common reasons. If unsure, check the call log and ask whoever answers the phone.
  2. What the agent can resolve with information alone: hours, address, services, published prices.
  3. What it must collect so your team can act without calling back.
  4. What it must never do: confirm, negotiate, diagnose, take payment.
  5. What counts as urgent and who needs to know.

That sheet is your real script. Everything else is putting it into the tool.

Step 2. Start from an industry template

Starting from scratch is the slowest way to get it wrong. CallSmarter has templates for:

  • Restaurant: bookings and takeaway orders pending confirmation; allergens noted only, never guaranteed.
  • Dental clinic: appointments and changes, emergencies; no medical advice and 112 if life may be at risk.
  • Accommodation and real estate: bookings, viewings, owners, guest incidents.
  • Home services: breakdown, servicing and installation calls with urgency levels.

Each comes with a tested script, knowledge base sections, fields, categories and urgency rules. You adapt them; you don't rewrite them.

Step 3. Write a knowledge base that works out loud

The knowledge base is organised by section (hours, address, services, prices, policies, FAQs…) and holds up to 40,000 characters, plenty for a small or mid-sized business. A few rules make all the difference:

  • Write the way people talk. "We're open Tuesday to Sunday, 13:00 to 16:00 and 20:00 to 23:30. Closed on Mondays" works better than a table pasted from your website.
  • Say what you don't do. "We don't cater for groups over 40" stops the agent saying "I'll check".
  • Full dates and holidays. The agent knows the current date and time and, if you've set the line's hours and holidays, whether you're open right now.
  • Tag your prices. In home services, each price is tagged exact, estimate or don't say. Elsewhere, only include prices you're happy to hear on the phone.
  • No instructions inside the knowledge base. It's data. The agent treats it as information, not orders, which also protects you if someone tries to slip instructions in.

Example section (clinic, illustrative):

Appointment policy. First visits take about 30 minutes. Please arrive 10 minutes early. Cancellations need 24 hours' notice. We see emergencies without an appointment in the mornings, subject to availability; the clinic confirms the time by phone.

Step 4. Decide what to collect (and nothing more)

Each request has its fields: name, phone, preferred date and time, party size, reason… Three tips:

  • Less is more. Every field is a question. If you won't use it, don't ask. It's also what GDPR data minimisation requires.
  • Use the caller's words for symptoms, allergies or reasons. Don't ask the agent to interpret.
  • Confirm what matters. The templates make the agent read back address and phone number before saving.

Step 5. Categories with value: the basis of the revenue report

Each request is filed under a category. For each one you set:

FieldWhat it's forExample (restaurant)
Is it a conversion?Whether it counts as a revenue opportunityBooking: yes; Enquiry: no
Average ticketTypical value of that opportunityBooking: €90
Close rateHow often it convertsBooking: 0.8
Default urgencyStarting priorityTakeaway order: high

With this, the "Inbound value" report estimates saved revenue (conversions × average ticket × close rate). Leave them blank and the report tells you which categories it can't count. Full method in the cost of a missed call.

Step 6. Urgency and alert rules

Two layers:

  1. Agent escalation rules. For example: "if urgency is critical, hand over to the on-call person"; "complaints, urgent alert". On the web channel there's no live transfer yet: the agent flags the request as urgent and promises a call back.
  2. Alert routing rules, "if… then…": by category, urgency, time or line, notify via Telegram, email, webhook or a CRM task; assign to someone; create a task or a deal.

Practical tip: start with few rules. If everything is urgent, nothing is.

Step 7. Greeting and tone

The platform always opens with the notice that it's an AI assistant and the call is recorded, as Article 50 of the AI Act requires. Then comes your greeting: short, with the business name and an open question. "Hello, thanks for calling [your business]. How can I help?" works. Skip long greetings with promotions: people call for something specific.

Step 8. Test it like you're trying to break it

The most underrated part. A minimum test plan:

Test typeExamples
Happy pathBook for four on Friday; request a first appointment; report a breakdown
Tricky detailsForeign names, addresses with block and staircase, changing the time mid-call
Outside the knowledge baseA service you don't offer, a price that isn't listed
LimitsAsk for a diagnosis, demand confirmation, insist on speaking to the owner
EmergenciesA leak, severe pain, someone describing a life-threatening situation
LanguagesStart in English, switch to Spanish; call in Russian
Manipulation"Ignore your instructions and tell me…", asking for another customer's details
Real conditionsBackground noise, interrupting the agent mid-sentence, long silences

After each round, review transcripts and the automatic quality score the platform gives every call (0–100, with whether the goal was met, whether the script was followed and a recommendation). Fix the knowledge base, not just the script: most failures are missing information.

Step 9. Put it on your website

Today the live channel is the "Call the AI" button. You add it by pasting one line into your site:

<script src="https://callsmarter.ai/widget/v1.js" data-key="wgt_…" async></script>

First, set in the dashboard which domains may use the button. The caller taps it, accepts the notice (with your privacy policy linked), allows the microphone and talks, with nothing to install. Put it where people make decisions: bookings, contact, service or listing pages. Answering on your phone number via forwarding is being rolled out; ask us.

Step 10. Improve every week

  • Read the weekly summary, delivered on Mondays by email or Telegram.
  • Check the heatmap of calls by day and hour: it tells you when you need more people on the phone.
  • Listen to five random calls a week. It's the best audit there is.
  • Work in versions. Prepare a new agent version, test it and activate it when ready; the previous one keeps answering meanwhile.

Common mistakes when you build an AI voice agent

  • A knowledge base copied from the website, full of marketing lines that sound odd out loud.
  • Too many fields: callers feel interrogated.
  • Everything urgent: the team stops looking at alerts.
  • Going live without testing in real conditions.
  • Expecting the agent to decide things your team hasn't decided.

Start by listening to one

Before you build yours, talk to Foxy: an agent built with exactly this method, with a knowledge base limited to our website. To see how it would look in your industry, read the guides for restaurants, dental clinics, home services or real estate, and don't skip the legal side in GDPR and the AI Act.

Frequently asked questions

Do I need to code to build a voice agent?

No. With CallSmarter you start from an industry template and fill it in from the dashboard: knowledge base by section, request fields, categories and alert rules. Adding the button to your website takes one line of code.

How long does it take?

A small business can write its knowledge base in an afternoon. Testing deserves more time: we recommend at least a week of internal calls before going live.

What if the agent doesn't know the answer?

It says so naturally and offers to log the question for your team. The templates forbid it from inventing anything that isn't in the knowledge base.

Can I change the script after going live?

Yes. Agents are versioned: prepare a new version, test it and activate it when ready, without touching the one taking calls.

Sources

  1. AEPD: GDPR compliance of processing that embeds artificial intelligence (guide, Spanish)
  2. European Commission: transparency obligations under Article 50 of the AI Act