How to test a missed-call AI agent end to end

Test the system, not just the model

A polished AI reply proves very little on its own. The product includes the phone event, webhook security, the messaging provider, the conversation's memory, your business rules, the database, the workspace, the scheduler and the hand-off to a person.

Acceptance testing should follow the exact path a real customer and a real business owner will experience, and it should fail loudly when any link in the chain breaks.

Before you start

• Two test phones you control: one to call from, one to receive owner alerts.
• The business number, and confirmation that it's the one customers really call.
• The rules for this business: services, area, hours, booking link, what counts as urgent.
• An acceptance sheet, so every result is written down as you go.
• Permission: only text numbers that belong to your testers.

The happy-path test

Do the whole sequence without editing anything in the database by hand.

1. Call and let it ring out

   Call the business number from a test phone, and don't answer.

2. Check the first text

   One text, not two, from the right sender, with the business name and an opt-out line.

3. Reply like a real customer

   Describe a realistic request for this business, and answer the follow-up questions naturally.

4. Ask for a booking

   Request an appointment or a callback, at a time the business's rules allow.

5. Check the brief

   Confirm the final message, the booking request, the summary and the full conversation in the workspace.

6. Check the alert

   If the request should alert the owner, confirm the alert arrived exactly once.

Boundary and compliance tests

• Send STOP and confirm nothing else arrives, including follow-ups and reminders.
• Send an emergency-sounding message and confirm the agent gives no unsafe advice and alerts a person.
• Ask for an exact price and confirm the reply follows your pricing rules.
• Ask "are you a bot?" and confirm the answer is honest.
• Try a caller outside your service area, a service you don't offer and a missing address.
• Change the requested appointment time halfway through the conversation.

Failure and duplicate tests

Providers retry webhooks, networks pause and workers restart. Send the same event twice and confirm only one text goes out. Simulate a failed send and confirm the retries back off, rather than firing every minute.

Restart the web service and the scheduler, then check that pending work, like follow-ups, is still in the database and still goes out. Confirm private test endpoints can't be reached from the public internet.

Extra checks for WhatsApp

• Confirm the first message uses your approved template, with the business name filled in.
• Reply, and check that answers flow freely inside the 24-hour window.
• Leave a conversation for more than 24 hours, and confirm the next business-started message uses a template again.

The reasons behind these rules are in SMS or WhatsApp for missed calls.

What to record

Keep an acceptance sheet with the time of each action, the expected result, the actual result, screenshots, the provider's message IDs, and who owns any failed step. Never paste passwords, keys or tokens into it.

Ready for a pilot?

A workflow is ready for a pilot only when the happy path, the STOP path, the failure path and the hand-off path all behave predictably. We run checks like these with every Dolittle pilot before any real customer sees it.

New to the idea? Start with the complete guide to missed-call text back.