History-aware conversation
Use a returning lead with a changed timeline. See whether the assistant uses existing facts without treating guesses as CRM data.
A good assistant should do useful work inside your team’s workflow. Use this checklist to compare tools, ask better demo questions, and measure a pilot without confusing activity with results.
“Mortgage AI” can mean lead follow-up, voice qualification, document processing, underwriting support, or servicing. First define the job you are buying: for example, continue borrower text conversations in Bonzo and coordinate a consultation with the right officer.
Then list must-have channels, systems, ownership rules, contact restrictions, and outputs. A large feature list is not a substitute for an assistant that reliably completes your particular handoff.
Use a returning lead with a changed timeline. See whether the assistant uses existing facts without treating guesses as CRM data.
Have an officer reply, pause the lead, or change a stage. Verify whether automation stops or adapts as intended.
Follow the result to the stored booking and assigned officer. Check the date, time zone, and notice destination.
Ask a question that needs a licensed officer or unavailable information. The assistant should not invent an answer to keep talking.
Test missing history, a failed provider action, or a contact restriction. Look for a visible outcome and an appropriate next step.
Ask for plan, volume, setup, support, messaging, telephony, taxes, and relevant exclusions in writing.
| Metric | Definition to agree before testing |
|---|---|
| Eligible lead | A contact allowed to participate under the same ownership and contact rules. |
| Useful conversation | A conversation that establishes a relevant goal, next step, or need for human help. |
| Confirmed appointment | A stored booking with the required officer, date, time, and time zone—not merely a positive reply. |
| Attended consultation | A consultation that actually took place, measured separately from booking. |
| Complete cost | Platform and required third-party charges for the same measurement period. |
Cost per confirmed qualified appointment equals complete evaluation cost divided by appointments meeting your agreed definition. Cost per useful conversation uses the same cost divided by useful conversations. If there are no qualifying outcomes, report that explicitly rather than presenting a misleading cost of zero.
For a business-case estimate, distinguish assumptions from observed outcomes. Hours saved, attendance, applications, and funded loans are separate measurements. A booking is not a funded loan, and a successful synthetic demo is not proof of a production conversion rate.
Use fictional data to verify basic behavior without exposing or contacting real borrowers.
Set scope, duration, contact rules, human coverage, success definitions, and the conditions that stop the pilot.
Inspect messages, delivery outcomes, CRM writes, bookings, notices, and exception handling.
Use observed results and remaining limitations to choose the next stage of rollout.