How do you choose an AI tool or vendor?

Start with the job

Before you look at any product, write one sentence describing the problem: "We miss calls after 5 PM," or "I spend an hour a day sorting inquiries." Then write how you will know it is solved: fewer missed calls, an hour back, faster first replies.

This keeps you in charge of the conversation. A demo will always look impressive. The question is whether it solves your problem, in your business, with your customers.

The question checklist

Ask these of every tool or vendor. Write down the answers so you can compare fairly.

What it does

• What exactly will this do for my business, step by step?
• What will it not do, and where does a person need to step in?
• Can I try it on my own real examples before committing?

Claims and evidence

• What results have you measured, and how? Can you show me, not just tell me?
• Have you tested it on businesses like mine?

The Federal Trade Commission's advertising guidance says advertisers must have evidence to back up their claims [1]. That principle has been applied to AI products. In February 2025, the FTC announced a final order against a company that marketed an "AI lawyer" service; according to the FTC's complaint, the company did not test whether it performed at the level of a human lawyer, and the order bars it from advertising that the service performs like a real lawyer unless it has sufficient evidence [2]. A trustworthy vendor will welcome evidence questions.

Your data

• Where is my data stored, and for how long?
• Is my data, or my customers' data, used to train or improve your AI or anyone else's? Can I turn that off?
• Can people at your company, or your providers, see my conversations or files?
• Which other companies' AI services does your product rely on?

Providers document these things differently by product and plan. For example, Google's Gemini privacy hub notes that work and school accounts may be subject to different data terms than personal accounts [3]. Ask any vendor to show you the equivalent written policy for exactly what you will use. (Lesson 4 covers what never to paste in, whatever the policy says.)

NIST, the U.S. standards agency, points out that AI products often depend on many third-party parts, such as models and data from other companies, and that errors in those parts can affect accuracy downstream [4]. That is why asking "what do you rely on?" matters.

Review and control

• Can I see what the tool said or did, such as transcripts or logs?
• Can I set rules about what it must never say or do?
• How do I correct it when it is wrong?
• Who do I contact when something breaks, and how fast do they respond?

Cost and exit

• What is the full monthly cost, including usage fees, setup and add-ons?
• What happens to cost if my volume doubles?
• How long is the contract, and how do I cancel?
• If I leave, can I export my data, and will you delete what you hold?

Warning signs

• Promises of guaranteed results, such as a set number of new customers or sales.
• Vague answers about data, or no written policy.
• No trial, or a demo that only uses the vendor's own perfect examples.
• Pressure to sign a long contract quickly.
• No way to review what the tool is doing in your name.

Run a fair trial

Pick two or three real tasks. Run each option on the same tasks. Score the results against the success sentence you wrote at the start. Have the person who will use the tool every day try it too. Their view on ease of use can decide whether a tool actually gets used.

Common mistakes

Choosing from the demo. Demos show the best case. Test on your own messy, real examples.

Skipping the data questions. Data terms differ widely and are hard to change after you sign.

Ignoring total cost. Low monthly prices can grow quickly with usage fees.

No exit plan. Know how to leave before you commit.

Worked example

A property management company wants help answering after-hours tenant messages. The owner writes the goal: "Every after-hours message gets a polite reply and is logged for the morning, and nothing is promised without a person."

She shortlists two options and asks both the checklist questions. One gives clear written data terms, shows full conversation logs and offers a two-week trial. The other promises "more satisfied tenants guaranteed," will not say whether messages are used for training, and requires a one-year contract.

She trials the first option on twenty real past messages. It handles routine questions well, and she adds rules so maintenance emergencies always reach the on-call person. She signs a month-to-month plan, with a review date on her calendar in sixty days.

Checklist

• I wrote down the problem and how I will know it is solved.
• I asked every option the same questions and recorded the answers.
• I have the vendor's data policy in writing for the exact plan I will use.
• I can review the tool's work and set rules for it.
• I know the full cost and how to leave.
• I tested it on my own real tasks before signing.

Try it

Choose one AI tool you use or are considering. Answer the "Your data" questions using only its official documentation. Note any question you cannot answer, and send those to the vendor. Their reply will tell you a lot.

Tips

• Ask for evidence, not adjectives.
• Get data terms in writing for the exact plan you will buy.
• Trial on your own real examples, side by side.
• Prefer short commitments until a tool has proven itself.

Sources

1. Federal Trade Commission, "Advertising FAQ's," FTC business guidance, https://www.ftc.gov/business-guidance/resources/advertising-faqs-guide-small-business
2. Federal Trade Commission, "FTC Finalizes Order with DoNotPay That Prohibits Deceptive 'AI Lawyer' Claims, Imposes Monetary Relief, and Requires Notice to Past Subscribers," press release, February 11, 2025, https://www.ftc.gov/news-events/news/press-releases/2025/02/ftc-finalizes-order-donotpay-prohibits-deceptive-ai-lawyer-claims-imposes-monetary-relief-requires
3. Google, "Gemini Apps Privacy Hub," Gemini Apps Help, last updated September 24, 2026, https://support.google.com/gemini/answer/13594961
4. National Institute of Standards and Technology, "Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile," NIST AI 600-1, July 2024, https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf