What is AI vendor diligence, and how do you do it without becoming the bottleneck?

An AI vendor sends a deck. The deck claims a proprietary model, shows a customer success story with redacted numbers, runs a demo on dummy data, and quotes twenty-four thousand dollars a year with a twelve-month minimum. You suspect parts of it are theatre. You are right. You also need to make a decision in two weeks, the COO has a budget line for AI tooling, and a junior engineer on your team likes the product. This piece is the framework I use to read that pitch in under an hour.

What AI vendor diligence actually is

AI vendor diligence is the structured assessment of a vendor pitch before you sign. It does four things:

1. Strips the marketing. Names what the product actually does in technical terms, separate from how the pitch describes it. “Proprietary AI” becomes “a system prompt over GPT-4o with a vector store and a queue UI.”
2. Tests against your use case. Generic assessments are worthless. The question is not whether the tool is good; it is whether it helps you specifically.
3. Surfaces the specific red flags. Not the generic list; the ones in the actual pitch in front of you.
4. Produces the questions to put to the vendor in writing. Verbal answers can be qualified later. Written answers cannot.

It ends in a recommendation that is one of three things: Go (the product fits, sign with named conditions), Maybe (the shape is right but a piloted test answers the open questions before the annual commitment), or Stop (the product is the wrong shape, the wrong price, or wrong on data; here are the alternatives).

The ten patterns that signal theatre

Vendor pitches in 2026 fall into ten recurring patterns. Any one of them is not a deal-breaker; three together usually is.

1. “Proprietary AI” with no model named. Ask which underlying model, who their contract is with, and what the data retention and training position is.
2. The demo runs on dummy data. Insist on a paid trial with your real data before any annual commitment. A confident vendor agrees on the call.
3. The data-storage question is dodged. Where the data lives, in which jurisdiction, and whether it is processed outside Australia are answerable in writing in one sentence. Dodging them is the answer.
4. The price is five to ten times raw API cost. That is fine if you are paying for UI, audit trail and support. The vendor should be able to articulate what you are paying for.
5. Twelve-month minimum, no pilot option. The counter-offer is a one-month paid pilot at roughly one twelfth of the annual quote. Confident vendors say yes.
6. No reference customer at your scale in your industry. “Lots of enterprise customers” is not a reference. Two named contacts running roughly your volume, in roughly your industry, live for more than six months, is.
7. Accuracy claims with no baseline. “Ninety percent accurate” on what dataset, measured how, by whom? Ask for first-pass accuracy on something like yours, plus the human-in-the-loop edit rate at month one versus month six.
8. The roadmap answers more questions than the product. Score the pitch on what ships today. The roadmap is a hope.
9. Cancellation terms are vague. What happens to your data, your prompts, and your audit trail the day you stop paying needs to be answered in the contract, not the FAQ.
10. The seller cannot explain how it works in plain English. If the explanation hides behind jargon twice in a row, the gap is usually in the product, not your understanding.

The free two-page checklist I use to read pitches is downloadable from the homepage. It carries the same ten patterns with the question to ask printed next to each one. Print it, take it into the vendor meeting, and watch which questions land and which slide.

The wrapper test

A common pattern in AI vendors in 2026: the underlying capability is a system prompt over a third-party model (almost always OpenAI’s GPT family or Anthropic’s Claude), with a custom UI, a queue or workflow on top, and an audit trail. This is a wrapper. Wrappers are not bad. Some are excellent products. The issue is paying enterprise-AI prices for one.

The wrapper test is two questions. Both answers belong in writing.

Which underlying model does your product use, and what is your contractual position with that model provider on data retention and training?

Beyond the underlying model, what specifically is your differentiated capability? Training, tuning, retrieval, eval, governance, or another concrete layer?

If the answer to the second question is “our prompt” or “our user interface,” you are buying a wrapper. The price should reflect that.

What good vendor diligence produces

A useful vendor memo is three pages and structured for a busy buyer. It contains, in order:

• A plain-English read of what the product actually does, with the marketing stripped.
• An honest assessment of whether it fits your specific use case.
• Red flags called out specifically, not generically. Generic red flags are worth less than specific ones.
• Five to seven sharp questions to put to the vendor in writing. Watch how fast they answer, and which ones they answer with specifics.
• Two to three alternative approaches with honest trade-offs. A paid pilot, an internal build on the raw API, or staying with the current process are all credible alternatives.
• A clear Go, Maybe, or Stop recommendation with rationale.

The DPEX AI Vendor Diligence tool produces exactly this format automatically from pasted pitch material, in about a minute, for ninety-five dollars per vendor. You can see a sample memo as a PDF before paying.

When the stakes justify having me in the room

For six-figure annual contracts, the memo is the prep, not the decision. The full version of vendor diligence is putting the questions to the vendor in writing, watching how they answer, and having an external read in the room when you do. That is a separate engagement, priced as a fixed fee per vendor.

The honest summary

AI vendor diligence is not a content marketing exercise; it is a procurement function that has not caught up with the shape of the market. The patterns above are reliable enough that you can run your own diligence with the checklist and a clear use case. For serious contracts, get a structured memo before the second meeting. For the largest contracts, get someone in the room when the questions are put.

The tool below produces a memo automatically. If you want to run one against a pitch you already have on your desk, paste it in; the memo is yours in about a minute.