You do not need to evaluate the technology. You need to evaluate the proposal.
An opinion piece, and labeled as one: the data is real, the take is mine.
There is an AI proposal sitting in your inbox right now that you are not sure about. This page is how I would evaluate it, and the method has nothing to do with evaluating the technology.
The grift works because failure is the base rate. When 95 percent of generative AI pilots deliver no measurable return, a vendor who delivers nothing looks statistically normal, and by the time you know, they are two invoices ahead. 42 percent of companies scrapped AI initiatives in 2025, up from 17 percent a year earlier, with average sunk costs in the millions. The flip side is the useful part: companies buying from specialized vendors succeed at roughly two thirds, while internal builds succeed at a third of that rate. It is not the AI that fails. It is the integration, and the grifter sells you the model story precisely to distract you from the integration work.
Ten percent of AI value is the algorithm, which is the part that demos beautifully. Twenty percent is data: preparation, access, permissions, pipelines, the unglamorous, unavoidable middle. Seventy percent is process and governance change: interviewing operators, redesigning workflows, training people. The part that actually pays.
Grifters always sell the 10 percent wrapped in the language of the 70 percent. Every red flag below is that same move in different dress. The corollary that will save you the most money: pick your AI partner the way you pick an ERP, not the way you pick a model.
Speed of close is inversely correlated with quality of diagnosis.
Custom builds fail at roughly twice the rate of integrating a proven platform. Buy the platform, build the integration.
If you cannot see how you would leave their platform, assume the opacity is intentional.
The payment schedule is a confidence test the vendor takes on their own product.
Data preparation is 60 to 80 percent of any real AI project. If it is not budgeted and staffed, the vendor either does not understand the work or is hiding the hardest part until after signature.
The tool attached to this playbook is a one-page scorecard: five dimensions, the exact question to ask, and what clean and dirty answers sound like. Score each answer 0 (flag present), 1 (mixed), or 2 (clean). Serious vendors score 8 to 10. Below 6, you are funding someone else's learning curve. And watch how the vendor reacts to being asked. The reaction is the data.
| Ask this | Clean sounds like | Dirty sounds like |
|---|---|---|
| "Walk me through how you will map our workflow before scoping." | Named operators, ride-alongs, a delivered process artifact. | "We'll start with a discovery call." |
| "Which proven platform are you integrating, and why that one?" | A named platform and a reasoned fit. | A custom build with no platform named. |
| "In one sentence: how do we leave you, and what do we keep?" | One sentence, data formats named. | A pivot to features. |
| "Tie payment to milestones. What is verified before phase two?" | Staged, verifiable. | Front-loaded, vague, early-pay discount. |
| "Show me the data-preparation line item." | Scoped, staffed, priced. | "Leveraging your existing data." |
The buyers who get this right do not pick a vendor; they build a market and shop in it. One capability bar (the five flags as positive criteria). One contract template with staged payment, an exit and portability clause, a named data-preparation scope, and audit rights: write it once, sign it five times with five vendors. Qualify three to five vendors so the market does price discovery while your framework does quality control. And remember that renewal is the only lever you actually have; scoping and portability go into the contract at renewal, not into a policy document nobody enforces.
Twenty minutes to score the proposal already in your inbox. One legal cycle to add three clauses to your template. The alternative is the base rate, and the base rate runs seven figures.
The five flags survived a year of real proposals unchanged, and the walk rule stands. What sharpened is my language on flag two. "Proprietary build" used to sound like the opposite of "buy the platform," and vendors learned to hide behind that: everything is a build if you squint. The real distinction is building versus integrating. Nobody at mid-market scale should be training models, running their own platforms, or writing their own harnesses. The legitimate work is plumbing: wiring proven platforms into your workflows and modifying them to function the way you need. You are customizing a car, not building one. A vendor proposing to build you the car is flag two, every time; a vendor scoping the plumbing, with the platform named, is what good looks like.
Phase Zero produces the workflow map and the data-readiness picture before any vendor scoping, so every proposal you receive gets scored against reality. If you have already mapped it yourselves, a Pilot Build picks up from there.
Start with a Phase Zero assessment Explore all engagements