The Vendor AI Claim Rubric

The situation you are in

Three vendors want fifteen minutes this week to show you the AI feature they added to something you already own. A fourth has a product that did not exist in March. Your finance lead wants to know why the software line moved. Someone on your team already signed up for a tool on a personal card because the official path was slower than their deadline, and they were right about that.

You are not an AI expert and you are not going to become one. You have a day job that this is a small part of. What you need is not expertise. It is a way to tell, quickly, whether the thing in front of you is worth buying, what it will actually cost, where your data ends up, and who is on the hook when it is wrong — and then a way to write that down so that in a year nobody has to reconstruct what you were told.

Right now the honest version of your process is probably this: the demo looks good, the price looks fine, the vendor says the right words about security, and you say yes because saying no requires a reason you do not have time to produce. That works until it does not.

What this kit is

A judging method and the paper to go with it. Eight chapters, about 11,400 words, and nine templates — one of which is a card that fits in a notebook.

The method is five questions, in a fixed order, that take about fifteen minutes and work on any product in any sector. Around them sits a scored rubric with a few hard gates, a one-page privacy checklist, a questionnaire you send before the demo, and a one-page memo format so the decision exists somewhere other than your memory.

It is opinionated where opinions help. It says an accuracy number without a test set is decoration. It says "a human reviews it" is only a control if that human has the time, the expertise, and the standing to say no. It says the default state of the next AI feature the vendor ships matters more than the one they are demoing. It says most of your answers should be no, and that a no with a reason costs less than a yes without one.

It is also short on purpose. A review process nobody can finish is a process nobody uses, and the tools you never saw are the ones that hurt you.

What is inside

• The guide. Eight chapters, about 11,400 words: what "AI-powered" usually means and the four kinds of claim; the five questions; data and where it actually goes; accuracy and who is accountable when it is wrong; control, logs, and exit; cost in three layers; running both the fifteen-minute review and the deeper one; and keeping a register you re-check annually.
• In the meeting. The Five Questions card — one page, the thing you will actually carry — and a demo script with the minute-by-minute, the three things to ask to see in the product, and what to leave out of the room.
• Deciding. The scored vendor rubric, weighted across data, accuracy and accountability, control and exit, cost, and fit, with pass/fail gates that a great demo cannot outscore. Plus the one-page data-privacy checklist and the vendor questionnaire you send three days before the meeting.
• On the record. A one-page decision memo format, with a filled invented example, and a tool register with the eleven columns worth keeping and four cheap ways to find the tools nobody told you about.
• Staying honest. An annual re-check checklist to run sixty days before each renewal, and a red-flags list: four that end a conversation, fourteen that make it longer.
• START-HERE. Which page to open fifteen minutes before the call, and what to read if you have an hour, a day, or a week.

Nine templates in all. Every one uses [BRACKETED PLACEHOLDERS], carries a "How to adapt" note, and is plain markdown, so it pastes into a document, a wiki, or a ticket without fighting formatting.

Who wrote it and why he can

Wayne Bridges, a public-sector solutions engineer who builds and runs AI agent systems daily.

That second half is the part that matters here. The questions in this kit about where data goes, what "we don't train on your content" leaves out, how a wrapper behaves when the model underneath it changes, and what an invoice does when adoption goes well are not questions collected from articles. They come from operating these systems, paying for them, and watching them fail in specific, unglamorous ways.

The first half matters too. He has spent years on the buying side of the table in organizations that have a purchasing office, an audit trail, and a board or council that asks questions — and where the honest answer is often "we have to decide this month with incomplete information." This kit is built for that meeting, not for a research budget.

He is not a lawyer, and every chapter that touches law, policy, HR, or procurement says so at the top.

What this is not

• Not legal or procurement advice. It tells you what to ask. Your counsel and purchasing office tell you what you may do.
• Not a product recommendation. No vendors are named anywhere. Examples are invented and labelled as such.
• Not a security review. It does not replace a SOC 2 report, a penetration test, or whatever your organization requires. It sits in front of them and tells you whether it is worth asking for them.
• Not sector-specific. The structure of the question is the same everywhere; the law on top of it is not. Where your sector adds a layer, the guide says so and points at it.
• Not anti-AI. The author builds with these tools every day. The point is to buy the good ones on terms you understand.

Who should not buy this

• Organizations with a mature vendor risk program, a security questionnaire process, and a privacy officer who already reviews every AI feature. You have this covered; you might still want the five-questions card for the managers who get pitched directly.
• Anyone looking for a list of approved tools or a vendor comparison. That list is yours to build, and the rubric is how.
• Anyone who wants the sector's policy language — board policy, council resolutions, staff acceptable use. That is what the sector kits are for. This is the judging method underneath them.
• Anyone hoping a rubric will settle an argument that is actually about something else. If leadership has already decided, the honest move is a memo saying so, not a score.

If you approve software and you are tired of nodding through AI pitches you cannot evaluate: read the five questions, take them to your next demo, and end that call on time.