Why a checklist beats a vendor pitch

When a sales engineer demos a chatbot that answers customer questions in 0.4 seconds, it is easy to think the hard part is the model. It is not. The hard part is whether your organisation can actually feed it the right data, accept its answers, and operate it for 36 months without it quietly degrading. A checklist forces you to look at the parts a vendor will never bring up: who owns the source data, who reviews the outputs, what happens when a process changes upstream, and how you will know whether it is still working in month 18. We have shipped AI work into companies of fewer than 200 employees and into divisions of 5,000-person enterprises, and the same five categories decide whether a project becomes infrastructure or a forgotten experiment.

1. Data readiness

You do not need a data warehouse. You need data that is consistent, retrievable, and trustworthy enough that you would let a junior employee make a decision based on it. In practice this means three things. First, the data lives in a system with an API or, at minimum, a clean export — not in PDFs scanned by a former intern. Second, the people closest to the data agree on what each field means: if "customer" sometimes means "buyer" and sometimes means "end user", you have to clean that up before any model touches it. Third, the data is current. A RAG assistant grounded in two-year-old policy documents will confidently quote rules that no longer apply. Before kicking off any AI project, write down the top five data sources it will rely on and rate them on those three axes from 1 to 5. Anything below 3 is a project risk that needs a remediation step in the plan.

2. Process clarity

AI shines on processes that are well understood and slightly tedious — invoice extraction, support triage, contract first drafts, internal Q&A. It struggles on processes that are messy, contested, or where humans are still arguing about what the right answer is. The honest test is this: if you handed the process to a smart but new employee with the documentation you have today, would they get it right? If not, you are about to ask a model to do something the team itself has not converged on. The fix is not a better model. The fix is sitting with the team, mapping the actual current process (not the one in the SOP), and deciding which steps the AI is actually being asked to take over versus only assist.

3. Technical readiness

You do not need a full-stack platform team. You do need a few specific things: someone who owns identity and access (so the AI assistant respects who can see what), an existing system the AI can plug into without forcing every user to switch tabs, and a basic logging discipline so you can see what the assistant did this week. The single biggest predictor of an AI project succeeding is not model quality. It is whether the team can answer "what did the assistant do yesterday and was it right" in under five minutes. If that is opaque, the project will drift. If it is observable, problems get caught early.

4. Change management

The unsexy half of every AI project. The team that owns the workflow needs to feel the AI is helping them, not replacing them or grading them. That is a deliberate communication choice from leadership, not a side-effect of a good launch email. Three concrete moves: include two or three users from the affected team in the design from week one; ship the assistant in a "draft mode" first where it writes but a human still sends; and make a public commitment that the goal is not headcount reduction (or, if it is, say that out loud — pretending otherwise is worse). Quiet sabotage of AI tools is real. We have seen support agents intentionally not log their improvements so the metric would not move. Trust costs nothing to invest in early and is very expensive to repair later.

5. Measurement and governance

Decide before launch how you will know it worked. Not "ROI" in the abstract — three concrete metrics, baselined now, that you will look at in 90 days. Examples: average time to first draft of a sales proposal; percentage of support tickets resolved without human escalation; days between a policy update and the assistant reflecting it. Pick numbers your team already cares about, not numbers that sound good in a board deck. On governance: even at 30 employees, write down who can change the system prompt, who reviews flagged outputs, and what happens if a customer complains the assistant said something wrong. Two pages, not twenty. The companies that struggle later are the ones that bolted governance on after a near-miss.

Putting it together

Score your project on the five dimensions on a 1–5 scale. Below 3 on any one of them is the place to spend the next two weeks before you sign anything. We have a longer version of this checklist with vendor questions, sample governance documents and the discovery interview script we use; if that would be useful, get in touch via the contact form and we will share it. The point of the checklist is not to slow you down. It is to make sure that when you do move, you move once.