Six gates that move an AI pilot to production | Ep. 14

About this episode

Everybody has an AI pilot. Almost nobody has a written path for getting one into production, and that gap is where this solo episode of Data vs. Commerce sits. Matt Johnson runs this one alone, with Floyd Blaikie out of the studio for the week.

ㅤ

Matt keeps hearing the same sentence from data, commerce, and supply chain leaders: we built this, we proved it works, and we can't figure out how to get it into production. His answer isn't a better tool or a smarter consultant. It's a stage gate process, six gates adapted from the years he spent leading R&D at a manufacturing company, applied to AI experiments in digital commerce.

ㅤ

The friction this week isn't data against commerce. It's experimentation against governance, and Matt argues most companies make the trade backwards. They restrict who gets to experiment, then leave the road to production undefined. He flips it. Train more people, give them permission, and put the checkpoints at the gates instead of at the door. He also prices out the free-for-all: duplicated work, abandoned projects, and tokens burned with nothing to show for them. Matt is a digital commerce leader at Pivotree.

ㅤ

📌 What We Cover

• Why pilots stall in silos, with the data team running one experiment, marketing running a pet project, and operations testing something nobody else has heard about
• The counterintuitive move: empowering more people to experiment, not fewer, because the ideas that matter come from whoever is closest to the friction
• What happens with no guardrails at all, from 50 uncoordinated experiments to burning through tokens like crazy and wondering whether AI is returning anything
• Gate one: a business sponsor, a rough business case, and a working prototype, and why none of it has to be precise yet
• Gate two and the council: the Shark Tank demo, real metrics like hours saved and costs reduced, and a budget approval that covers tokens and internal time
• Gate three and four: running the pilot as a controlled science experiment, 10,000 SKUs through AI and through the traditional process, then letting the data decide, including the decision to kill it
• Gate five as a checklist: documentation, integration with legacy systems, who owns the nine o'clock request, and whether it complies with your data governance policies
• Gate six and the seventh that isn't really a gate: adoption plan, success metrics, feedback loop, and continuous improvement

ㅤ

🔗 Resources Mentioned

• Pivotree
• OpenAI
• Anthropic
• Claude Code