Baseline first
The discipline that lets ML projects survive contact with operations: measure what you are replacing before you build the thing that replaces it.
Most ML projects that die in production die for the same reason: nobody measured what they were replacing.
So before I build a model, I quantify the accuracy of the current human or rule-based process first. Without that baseline, leadership can’t tell whether the model is actually better than what it replaces — and “better than nothing” isn’t a number you can defend in a capital review.
The baseline does three things:
- Gives the model a concrete target to beat.
- Turns a vague “AI project” into a measurable business case.
- Tells you when to walk away.
That last one matters most. I once took several swings at a capital-project risk-optimization model. It never shipped — the upstream process simply wasn’t capturing enough of the right signal to reduce the estimation variance. Instead of shipping a bad model, I handed the diagnosis back to the source teams as a set of recommendations for what to start collecting, so a future model could succeed.
Walking away from a use case when the data isn’t there is the senior judgment call most practitioners don’t make. The baseline is what makes that call legible.