In the last four years we have delivered fourteen machine learning projects. Five returned multiples of their cost. Six broke even or better. Three died quietly within a year of launch. When we lined them up afterwards, the winners and losers were distinguishable before any model was trained, from the shape of the problem alone.
The pattern behind every ML win
Every project that paid off shared four traits. It was a repeated decision, made hundreds or thousands of times a month. Each decision had a measurable cost of being wrong. There was historical data recording both the decision and the outcome. And a wrong prediction was recoverable: a human could catch it, or the cost of one bad call was small.
Miss any one of those and the economics collapse. A decision made twelve times a year cannot repay a model. A decision with no recorded outcomes cannot train one.
Four use cases with real returns
Credit and risk scoring. Prosper Capital was manually reviewing every loan application. We trained a scoring model on four years of application and repayment history. It now auto-approves the clearly good 55 percent and auto-declines the clearly bad 15 percent, leaving humans the genuinely hard middle. Review time per application dropped from 40 minutes to 9.
Demand forecasting. Retail and food businesses order stock against guesses. A forecasting model that beats the buyer’s gut by even 10 percent shows up directly as less waste and fewer stockouts. This is usually the cheapest ML win available because sales history already exists in clean form.
Churn prediction. If you have subscription revenue, you have churn, and the behavioural signals almost always exist in your product data: logins dropping, support tickets rising, feature usage narrowing. Ranking customers by churn risk lets a small retention team spend its time where it matters.
Document and message routing. Classifying incoming email, claims, or support tickets and sending each to the right queue. Unglamorous, quick to build, and it compounds daily.
55%
of loan applications auto-decided at Prosper Capital
9 min
average review time, down from 40 minutes
14 weeks
from kickoff to the model running in production
Where ML disappoints
- One-off strategic decisions. Whether to open a Durban branch is a spreadsheet and a debate, not a model.
- Problems where you have under a few thousand labelled examples. The model will memorise noise.
- Anything where an unrecoverable wrong answer is catastrophic and no human checks the output.
- Problems a rule solves. If "flag orders over R50 000 from new accounts" catches 95 percent of fraud, ship the rule and move on.
- Projects motivated by the board wanting "an AI story" rather than by a costed decision.
How to score a candidate project
- 1Count the decisions per month. Under a few hundred, stop.
- 2Put a rand value on a wrong decision. If nobody can, stop.
- 3Confirm you have at least a year of data recording decisions and outcomes. If not, start logging now and revisit in six months.
- 4Write the afternoon rule and measure it. This is your baseline.
- 5Estimate the model uplift conservatively at 10 to 20 percent over the baseline, then check whether that uplift, in rand, repays the build within 18 months.
Start smaller than you think
Our successful projects averaged 12 to 16 weeks for the first production version, built by two people. The failures were mostly ambitious platforms meant to do everything at once. A narrow model, in production, learning from real feedback, beats a broad model in a slide deck every time.
Ongoing cost matters too: models drift as the world changes, so budget roughly 15 percent of the build cost per year for monitoring and retraining. A model nobody watches degrades silently, and silent degradation is worse than no model.
If you have a decision that fits the pattern, our AI and machine learning practice will run the scoring exercise above with you in a single workshop. And if your data is not ready yet, the honest first step is usually the warehouse work we covered in data warehouse or data lake.