After all these years of data analysis, most of the output answers one question: what happened. How much closed last month, which route grew, which salesperson converts poorly.
Useful, but still one step short of a decision. After reading the report, someone still has to work out: so which one do we pick now? And what does picking wrong cost?
I now prefer to split this in two: Data Insight answers "what happened"; Decision Insight answers "which one now, and what does a wrong pick cost". The second is what the business is actually short of.
Build the decision list before the insight platform
The usual opening move for "insights" is to build a platform: connect the data, add dashboards, add Q&A, then wait for people to ask. As I've written before, people don't have that many questions, and platforms like that mostly end up waiting.
Do it the other way round: first list the decisions the company actually has to make — who makes each one, how often, and on what data. Taking private-aviation sales as an example, a first draft looks roughly like this:
| Frequency | Decision | Depends on | Who decides |
|---|---|---|---|
| Per deal | What margin to quote, when to give on price | Closing prices on the same route and type, price sensitivity | Sales |
| Per deal | Which operators to send the inquiry to | Aircraft actually flying, reply speed, refusal and swap history | Sales |
| Daily | Whether to chase a lost deal, and how | Whom the customer ended up flying with, repeat patterns | Sales lead |
| Weekly | Which customers are about to churn | Customers' flying patterns, market-wide flight data | Sales lead |
| Quarterly | Which routes and types are worth locking capacity on | Demand forecast, empty-leg distribution, margin tiers | Management |
| Yearly | Which market to open next | Market share, unserved demand | Management |
The top half is tactical decisions: high-frequency, deciding how each deal is played. The bottom half is opportunity insight: low-frequency, deciding direction. They need different data, a different delivery rhythm, and carry different costs when wrong. They don't belong on the same dashboard.
Once the list exists, you don't build all of it. Fill it out to fifteen or twenty decisions, then start with the three that move the most money.
Every recommendation delivers four things
I require every Decision Insight to come with all four:
- The recommendation: which one, in a single sentence.
- Confidence: including the historical hit rate of similar recommendations. Not the confidence the model reports about itself — how often it has actually been right.
- Evidence: every fact cited traces back to a source.
- The overturn condition: under what circumstances this recommendation no longer holds.
The first three occur to everyone. The fourth is the easiest to drop, and the most important.
A recommendation without an overturn condition leaves the reader two choices: believe it or don't. With one, the reader knows what to check: "Send it to these three, but if the customer insists on a same-day return, the second one is out because of its airport's curfew." The second half of that sentence sounds far more like a senior salesperson than the first.
A judgment you can entrust is one that can say when it would be wrong.
Let the hit rate grow on its own
The "historical hit rate" in item two doesn't exist at the start. It has to be written back: for every recommendation, whether sales took it and how it turned out.
This is the same thing as the decision trace. After a while, what management sees is no longer "the AI says pick this one" but, say, "on this kind of question the AI's recommendations have been right eight times in ten; on that kind, only half the time." Trust in AI should be granted separately by type of question, and granted by data.
One rule for the review: a recommendation may only be judged against what was known when the decision was made. What you learned afterwards can't be used to call the original recommendation right or wrong.
How this relates to control points
In the previous piece I wrote that an insight not wired to a control point is just a notification. The decision list is really a search for control points: each decision is a real node in a business pipeline, one that can be changed.
The list is what tells an insight where to connect; the four deliverables are what make it worthy of that position once it's connected.
So
Next time someone says they want to build an "intelligent insight platform", I'll ask two questions first:
Which decision, made by whom, is it meant to help with? And will its recommendations tell you when they stop being true?
Can't answer the first, and it's a dashboard. Can't answer the second, and it's a very confident dashboard.