Explicit Rules or Statistical Inference: Choosing How to Solve a Prediction Problem
- Difficulty
- Moderate
- Time to result
- ~weeks to results
- Steps
- 6
- Confidence
- —
The decision rule underneath the entire history Cukier tells: classical AI tried to write down every rule a human uses, and it hit a wall because reality has infinite exceptions. Statistical machine learning reversed it — give the machine a large body of examples and let it work out the system for itself. The test for which approach applies is whether you can explicitly describe the phenomenon you are trying to predict. If you can enumerate the rules, code them and keep full explainability. If the exceptions explode — every misspelling of a spam word, every cat photographed tailless from behind — stop writing rules and start collecting examples. The method then transfers across domains: the same engineers who built spam filters published on reading retina scans. Accepting probabilistic error is the price, and the acceptable error rate is set by the stakes and the time saved.
Origin
Extracted from Deep Dive with Ali Abdaal
How to run it
- 1
Try to write the rules explicitly
Attempt the decision tree first. If the task is genuinely enumerable, explicit instructions give you a program you can read on a piece of paper and full explainability of every output.
- 2
Test whether the exceptions explode
Push on the edges. Can you code every permutation a human reads as the same word when digits replace letters? Can you describe a cat when it has no tail and is seen from behind? If each fix spawns another exception, the explicit route is fools gold.
Pro tip Cukier's tell: the early gains come fast, then it gets harder and harder to wring out any further benefit.
- 3
Collect a large body of labelled examples
Gather examples under all the conditions the real world produces — every handwriting style in a postcode, several million cats under all lightings. Scale is not optional here; the technique fails outright on a small sample.
Watch out You cannot do this with a small body of data; you simply would not have enough information for an inference to hold.
- 4
Let the system infer the distinguishing traits
Overlay the examples and let the algorithm find the traits that best predict one thing versus another, layer by layer from contrast to edge to shape. You are converting the problem into a large probability table rather than a rulebook.
Pro tip You do not need to know which variables and coefficients it used — the intricacy exceeds human capacity to grok, and the effect is that it works better than the alternatives.
Watch out Accept the loss of explainability consciously; this is an inference, not an answer.
- 5
Set the acceptable error rate by the stakes
Quantify the inaccuracy and weigh it against time saved and the alternative. A day's delay on a lung cancer diagnosis changes nothing; a brain haemorrhage answer in four seconds instead of forty minutes is worth a one-in-five misdiagnosis rate. The alternative in a rural clinic may be no diagnosis at all.
Pro tip Define the workflow question explicitly: what does the consumer expect, under what conditions, and does a human vet the output?
- 6
Transfer the method to a new domain
Once the pattern works, port it. Train on what you know, then predict on what you do not — the same approach that stopped spam went on to spot the traits of cancer years before the cancer forms, often from data not obviously related to medicine at all.
In the wild
Nobody can hard-code every way a human still reads Viagra when letters become numerals. So spam filters stopped trying and made a Bayesian inference from a large flow of scored examples, with a handful of people marking items as spam teaching the system for everyone. Cukier's punchline is the transfer: the same Google engineers who worked on the spam filter published a paper on reading a retina scan to identify who will develop disease. The method — train on the known, infer on the unknown — did not care that the domain changed.
→ Hundreds of millions of spam items stopped, and a decade of academic papers diagnosing medical conditions from proxy data.
To recognise the letter N you could define it on graph paper as a vertical line, a diagonal, another vertical — until someone else's handwriting breaks it, and then you rewrite the rules again. Instead, take all 25,000 people in a London postcode, have them write A to Z, and overlay every sample. You get thick blurry lines, no archetypal N, and an algorithm that infers N from W and usually from R too. With a million samples the accuracy climbs. By 2000 no Western post office was reading envelopes by hand.
→ Automated sorting at national scale, from a problem that was unsolvable by explicit rules.
Cukier's stakes test made concrete: for a suspected brain haemorrhage you can quantify the model's inaccuracy — say one in five misdiagnosed in either direction — but you get the answer in four seconds rather than forty minutes. In an emergency where the delay itself kills, he calls that a no-brainer. Change the case to lung cancer, where an accurate diagnosis matters more than a 24-hour delay, and the workflow should look completely different, with human verification built in.
→ Two opposite deployment decisions from the same model quality, driven entirely by the stakes and time sensitivity.
Common mistakes
Trying to code every exception
The real world produces more edge cases than you can enumerate. Each patch buys diminishing returns and the project stalls — the pattern that produced the AI winters when funding dried up.
Attempting inference on a small dataset
The whole approach depends on scale: things become possible with a large body of data that are fundamentally impossible with a smaller one. Below that threshold you get noise dressed as prediction.
Expecting explainability from an inference
The trade for accuracy is that the variables and coefficients exceed human capacity to follow. If you need to show the decision path on a piece of paper, you needed the explicit route.
From the transcript
“how would you design a system that that and a program for a computer to follow a set of instructions to say that that is…”
“just give the machine a lot of data let it work out the system for itself and make an inference”
“it's also incredibly humbling because it presumes that the human being cannot explicitly describe the phenomenon it's trying to accurately predict”
From the episode
Economist's Insider Reveals How Humanity Can Outsmart Risks Of Artificial Intelligence