The Data Duty of Care Co-Pilot Loop
- Difficulty
- Advanced
- Time to result
- ~months to results
- Steps
- 7
- Confidence
- —
Cukier's proposal for how institutions should actually use data in decisions. Today a professional makes a judgement and then hunts for a sliver of data that confirms it. He inverts that: the human decides, a machine trained on every comparable historical case decides independently, the two are blinded and compared, and disagreements escalate to a third opinion rather than being silently overruled. The machine reports its confidence rather than a verdict, the eventual outcome is fed back so the system improves like Google's click-ranking, and the whole thing sits inside a mindset shift he calls a data duty of care: applying available evidence is not a nice-to-have but a legal and moral obligation, and failing to apply it is negligence. He notes it also requires loosening privacy rules in narrow, governed ways for this social goal.
Origin
Extracted from Deep Dive with Ali Abdaal
How to run it
- 1
Assemble the comparable cohort
Instead of the clipboard at the bedside, pull every historical case that meets the same criteria as the decision in front of you — in his example, every patient with that exact condition going back a decade.
Pro tip The unit of analysis is the agglomeration, not the individual data point: things become possible at large scale that are impossible with a small sample.
- 2
Let the human decide first
The practitioner forms and records their own judgement before seeing the machine's. Early-career people especially need to make — and sometimes get wrong — their own calls.
Watch out Cukier is explicit that you may not want a full co-pilot at the outset of a career: mistakes made young are how judgement is built.
- 3
Blind the two decisions and compare
Run the machine's estimation independently and compare it with the human's. Agreement is confirmation; disagreement is the signal worth spending time on.
- 4
Break disagreements with a third opinion
Where human and machine differ, add a third decider — his 'Minority Report' — to vote in one direction or the other rather than letting either side automatically win.
Watch out Outcomes are rarely black or white; treat it as a spectrum of rightish and wrongish, and remember you never see the counterfactual.
- 5
Make the system prompt, not pronounce
Design the output as a nudge with a stated confidence level — 'have you considered sepsis, at roughly 60% likelihood' — so the human can override it for reasons the model cannot see, such as a drug's toxicity for that patient.
Pro tip A confidence number turns the machine from an authority into a colleague you can argue with.
- 6
Feed the outcome back in
Record what actually happened — was the diagnosis or therapy right — and route it back into the model, the way each click on a search result acts as a vote that reorders Google's rankings.
Pro tip Copy the mechanism, not the domain: any repeated decision with an observable result can be ranked by feedback.
- 7
Flip the burden of proof
Reframe applying data as a duty rather than an option: a decision made without consulting the available evidence is a failure of care and carries liability. Pair this with narrow, still-governed loosening of privacy rules for the social goal.
Watch out Privacy law was written for commercial rapaciousness, not statistics; changing it is politically slow, so start with the mindset change inside your own institution.
In the wild
A junior doctor on a night shift decides a patient has gallstones. A supervisor who had seen it a mile away would have said sepsis, but it is four in the morning and nobody senior is there. In Cukier's design the system does not overrule the doctor; it prods — have you considered sepsis — and attaches its confidence, say sixty percent. The doctor reconsiders, or overrides it because giving that particular drug would be toxic for this patient. The human keeps the decision; the data removes the blur-eyed blind spot.
→ The junior clinician's own judgement is preserved while the specific failure mode of an unsupervised 4am decision is caught.
Cukier points at search as the working proof of the feedback step. If everyone searching a given term starts clicking the second result — or better, the eighth — the algorithm promotes it, because each click is a vote on whether the previous ranking was right. No human re-ranks anything; the outcome of past decisions reorders future ones automatically. His question is why healthcare, which generates outcome data on every single patient, does not do the same thing with diagnoses and therapies.
→ A ranking system that improves continuously from real outcomes rather than from expert opinion.
Ali describes working at Addenbrooke's in Cambridge, where bloods, observations, scans and drug charts were all in one iPad app. He could run a whole night shift from the cafe downstairs because every piece of information about a patient was one tap away. It was not a co-pilot — nothing told him what to do next — but it was the base layer the loop requires. In most of the UK the same data is split across four apps and some of it is still pen and paper, so the loop cannot even be attempted.
→ Demonstrates that integration of records is step zero; without it none of the comparison or feedback steps are possible.
Common mistakes
Using data only to confirm a decision already made
The default pattern is to reach a judgement and then find a smidgen of information that supports it. That is confirmation dressed as evidence. The loop only works if the data-derived view is generated independently and blinded before comparison.
Handing the decision to the machine outright
The design is a co-pilot, not an arbiter. Removing the human removes both the accountability and the flexibility to override for reasons outside the model, such as a patient-specific contraindication.
Skipping the outcome feedback
Without routing what actually happened back into the system, the model never improves and the loop degrades into a static second opinion. The feedback edge is what made search ranking work.
From the transcript
“we need if you will a data duty of care mindset”
“here's my decision and if I didn't use data to confirm it and to validate it I have not performed my duty of care to…”
“we want the AI system and or in this case the Big Data System to Simply prod you and say have you considered septus”
From the episode
Economist's Insider Reveals How Humanity Can Outsmart Risks Of Artificial Intelligence