Seven scientists and other experts went on trial on manslaughter charges Tuesday for allegedly failing to sufficiently warn residents before a devastating earthquake that killed more than 300 people in central Italy in 2009.
Hindsight bias! Get yer hindsight bias, piping hot and fresh from Italia! Cheaper when purchased by the gallon!
Let's put aside for a minute any question of actual moral culpability, and whether it's fair to hold people criminally liable for scientific predictions. Instead, let's assume that we're just interested in saving as many lives as possible and increasing the accuracy of earthquake forecasts (although astute readers will note that even this phrasing involves tradeoffs between the two goals). Does arresting seismologists who fail to predict earthquakes sound like good policy?
The trouble with incentives is that you need to think very carefully about whether you're actually creating the incentives you think you're creating.
I imagine the Italian authorities have the following mental model:
1. Massively increase the penalties for wrong predictions
2. Lazy and corrupt seismologists put more effort into getting the right predictions
3. Save lives!
But this is assuming the continued existence of seismologists making predictions in the first place. If you don't do this, then you have the following:
1. Massively increase the penalties for wrong predictions, while not increasing the payoff from correct predictions
2. Result is massively reduced payoffs from making any seismological predictions at all.
3. Seismologists quit en masse, or refuse to provide any guidance.
4. No more predictions.
5. Earthquake deaths.
Not quite the same thing, is it?
Okay, let's put Mussolini in charge! He'll chain those lazy seismologists to their desks and force them to keep making forecasts! Surely we'll be okay then, right?
Or maybe you've then set up the following incentives:
1. Massively increase the penalties for type II errors (i.e. false negatives: saying there won't be an earthquake when there will be one), but not increasing the penalties for type I errors (i.e. false positives: saying there will be an earthquake when there won't be one).
2. Seismologists predict an earthquake every single day of the year
3. People quickly learn to ignore all the warnings
4. Signal to noise ratio in earthquake predictions goes to zero, meaning that there are effectively no more meaningful earthquake predictions
5. Earthquake deaths.
Bureaucrats seem to think that ramping up penalties will create scientific knowledge where there was none. But it won't. All it will do is shift around the incentives and behaviour of agents in ways that are completely predictable, even if they are almost certainly not predicted by the bureaucrats.
In the mean time, you can expect displays of righteous indignation by prosecutors, giddy with hindsight bias, exactly sure that they knew there was going to be an earthquake, so why didn't those lazy seismologists? What you won't be able to get from them is any useful guidance on when the next earthquake is actually going to happen, notwithstanding their amazing hindsight powers.
Let's put aside for a minute any question of actual moral culpability, and whether it's fair to hold people criminally liable for scientific predictions. Instead, let's assume that we're just interested in saving as many lives as possible and increasing the accuracy of earthquake forecasts (although astute readers will note that even this phrasing involves tradeoffs between the two goals). Does arresting seismologists who fail to predict earthquakes sound like good policy?
The trouble with incentives is that you need to think very carefully about whether you're actually creating the incentives you think you're creating.
I imagine the Italian authorities have the following mental model:
1. Massively increase the penalties for wrong predictions
2. Lazy and corrupt seismologists put more effort into getting the right predictions
3. Save lives!
But this is assuming the continued existence of seismologists making predictions in the first place. If you don't do this, then you have the following:
1. Massively increase the penalties for wrong predictions, while not increasing the payoff from correct predictions
2. Result is massively reduced payoffs from making any seismological predictions at all.
3. Seismologists quit en masse, or refuse to provide any guidance.
4. No more predictions.
5. Earthquake deaths.
Not quite the same thing, is it?
Okay, let's put Mussolini in charge! He'll chain those lazy seismologists to their desks and force them to keep making forecasts! Surely we'll be okay then, right?
Or maybe you've then set up the following incentives:
1. Massively increase the penalties for type II errors (i.e. false negatives: saying there won't be an earthquake when there will be one), but not increasing the penalties for type I errors (i.e. false positives: saying there will be an earthquake when there won't be one).
2. Seismologists predict an earthquake every single day of the year
3. People quickly learn to ignore all the warnings
4. Signal to noise ratio in earthquake predictions goes to zero, meaning that there are effectively no more meaningful earthquake predictions
5. Earthquake deaths.
Bureaucrats seem to think that ramping up penalties will create scientific knowledge where there was none. But it won't. All it will do is shift around the incentives and behaviour of agents in ways that are completely predictable, even if they are almost certainly not predicted by the bureaucrats.
In the mean time, you can expect displays of righteous indignation by prosecutors, giddy with hindsight bias, exactly sure that they knew there was going to be an earthquake, so why didn't those lazy seismologists? What you won't be able to get from them is any useful guidance on when the next earthquake is actually going to happen, notwithstanding their amazing hindsight powers.
While I agree with everything you've written can we at least burn a few scare mongering climate scientists at the stake before we cancel the inquisition?
ReplyDeleteAh yes, the age-old 'burn the scientists in the name of science' shtick. Not sure if comment is outright satire of right-wing people, or directionally sincere but highly facetious.
ReplyDeleteEither way, I shall respectfully pass. It's far more useful and entertaining to just be the stickler pointing out how the results of the CERN cloud experiment are a bit awkward for the most gung-ho supporters of carbon taxes.