I have recently been working my way through the excellent History of Rome podcast series. I had been meaning to do this for some time as my previous knowledge had been mostly from another excellent series, The Fall of Rome , which I've written some thoughts from before. I realized in hindsight that I was doing things in a mistaken order. Perhaps due to my own slightly pessimistic nature and preoccupation with the current political situation, the narrative of decline and why it happened was very appealing. But before you can delve into the big picture forces (which Wyman does well in the Fall of Rome), it's helpful to understand the basic sequence of what happened when. Not only that, but you can't really hope to understand the fall of Rome unless you also understand its rise, and all the times when it could have fallen, but didn't. Going through the story made me want to go back and read all sorts of things again with a better knowledge of the events, from the Asterix comics, to Horatius at the Bridge by Macauley, to The God Abandons Antony by Cavafy, to Blog.Jim's posts on normality bias.
One of the interesting challenges when listening to the narrative is to try to invert the ex post story back into the ex ante perspective at the time - what people would have or should have thought, knowing only what they knew at the time. History is told backwards, but must be lived forwards. The simplest version of history tells things as a story, describing the important events that happened. But quickly students of history want to move from what happened, to why it happened. In the language of statistics, this means fitting the right ex-post model to the data, so you can understand what variation drove what outcome. Even this is hard to do - you might overfit the model, or select the wrong variables (and you don't get to re-run things to find out if you're right.) The causes you identify are probably there, and perhaps even contributed, but are they actually the important ones? This is a lot of the challenge of historians. But from a statistical point of view, the next step is the ex-ante one. If you'd run this same model using only the data you'd observed up to that point, even if you'd thought of the same variables, what relationship would you have estimated? Price to dividend ratios predict market returns reasonably well at long horizons ex-post, if you believe Campbell and Shiller. If you run it out of sample, Goyal and Welch say they don't.
The statistician's version of this is quite easy - just run the same model on less data, and see what it produces. So why is it so hard as a historian to do the same? Because you're not really running models, you're evaluating things according to your own judgment. This isn't a knock on the field, per se. Some bits of history lend themselves to quantification, like Robert Fogel did, but others (including a number that you really care about) simply don't. When you form your own judgment of things, it's hard not to fall victim to the curse of knowledge. That is, when you know something, it is very difficult to credibly put yourself in the position of someone who doesn't know the thing. It will always seem like things that you know after the fact should have been easy to forecast at the time, but they often aren't.
In history, we always know how the story ends, so identifying what counts as a major event, or a turning point, or a transition, is always made with the benefit of hindsight, so as to give the most informative narrative. This leads people to make a significant mistake when translating history into their own lives. They assume that when some major shift occurs, there will be lots of signs to indicate this fact. But there might not be. Maybe what's important won't be obvious until much later. Reading about the last days of the Roman republic, one of the interesting aspects is that what in hindsight seem like important turning points. When the Gracchi brothers started using mobs of plebians as threats to get their political will, it might not have seemed that shocking. But it draws a line to Marius becoming consul seven times and leading an army into Rome to institute a reign of terror, and then Sulla being declared dictator for life. Except Sulla stepped down, and attempted to restore the Republic. You can imagine that things might have seemed back to normal then. But instead, this is described as more steps towards empire. Even Caesar Augustus, who consolidated power single-handedly more than anyone since Tarquin, kept a lot of the forms of the Republic, and only changed his status quite gradually. There was still a senate, and consuls, and praetors. To someone wanting to convince themselves that things weren't actually that different, it was probably easier than you might think. Indeed, one narrative of Julius Caesar's downfall is that he attempted to shift power to himself too quickly, and got stabbed to death by the Senate for his troubles, even after all his triumphs. This seems to suggest that the prudent strategy is probably to maintain the old forms, and pretend like they're still in operation, even as they're gradually undermined.
Which perhaps should make you wonder - has this... happened in America? Almost certainly. As Moldbug describes, America has gone through at least four versions of the Republic since its founding. The main reason people don't notice this is that they all swear fealty to the same piece of paper. But look around! Does the paper actually describe the government? If it does, why is the government so radically different, even as the paper is the same? Try explaining the CIA to George Washington, or the modern interpretation of the Commerce Clause to Thomas Jefferson. (As a party joke, I enjoy asking law students to list as many hypothetical pieces of legislation as they can that they're sure would not be justified by the Commerce clause. There's, uh, gun-free school zones? And ... hmmm, did I already mention gun-free school zones?).
Is FDR delivering his inaugural address, which basically demanded absolute authority from Congress under threat he'd just take it anyway, and threatening to pack the Supreme Court when he didn't get his desired judgments, and serving an unprecedented four consecutive terms, breaching the 150 year old norm of only two, basically equivalent to a less violent form of a Caesar? The case is at least arguable, but you can be damn sure that you won't read this argument in your high school civics class. Was Nixon a corrupt figure that was justly impeached, or was he stitched up in a deep state coup? Also at least arguable.
Or, to take one that's not yet a fait accompli in where it will end. You can also observe a gradual breaking down of existing norms and compromises that served to keep the parties' relationships with each other civil. The Democrats breach the previous norm that presidents basically get their Supreme Court nominations, by filibustering the eminently qualified Robert Bork. Republicans targeted Clinton, first with special prosecutors empowered to go on endless fishing expeditions (like starting out looking into dodgy Arkansas land deals and ending up looking into semen-stained dresses), and impeachment over purely process crimes like perjury when the underlying events were not actually criminal. Or the FBI illegally wiretapping the Trump campaign and Carter Page. Or Trump calling mobs to the capital to protest what he (and I) saw as election fraud. I happened to think that the January 6th mob was obviously going to be useless for anything other than theater with no coherent plan. But still, it is a notable shift from previous norms. Just like the Gracchi brothers. Maybe this is one more step towards perdition. Maybe it's just rumblings that will eventually settle down, like the secession of the plebs.
In other words, we expect changes of government to look like America and Russia turning up in Berlin in 1945 - the game is over, and everyone knows it. But even the collapse of the western Roman empire doesn't quite work like this. One might think that when Rome gets sacked, that's basically the end. But Rome got sacked by the Gauls in 390 BC and bounced back. It got sacked by the Visigoths in 410 AD which was bad, but things still limped along. It got sacked again by the Vandals in 455 AD, by which time things were looking pretty dire indeed, but Odoacer declaring himself King was still twenty years away.
The challenge, in other words, is to be able to estimate the versions of history that could have happened but didn't, and the probabilities one should have attached to them.
And when people imagine the idealised version of what this could be like if done well, those with a sci-fi bent will immediately think of Isaac Asimov's psychohistory. Imagine a fully worked out statistical model of psychology, sociology, and economics. Asimov's idea was that the perfect version of the social sciences should operate akin to the gas laws. The behavior of any one person is random, just like the movement of any one gas molecule. But the behavior of quadrillions of gas molecules or people is highly regular, and thus can be predicted quite well.
I am a huge Asimov fan, and found his writing highly influential in my teenage years. But the more I've pondered it, the more I think the idea of psychohistory has a tendency to lead people badly astray as to what ought to be possible, even in theory.
The version of psychohistory in the first foundation novel starts out with a version that presents the science as statistical, in the sense of assigning probabilities.
Gaal said, "Indeed? In that case, if Dr. Seldon can predict the history of Trantor three hundred years into the future -"
"He can predict it fifteen hundred years into the future."
"Let it be fifteen thousand. Why couldn’t he yesterday have predicted the events of this morning and warned me. -No, I’m sorry." Gaal sat down and rested his head in one sweating palm, "I quite understand that psychohistory is a statistical science and cannot predict the future of a single man with any accuracy. You’ll understand that I’m upset."
"But you are wrong. Dr. Seldon was of the opinion that you would be arrested this morning."
"What!"
"It is unfortunate, but true. The Commission has been more and more hostile to his activities. New members joining the group have been interfered with to an increasing extent. The graphs showed that for our purposes, matters might best be brought to a climax now. The Commission of itself was moving somewhat slowly so Dr. Seldon visited you yesterday for the purpose of forcing their hand. No other reason."
Gaal caught his breath, "I resent -"
"Please. It was necessary. You were not picked for any personal reasons. You must realize that Dr. Seldon’s plans, which are laid out with the developed mathematics of over eighteen years include all eventualities with significant probabilities. This is one of them. I’ve been sent here for no other purpose than to assure you that you need not fear. It will end well; almost certainly so for the project; and with reasonable probability for you."
"What are the figures?" demanded Gaal.
"For the project, over 99.9%."
"And for myself?"
"I am instructed that this probability is 77.2%."
I think this gets to the second part, which Asimov probably couldn't have known. Specifically, he was writing in the 1940s and 50s, before the age of cheap computing power and large, easily available datasets. That is to say, Asimov almost certainly had no experience actually constructing and testing statistical models. He couldn't have! Unless he was inverting the matrixes for the OLS estimator by hand.
And as a result, he missed out on the single most important lesson you get from actually testing quantitative models. Namely, you find out how often your intuition about the world is just completely wrong. Or the effects are kinda-sorta there but much weaker than you thought. Or you start to worry about which of the many variations on some predictor variable you should be coding up, or whether there might be data errors, or whether linear models really are the right choice here, or whether there's reverse causality going on, and a thousand other things. You learn, in other words, that predicting almost anything you actually care about is surprisingly hard. And the work of doing so doesn't look at all like psychohistory, where one mathematical genius comes along, and suddenly you've got perfect predictions. Rather, it's about the slow grind of finding one new variable to improve the R-squared, or some new estimation technique to get the mean-squared error down. Sometimes a new discovery improves things a fair bit. But you never have a sense that you'll get the R-Squared to one, until your number of predictors equals the number of observations, at which point you've played yourself, as the Beastie Boys put it. Indeed, you quickly hit the law of diminishing returns on this type of thing. Initially, you add the large, big picture effects that have the most predictive power. But then what's left over is increasingly random and noise-driven, coming from outside forces and quirks mostly orthogonal to what you're interested in studying. Like Zeno's paradox, you might get closer and closer, but there's no hope of getting to the final goal.
Even this is in the idealized version! Often you come away just convinced of your own deep epistemic uncertainty about the universe. You'll never really even perfectly explain what happened in a dataset, let alone forecast out of sample, because the world is just a shockingly complicated place. And all the bits you leave out end up in your residual term.
Nobody who has ever run a regression could really believe in psychohistory, if they thought about it hard. But many, (like me until very recently), suspended their disbelief in the face of the wonderful story, and just didn't think about it. But when you do, you realize it's just not how prediction works. Not in practice, and without independence, not even in theory. Models don't fail just because the Mule comes along. They fail because the task itself can only be statistical and uncertain.
But if you never actually run these tests, you'll evaluate your theories of history on a heuristic basis, and make all sorts of kludges and exceptions, and be surprised when the world doesn't work out with as much certainty as a history book.
What does genuine, significant, high stakes predictions actually look like, in the heat of the moment? Before you actually know how it's going to go down?
When Russia first invaded Ukraine I spent an inordinate amount of time trying to figure out what the probability was that this would turn into a nuclear exchange. Because if the probability were non-trivial, it was time to get out of America, for at least weeks, to see how it was going to pan out. The important question is not "was it going to happen". Rather, the better question was, and is, "what would be the trigger events that I could observe at the time that would indicate a significant increase in the probability of dramatic escalation?"
My guess was that the highest probability path to large scale war between Russia and America (and thus potentially going nuclear) was the US imposing a no-fly zone. Which is to say, shooting down Russian jets. The chances that this might spiral out of control, and quite fast, seems decent. Large overt NATO troop present in Ukraine would also be in the category. Other wild-card events like Poland unilaterally sending troops might also could, but it's harder to know where that goes.
As it turns out, thankfully none of this seems likely any more. Whether it was ever likely is a separate question, but the identification of trigger events doesn't hinge on this question hugely, except for the question of whether the mental exercise is worth your time. Fortunately, the Biden administration repeatedly said early on that it wasn't interested in a no-fly zone, something we can all be very grateful for. But even if he had declared one, I suspect you probably would have had time to get on a plane out of America within 24 hours if you acted immediately, as things probably don't go nuclear at the first downed plane. On the other hand, it seems highly likely to me that most people wouldn't act, and would just sit there. Which is lucky really - the plane capacity to leave America each day is only a tiny, tiny fraction of the population. The plan only works if nobody else acts. But this in turn means that you need to act, and quickly, exactly at the point when everyone else thinks you're weird and paranoid. How else could it work? If you wait until the air raid alerts are being sent out, you will probably just die getting incinerated in your car, stuck in the biggest traffic jam in the soon-to-be-concluded history of the city.
This is why forecasting things usefully ex-ante is hard. People expect there to be a big glaring sign that everyone will see. But there probably won't, at least until the historians write about you in 200 years' time.
