Hi,
"The fox knows many things but the hedgehog knows one big thing."
I’m thinking a lot about this quote these days. It comes from a more than 2,600 year old Greek poem by the war poet Archilochus and is often quoted in the literature about prediction. Philip Tetlock writes about it in his book, Superforecasting (with Dan Gardner), and data journalist Nate Silver uses it in The Signal and the Noise.
I’m reminded of the fox and the hedgehog as I follow the public debate on corona. Some people are so confident about their views – the country must open! The country must close! 5G did it! – those are the hedgehogs. They follow one big idea and can therefore get away with very little nuance.
It makes for nice TV, but bad predictions.
Dart-throwing chimpanzees
This is evident from the work of Philip Tetlock, who has been researching predictions for 30 years. Our world, he argues, depends on predictions. Insurers estimate whether someone will die, security services estimate whether a war will break out and ordinary people – you and I – estimate whether it is wise to buy a house, marry that one person or pursue a career change.
And now we, all of us, need to predict which measures will best guide us through the pandemic.
It’s quite important, then, that those predictions are correct. But Tetlock found something astonishing: the average expert predicted no better than a dart-throwing chimp. He examined 300 people for whom forecasting is part of their job and found that when it comes to making predictions, they might as well have flipped a coin.
The talking heads of this world are dart-throwing chimpanzees – it’s no surprise that the media loved Tetlock’s research. But they overlooked one thing: in the research there was a small group that predicted better than chimpanzees.
Superforecasters
Sure, they weren’t oracles, but they did it a little less bad than the rest. Tetlock decided to gather a team of volunteers to compete in the biggest forecasting competition ever, organised by the US security service.
In year one of the league it turned out that Tetlock’s team did better than the other teams. Much better. It stayed that way for the remainder of the game. Four years, 500 questions and more than a million predictions later, it was certain. Tetlock had found them: superforecasters.
These superforecasters weren’t necessarily super smart, nor did they know much about the topics that came up in the game. Their jobs often had little to do with predicting. There was a filmmaker, a ballroom dancer, a retired computer programmer.
"It’s not really who they are. It is what they do." says Tetlock in Superforecasting. Almost every superforecaster, he saw, took a number of steps that anyone can learn.
Five steps
Those steps can help you to predict better, and to stay clear-headed when dealing with information in the corona crisis.
- Chop the problem up into manageable pieces. A question like "How many people will die of Covid-19 in the United States?" is too big. Make an overview of all the chunks of information you need, like: "What is the age structure?"
- Look at the problem from the outside. Don’t get into the specific situation right away, but take a broader look at similar situations. Suppose you believe that your startup is going to be a success. How realistic is that prediction? Ignore the details about your own company and take a broader view: how often does a startup succeed.
- Find a good balance between the outside and the inside view. Complete the outside view with information about the specific situation. What are the chances that Boris Johnson will still be prime minister in 2022? Don’t just look at historical data, but also at the current situation in the UK. Superforecasters have a good sense of how they should take information into account.
- Don’t correct too much, but also not too little. "Belief updating is to good forecasting what brushing and flossing are to good dental hygiene. It can be boring, occasionally uncomfortable, but it pays off in the long term," says Tetlock. Don’t get carried away by every news item (a new vaccine for example), but don’t stay too inflexible in your ideas either.
- Look for as many perspectives as possible. Here comes the fox. Gather as much information as you can, don’t hesitate to change your prediction and keep looking for counterarguments for your reasoning. And always stay in doubt.
I’m curious: who do you think are the hedgehogs in the public debate about Covid-19? And who are the foxes? I’d love to hear from you in the contributions under this newsletter.
I can wholeheartedly recommend the book Superforecasting, it taught me a lot. And would you like to know more about forecasting the corona crisis? In mid-April, the BBC4 podcast More or Less made an episode about it.
#coronafree
I’ve been rediscovering gaming lately. And I’m not the only one: "coronavirus triggers video game boost" wrote the BBC.
Personally, I hadn’t touched a game since these days:
And, frankly, I associated gaming with shooting and racing adolescents.
But I was mistaken. There is so much more. I started to play The Gardens Between, a beautifully designed puzzle game about two friends who have to find their memories.
After I finished that, I continued with Stardew Valley. You inherit a farm from your grandfather, quit your boring office job and try to make it as a new farmer. In addition, you gain friends, go on dates and visit festivals in the village. It’s extremely addictive.
Besides that my boyfriend and I played Firewatch on the Xbox. Henry’s taking a job as a groundskeeper in a national park in Wyoming, US. But something’s off. You go exploring the wilderness and find yourself in increasingly creepy situations. This game is perfect if you’re longing for distant travels.
Before you go ...
Last week I wrote a column about corona dashboards that are mushrooming throughout the world. My main point: they shouldn’t be a technical fix for a political problem.
Part of this newsletter is an abridged version of a (Dutch) article I wrote in 2018 about predicting the World Cup.
Prefer to receive this newsletter in your inbox?
Follow my weekly newsletter to receive notes, thoughts, or questions on the topic of Numeracy and AI.