Illustrations by Jacob Stead for The Correspondent

Imagine you take your toddler to the hospital. You’re worried. He’s been inconsolable for hours, and he’s burning up. You are all whisked off to an examining room and three doctors come examine your son.

  • The first doctor says, “I have no idea if your son has a fever.”
  • The second says, “His temperature is probably somewhere between 100 and 106.”
  • The third states, “It’s 104. Your son has a high fever.”

Which of these people would you want treating your child? The scenario seems pretty clear-cut: the first doctor is totally incompetent, and the second is using a dubious guesswork method with a huge margin of error. It’s highly likely you’d choose the third.

But what if you lived in a world where thermometers were inaccurate, or even nonexistent? Would you want your child treated by a doctor who made decisive pronouncements based on information you couldn’t verify? Or would you choose a physician willing to admit they didn’t know exactly, or at all?

Statistics schmatistics

The world is doing well. The percentage of poor and malnourished people has fallen by half over the last 25 years. Africa – once the The Economist called it that in 2000. In 2013, the magazine published a special “Africa Rising” issue. – is doing great: it can claim five of the world’s ten Read more about the fastest-growing economies in The Economist. fastest-growing economies. We’re living in Read more about Africa’s turn-around here. “the African century”;  the continent is “the China of the future.”

At least, that’s what we think. This rosy picture is the product of data furnished by international organizations like the United Nations, the World Bank, and the International Monetary Fund. From child mortality to economic growth, deforestation to unemployment, name the phenomenon and they’ve got a table comparing rates country by country.

But there’s a big problem. When it comes to data from developing countries, all too often, the thermometers are useless. Whether you’re looking at economic growth, hunger, or poverty, the numbers are frequently inaccurate or simply nonexistent.

“Africa rising”

When it comes to economic growth in Africa, in some respects, we know as little as The Scotsman David Livingstone (1813–1873) was one of the most famous European explorers of southern Africa. did going up the Nile. Figures are often absent, of poor quality, or contradictory. To understand why, you have to look at the source: the various national statistics offices. They’re charged with providing international organizations with data, but they’re hampered by a chronic lack of funding, staffing, and the knowledge necessary for collecting the required Calculating GDP in African countries is made even harder by the fact that they often harbor vast informal economies. An informal economy is defined as a set of legal and illegal activities operating below the authorities’ radar and outside the tax system. Since there’s no official data on these doings, it’s hard to factor them in. An estimated 55% of GDP in sub-Saharan Africa comes from informal activities, and 80% of the working population is thought to be employed in the sector.

In the book Poor Numbers, Morten Find out more about Jerven’s book, Poor Numbers. Jerven recounts a visit to the Zambian statistics office in 2010. There, one man bore responsibility for calculating the National accounts track a country’s economic activity through measures such as income distribution, GDP, and expenditures. including gross domestic product (GDP). “What happens when I’m gone?” the Zambian official wondered aloud.

The cases of Ghana and Nigeria make the inadequacy of African growth data painfully apparent. In 2014, Nigeria updated the way it calculated GDP is determined using a “base year,” after which a country’s economic structure is assumed to have remained unchanged – for example, the agriculture and telecom industries are assumed to have stayed the same size. Nigeria’s previous base year was 1990. When officials bring forward the base year, they reassess the country’s economic makeup and make new calculations based on the updated picture. Developed countries do this about every five years. It was about time: the country had been relying on statistical methods and systems dating back to 1990. Telecommunications and the Nollywood film industry, for instance, were still treated as small, yet both sectors had seen explosive growth in the ensuing 25 years. After the recalculation, Nigeria looked almost twice as rich. In fact, with a GDP increase of 89% over the previous year, it outstripped South Africa as the continent’s wealthiest country. That’s like the average Slovak suddenly becoming as rich as the average American.

Sometimes the World Bank estimates missing data using an unknown method. No one knows precisely what happens inside the black box, but eventually, the figures come out

Ghana had previously revised its figures and seen its GDP soar by an almost-as-impressive 60%. Both countries had modernized their methods, and that was good news. But the striking differences in results offered concrete proof that African GDP statistics could be of dubious quality. Were methods being used elsewhere even more sorely in need of improvement? How did things stand in nations less prosperous than Ghana and Nigeria?

It’s not just the quality of GDP figures that’s a problem; their timeliness is too. Procedures are so sluggish that national statistics offices often take years to release figures. This is particularly an issue for the World Bank, which wants current data, says Jerven.

Though not authorized to do so by any international agreement, the organization began publishing its own GDP data in the 1980s. Sometimes it bases its figures on trips to a country, where employees talk with high-ranking officials and obtain access to data others aren’t privy to. Other times, however, the World Bank estimates missing data points using an unknown method. No one knows precisely what happens inside the black box, but eventually, the figures come out. This data is then The World Bank’s figures are less popular among academics: two-thirds of cross-country growth comparisons use data from the Penn World Table, according to Simon Johnson et al. in “Is Newer Better? Penn World Table Revisions and Their Impact on Growth Estimates” (Journal of Monetary Economics, 2013). by numerous policymakers and journalists.

As a result, the World Bank’s figures deviate from national estimates. The graphic below shows percentage differences between the World Bank’s GDP estimates and those of the countries themselves. For some nations, the figures are the same, but for most, the two estimates differ. The World Bank calculates Guinea-Bissau’s GDP at more than 40% higher than the country itself does.

Source: Poor Numbers by Morten Jerven

Comparing World Bank figures with those of two other authoritative databases, the Penn World Table The Penn World Table gives macroeconomic indicators by country since 1950, the most important one being GDP. Formerly maintained by the University of Pennsylvania, the database recently became the responsibility of scholars at the University of Groningen and University of California, Davis. and the Maddison University of Groningen professor Angus Maddison (1926–2010) maintained a database of GDP data back to the year 1 AD. After his death in 2010, colleagues took over the project. also reveals big differences. For instance, Maddison ranks Guinea as Africa’s seventh poorest country, while the PWT puts it just behind the continent’s ten richest nations. The World Bank lists Guinea in The data in this paragraph and the one before it come from the book Poor Numbers.

Extreme poverty “reduced by half”

“The world has reduced extreme poverty by half,” Read the UN report on the Millennium Goals here. the UN wrote in its evaluation of the Millennium Development Goals in 2014. That meant one of the main targets, halving the percentage of people living below the It should be noted that the drop in numbers of poor people was primarily a consequence of China’s economic prosperity. If you take China out of the equation, extreme poverty in developing countries hasn’t dropped by half. had been met.

But poverty statistics, like those for economic growth, are often unreliable. That’s because it’s extremely hard to determine how many people are living below the line. First of all, you have to conduct mass surveys, and administering and processing these questionnaires can take years. Botswana, for instance, conducted a survey in 2002–2003; but 13 years later, the data still hasn’t been published. Poverty estimates are still based on figures from 1994.

On top of that, since countries design and carry out surveys in different ways, their numbers are hard to compare. Some measure income, for instance, while others look at how much a household consumes. The latter method is more suited to many developing countries, where incomes can fluctuate widely from month to month and purchases are often made in kind. Questionnaires also differ, for example in the number and wording of One way they can vary is in how much they ask subjects to remember. Do they ask people to report purchases for the last seven days, two weeks, or a year? Another variable is how many items people are expected to report expenditures for. Is it a long, detailed list – fried chicken, shoelaces – or a shorter summary of purchases by type? Another difference is whether subjects report spending after the fact or keep a diary. These variables can have a big impact. In Tanzania, an experiment that used different methods came up with poverty estimates that varied from 47.5% to Read more about the Tanzanian study here. 66.8%.

Where national figures are available, they have to be converted and adjusted so they can be measured against the global poverty line of $1.90 a day. This is necessary because purchasing power varies between countries: $1.90 buys more in Thailand than it does in Brazil. To make these adjustments, you need other sources of data, which contain Along with surveys, the following are needed for obtaining internationally comparable poverty figures: census data, national accounts, consumer price indexes, and International Comparison Program (ICP) data. The ICP generates so-called purchasing power parities – values for different countries representing how far money goes there. of their own. So measurement errors accumulate, and the figures become more and more unreliable.

But not only is the soundness of the measurements questionable, the data often isn’t even available in the first place. Of the 161 developing countries the UN looked at, there was only enough information on hand to Details of data availability can be found here. determine poverty trends for 84 countries – that’s just 52%. The World Bank – from which the UN gets its poverty figures – solves the problem of missing data by filling in the holes: it assumes a country’s poverty trend will continue in the same manner, or that it’s the same as it is elsewhere on the Find out more about how poverty figures are calculated. continent. But the poorest countries are the ones that have the hardest time collecting data, which means it’s easy to form an unduly optimistic picture of global poverty.

Hunger “has declined” since 1990

Another key UN goal is ending world hunger. One of the millennium targets was to cut the percentage of undernourished people in half by 2015. In May of that year, the Food and Agriculture The Food and Agriculture Organization (FAO) is a specialist UN agency that leads efforts to end global hunger. It also fights poverty and works toward a more sustainable use of land and water. published its annual report “The State of Food Insecurity in the World,” also known as Here’s the latest SOFI report. SOFI. The most important findings: the number of undernourished people in the world had dropped to 795 million, and hunger had declined in 72 of the 129 countries mentioned in the report. In all developing countries combined, the percentage of people going hungry fell from 23.3% to 12.9%. A halving of the figure to 11.7% was therefore not quite achieved.

As with GDP and poverty statistics, data availability and quality make malnutrition figures problematic. But how concepts are defined also makes a big difference. The case of hunger serves as an example of how varying definitions can dramatically change data patterns.

In 2012, the FAO decided to change the way it measured hunger and retrospectively recalculated its statistics back to 1990. First, it adjusted the formulas used for The FAO modified its assumptions about how calorie consumption was distributed in the population. It changed the shape of its estimated distribution and the way it arrived at other parameters, such as variance. For more on the changes, see Annex 2 of the report. Second, it switched to better data on matters such as food supply and population figures. For instance, new estimates indicated 25 million more people had lived in China in the 1990s than was assumed in the original calculations.

The new methodology dramatically changed the view of hunger. Suddenly, it turned out 150 million more people had gone without enough food in 1990 than was previously thought. Though the old data had shown an increase in the number of undernourished people over the past 20 years, the new figures indicated a steep drop. The UN’s target had seemed a distant dream, but after the change in method, voilà: the goal was within reach, according to the FAO. Since things in 1990 had been much worse than thought, Read more in the FAO’s 2012 report. (See p. 10 for this particular point.) it said, more progress had been made than previously assumed.

Source: UN Food and Agriculture Organization

Even if you accept the new methodology, there are other factors affecting whether or not progress can be said to have been made. For one thing, what exactly does the FAO mean by hunger? It defines a person as undernourished if he or she consumes too few calories for a year. A minimum calorie intake is determined for each country on the basis of average height, age, and sex. In calculating it, the FAO assumes people have a “sedentary lifestyle”: that is, they spend their days in a cubicle or someplace similar. In fact, though, poor people in developing countries often do physically demanding work. According to the guidelines, someone in Uganda needs That was the estimate for the period 2006–2008. That might be enough for a Victoria’s Secret model, but not for a hardworking Ugandan farmer.

Up the calorie limit slightly and the hunger pattern suddenly looks very different. If you assume people are living moderately active lives, then there are as many people without enough to eat today as there were in 1990 (see graph). If you suppose more intense activity levels, that means there are even more.

And the UN doesn’t look at absolute numbers but at the global percentage of undernourished people, which changes the picture again, making it look a lot more positive, since the world’s population has For a detailed critique of malnutrition statistics, see “Framing Hunger.” grown. But the choice between percentages and absolute numbers isn’t straightforward: After all, it doesn’t help a starving person to know he or she has become In response to the 2012 SOFI report, a group of scholars and organization leaders led by Frances Moore Lappé authored the document “Framing Hunger.” It criticized the FAO’s methodology and argued for a return to talking about absolute numbers (as at the 1996 World Food Summit, where 186 countries pledged to halve the number of hungry people. The Millennium Development Goals reframed the issue in terms of percentages). .

Source: UN FAO, 2012

A picture blurred by false certainties

Are poor countries prospering? Is Africa rising? Have poverty and hunger halved? Those are the claims, and they could be true, but the reality is, there’s a lot we don’t know. Too many figures are unreliable, outdated, or simply nonexistent. That doesn’t mean we should give up trying to get accurate data. Though whoever said “Not everything that counts can be counted” (often attributed to Einstein) had a point, we still need reliable figures to make things better in developing countries.

Fortunately, there are bright spots. Data collection has improved significantly in the new century. In 1990, for instance, poverty data was available for just 22 developing nations. The figure has since risen to 127. Statistics offices have gotten better at what they do, partly thanks to investment by international organizations such as the World Bank. And new technologies like satellites and mobile phones have made it easier to gather sound data.

International organizations are being more frank about the quality of their figures, too. If you examine the reports, you come across plenty of caveats. For instance, the Millennium Development Goals report cautions, “Despite considerable advancements in recent years, reliable statistics for monitoring development remain inadequate in many countries.”

The World Bank writes, “Many statistical offices, especially those in developing countries, face severe limitations in the resources, time, training, and budgets required to produce reliable and comprehensive series of national accounts statistics.” And the FAO acknowledges in its 2015 report that its indicator is based on a narrow definition of hunger, which is why it uses a set of food security indicators to create a more representative picture. The organization also measures iodine deficiency in populations and the percentage of underweight children.

But such mentions of limitations and and nuances are often buried in footnotes and appendices. When you get down to it, international organizations appear to be ignoring the gravity of the situation. This is painfully obvious in the UN’s Here’s the UN’s website on the new goals. new development goals. Now, developing countries will have to measure 169 different indicators.

That seems overly ambitious – even though the corridors of the UN are abuzz with talk of a The UN’s report on the data revolution can be found here. “data revolution.” After all, collecting sound data for the previous 48 indicators proved impossible. Less than 70% of the information needed was available, and half of that consisted of estimates, since national data was lacking.

Meanwhile, international organizations seem to be acting as if the hypothetical thermometer is working just fine – and able to read your blood pressure and take your pulse, too. It’s not their fault the data’s sketchy; getting hold of accurate figures in developing countries is difficult. But international organizations – along with journalists and policymakers – should be forthright about the shortcomings instead of blurring our view of the world with false certainties.

— Translated from Dutch by Laura Martz and Erica Moore

More from The Correspondent:


Poverty 101: How can we end global poverty once and for all? By 2030 the world must be free of extreme poverty. The question is how? Of course there’s no single answer. In this crash course on global poverty, we bring together as many answers as we can. We’ll look at the extent of the problem, the causes, and possible solutions. Go to Poverty 101 This is the Berlin Wall of our time Syrian refugees continue to flee the country in droves, but a solution to the conflict there remains elusive. Why? The answer may seem surprising, but our greatest obstacle on the road to peace in the world could well be veto rights in the United Nations. Read Maite’s story here The bizarre tale of President Nixon and his basic income bill In 1969 President Richard Nixon was on the verge of implementing a basic income for poor families in America. It promised to be a revolutionary step – had the President not changed his mind at the last minute. This is the incredible and largely forgotten tale of just how close the U.S. came to stamping out poverty altogether. Read this story from Rutger’s Utopia series here The world’s not changing faster than ever at all The world is changing faster than ever. And we’re going to have to adapt faster than ever too, just to keep up. At least that’s the cliché. But a new book gives short shrift to that notion. We’re not living in a period of technological revolution after all, author Robert J. Gordon argues, but in a period of technological stagnation. Read Jesse’s story here

Facebook
Twitter
LinkedIn
Whatsapp
Email