Measuring Poverty for Education

In researching our upcoming podcast on the US education system, I came across an interesting debate about how to best measure poverty. Since poverty is correlated with poor academic performance (see chart below), the dialogue was focused on whether the US has higher than average poverty rates compared to other countries, which could explain the US’ low test scores.

One side argued that relative rates are the best way to understand the prevalence of poverty in a society, whereas the other preferred absolute rates (description of both of these below). The debate was interesting because it was an open and relatively cordial disagreement about method, which, to my knowledge, doesn’t happen too frequently outside of academia.

Relative or absolute?

Most poverty experts use relative poverty rates. Relative poverty is defined as a level of income that is less than half of the median income. Medians can be of state income (i.e. income that is less than half of the median state income) or national income (i.e. income that is less than half the median national income). In a National Review piece, Michael Petrilli and Brandon Wright propose that relative rates are not the best way to think about poverty, and instead propose using absolute rates.

Absolute poverty can be defined a couple of ways, but the description that Petrilli and Wright use is: the percentage of individuals (or households) living below the federal poverty rate of a given country. Petrilli and Wright contend that absolute rates are a better measure of poverty, whereas relative rates are a better measure of inequality, since a state (or country) with greater inequality will positively skew the median income without hurting purchasing power. Greater inequality by definition increases the relative poverty rate. But having wealthier individuals in a group doesn’t necessarily mean that the additional individuals in the middle that are captured by a higher median are worse off.  

This may seem like an academic distinction, but there are real policy implications. If the US has above average poverty rates than other countries, then, since poverty is correlated with below-average performance, that could point to unusually high poverty as a factor causing depressed testing performance. However, while the US has an above average relative poverty rate, it does not have an above average absolute poverty rate–in fact, it’s quite on par with some countries like Finland and Germany whose PISA academic performances consistently beat those of the US.

Since, on an absolute basis, the US does not have above average poverty rates, making the case that above average poverty is the cause of our below average test scores becomes a more difficult proposition.

The rebuttal

Dylan Matthews of Vox disagrees that absolute rates are a better way to measure poverty, and wrote a critique of Petrilli and Wright’s work. He argues that using absolute rates to compare countries doesn’t take cost of living differences into consideration:

“That’s because poverty, in developed nations, is an essentially comparative notion. In rich countries, it doesn’t make much sense to define poverty as ‘not having enough to meet basic material needs.’ Almost no one is that poor in America – not even those earning less than $2 a day. What rich countries mean by poverty is something more like ‘having enough to have the bare minimum life necessary to be a party of your society.’ ”

Matthews points to relative purchasing power as an example of why some poverty experts don’t like using absolute rates. Additionally, Matthews also calls Petrilli and Wright out on what he considers to be poor methodology in calculating their absolute poverty rate figures. He claims that, by using an alternative absolute poverty rate metric, the US does in fact appear poorer than other OECD nations:

Counter Rebuttal

Petrilli and Wright then responded to Matthews’ article, providing greater clarity into their methodology. The primary difference in how they and Matthews represent absolute poverty is the cutoff line for the percent of the US federal poverty rate. Since cost of living in some OECD (especially in Scandanavia) countries are higher than in the US, they claim that using an absolute poverty rate equal to 100% of the US federal poverty line may not make sense when making international comparisons. Petrilli and Wright admit that using an absolute poverty rate threshold of 100% yields figures similar to what Matthews showed in his Vox article. Using a 125% cutoff, however, looks quite different(1):

The key question here is: which threshold should be used? As you can see from the above chart, a small difference paints a very different picture.

Greater detail

Petrilli and Wright published a follow-up piece in Education Next expanding upon their original National Review piece. In it, they use state-level data in the US to compare relative and absolute poverty rates:

While some states have greater poverty regardless of how it’s measured, some vary significantly depending on which metric is used. For example, Massachusetts and Connecticut have absolute poverty rates that are “among the lowest in the country. But their relative poverty rates are above the average – higher than Texas, Tennessee, and Oklahoma. Massachusetts has a higher relative poverty rate than Georgia, Kentucky, and Alabama.” Petrilli and Wright draw the distinction between poverty and inequality, saying that Massachusetts is not actually poorer than Alabama, but more unequal.

Which metric is better?

It’s difficult to tell. I’m inclined to think that using absolute rates to describe poverty across countries will fail to take into account all cost of living differences. Remember, though, that neither of these articles are ultimately about poverty, but rather or not above-average poverty is correlated with educational success. We can then reframe the question “which metric better measures poverty” to “which poverty metric has a higher correlation with poor test scores?” Here there is a clear winner, at least on a state-level:

In the US, absolute poverty has a greater correlation with lower test scores than relative poverty. Making the claim that poverty leads to worse test scores may make some sense, but it makes more sense to use the absolute rate due to the closer fit. As shown in Figure 3 above, the US is not poorer than other OECD countries on an absolute basis, which makes the argument that poverty is a leading cause of lower national average test scores less compelling. That said, Petrilli and Wright could have made a stronger case if they showed the correlations for international relative and absolute poverty rates and test scores.

So what causes the US’ lower test scores? That’s a hard question to answer, and unfortunately the data may not yet exist to answer that question. However, finding the correct cause is critical, since if we are to expend physical resources and political capital pushing a solution through Congress or at the state level, we better be sure that we’re fixing the right thing. If the data shows that the US is not poorer than average compared to similar countries, then the claim that poverty is driving lower test scores doesn’t hold up as well. There’s more research to be done.

1. If data from 1999-2000 looks old to you, you’re right. Another contention of Petrilli and Wright is that recent data is scarce, which forced them to make methodological estimates.

— Xander Snyder Reconsider Podcast Co-host Politics: but we don’t do the thinking for you reconsidermedia.com/podcast Subscribe on iTunes

Learning More

I go in-depth with Erik on this topic in our podcast episode, “Education in America, Pt I.” Pt. II coming soon!