Data in Policy Debate: Organizing and Framing

In our ongoing series about data in policy debate, we got a few requests from readers with respect to creating a more comprehensive guide to interpreting politics graphs we see in the wild. 

We think it’s a great idea. We’re going to get started with the seeds of a new chapter. We’ve already discussed a few key ways that data can be presented in policy debate in order to nudge us towards making a certain conclusion:

1. Correlating two variables and implying causality
2. Carefully selecting what variables go on graph axes
3. Looking at a variable out of context of the “bigger point”
4. More fishy variable definitions
5. More questions of which variables to correlate
6. Problems of statistical significance

Today we want to focus on “organizing and framing,” which is likely to be a chapter on its own if and when this becomes a book. We’ll look at an example near and dear to our hearts as we crank through the editing of Wedged. 

There’s an oft-cited statistic that in the United States, there are 32,000 gun deaths per year.  This is true. But one little statistic can carry with it a whole lot of meat to chew on.

There’s already a framing sin afoot here: talking about raw numbers instead of rate (per population) is fairly unforgivable when trying to think rationally about policy. The United States is a big country, so its numbers are all going to be big compared to other countries’. 

There are a few other questions we need to ask. Are these gun deaths homicides, accidents, suicides, etc? Turns out it’s all three together. Suicides outnumber homicides about 2:1 (accidental deaths make up less than 2% of gun deaths). And as gun homicides have dropped (along with the general drop in homicide in the US), gun suicides have increased. 

So if we’re just querying “gun deaths,” we’d see that they’re largely unchanged in the United States over the past 7 years. We would miss completely that homicide rates (gun and otherwise) have been dropping significantly, and that suicide rates have been increasing just as quickly. 

So what’s the issue with the framing here? When we talk about gun control, we’re generally talking about measures that will affect the murder rate: background checks, trigger locks, assault weapon bans, clip size restrictions. None of this would have any impact on suicides, which make up a 2/3 (and growing) share of gun deaths. With regard to policy, lumping homicides and suicides together as “gun deaths” is deceptive and unhelpful.

Looking specifically at gun suicides, we may imagine that the US has a major problem compared to its peers. The US has by far the highest gun suicide rate of the OECD.

Look all the way to the right at South Korea and Japan, with the lowest gun suicide rates in the OECD. Now let’s look at the suicide rate, regardless of tool:

South Korea and Japan are #1 and #3 for suicide overall. The United States is below average, under traditionally-happy countries like Canada and Sweden. 

In the case of suicide, focusing on the tool can cause us to look at data in an unhelpful way: it can imply that guns are a primary or decisive driver of suicide; looking at the big picture reveals that in some countries, guns are part of the picture, and in some countries, suicide rates can be very high despite almost no guns at all. 

The lesson is similar in the case of homicide rates: we’ve seen homicide rates drop in the US despite a higher gun ownership rate; guns are part of the story, but far from the whole thing.

Why would we want to reduce deaths brought about by a certain tool, rather than premature deaths? Why might we care about gun suicides more than other suicides? 

The answer is that we obviously don’t. Focusing on guns specifically is meant to imply a certain policy path. Organizing many kinds of death into a certain lump like “gun deaths” obfuscates a complex issue and implies that the tool involved is the only factor worth considering.

Do you have other ideas for a Data in Policy Debate book? Let us know in comments.

Other news: we’ve been invited to host our column on the illustrious Byline: a fully-independent journalism platform with crowdsourcing. Like what you’re reading and want it to keep going? Pitch in a few bucks a month and help us pay the bills. We have special perks at certain support levels that let you twist our arm into writing about different topics, in case anything’s hot on your mind.