Data in Policy Debate: Very Deceptive Polling

Donald Trump's supporters seem to break the typical ideological or religious bounds that link voters to candidates: they tend not to be particularly politically conservative or moderate, and don't seem to lean particularly Evangelical, Catholic, or otherwise.

Pollsters therefore have put a lot of effort into trying to find the common thread that unites them. The New York Times dug into a YouGov/Economist poll and found that 20% of Trump's supporters disapproved of the Emancipation Proclamation.

Cue expletives about racism from Daily Kos and TIme getting some easy clicks by running a piece titled, "Nearly 20% of Trump Fans Think Freeing the Slaves Was a Bad Idea." At first blush this is pretty amazing.

But putting aside any deeper-dive into the souls of those who support various candidates, it's very much worth a deeper dive into the poll. First, David Mastio finds a few other oddities in this poll, including:

  • 15% of Hispanics and 5% of blacks also agreed that the Emancipation Proclamation was a bad idea (and another 24% of blacks weren't sure).
  • 32% of blacks--about the same number as Trump voters--supported the internment of Japanese Americans in World War 2.
  • 43% of likely Democratic primary voters approve of George W Bush's pro-torture executive order which came after 9/11.

This should raise some eyebrows. Such positions seem, at best, incongruous. If we actually dig into the poll, we can find out why such wacky results came up.

The Emancipation Proclamation poll question reads like this:

"Do you approve of the executive order which freed all slaves in the states that were in rebellion against the federal government?"

So already we're bringing in politically hot buzzwords like "executive order" and "states that were in rebellion." This question came packed in a bunch of other questions on executive orders. To make matters worse, it came right after the pollster asked if one approved of executive orders in general, and whether one thought they were constitutional. 41% of Republicans declared that all executive orders were unconstitutional... right before being asked if they approved of a big list of executive orders. (21% overall said they were unconstitutional.)

So we've gotten people to say out loud that executive orders are unconstitutional, then we ask about one. We're now narrowing down to Trump supporters, who are part of that group of 41% of Republicans. It's actually a bit of a miracle that more than half of them change their minds (19% disapproved of the Emancipation Proclamation). 

The "torture" question asked whether one approved of "an executive order which authorized enhanced interrogation techniques and established military tribunals to try foreign enemy combatants." It didn't say, "do you approve of torturing people?" 17% of Republicans said they disapproved of that one, too--just about the same number.

So maybe after getting Republicans to declare that all executive actions are unconstitutional, some of them might stick to their guns when asked about specific ones. Of this group--including that 5% of blacks--maybe they approved of freeing the slaves but not of the executive action (remember the 14th Amendment passed soon after the Proclamation, but we didn't ask about that in order to get clarification). Maybe they just thought Lincoln was a tyrant. Or maybe 20% of Trump's supporters agree with 7% of Democrats and 5% of blacks that America would be better off if blacks were literally property.

This kind of really terrible polling is frustratingly pervasive. YouGov probably didn't try to manufacture this result, but boy, they did a great job of it. Time Magazine and The New York Times were probably just lazy, rather than actively trying to deceive the American public--we'd at least like to think so. Daily Kos, like so many of us, might just love an opportunity to get angry and write about it on the Internet.

Another eye-rollingly frustrating poll result got the self-congratulatory headline by The Guardian, "Poll: 30% of GOP voters support bombing Agrabah, the city from Aladdin." Oh man, they're pretty stupid! Now, the headline (and entire article) leaves out that 19% of Democrats also support bombing Agrabah. 

What's really going on here? Look: most Americans (and Europeans, but we don't like bashing on them as hard as we like bashing on ourselves) can't find Syria on a map, which in itself isn't a problem. But if you're not a war nerd, you're not going to be able to name most towns in the Middle East. Or maybe you're keeping up with the news, and you've heard ar-Raqqah (which sounds suspiciously like Agrabah to those of us not fluent in Arabic) is the capital of ISIS, and it's chock-full of command centers, weapons caches, and training grounds. When asked, "do you support coalition airstrikes on the command, training, and weapons facilities of ISIS in its capital," what kind of answer would you get? 

But even for those that don't keep up: when you get a phone call from a credible polling source that primes you with information about the war against ISIS, and then immediately after asks if you support bombing some Middle-Eastern-sounding city, you're in trouble. Are you going to say, "I've never heard of that city, could you explain?" Probably not: we're expected to have opinions on everything, and we're assuming that this isn't an episode of Punk'd where otherwise-reputable journalists are trying to have a giggle at a "gotcha" moment. We're answering whether we generally support airstrikes in the Middle East against ISIS. Frankly, it's surprising that the number wasn't higher: maybe most Republicans and Democrats are Disney aficionados and sniffed out the dirty trick. The Washington Post's Volokh Conspiracy has a piece on "political ignorance" that's worth a read. 

Propagating this stuff isn't harmless. It gets spread about the Internet like so much manure. It exacerbates polarization and mutual antipathy. Most of us are going to read a headline or see something on Facebook and walk away saying, "these people are idiots that want to literally enslave black people." It takes a particularly ornery, dedicated politics nerd to dig up that these polls are hopelessly flawed--and then rarely do people run around sharing the correction to this misinformation, because it doesn't make our blood boil. 

What can you do about it? You could shame these guys. When our chicken nuggets are under-cooked, we get in a huff, take a picture, post it on Twitter, and call out McDonald's. They hear that--they care about their online reputation. If you want to make the world a better place, you can do the same here--where it matters. Sloppy journalism is a plague, but it'll keep going as long as we keep gleefully buying into it to feel smug. If you're uninterested, we've got a great site for you that warns Americans about the dangers of the deadly chemical Dihydrogen Monoxide, which is in your tap water and can kill you. 


Erik Fogg

We do politics, but we don't do the thinking for you.