Maybe Americans really did vote on Tuesday for a massive shake-up of a stale and sclerotic Washington system. Maybe elevating Donald Trump to the Oval Office is the kill-or-cure remedy the United States needs.
Whether or not you agree that the best thing that could happen to Washington is that all sorts of old traditions and norms get thrown out the window, the fact is that it’s happening. And since we’re going to be clearing out some space, here’s one thing we ought to get rid of for good: The ubiquitous polling question giving people a binary choice about the condition of the country as a whole.
Related: Did Third Party Candidates Help Sink Clinton’s Campaign?
You’ve heard it in its many variations: “Is the country going in the right direction, or has it gone off on the wrong track?” or “Are you satisfied or dissatisfied with the way things are going in the United States at this time?”
The morning after Donald Trump was elected president, House Speaker Paul Ryan held a press conference to declare that the election results had given the Republican Party a “mandate” for massive change. In support of his claim, he fell back on right track/wrong track polling. “Seven out of ten Americans, they do not like the direction our country is going,” he said.
But elementary school math will tell you that if 70 percent of people are unhappy with the direction of the country, and only 48 percent of voters chose Donald Trump as president, there’s some pretty widespread disagreement about what direction the country ought to be headed in.
That’s the problem with vague questions and binary choices. Back when the Affordable Care Act was passed, pollsters frequently found that large majorities of the public were dissatisfied with the new law, making it appear wildly unpopular. The few who pushed deeper found that the “dissatisfied” respondents didn’t represent a single group: Many hated the law because they didn’t want the government involved in health care, but others were only dissatisfied because they believed the law hadn’t gone far enough. If you added the latter group to the “satisfied” column, public opinion was actually slightly in favor of the new law.
Related: Ryan Declares GOP Has a ‘Mandate’ to Enact Sweeping Changes
But back to the right track/wrong track polling question. What’s most aggravating about it is that even professionals recognize that it probably isn’t really a useful metric of much of anything.
Asked what the question really measures, Michael Traugott, a research professor in the Center for Political Studies at the University of Michigan said, “I would begin by saying we’re not sure exactly. We know that the predictive power of the right track/wrong track question for voting behavior is not that great certainly not in presidential elections.”
Traugott, who also serves as a senior scientist with the Gallup polling company, said that the question was originally designed to gauge respondents’ perception of the economy. But over the decades since pollsters started asking it, he said, the context has changed pretty dramatically.
“I think as the United States has become more of a multicultural society and issues about gender and all its varieties and same sex marriage have become part of the mix, that part of what people think of as being on the wrong track has to include that as well.”
Related: Who Called It Correctly for Trump? A Poll, a Professor and Michael Moore
He said that even among respondents who respond to it as a purely economic question, it’s problematic.
“There is also the question of how unusual the recovery from the great recession has been, compared to other economic downturns in the United States,” he said. “So the wrong track response, in terms of a cyclical decline, might be different than the response in a period of very, very slow recovery.”
So, why do pollsters keep asking the question?
“There is a tendency among pollsters ... to sometimes ask the same question repeatedly for the sake of maintaining a time series and therefore facilitating comparisons over time,” Traugott said. “But the basic assumption there is that there is a consistent validity to the measurement. And I think in this particular case, this may be a faulty assumption.”
America, it’s time to make opinion polling great again. Retire the right track/wrong track question.
