I’d like to talk to you today about math and politics. Don’t go running to the sports page. This won’t hurt. Well, it might, if you learn that certain rather important numbers may not be telling you what you think they are.
Let’s dispense immediately with call-in polls local television stations often do: They put a question on the screen and invite viewers to call or text their views. Then, they broadcast a bar graph or something giving “results.” Statistically, all those numbers do is reflect the views of the several dozen who called or texted. They have no relation whatsoever to what all voters think; unfortunately, too many viewers believe that they do.
Now, to “real” polls, with carefully and scientifically chosen population samples:
Longtime Valley pollster Bruce Merrill has been gauging public opinion for 40 years. The now-retired professor is best known for his years conducting the Cronkite/Eight poll at Arizona State University. Today he is a senior research fellow at ASU’s Morrison Institute for Public Policy.
He says he still finds many people are misled because they often do not get the full story of the results of real polls from journalists, who report far too little about them.
Merrill spoke to a small gathering of Valley journalists in central Phoenix Wednesday about understanding polling statistics. More media folks should have been there. They might have learned that if they only report that, say, Candidate A leads Candidate B by three points, they are leaving out some important information.
Take the “margin of error.” It’s often a single-digit number, say, 4 percent. That’s plus-or-minus 4 percent, of course, meaning whatever percentage of respondents is reported to be supporting a certain candidate is simply sitting at the center of a range of 8 percent, Merrill said. But even if an article mentions the margin of error, here’s what most of them don’t:
Merrill used as an example last week’s results of a poll conducted by another polling service. In the contest for U.S. senator, this poll had Republican Jeff Flake leading Democrat Richard Carmona, 44 percent to 43 percent. Republicans can cheer Flake’s lead while Democrats can applaud Carmona’s support keeping pace with that of a well-known congressman.
But both sides can agree that this poll shows the race is tight, right? Maybe and maybe not, Merrill says.
Flake’s 44 percent figure means that – 95 percent of the time, by the way, not 100 percent – his strength among all voters could be as low as 40 percent and as high as 48 percent, Merrill said, while Carmona’s 43 percent number means his support among all voters could be as low as 39 percent and as high as 47 percent.
This means that among all voters it’s as likely that Flake could be swamping Carmona, 48-39, or Carmona could be dominating Flake, 47-40, as it could be 44-43.
This is not something you often hear on TV or read in the paper. That’s because most reporting does not include mention of the components of a poll, called the methodology.
Those who refuse to believe the well-settled mathematical determination that you can get a reliable idea, within the margin of error, of what all voters in a state believe from a properly selected sample of, say, 400 voters, can stop reading here. For the rest of us, Merrill said that one fact voters rarely see in news reporting is whether that sample of 400 is an approximate microcosm of all voters.
Currently in Arizona, voter registration is about 35 percent Republican, 34 percent independent and 31 percent Democratic. But, Merrill asked, what if the sample of 400 people surveyed consisted of 45 percent Republicans? If it was that Flake-Carmona poll, might that 44-43 result hide the fact that Flake’s number was made up of a larger percentage of GOP voters than Arizona as a whole?
And because poll results are only a reflection of what people are thinking on that day – which in most cases is not near Election Day – polls should not be used to predict election outcome, Merrill said, which is unfortunately what the media and the public do with them.
“Polls not only measure public opinion, they drive it,” Merrill said.
But, he said, those polls taken right before elections are quite accurate compared to the actual outcome. He said that since polling in presidential elections began in the 1930s, all such election-eve polls have accurately predicted their outcomes 99.3 percent of the time.
The talking heads on cable TV? Well, let’s just say their rate of accuracy is like the manufacturer’s estimation of your car’s highway mileage versus what mileage it actually gets. It’s probably less.
Read Tribune contributing columnist Mark J. Scarp’s opinions here on Sundays. Reach him at firstname.lastname@example.org.