Posts Tagged ‘polls’
“Public opinion polls are rather like children in a garden, digging things up all the time to see how they’re growing”*…
As the press continues to treat this year’s all–too–consequential election as a horse race, your correspondent is re-visiting a topic touched a few weeks ago: the prevalence of polling data in election coverage. Rick Perlstein weighs in with a (fascinating) history of presidential election polling, then turns to it implications…
… That polls do not predict Presidential election outcomes any better now than they did a century ago is but one conclusion of this remarkable history. A second conclusion lurks more in the background—but I think it is the most important one to absorb.
For most of this century, the work was the subject of extraordinary ambivalence, even among pollsters. In 1948, George Gallup called presidential polling (as distinguished from issue polling, which has its own problems) “this Frankenstein.” In 1980, Elmo Roper admitted that “our polling techniques have gotten more and more sophisticated, yet we seem to be missing more and more elections.” All along, conventional journalists made a remarkably consistent case that they were empty calories that actively crowded out genuine civic engagement: “Instead of feeling the pulse of democracy,” as a 1949 critic put it, “Dr. Gallup listens to its baby talk.”
Critics rooted for polls to fail. Eric Sevareid, in 1964, recorded his “secret glee and relief when the polls go wrong,” which might restore “the mystery and suspense of human behavior eliminated by clinical dissection.” If they were always right, as James Reston picked up the plaint in 1970, “Who would vote?” Edward R. Murrow argued in 1952 that polling “contributed something to the dehumanization of society,” and was delighted, that year, when “the people surprised the pollsters … It restored to the individual, I suspect, some sense of his own sovereignty” over the “petty tyranny of those who assert that they can tell us what we think.”
Still and all, the practice grew like Topsy. There was an “extraordinary expansion” in polls for the 1980 election, including the first partnerships between polling and media organizations. The increase was accompanied by a measurable failure of quality, which gave birth to a new critique: news organizations “making their own news and flacking it as if it were an event over which they had no control.”
And so, after the 1980 debacle, high-minded observers began wondering whether presidential polls had “outlived their usefulness,” whether the priesthood would end up “defrocked.” In 1992, the popular columnist Mike Royko went further, proposing sabotage: Maybe if people just lied, pollsters would have to give up. In 2000, Alison Mitchell of The New York Times proposed a polling moratorium in the four weeks leading up to elections, noting the “numbing length … to which polling is consuming both politics and journalism.”
Instead, polling proliferated: a “relentless barrage,” the American Journalism Review complained, the media obsessing over each statistically insignificant blip. Then, something truly disturbing started happening: People stopped complaining.
A last gasp was 2008, when Arianna Huffington revived Royko’s call for sabotage, until, two years later, she acquired the aggregator Polling.com and renamed it HuffPost Pollster. “Polling, whether we like it or not,” the former skeptic proclaimed, “is a big part of how we communicate about politics.”
And so it is.
Even as the resources devoted to every other kind of journalism atrophied, poll-based political culture has overwhelmed us, crowding out all other ways of thinking about public life. Joshua Cohen tells the story of the time Silver, looking for a way to earn eyeballs between elections, considered making a model to predict congressional votes. But voters, he snidely remarked, “don’t care about bills being passed.”
Pollsters might not be able to tell us what we think about politics. But increasingly, they tell us how to think about politics—like them. Following polls has become our vision of what political participation is. Our therapy—headlines like the one on AlterNet last week, “Data Scientist Who Correctly Predicted 2020 Election Now Betting on ‘Landslide’ Harris Win.” Our political masochism: “Holy cow, did you hear about that Times poll.” “Don’t worry, I heard it’s an outlier …”
The Washington Post’s polling director once said, “There’s something addictive about polls and poll numbers.” He’s right. When we refer to “political junkies,” polls are pretty much the junk.
For some reason, I’ve been able to pretty much swear off the stuff, beyond mild indulgence. Maybe it’s my dime-store Buddhism. I try to stay in the present—and when it comes to the future, try to stick with things I can do. Maybe, I hereby offer myself as a role model?
As a “political expert,” friends, relatives, and even strangers are always asking me, “Who’s going to win?” I say I really have no idea. People are always a little shocked: Prediction has become what people think political expertise is for.
Afterward, the novelty of the response gets shrugged off, and we can talk. Beyond polling’s baby talk. About our common life together, about what we want to happen, and how we might make it so. But no predictions about whether this sort of thing might ever prevail. No predictions at all…
Presidential polls are no more reliable than they were a century ago. So why do they consume our political lives?
Eminently worth reading in full. Presidential polls are no more reliable than they were a century ago. So why do they consume our political lives? “The Polling Imperilment,” from @rickperlstein in @TheProspect.
Pair with: “The Problems with Polls.”
For more on why today’s polls are so flawed, see “A public-opinion poll is no substitute for thought.”
Apposite: from the estimable James Fallows: “Election Countdown, 38 Days to Go: What Is Wrong With Our Leading Paper?“
* J. B. Priestley
###
As we pray for more consequential coverage, we might recall that it was on this date in 1936 that the (then-venerable) Literary Digest mailed out return postcard to 2,000,000 Americans, asking them to return the card with an indication for whether they would be voting in the upcoming presidential election for incumbent, Franklin D. Roosevelt or challenger Alf Landon. They published the results of their anxiously-anticipated poll in their October 31 issue: a massive victory for Landon. In the event, of course, Roosevelt defeated Landon in an unprecedented landslide.
“A public-opinion poll is no substitute for thought”*…
Opinion polls are a key accelerant in the inflamed civil discourse of our time. And, as Teresa Carr explain, that’s a problem…
Last December, a joint survey by the Economist and the polling organization YouGov claimed to reveal a striking antisemitic streak among America’s youth. One in five young Americans thinks the Holocaust is a myth, according to the poll. And 28 percent think Jews in America have too much power.
“Our new poll makes alarming reading,” declared the Economist. The results inflamed discourse over the Israel-Hamas war on social media and made international news.
There was one problem: The survey was almost certainly wrong. The Economist/YouGov poll was a so-called opt-in poll, in which pollsters often pay people they’ve recruited online to take surveys. According to a recent analysis from the nonprofit Pew Research Center, such polls are plagued by “bogus respondents” who answer questions disingenuously for fun, or to get through the survey as quickly as possible to earn their reward.
In the case of the antisemitism poll, Pew’s analysis suggested that the Economist/YouGov team’s methods had yielded wildly inflated numbers. In a more rigorous poll posing some of the same questions, Pew found that only 3 percent of young Americans agreed with the statement “the Holocaust is myth.”
These are strange times for survey science. Traditional polling, which relies on responses from a randomly selected group that represents the entire population, remains the gold standard for gauging public opinion, said Stanford political scientist Jon Krosnick. But as it’s become harder to reach people on the phone, response rates have plummeted, and those surveys have grown exponentially more expensive to run. Meanwhile, cheaper, less-accurate online polls have proliferated.
“Unfortunately, the world is seeing much more of the nonscientific methods that are put forth as if they’re scientific,” said Krosnick…
…
… headlines as outrageous as they are implausible continue to proliferate: 7 percent of American adults think chocolate milk comes from brown cows; 10 percent of college graduates think Judge Judy is on the Supreme Court; and 4 percent of American adults (about 10 million people) drank or gargled bleach to prevent Covid-19. And although YouGov is one of the more respected opt-in pollsters, some of its findings — one third of young millennials aren’t sure the Earth is round, for example — strain credulity.
Amidst a sea of surveys, it’s hard to distinguish solid findings from those that dissolve under scrutiny. And that confusion, some experts say, reflects deep-seated problems with new methods in the field — developed in response to a modern era in which a representative sample of the public no longer picks up the phone.
The fractious evolution in polling science is likely to receive fresh attention as the 2024 elections heat up, not least because the consequences of failed or misleading surveys can go well beyond social science. Such “survey clickbait” erodes society’s self-esteem, said Duke University political scientist Sunshine Hillygus: It “undermines people’s trust that the American public is capable of self-governance.”
Veteran pollster Gary Langer compares traditional randomized polling methods, known as probability polling, to dipping a ladle into a well-stirred pot of minestrone soup. “We can look in and see some cannellini beans, little escarole, chunks of tomato,” he said. “We get a good representation of what’s in the soup.”
It doesn’t matter if the pot is the size of Yankee Stadium, he said. If the contents are thoroughly mixed, one ladle is enough to determine what’s in it. That’s why probability surveys of 1,000 people can, in theory, represent what the entire country thinks.
The problem is that getting a truly representative sample is virtually impossible, said YouGov’s Douglas Rivers, who pointed out that these days a good response rate to a randomized poll is 2 percent…
…
… with the appropriate guardrails against fraud, YouGov chief scientist Rivers said, such methods offer a practical alternative to conventional probability sampling, where the costs are too high, and the response rates are too low. In some sense, he suggested, most polling is now nonprobability polling: When only 2 out of 100 people respond to a survey, it’s much harder to claim that those views are representative, said Rivers. “Sprinkling a little bit of randomness at the initial stage does not make it a probability sample.”
“Our approach has been: Let us assemble a sample systematically based on characteristics,” said Rivers. “It’s not comparable to what the census does in the current population survey, but it’s performed very well in election polling.” Rivers pointed to YouGov’s high ranking on the website FiveThirtyEight, which rates polling firms based on their track record in predicting election results and willingness to show their methods.
Gary Langer was not particularly impressed by high marks from FiveThirtyEight. (His own firm, Langer Research Associates, also gets a top grade for political polling they conduct on behalf of the partnership between ABC News and The Washington Post.) “Pre-election polls, while they get so much attention, are the flea on the elephant of the enterprise of public opinion research,” he said. The vast majority of surveys are concerned with other topics. They form the basis of federal data on jobs and housing, for example, and can reflect the public’s views on education, climate change, and other issues. “Survey data,” he said, “surrounds us, informs our lives, informs the choices we make.”
Given the stakes, Langer relies exclusively on probability polling. Research shows that opt-in polls just don’t produce the same kind of consistent, verifiable results, said Langer…
Research suggests that widely used nonprobability methods, in particular online opt-in polls such as the Economist/YouGov survey, have inherent vulnerabilities.
The prospect of cash or rewards can incentivize some people to complete surveys quickly and with as little effort as possible. “They’re giving you data and answers that just can’t possibly be true,” said Kennedy.
For example, in one test of opt-in polling, 12 percent of U.S. adults younger than 30 claimed that they were licensed to operate a nuclear submarine. The true figure, of course, is approximately 0 percent…
…
… Media consumers should be skeptical of implausible findings, said Krosnick. So should reporters, said Langer, who spent three decades as a journalist, and who said news outlets have a responsibility to vet the polls they report on: “Every newsroom in the country — in the world — should have someone on their team evaluate surveys and survey methodologies.”
In the end, people need to realize that survey research involves some degree of uncertainty, said Joshua Clinton, a political scientist at Vanderbilt University, who noted that polls leading up to the 2024 election are bound to get something wrong. “My concern is what that means about the larger inferences that people make about not only polling, but also science in general,” he said. People may just dismiss results as a predictable scientific failure: “‘Oh, the egghead screwed up again.’” Clinton said he wants people to recognize the difficulty of doing social science research, rather than to delegitimize the field outright.
Even Rivers, whose firm produced the Economist poll that made headlines, acknowledged that readers should be cautious with eye-catching headlines. “We’re in a challenging environment for conducting surveys,” he said. That means that people need to take survey results — especially those that are provocative — with a grain of salt.
“The tendency is to overreport polls,” said Rivers. “The polls that get reported are the ones that are outliers.”…
It’s very difficult to get anyone to answer a phone call—and that’s skewing data on everything from chocolate milk to antisemitism: “We’re in a New Era of Survey Science,” from @TeresaRCarr in @undark via @Slate. Eminently worth reading in full.
* Warren Buffett
###
As we take it with a grain of salt, we might recall that it was on this date in 1941 that Tom and Jerry first appeared on screen with those names in the MGM cartoon “The Midnight Snack,” though it was in fact their second screen appearance.
In 1940, MGM had produced “Puss Gets the Boot,” based on Hanna’s and Barbera’s pitch for a story rooted in two “equal characters who were always in conflict with each other.” It was the first collaboration between William Hanna and Joseph Barbera (founding a partnership that would last over 50 years and yield such treasures as The Flintstones, Huckleberry Hound, The Jetsons, Scooby-Doo, Top Cat, and Yogi Bear); at over nine minutes in length, it’s the longest T&J ever produced– and the first of three T&J essays (with “Puss n’ Toots” and “Puss ‘n’ Boats”) to pun it’s title on the fairy tale “Puss in Boots.” “Puss Gets the Boot” was nominated for an Academy Award– the first of Hanna and Barbera’s many Oscar nominations.
The cat in “Puss Gets the Boot” was actually named “Jasper”; the mouse, “Jinx.” But when the pilot got the go-ahead to become a series, animator John Carr won a studio-wide naming contest with his suggestion: “Tom and Jerry.” The cat’s owner, “Mammy Two-Shoes,” was voiced by June Foray— who later earned immortality as the voice of Rocky J. Squirrel.
“As democracy is perfected, the office of president represents, more and more closely, the inner soul of the people. On some great and glorious day the plain folks of the land will reach their heart’s desire at last and the White House will be adorned by downright moron.”*…
This is a year, Niccolo Conte demonstrates, in which that possibility is especially present around the world…
With almost half of the world’s population residing in countries holding executive or legislative elections in 2024, it’s set to be the busiest election year ever recorded…
Many people are already aware of the U.S. presidential and legislative elections set to be held on November 5th, especially due to American influence on the global political stage and media coverage.
But two governments affecting larger populations, India and the European Union, are also slated to have elections in 2024…
A few notable elections have already occurred. Taiwan held general elections on January 13th, with the more anti-China Democratic Progressive Party retaining the presidency but losing its majority in the legislature.
[And in February, Indonesia held general elections; while the results are still being tabulated, early indications are that it could make for some material changes in the world’s third-largest democracy.]
Pakistan also held elections on February 8th, with former Prime Minster Imran Khan’s party and affiliates winning a plurality of seats but losing power to a military-backed coalition.
Pakistan’s election results were cast into doubt by foreign observers and media, with Khan having been arrested and sentenced to prison on corruption charges. It is far from the only country holding controversial and potentially undemocratic elections in 2024.
Bangladesh’s landslide January 7th elections were boycotted by the opposition and voters, and Russia’s March 15th elections had three anti-war presidential candidates barred from competing, including Alexei Navalny before his controversial death in February…
The biggest global election year on record: “Mapped: 2024 Global Elections by Country,” from @Niccoloc in @VisualCap.
* H. L. Mencken
###
As we peruse the polls, we might recall that it was on this date that it was on this date in 12 BCE that Caesar Augustus (AKA Octavian), the first Roman Emperor, was elected Pontifex maximus (the chief high priest of the College of Pontiffs [Collegium Pontificum] in ancient Rome)– adding stature as head of Rome’s state religion to his imperial credentials.
“In America, everyone is entitled to an opinion, and it is certainly useful to have a few when a pollster shows up”*…
In the last couple of decades, opinion polling in the U.S. has exploded; the number of national pollsters has more than doubled. Over the same period, American lifestyles have changed in ways that have challenged pollsters– and led them to innovate in a quest for accuracy. Indeed, after the embarrassment of the election of 2016, 61% of national pollsters have changed their methods…
The pollsters at The Pew Research Center— arguably the best of bunch– have polled the pollsters…
The 2016 and 2020 presidential elections left many Americans wondering whether polling was broken and what, if anything, pollsters might do about it. A new Pew Research Center study finds that most national pollsters have changed their approach since 2016, and in some cases dramatically. Most (61%) of the pollsters who conducted and publicly released national surveys in both 2016 and 2022 used methods in 2022 that differed from what they used in 2016. The study also finds the use of multiple methods increasing. Last year 17% of national pollsters used at least three different methods to sample or interview people (sometimes in the same survey), up from 2% in 2016.
This study captures what changes were made and approximately when. While it does not capture why the changes were made, public commentary by pollsters suggests a mix of factors – with some adjusting their methods in response to the profession’s recent election-related errors and others reacting to separate industry trends. The cost and feasibility of various methods are likely to have influenced decisions.
This study represents a new effort to measure the nature and degree of change in how national public polls are conducted. Rather than leaning on anecdotal accounts, the study tracked the methods used by 78 organizations that sponsor national polls and publicly release the results. The organizations analyzed represent or collaborated with nearly all the country’s best-known national pollsters. In this study, “national poll” refers to a survey reporting on the views of U.S. adults, registered voters or likely voters. It is not restricted to election vote choice (or “horserace”) polling, as the public opinion field is much broader. The analysis stretches back to 2000, making it possible to distinguish between trends emerging before 2016 (e.g., migration to online methods) and those emerging more recently (e.g., reaching respondents by text message)…
Fascinating– and important: “How Public Polling Has Changed in the 21st Century,” from @pewresearch (via friend PH).
* Neil Postman, Amusing Ourselves to Death
###
As we consider our answers, we might recall that it was on this date in 2016 that Pew Research Center published the results of a poll on voter satisfaction with U.S. Presidential candidates:
Voter satisfaction with the choice of presidential candidates, already at a two-decade low, has declined even further. A new survey finds that just a third of registered voters say they are very or fairly satisfied with the choices, while 63% say they are not too or not at all satisfied. That represents a 7-percentage-point drop since June in the share of voters expressing satisfaction with their candidate choices…
Already-low voter satisfaction with choice of candidates falls even further
You must be logged in to post a comment.