In recent years, content shared via social media from conflict war zones has allowed us to gain a far deeper understanding of the on-the-ground realities of specific conflicts than previously possible. This presents a real opportunity for providing robust evidence which can underpin foreign and security policymaking about emerging, or rapidly escalating, conflict zones. Despite this opportunity, however, policymakers have generally been slow to adapt to the volume of information disseminated on social media from various armed groups in Syria and slower still to use this as an evidence base for policy generation.
In the case of the Syrian conflict, for example, far more information in the public domain comes not from journalists on the ground (who, it goes without saying face extreme danger in attempting to report from many parts of Syria) but from social media and user generated content online.
In 2012, the video streaming site Bambuser was adopted by groups opposing the Syrian government to livestream videos of protests and violence from across Syria. The situation changed as violence escalated and internet access in opposition-controlled areas became increasingly limited. Today, this means that the vast majority of social media accounts being used by the Syrian opposition are associated with specific locations, media centres, or armed groups. This provides researchers with a real opportunity to understand the actors and events taking place on the ground.
This approach has its advantages: because comparatively few people in opposition areas are using social media, and they tend to be associated with specific groups, it is possible to collect and organise the majority of the social media accounts that are being used in opposition controlled areas. These accounts can be monitored for activity, and cover the majority of information coming from opposition controlled areas through social media. This can provide a detailed and dynamic picture of the conflict as well as a detailed understanding of the various armed groups.
There are challenges in this approach, however. The most obvious, in the Syrian context, is that researchers are limited to what those groups post online – and perhaps more importantly, that this often represents a narrow perspective on the conflict in opposition areas. A second problem is neatly illustrated by the Bellingcat research on the current conflict in Ukraine. Here, the opposite is true: because internet access is not limited, anyone can post anything they like online and they can post to a wider variety of social media sites. Granted, this vast amount of information posted from a huge variety of sources provides a richer picture, but it is also one that is harder to access through the ‘noise’ of other material.
Open source intelligence analysis has the potential not only to inform us about various actors and events on the ground, but also to allow us to piece together disparate pieces of information into a wider, more investigative, piece of research. In Bellingcat’s MH17 investigation, the research team compiled information from dozens of different sources, discovered by searching through thousands of social media accounts from Ukraine and Russia, on a wide variety of social media sites. Using this information it was possible to track the movements of a Buk missile launcher on July 17th through separatist-controlled territory to the likely launch site of the missile that downed MH17. The team was also able to find videos from multiple sources of exactly the same missile launcher travelling as part of a convoy through Russia towards the Ukrainian border a few weeks before MH17 was downed. The evidence was sufficiently robust to allow us to conclude that the Buk seen in Ukraine on July 17th originated from the Russian military.
Our research on the Buk missile launcher demonstrates that not only is there a wealth of largely untapped information available online and especially on social media, but also that a relatively small team of analysts is able to derive a rich picture of a conflict zone. Clearly, research of this kind must be underpinned by an understanding of the way in which content is being produced, who is sharing it, and, crucially, how to verify it – and these are methodological challenges which need to be addressed systematically. Nonetheless, the overarching point is that there is a real opportunity for open source intelligence analysis to provide the kind of evidence base that can underpin effective and successful foreign and security policymaking. It is an opportunity that policymakers should seize.
Eliot Higgins is the founder of Bellingcat and the Brown Moses Blog. Eliot focuses on the weapons used in the conflict in Syria, and open source investigation tools and techniques.
Speaking of systematic methodology, how would it be to validate the source of pictures and obtain originals, instead of working with copies.
The issue I see with these OSINT approaches is, that there is absolutely no way of trusting relationships, like in real intelligence scenarios.
The simple reason is lack of accountability due to anonymous sources… These resources have a great potential, but relying real world policy questions on them is very dangerous, as these are mostly not marketing based topics, but effecting peoples lives.
You can be sure, that intelligence agencies are ahead of your ideas of sourcing open information and more importantly connecting them.
But they simply have to be accountable for their results, so have to omit a lot of information which simply isn’t possible to verify.
So trying to emulate these methods through crowds is very naive, as it will simply just be another opinion generator and multiplier.
Obviously your company will be able to generate some money with this, by selling these “investigations” to news outlets, but this will only lead to a further deterioration of journalism, which will not have the resources to actually investigate topics.
So you might call this revolutionary, I would call this the wet dream of every
Polling may never have been less reliable, or more influential, than it is now.CREDIT
ILLUSTRATION BY MATT CHASE
“I am who I am,” Donald J. Trump said in August, on the eve of this season’s first G.O.P. Presidential debate, and what he meant by that was this: “I don’t have a pollster.” The word “pollster,” when it was coined, was meant as a slur, like “huckster.” That’s the way Trump uses it. Other candidates have pollsters: “They pay these guys two hundred thousand dollars a month to tell them, ‘Don’t say this, don’t say that.’ ” Trump has none: “No one tells me what to say.”
Every election is a morality play. The Candidate tries to speak to the People but is thwarted by Negative Campaigning, vilified by a Biased Media, and haunted by a War Record. I am who I am, the Candidate says, and my Opponents are flunkies. Trump makes this claim with unrivalled swagger, but citing his campaign’s lack of a pollster as proof of his character, while fascinating, is utterly disingenuous. The Path to Office is long. To reach the Land of Caucuses and Primaries, the Candidate must first cross the Sea of Polls. Trump is a creature of that sea.
Lately, the Sea of Polls is deeper than ever before, and darker. From the late nineteen-nineties to 2012, twelve hundred polling organizations conducted nearly thirty-seven thousand polls by making more than three billion phone calls. Most Americans refused to speak to them. This skewed results. Mitt Romney’s pollsters believed, even on the morning of the election, that Romney would win. A 2013 study—a poll—found that three out of four Americans suspect polls of bias. Presumably, there was far greater distrust among the people who refused to take the survey.
The modern public-opinion poll has been around since the Great Depression, when the response rate—the number of people who take a survey as a percentage of those who were asked—was more than ninety. The participation rate—the number of people who take a survey as a percentage of the population—is far lower. Election pollsters sample only a minuscule portion of the electorate, not uncommonly something on the order of a couple of thousand people out of the more than two hundred million Americans who are eligible to vote. The promise of this work is that the sample is exquisitely representative. But the lower the response rate the harder and more expensive it becomes to realize that promise, which requires both calling many more people and trying to correct for “non-response bias” by giving greater weight to the answers of people from demographic groups that are less likely to respond. Pollster.com’s Mark Blumenthal has recalled how, in the nineteen-eighties, when the response rate at the firm where he was working had fallen to about sixty per cent, people in his office said, “What will happen when it’s only twenty? We won’t be able to be in business!” A typical response rate is now in the single digits.
Meanwhile, polls are wielding greater influence over American elections than ever. In May, Fox News announced that, in order to participate in its first prime-time debate, hosted jointly with Facebook, Republican candidates had to “place in the top ten of an average of the five most recent national polls.” Where the candidates stood on the debate stage would also be determined by their polling numbers. (Ranking in the polls had earlier been used to exclude third-party candidates.) Scott Keeter, Pew’s director of survey research, is among the many public-opinion experts who found Fox News’s decision insupportable. “I just don’t think polling is really up to the task of deciding the field for the headliner debate,” Keeter told me. Bill McInturff doesn’t think so, either. McInturff is a co-founder of Public Opinion Strategies, the leading Republican polling organization; with its Democratic counterpart, Hart Research Associates, he conducts the NBC News/Wall Street Journal poll. “I didn’t think my job was to design polling so that Fox could pick people for a debate,” McInturff told me. Really, it’s not possible to design a poll to do that.
Even if more people could be persuaded to answer the phone, polling would still be teetering on the edge of disaster. More than forty per cent of America’s adults no longer have landlines, and the 1991 Telephone Consumer Protection Act bans autodialling to cell phones. (The law applies both to public-opinion polling, a billion-dollar-a-year industry, and to market research, a twenty-billion-dollar-a-year industry.) This summer, Gallup Inc agreed to pay twelve million dollars to settle a class-action lawsuit filed on behalf of everyone in the United States who, between 2009 and 2013, received an unbidden cell-phone call from the company seeking an opinion about politics. (Gallup denies any wrongdoing.) In June, the F.C.C. issued a ruling reaffirming and strengthening the prohibition on random autodialling to cell phones. During congressional hearings, Greg Walden, a Republican from Oregon, who is the chair of the House Subcommittee on Communications and Technology, asked F.C.C. chairman Tom Wheeler if the ruling meant that pollsters would go “the way of blacksmiths.” “Well,” he said, “they have been, right?”
Internet pollsters have not replaced them. Using methods designed for knocking on doors to measure public opinion on the Internet is like trying to shoe a horse with your operating system. Internet pollsters can’t call you; they have to wait for you to come to them. Not everyone uses the Internet, and, at the moment, the people who do, and who complete online surveys, are younger and leftier than people who don’t, while people who have landlines, and who answer the phone, are older and more conservative than people who don’t. Some pollsters, both here and around the world, rely on a combination of telephone and Internet polling; the trick is to figure out just the right mix. So far, it isn’t working. In Israel this March, polls failed to predict Benjamin Netanyahu’s victory. In May in the U.K., every major national poll failed to forecast the Conservative Party’s win.
“It’s a little crazy to me that people are still using the same tools that were used in the nineteen-thirties,” Dan Wagner told me when I asked him about the future of polling. Wagner was the chief analytics officer on the 2012 Obama campaign and is the C.E.O. of Civis Analytics, a data-science technology and advisory firm. Companies like Civis have been collecting information about you and people like you in order to measure public opinion and, among other things, forecast elections by building predictive models and running simulations to determine what issues you and people like you care about, what kind of candidate you’d give money to, and, if you’re likely to turn out on Election Day, how you’ll vote. They might call you, but they don’t need to.
Still, data science can’t solve the biggest problem with polling, because that problem is neither methodological nor technological. It’s political. Pollsters rose to prominence by claiming that measuring public opinion is good for democracy. But what if it’s bad?
A “poll” used to mean the top of your head. Ophelia says of Polonius, “His beard as white as snow: All flaxen was his poll.” When voting involved assembling (all in favor of Smith stand here, all in favor of Jones over there), counting votes required counting heads; that is, counting polls. Eventually, a “poll” came to mean the count itself. By the nineteenth century, to vote was to go “to the polls,” where, more and more, voting was done on paper. Ballots were often printed in newspapers: you’d cut one out and bring it with you. With the turn to the secret ballot, beginning in the eighteen-eighties, the government began supplying the ballots, but newspapers kept printing them; they’d use them to conduct their own polls, called “straw polls.” Before the election, you’d cut out your ballot and mail it to the newspaper, which would make a prediction. Political parties conducted straw polls, too. That’s one of the ways the political machine worked.
Straw polls were usually conducted a few days or weeks before an election. This August, to cull the field for the first G.O.P. debate, Fox News used polls conducted more than four hundred and sixty days before the general election. (These early polls have become so unreliable that neither Gallup nor Pew conducts them.) The question asked ordinarily takes the form of “If the election were held tomorrow . . .” The circumstances under which the next U.S. Presidential election would actually be held tomorrow involve, essentially, Armageddon. Trump won. All flaxen was his poll.
A century ago, newspapers that wanted to predict the outcome of a Presidential election had to join forces. In 1908, the New York Herald, the Cincinnati Enquirer, the Chicago Record-Herald, and the St. LouisRepublic tallied their straws together. William Randolph Hearst’s newspapers did the same thing. But the best predictions were made by a national magazine, the Literary Digest, beginning in 1916. It regularly miscalculated the popular vote, but for a long time it got the Electoral College winner right. In 1920, the Digest mailed out eleven million ballots. By 1932, its mailing list had swelled to twenty million. Most of those names were taken from telephone directories and automobile-registration files. George Gallup was one of the few people who understood that the Digest risked underestimating Democratic votes, especially as the Depression deepened, because its sample, while very big, was not very representative: people who supported F.D.R. were much less likely than the rest of the population to own a telephone or a car.
Gallup was borrowing from the insights of social science. Social surveys, first conducted in the eighteen-nineties, had been a hallmark of Progressive Era social reform. In 1896, W. E. B. Du Bois went door to door in Philadelphia’s Seventh Ward and interviewed some five thousand people in order to prepare his study “The Philadelphia Negro.” In the nineteen-thirties, social scientists argued for the merits of a shortcut that relied on statistical science: surveying a tiny but representative sample of a population.
Gallup had always wanted to be a newspaper editor, but after graduating from the University of Iowa, in 1923, he entered a Ph.D. program in applied psychology. In 1928, in a dissertation called “An Objective Method for Determining Reader Interest in the Content of a Newspaper,” Gallup argued that “at one time the press was depended upon as the chief agency for instructing and informing the mass of people” but that newspapers no longer filled that role and instead ought to meet “a greater need for entertainment.” He therefore devised a method: he’d watch readers go through a newspaper column by column and mark up the parts they liked, so that he could advise an editor which parts of the paper to keep printing and which parts to scrap.
In 1932, when Gallup was a professor of journalism at Northwestern, his mother-in-law, Ola Babcock Miller, ran for secretary of state in Iowa. Her late husband had run for governor; her nomination was largely honorary and she was not expected to win. Gallup had read the work of Walter Lippmann. Lippmann believed that “public opinion” is a fiction created by political élites to suit and advance their interests. Gallup disagreed, and suspected that public opinion, like reader interest, could be quantified. To get a sense of his mother-in-law’s chances, Gallup began applying psychology to politics. The year of the race (she won), Gallup moved to New York, and began working for an advertising agency while also teaching at Columbia and running an outfit he called the Editors’ Research Bureau, selling his services to newspapers. Gallup thought of this work as “a new form of journalism.” But he decided that it ought to sound academic, too. In 1935, in Princeton, he founded the American Institute of Public Opinion, with funding provided by more than a hundred newspapers.
In 1936, in his syndicated column Gallup predicted that the Literary Digest would calculate that Alf Landon would defeat F.D.R. in a landslide and that the Digest would be wrong. He was right on both counts. This was only the beginning. “I had the idea of polling on every major issue,” Gallup explained. He began insisting that this work was essential to democracy. Elections come only every two years, but “we need to know the will of the people at all times.” Gallup claimed that his polls had rescued American politics from the political machine and restored it to the American pastoral, the New England town meeting. Elmo Roper, another early pollster, called the public-opinion survey “the greatest contribution to democracy since the introduction of the secret ballot.”
Gallup’s early method is known as “quota sampling.” He determined what proportion of the people are men, women, black, white, young, and old. The interviewers who conducted his surveys had to fill a quota so that the population sampled would constitute an exactly proportionate mini-electorate. But what Gallup presented as “public opinion” was the opinion of Americans who were disproportionately educated, white, and male. Nationwide, in the nineteen-thirties and forties, blacks constituted about ten per cent of the population but made up less than two per cent of Gallup’s survey respondents. Because blacks in the South were generally prevented from voting, Gallup assigned no “Negro quota” in those states. As the historian Sarah Igo has pointed out, “Instead of functioning as a tool for democracy, opinion polls were deliberately modeled upon, and compounded, democracy’s flaws.”
Ever since Gallup, two things have been called polls: surveys of opinions and forecasts of election results. (Plenty of other surveys, of course, don’t measure opinions but instead concern status and behavior: Do you own a house? Have you seen a doctor in the past month?) It’s not a bad idea to reserve the term “polls” for the kind meant to produce election forecasts. When Gallup started out, he was skeptical about using a survey to forecast an election: “Such a test is by no means perfect, because a preelection survey must not only measure public opinion in respect to candidates but must also predict just what groups of people will actually take the trouble to cast their ballots.” Also, he didn’t think that predicting elections constituted a public good: “While such forecasts provide an interesting and legitimate activity, they probably serve no great social purpose.” Then why do it? Gallup conducted polls only to prove the accuracy of his surveys, there being no other way to demonstrate it. The polls themselves, he thought, were pointless.
Donald Trump doesn’t have a campaign pollster, but, while he was leading them, his campaign loved polls. Polls admitted Trump into the first G.O.P. debate and polls handed him a victory. “Donald J. Trump Dominates Time Poll,” the Trump campaign posted on its Web site following the August debate, linking to a story in which Time reported that forty-seven per cent of respondents said that Trump had won. Time’s “poll” was conducted by PlayBuzz, a viral-content provider that embeds quizzes, polls, lists, and other “playful content” items onto Web sites to attract traffic. PlayBuzz collected more than seventy-seven thousand “votes” from visitors to Time’s Web site in its instant opt-in Internet poll. Timeposted a warning: “The results of this poll are not scientific.”
Because most polls do not come with warnings, many reporters and news organizations have been trying to educate readers about polling methods. The day after the first G.O.P. debate, Slate published a column called “Did Trump Actually Win the Debate? How to Understand All Those Instant Polls That Say Yes.” This, though, didn’t stop Slate from conducting its own instant poll. “TV talking heads won’t decide this election,” Slate’s pollster promised. “The American people will.”
The statistician Nate Silver began explaining polls to readers in 2008; the Times ran his blog, FiveThirtyEight, for four years. Silver makes his own predictions by aggregating polls, giving greater weight to those which are more reliable. This is helpful, but it’s a patch, not a fix. The distinction between one kind of poll and another is important, but it is also often exaggerated. Polls drive polls. Good polls drive polls and bad polls drive polls, and when bad polls drive good polls they’re not so good anymore.
Laws govern who can run for office and how. There are laws about who can vote, and where, and when. Seven constitutional amendments and countless Supreme Court cases concern voting. But polls are largely free from government regulation, or even scrutiny. (This is not true in other countries; Canadian election law, for instance, regulates the disclosure of election polls.)
This wasn’t always the case. In the nineteen-thirties and forties, motions were regularly introduced in Congress calling for an investigation into the influence of public-opinion polling on the political process. “These polls are a racket, and their methods should be exposed to the public,” Walter Pierce, a Democratic member of the House, wrote in 1939, the year Time first called George Gallup a “pollster.” One concern was that polls were jury-rigged. In the Presidential election of 1944, George Gallup underestimated Democratic support in two out of three states. When Congress called him in for questioning to answer the charge that “the Gallup poll was engineered in favor of the Republicans,” Gallup explained that, anticipating a low turnout, he had taken two points off the projected vote for F.D.R. In another instance, a congressman voiced concern that polls “are in contradiction to representative government”: pollsters appeared to believe that the United States is or ought to be a direct democracy.
Social scientists began criticizing pollsters, too. In 1947, in an address to the American Sociological Association, Herbert Blumer argued that public opinion does not exist, absent its measurement. Pollsters proceed from the assumption that “public opinion” is an aggregation of individual opinions, each given equal weight—an assumption Blumer demonstrated to be preposterous, since people form opinions “as a function of a society in operation.” We come to hold and express our opinions in conversation, and especially in debate, over time, and different people and groups influence us, and we them, to different degrees.
Gallup got his back up. In 1948, the week before Election Day, he said, “We have never claimed infallibility, but next Tuesday the whole world will be able to see down to the last percentage point how good we are.” He predicted that Dewey would beat Truman. He was quite entirely wrong.
Gallup liked to say that pollsters take the “pulse of democracy.” “Although you can take a nation’s pulse,” E. B. White wrote after the election, “you can’t be sure that the nation hasn’t just run up a flight of stairs.”
In the wake of polling’s most notorious failure, the political scientist Lindsay Rogers published a book called “The Pollsters: Public Opinion, Politics, and Democratic Leadership.” Rogers, the Burgess Professor of Public Law at Columbia, had started out as a journalist and, as a scholar, he was a humanist at a time when most students of government had turned away from the humanities and toward social science. (Amy Fried, in an essay about what was lost in that abandonment, has called him “the Forgotten Lindsay Rogers.”) He had drafted “The Pollsters” before the election debacle; his concern had very little to do with miscalculation. Where Blumer argued that polling rests on a misapplication of social science, Rogers argued that it rests on a misunderstanding of American democracy. Even if public opinion could be measured (which Rogers doubted), he believed that legislators’ use of polls to inform their votes would be inconsistent with their constitutional duty. The United States has a representative government for many reasons, among them that it protects the rights of minorities against the tyranny of a majority. “The pollsters have dismissed as irrelevant the kind of political society in which we live and which we, as citizens, should endeavor to strengthen,” Rogers wrote. Polls, Rogers believed, are a majoritarian monstrosity.
The alarms raised by Blumer and Rogers went unheeded. Instead, many social scientists came to believe that, if the pollsters failed, social science would fail with them (not least by losing foundation and federal research money). Eight days after Truman beat Dewey, the Social Science Research Council appointed an investigative committee, explaining that “extended controversy regarding the pre-election polls among lay and professional groups might have extensive and unjustified repercussions upon all types of opinion and attitude studies and perhaps upon social science research generally.” The committee concluded that the problem was, in part, quota sampling, but, in any case, the main work of the report was to defend the sample-survey method, including a landmark project founded at the University of Michigan in 1948, which became the most ambitious and most significant survey of American voters: the American National Election Survey.
In 1952, Eisenhower unexpectedly defeated Stevenson. “Yesterday the people surprised the pollsters, the prophets, and many politicians,” Edward R. Murrow said on CBS Radio. “They are mysterious and their motives are not to be measured by mechanical means.” But politicians don’t want the people to be mysterious. Soon, not only political candidates but officeholders—including Presidents—began hiring pollsters. Meanwhile, pollsters claim to measure opinions as elusive as Americans’ belief in God, as the sociologist Robert Wuthnow points out in a compelling and disturbing new book, “Inventing American Religion: Polls, Surveys, and the Tenuous Quest for a Nation’s Faith.” In 1972, when Congress debated a Truth-in-Polling Act, longtime pollsters like Gallup attempted to distance themselves from campaign and media pollsters. Called to testify, Gallup supported the bill, objecting only to the requirement that pollsters report their response rates. That same year, in “Public Opinion Does Not Exist,” the French sociologist Pierre Bourdieu revisited arguments made by Herbert Blumer. As these and other critics have demonstrated again and again, a sizable number of people polled either know nothing about the matters those polls purport to measure or hold no opinion about them. “The first question a pollster should ask,” the sociologist Leo Bogart advised in 1972, is “ ‘Have you thought about this at all? Do you have an opinion?’ ”
Despite growing evidence of problems known as non-opinion, forced opinion, and exclusion bias, journalists only relied on Gallup-style polling more, not less, and they began, too, to do it themselves. In 1973, in “Precision Journalism,” Philip Meyer urged reporters to conduct their own surveys: “If your newspaper has a data-processing department, then it has key-punch machines and people to operate them.” Two years later, the Times and CBS released their first joint poll, and we’ve been off to the races ever since, notwithstanding the ongoing concerns raised by critics who point out, as has Gallup Poll’s former managing editor David Moore, that “media polls give us distorted readings of the electoral climate, manufacture a false public consensus on policy issues, and in the process undermine American democracy.” Polls don’t take the pulse of democracy; they raise it.
By the end of August, Trump, faltering, revealed that he is of course obsessed with his standing in the polls. “I won in every single poll of the debate,” he boasted. “I won in Timemagazine.” Trump’s lead in the polls had taken so many political reporters by surprise that some people who cover polls—“data journalists” is, broadly, the term of art—began turning to data-science firms like Civis Analytics, wondering whether they, too, saw Trump in the lead.
If public-opinion polling is the child of a strained marriage between the press and the academy, data science is the child of a rocky marriage between the academy and Silicon Valley. The term “data science” was coined in 1960, one year after the Democratic National Committee hired Simulmatics Corporation, a company founded by Ithiel de Sola Pool, a political scientist from M.I.T., to provide strategic analysis in advance of the upcoming Presidential election. Pool and his team collected punch cards from pollsters who had archived more than sixty polls from the elections of 1952, 1954, 1956, 1958, and 1960, representing more than a hundred thousand interviews, and fed them into a UNIVAC. They then sorted voters into four hundred and eighty possible types (for example, “Eastern, metropolitan, lower-income, white, Catholic, female Democrat”) and sorted issues into fifty-two clusters (for example, foreign aid). Simulmatics’ first task, completed just before the Democratic National Convention, was a study of “the Negro vote in the North.” Its report, which is thought to have influenced the civil-rights paragraphs added to the Party’s platform, concluded that between 1954 and 1956 “a small but significant shift to the Republicans occurred among Northern Negroes, which cost the Democrats about 1 per cent of the total votes in 8 key states.” After the nominating convention, the D.N.C. commissioned Simulmatics to prepare three more reports, including one that involved running simulations about different ways in which Kennedy might discuss his Catholicism.
In 1964, a political scientist named Eugene Burdick wrote a novel called “The 480,” about the work done by Simulmatics. He was worried about its implications:
There is a benign underworld in American politics. It is not the underworld of cigar-chewing pot-bellied officials who mysteriously run “the machine.” Such men are still around, but their power is waning. They are becoming obsolete though they have not yet learned that fact. The new underworld is made up of innocent and well-intentioned people who work with slide rules and calculating machines and computers which can retain an almost infinite number of bits of information as well as sort, categorize, and reproduce this information at the press of a button. Most of these people are highly educated, many of them are Ph.D.s, and none that I have met have malignant political designs on the American public. They may, however, radically reconstruct the American political system, build a new politics, and even modify revered and venerable American institutions—facts of which they are blissfully innocent. They are technicians and artists; all of them want, desperately, to be scientists.
Burdick’s dystopianism is vintage Cold War: the Strangelovian fear of the machine. (Burdick also co-wrote “Fail Safe,” in which a computer error triggers a nuclear war.) But after 1960 the D.N.C. essentially abandoned computer simulation. One reason may have been that L.B.J. wasn’t as interested in the work of M.I.T. scientists as Kennedy had been. For decades, Republicans were far more likely than Democrats to use computer-based polling. In 1977, the R.N.C. acquired a mainframe computer, while the D.N.C. got its own mainframe in the eighties. The political scientist Kenneth Janda speculates that the technological advantage of the Republican Party during these years stemmed from its ties to big business. Democratic technological advances awaited the personal computer; the R.N.C. is to I.B.M. as the D.N.C. is to Apple. Then came the Internet, which, beginning with the so-called MoveOn effect, favored Democrats but, as Matthew Hindman argued in “The Myth of Digital Democracy,” has not favored democracy.
Douglas Rivers is a professor of political science at Stanford who is also the chief scientist at YouGov. He started trying to conduct public-opinion surveys via the Internet in the nineties, and has done much of the best and most careful work in the field. When he co-founded Knowledge Networks and conducted polls through Web TV, he used probability sampling as an alternative to quota sampling. The initial response rate was something like fifty per cent, but over time the rate fell into the single digits. Then came the Internet crash. “We slimmed down,” Rivers told me when I visited him in Palo Alto. “I went back to teaching.”
Rivers then started a company called Polimetrix, which he sold to YouGov for an estimated thirty-five million dollars. There he developed a method called “matched sampling”: he uses the U.S. Census Bureau’s American Community Survey, which surveys a million people a year, to generate a random sample according to “fifteen variables of representativeness” and to determine who will participate in polls. “You get a million people to take the poll, but you only need a thousand, so you pick the thousand that match your target population,” he explained to me.
Sometimes when political scientists are hired by corporations their research becomes proprietary. “When I say I don’t know the secret sauce, I really don’t know it,” Arthur Lupia says of political scientists who sell their research to businesses rather than publish it in journals that would require them to reveal their methodologies. Lupia is a professor of political science at the University of Michigan, a former director of the American National Election Survey, and the lead author of “Improving Public Perceptions of Political Science’s Value,” a 2014 report prepared by a task force established by the American Political Science Association. Where once social scientists avidly defended the polling industry, many have grown alarmed that media-run horse-race polls may be undermining the public’s perception of the usefulness of social-science surveys. (Lupia jokes that horse-race polls ought to have a warning label that reads “For entertainment purposes only.”) Like Rivers, Lupia ardently believes in the importance of measuring public opinion. “It is critical for a nation that cherishes its democratic legitimacy to seek credible measures of how citizens think, feel, and act in electoral contexts,” Lupia and the political scientist Jon Krosnick have written. Otherwise, “there will be no strong evidentiary basis for differentiating propagandistic tall tales from empirically defensible and logically coherent readings of electoral history.”
It’s an important point. But it may be that media-run polls have endangered the academic study of public opinion and of political behavior. Public disaffection with the polling industry has contributed to a plummeting response rate for academic and government surveys.
Those surveys are invaluable, the political scientist Sidney Verba has argued. “Surveys produce just what democracy is supposed to produce—equal representation of all citizens,” Verba said in a presidential address before the American Political Science Association in 1995. “The sample survey is rigorously egalitarian; it is designed so that each citizen has an equal chance to participate and an equal voice when participating.” Verba sees surveying public opinion not only as entirely consistent with democratic theory but as a corrective to democracy’s flaws. Surveys, Verba argues, achieve representativeness through science.
The best and most responsible pollsters, whether Democratic, Republican, or nonpartisan, want nothing so much as reliable results. Today, with a response rate in the single digits, they defend their work by pointing out that the people who do answer the phone are the people who are most likely to vote. Bill McInturff, of Public Opinion Strategies, told me, “The people we have trouble getting are less likely to vote.” But the difficulty remains. Surveying only likely voters might make for a better election prediction, but it means that the reason for measuring public opinion, the entire justification for the endeavor, has been abandoned. Public-opinion polling isn’t enhancing political participation. Instead, it’s a form of disenfranchisement.
“There are all kinds of problems with public-opinion research, as done by surveys,” Lupia admits. “But a lot of the alternatives are worse. A lot of what we’d have would be self-serving stories about what’s good for people. ‘When given a clear choice between eggs and bananas, ninety-eight per cent of the people prefer one or the other.’ Prior to the polls, I can say that, and you have no check on me. But if there’s a poll you have a check.”
That’s a good point, too, except that there isn’t much of a check on political scientists who don’t reveal their methods because they’ve sold their algorithms to startups for millions of dollars. Whether or not they’re making money, people who predict elections want to be right, and they believe, as fiercely as Lupia does, that they are engaged in a public good. I asked Doug Rivers what role the measurement of public opinion plays in a democracy. He said, “The cynical answer is ‘Once the rockets are up, who cares where they come down.’ ” (He was quoting a Tom Lehrer song.) But Rivers isn’t cynical. He believes that polling “improves the quality of representation.” I asked him to give me an example. He said, “You couldn’t have had the change in gay marriage without the polling data.” Everyone cares where the rockets come down.
The day I visited Crowdpac, at the back of a one-story office building in Menlo Park, the staff was having a debate about what kind of takeout to order during the G.O.P. debate. “What is G.O.P. food? BBQ?” A piece of computer hardware labelled “Hillary’s Hard Drive: HEAVY USE: Now Perfectly Clean” rested on a coffee table. There were Bernie Sanders posters on the walls and cutouts of Rand Paul’s head popping out of a jar of pencils. Crowdpac is the brainchild of Steve Hilton, a former senior adviser to David Cameron, and Adam Bonica, a young Stanford political scientist. Their idea is to use data science to turn public-opinion polling upside down. “There had been an explosion in the use of data, all structured to advance campaigns,” Bonica says. “They’d take information from voters and manipulate it to the politicians’ advantage. But what if it could go the other way?” The company’s unofficial motto on its Web site used to be “Now you can get the data on them!”
Crowdpac is just getting off the ground, but it has provided an interactive Voter’s Guide for several federal, state, and citywide elections from Philadelphia to San Francisco and encouraged people to run for office. Liz Jaff, Crowdpac’s Democratic political director (she has a Republican counterpart), showed me a beta site she’d set up, whereby visitors who supported Planned Parenthood could look up all the unopposed G.O.P. candidates who have promised to defund Planned Parenthood and then pledge money to anyone who would run against them. The pledges would be converted to donations automatically, as soon as someone decided to run. Candidates could see how much money they would have, right out of the gate, and their opponents could see, too. “If you get a tweet saying you just got five hundred thousand dollars pledged against you, that sends a message,” Jaff said.
“We are trying to figure out what drives people to be interested in politics,” Hilton told me. “We are working on tools that help people get engaged with particular issues. If you care about fracking—for or against—what should you do? What candidate should you give money to? What people should you urge to run for office? We are uncovering the hidden political wiring of politics.”
I asked him if that wasn’t the role of the press.
“Maybe once,” he said.
Data science may well turn out to be as flawed as public-opinion polling. But a stage in the development of any new tool is to imagine that you’ve perfected it, in order to ponder its consequences. I asked Hilton to suppose that there existed a flawless tool for measuring public opinion, accurately and instantly, a tool available to voters and politicians alike. Imagine that you’re a member of Congress, I said, and you’re about to head into the House to vote on an act—let’s call it the Smeadwell-Nutley Act. As you do, you use an app called iThePublic to learn the opinions of your constituents. You oppose Smeadwell-Nutley; your constituents are seventy-nine per cent in favor of it. Your constituents will instantly know how you’ve voted, and many have set up an account with Crowdpac to make automatic campaign donations. If you vote against the proposed legislation, your constituents will stop giving money to your reëlection campaign. If, contrary to your convictions but in line with your iThePublic, you vote for Smeadwell-Nutley, would that be democracy?
A worried look crossed Hilton’s face. Lindsay Rogers has long since been forgotten. But the role of public-opinion measurement in a representative government is more troubling than ever.
Hilton shook his head. “You can’t solve every problem with more democracy,” he said.
To winnow the field of candidates who would hold the main stage in the second G.O.P. debate, in September, CNN had intended to use the average of national polls conducted over the summer. But after Carly Fiorina’s campaign complained that the method was unfair CNN changed its formula. The decision had very little to do with American democracy or social science. It had to do with the practice of American journalism. It would make better television if Fiorina was on the same stage as Trump, since he’d made comments about her appearance. (“Look at that face!” he said.)
“No one tells me what to say,” Trump had said in August. By September, on the defensive about Fiorina, he insisted—he knew—that he had the will of the people behind him. “If you look at the polls,” he said, “a lot of people like the way I talk.”
Donald Trump is a creature of the polls. He is his numbers. But he is only a sign of the times. Turning the press into pollsters has made American political culture Trumpian: frantic, volatile, shortsighted, sales-driven, and anti-democratic.
He kept his lead nearly till the end of October. “Do we love these polls?” he called out to a crowd in Iowa. “Somebody said, ‘You love polls.’ I said that’s only because I’ve been winning every single one of them. Right? Right? Every single poll.” Two days later, when he lost his lead in Iowa to Ben Carson, he’d grown doubtful: “I honestly think those polls are wrong.” By the week of the third G.O.P. debate, he’d fallen behind in a national CBS/NYT poll. “The thing with these polls, they’re all so different,” Trump said, mournfully. “It’s not very scientific.” ♦
