a blog of data, dystopia and despair

  • Can we finally stop relying on primary polls?

    The Iowa Caucus results1 were yet another example of what many of us data nerds have been insisting upon for a long time: polling the presidential primary season – especially in the early stages– is an increasingly worthless endeavor.

    Hillary Clinton in 2008 was seen as inevitable and unbeatable in the Democrat primaries, according to polls, until she wasn’t and fell to Barack Obama. The polls showed Mitt Romney was slated to beat Obama in 2012, until he didn’t. Strategists and pundits leaning on single polls like a crutch for political analysis mocked statistician Nate Silver for his extremely accurate 2012 elections predictions based on polling aggregation and analysis, plus other weighted factors.2 The 2014 midterm was disastrous for media polling as GOP victories swept across the country. Then, of course, the 2016 Iowa Caucus and likely the caucuses and primaries that follow have demonstrated support for certain candidates are undervalued while others are exaggerated when viewed through the lenses of rolling and aggregated polls.

    The data is too sparse and unreliable and the methodology suspect in an age where landline phones near extinction and Internet surveys are only slightly more believable than Internet polls.

    There was a heyday for election polling spanning the 20th Century, when willing participants were more common and more easily reached. But barring a shift away from outdated, inadequate or underfunded research models, polling seems to be in crisis and simply cannot be relied upon as a trustworthy means of taking the electorate’s temperature.

    Pollsters well recognize3 this reality as more people use unlisted cellphones and increasingly don’t take the time to answer surveys, meaning that conducting quality research has become more difficult and expensive. This narrows the demographics of respondents to those older and less technologically connected, which is increasingly unrepresentative of the electorate.

    I had hoped media organizations would have been given more pause when Gallup, which has tracked horse races since it accurately predicted Franklin Roosevelt’s 1936 landslide, announced it wouldn’t be engaging in primary polling4 this election cycle due to significant mismatches between reporting and outcomes.

    There is a difference between a poll being accurate and one being predictive. Did the 600 people polled tell the truth about their candidate preferences? Probably, so the poll could be fairly accurate within the scope of the sample. But where the wheels fall off that bus is when media pundits and analysts try using such a snapshot to predict a race’s outcome when people start voting, and I would argue there simply isn’t enough reliable information to make that leap, especially in cases where actual voter turnout5 isn’t being accounted for.

    Many data journalists and statisticians, including the aforementioned Nate Silver of FiveThirtyEight, have been maligned for suggesting Donald Trump is unlikely to win the GOP nomination and that his soaring national poll numbers are likely inflated. This may or may not prove to be true. But Iowa was the first salvo showing Silver is probably more right than not, since Trump and Ted Cruz’s fortunes were reversed from expectations and Marco Rubio’s totals were significantly higher than polling aggregates suggested.

    Source: HuffPost Pollster

    I would argue any resemblance between the media’s polling and actual vote counts in primaries and caucuses is purely incidental. Even if we argue that aggregate polls for the Democratic race in Iowa showed a clear, narrowing trend line between the top two candidates over time, none suggested the series of coin tosses Hillary Clinton and Bernie Sanders found themselves engaged in. It was never supposed to be this close according to what the constant flow of media polls suggested.

    Source: HuffPost Pollster

    We only have Iowa under our belt thus far this primary season, so sure, every other state could somehow fall into line with horse race trends reported by various media outlets. But I’m not holding my breath.

    Ann Seltzer – hailed as the best pollster in Iowa for the Des Moines Register, who has a very good track record6 – showed a comfortable Trump caucus win in her final survey result (though Cruz has led the pack at various points over the last few weeks and the drill-down suggested he was more popular). Rather than a knock on a very good pollster who has done well in a very hard to gauge state7, the Register’s poll uncharacteristically missing the mark simply adds to growing doubts about the practice of horse race polling overall and further suggests that Gallup knows something many media outlets aren’t really widely acknowledging: primary polling is rather inaccurate.

    Do I have a solution to close the gap between primary polls and voting outcomes, a golden measurement to predict political futures? No. And there are pretty good polls and surveys out there outside of the realm of early presidential races. Plus, as the campaigns wear on and the field narrows into the general election, polls become slightly more valuable and even more predictive – though even then aren’t beyond suspicion.8

    Rather than tossing election polls out as utterly worthless just yet, I instead simply append a mental asterisk to new numbers and don’t let them guide my expectations. Because beyond the realm of data there are other, better indicators to determine how elections, nominations, primaries and caucuses will turn out – like waiting for people to vote.

    1. Amanda Cox, Alicia Parlapiano and Derek Watkins. “Iowa’s Democratic Caucus Results, Precinct by Precinct” The New York Times. February 2, 2016. Link 

    2. Taylor, Chris. “Triumph of the Nerds: Nate Silver Wins in 50 States” Mashable. November 7, 2012. Link 

    3. Zukin, Cliff. “What’s the matter with polling?” The New York Times. June 20, 2015. Link 

    4. Shepard, Steven. “Gallup gives up the horse race” Politico. October 7, 2015. Link 

    5. Phillip, Abby. “ 

    6. Enten, Harry. “The Best Pollster In Iowa Just Released Its Final Survey — How Accurate Has It Been?” FiveThirtyEight. Janury 30, 2016. Link 

    7. Silver, Nate. “Iowa Is The Hardest State To Poll” FiveThirtyEight. February 1, 2016. Link 

    8. Clement, Scott. “Gallup explains what went wrong in 2012” The Washington Post. June 4, 2013. Link 


  • London Eyes

    To paraphrase Alan Moore, his “V for Vendetta” depicted a world under constant audio and visual survelliance – and obviously, someone in London thought that was a fantastic idea.

    London has been the scene for many works of dystopian fiction, from George’s Orwell’s “Nineteen Eighty-Four,” Alan Moore’s “V for Vendetta” and P.D. James’ “The Children of Men.” Each has centralized government surveillance as hallmarks of future societies. It would seem London has already crossed the threshold of establishing omnipresent eyes tracking the activities of average citizens.

    A vast network

    The cameras really are everywhere and have simply become part of public life, peppered throughout the Underground and every London borough. Most paradoxical are signs prohibiting photography in certain public buildings, posted right next to the CCTV advisory. They can record us, but we must not record them.

    About three years ago now, the BBC plotted a density map1 of CCTV cameras in England. The Borough of Wandsworth had the highest number of cameras per thousand people, totaling about 1,113 individual devices, outnumbering the surveillance networks in Boston, Dublin and Johannesburg. Islington has about one camera per every 1,000 sets of feet walking the streets. The City of London has about 619 cameras to cover a population of 9,000, a staggering 68.7 cameras per 1,000 people.

    Creeping feelings of being watched extend beyond London’s vast limits too. The Shetland Islands and Corby –  two of the smallest local authorities in the United Kingdom – both have more CCTV cameras than are run by the San Francisco Police Department.

    Security and transparency

    CCTV recordings are subject to the Data Protection Act2, so people can request the footage of themselves going about their business, dancing with a sign or whatever. Similar to U.S. Freedom of Information Act requests, these can be frustrating to file and wait for.

    Police have argued that to provide full maps of CCTV placements would be useful to terrorists, so few complete listings exist. Some boroughs, like Lambeth, do have local maps available.3 Artist Manu Luksch mapped the cameras around Whitehall4, though not without encountering significant police resistance.

    The paranoia is somewhat understandable, since the British seem genetically predisposed to being blown up after years of battling the IRA, surviving the Nazi Blitz and more recently, the July 7, 2005 London Tube bombings. In that latter case, CCTV did nothing to prevent the attacks. For the July 21, 2005 bomb attempts to follow, authorities could merely hit rewind and see what happened, which is useful5 from the standpoint of prosecuting those responsible.6 But the constant drone of overhead voices throughout the Underground is how CCTV cameras hang from the ceiling “for your safety and security.”

    Privacy advocates view CCTV as an affront to personal liberties. Though do people really have a right to privacy in public spaces? A photojournalist would claim snapping pictures of someone on the streets for publishing is fair and legal practice. Is it really any different when the government does it?

    Debate rages over whether CCTV impacts crime statistics, with some reports showing little effect7 save for on car-related infractions, even while three quarters of anti-crime spending is allocated to the networks. Critics claim it’s used to spy on legal protests8, classrooms9 and pubs.9 Studies have argued the impact is difficult to measure10 because camera presence can lead to increases in reported criminal activity, which can skew numbers of actual crimes committed, prevented or punished.

    Senior police officials have both supported CCTV as an investigative tool while others have questioned whether it’s being used effectively.11

    Some Brits seem convinced it’s meant only as a deterrent, explaining why signs and voices are constantly warning people whenever CCTV is in use. If police really wanted to capture criminals in the act and prove the system’s effectiveness, wouldn’t they be more shifty about it? The cameras aren’t always turned on either, and the sheer number of devices on the network make it inconceivable they can all be individually and constantly monitored anyway, as demonstrated by the Tube bombings.

    Wandsworth, with its very high concentration of cameras, had a 8.69 percent crime rate in May 2012.12 Neighboring Richmond had 7.07 percent and is in the bottom ten boroughs for number of cameras. Comparing the Metropolitan Police’s London crime map13 and City of London statistics14 against the BBC’s camera density findings1 and ONS population statistics15 doesn’t seem to yield a direct connection between CCTV presence and criminal incidents. There are varying socioeconomic factors contributing to crime rates, but can data be produced demonstrating CCTV’s direct impact on those numbers? Or is it merely a data collecting system to better prosecute criminals after the fact?

    1. “The statistics of CCTV” BBC News. July 20, 2009. Link  2

    2. “Data Protection Act 1998” The National Archives. Link 

    3. “Cameras in Lambeth, London” Lambeth. Retrieved July 2012. Link 

    4. “Mapping CCTV around Whitehall” AmbientTV. Link 

    5. “Keep an eye on our growing surveillance culture” The Guardian. February 8, 2009. Link 

    6. “21 July 2005 London bombings” Wikipedia. Link 

    7. Travis, Alan. “CCTV schemes in city and town centres have little effect on crime, says report” The Guardian May 17, 2009. Link 

    8. Doctorow, Cory. “How British cops are criminalising peaceful protest” BoingBoing. March 27, 2009. Link 

    9. Doctorow, Cory. “English schoolkids go on strike until CCTVs are removed from classes” BoingBoing. May 22, 2009. Link  2

    10. Doctorow, Cory. “London police demand that new pub owner installs a CCTV as a licensing condition” BoingBoing. February 9, 2009. Link 

    11. “The Effectiveness of Public Space CCTV: A Review of Recent Published Evidence Regarding the Impact of CCTV on Crime” Scottish Government. December 2009. Link 

    12. “Police ‘not using CCTV properly’” BBC News. July 20, 2009. Link 

    13. “Metropolitan Police Crime Mapping” Metropolitan Police Department. Retrieved July 2012. Link 

    14. “Crime map” Metropolitan Police Department. Retrieved July 2012. Link 

    15. “2010 Population Statistics” London Datastore. Retrieved June 2012. Link 


Return to Top

Datamancy is the work of Frey Hargarten, a data journalist and artist.

Don't take my word for anything -- do lots of research instead. My work here is offered under the Creative Commons license.

Opinions are mine and don't reflect the stances of my employer.