a blog of data, dystopia and despair

  • Leave a Comment

    I report, write and design news things. I get a lot of feedback from readers. Some is good. Some is bad. Some is threatening, completely insane or both.

    At my publication, the emails and phone numbers of writers are attached to our content, so we occasionally get some quality one-on-one time with our readers. It can be valuable and enriching to get that kind of feedback. But other times, the interaction can be frustrating, and getting some people off the phone without being completely rude is nigh impossible. But for those who don’t like talking to people, the news media might be the wrong industry to work in.

    It’s the other kind of feedback that’s created a real problem though, one that’s been with us for a while and entirely borne of the digital age: online comments.

    Those two words fill many a digital editor with both dread and amusement and have simultaneously become a joke and a major concern in newsrooms across the world.

    The window of discussion on many comments often ranges from passive-aggressive to outright hostile and deranged, touching upon conspiracy theories, all the things we can thank or blame President Obama for and how every topic — no matter how seemingly benign — touches on political, cultural, racial, gender and other divisions.

    None of this is news to anyone, particularly those in the media.

    We frequently have had to shut down comment threads that have descended into personal attacks, threats and disgusting, bigoted language, only to have the same discussion move away from a political story and continue on stories somewhere like the Home and Garden section. There are some kinds of stories — particularly those involving crime or race — that simply have comments disabled from the get-go. And the emails sent to content listservs whenever we shutdown comments on a controversial story can lead to us being compared to Hitler via email, proving again and again the validity of Godwin’s Law.1

    The other law that frequently applies is Poe’s Law,2 where extreme views and parodies of those who actually hold such views can seem indistinguishable, devolving discourse into a mush pile of smirks and rage.

    This plague of trolls, hatemongers and other assorted habitually angry readers has prompted some major publications like Popular Science3 and Bloomberg4 to kill comments all together, while the Huffington Post5 and others vanquished anonymity in discussion sections. Some have chosen simply not to engage with the problem: Vox and the Verge launched without in-house comments at all. Newsrooms across the country are navel-gazing on what to do about this daily problem on their websites and social media pages.

    And therein lies a struggle as the free press deals with balancing open dialogue against the public incitement of hatred by commenters taking place on their digital turf. Journalism, the only profession explicitly protected by the First Amendment, has to delicately walk the line between protecting the spirit of its sister rights and protecting itself and its readers from a groundswell of hateful and oftentimes personal commentary.

    The issue might be“resolved” across the pond where the European Court of Human Rights ruled in summer 2015 that news websites can be fined for the content of their comments sections.6 But in the United States, that would be a dicey solution.

    Of course, First Amendment rights pertain to the government’s ability to make and enforce laws and don’t necessarily apply to private entities. But for journalism, which is viewed as a public trust — aka the Fourth Estate — there does seem to be an implicit expectation of free and open communications that run both ways between media entities and the citizenry they claim to serve.

    In a culture full of safe spaces and growing sensitivity and awareness of oppressed and marginalized communities, it’s easy to forget that vitriol can be a constructive part of public discourse. Pathos, Ethos and Logos are each important parts of making an argument with different impacts on different audiences even if they can sometimes descend into ad hominem attacks

    That’s not an excuse for racism or bigotry or threats or illegal content, of course. Those things we could do without. But even trolls and the harshest of pundits can make a point.

    Those looking to strike a compromise to keep comments sections while reducing the poison found therein have advocated axing anonymous comments7, which about 25 percent of people have made.8 Research does show that people are more likely to say horrible things about individuals and groups if they can hide behind the shroud of an anonymous screen name.9 Surely if they have to stand by their comments with their real name and perhaps even their photo, discussions would moderate themselves.

    That logic seems sound. But upon entering the fray of the endless digital rage war, that proposition starts to lose steam. One has to look no further than Facebook on any given hour to see people have absolutely no problem making the same kinds of dreadful commentary using fully identifying credentials. It’s lead to media organizations, even those that have abandoned native commenting threads, to hire stringers or assign editors to constantly police Facebook comments — which can be an exhausting and repetitive exercise.

    There have long been tensions between the advocacy of free speech and a social desire to crackdown on hate speech. Every time the Westboro Baptist Church shows up at a soldier’s funeral, there are always those vocally wishing their presence wasn’t legal. SCOTUS hasn’t outlined restrictions on hate speech like they have against defamation, fighting words, incitement, obscenity and other First Amendment exceptions. Part of that is rooted difficulties defining exactly what hate speech is.

    So where do we draw the line? When it comes to discussion communities, on one extreme rests sites like 4chan with nearly complete anonymity and few posting rules. On the other extreme are mainstream news websites that have disabled comments entirely. And when it comes to trolls themselves, there are those merrily playing devil’s advocate trying to push discussions and reveal people’s attitudes and kneejerk reactions — and then there are the racists, bigots and assorted crazy people spewing threats and hatred that promote a chilling effect on constructive discussion.

    Is there a middle ground?

    I have no idea for certain and smarter people than myself may eventually come up with some good solutions. A couple things cross my mind though that are drawn from my own experience:

    1.) For several of my childhood years, I was a moderator for gaming-related bulletin boards — yeah, good ol’ BBS, a long-forgotten yet still-present throwback compared to today’s social media landscape. Boards like those I worked for and sites like Something Awful — a granddaddy of online communities — knew that digital communities thrived best when those who care the most about them were allowed to tend them. The best and most constructive commenters were promoted to community moderators and bestowed powers to police threads and to wield their banhammers with discernment. They knew who the real destructive elements in the community were and who were merely the harmless trolls poking the bear. They had a vested interest in keeping communities going and actively shepherding constructive, on-topic commentary. Some — including myself — even got paid a little something, depending upon the website.

    I have seen it tried at various new media websites but lack the data to know how well it worked.

    Is it a model that can be adopted in a world of Disqus, Facebook-driven comments and other social media platforms? It’s hard to say and might require some adjustments to commenting platforms, and perhaps it’s better suited for websites that have retained in-house commenting systems.

    But I’m interested to know whether it’s worth a try to combat a growing problem of toxic commentary while preserving the constructive, open discussions vital for our democracy.

    2.) I’ve also kicked around the potential value of installing a quiz at the end of each article that would-be commenters must answer before posting.

    It’s been clear over the years that the worst comments I get on my articles are from those who clearly have not read a single word of the piece beyond perhaps the headline. Could a basic quiz about facts from the story act both as a filter against those who just want to incite hatred while also providing a brief cooling down period before people write anything?

    Many comment threads seem to be drowning in ill-conceived hot takes and staggering misconceptions about the subject matter, which cause emotions and words to run wild. Maybe being required to take a moment to breathe and ponder before posting could help?

    These suggestions could be completely off base, but might be ways to mitigate at least some of the hatred while preserving online discourse surrounding the news.

    Further discussion welcomed. Feel free to leave a comment.

    1. “Godwin’s law” Wikipedia. Link 

    2. “Poe’s law” Wikipedia. Link 

    3. LaBarre, Suzanne. “Why we’re shutting off our comments” Popular Science. September 24, 2013. Link 

    4. Bode, Karl. “Bloomberg Latest To Kill Comments Because Really, Who Gives A Damn About Localized User Communities?” TechDirt. February 2, 2015. Link 

    5. Soni, Jimmy. “The Reason HuffPost Is Ending Anonymous Accounts” The Huffington Post. October 26, 2013. Link 

    6. Vaas, Lisa. “News sites can be held responsible for user comments” naked security. June 18, 2015. Link 

    7. Wallsten, Kevin and Tarsi, Melinda. “It’s time to end anonymous comments sections” The Washington Post. August 19, 2014. Link 

    8. Beaujon, Andrew. “25% of people have posted anonymous comments, Pew finds” Poynter. September 5, 2013. [Link](http://www.poynter.org/news/mediawire/222912/25-of-people-have-posted-anonymous-comments-pew-finds/0 

    9. Santana, Arthur D. “Virtuous or Vitriolic: The effect of anonymity on civility in online newspaper reader comment boards” Journalism Practice. Vol. 8 , Iss. 1,2014 Link 


  • Can we finally stop relying on primary polls?

    The Iowa Caucus results1 were yet another example of what many of us data nerds have been insisting upon for a long time: polling the presidential primary season – especially in the early stages– is an increasingly worthless endeavor.

    Hillary Clinton in 2008 was seen as inevitable and unbeatable in the Democrat primaries, according to polls, until she wasn’t and fell to Barack Obama. The polls showed Mitt Romney was slated to beat Obama in 2012, until he didn’t. Strategists and pundits leaning on single polls like a crutch for political analysis mocked statistician Nate Silver for his extremely accurate 2012 elections predictions based on polling aggregation and analysis, plus other weighted factors.2 The 2014 midterm was disastrous for media polling as GOP victories swept across the country. Then, of course, the 2016 Iowa Caucus and likely the caucuses and primaries that follow have demonstrated support for certain candidates are undervalued while others are exaggerated when viewed through the lenses of rolling and aggregated polls.

    The data is too sparse and unreliable and the methodology suspect in an age where landline phones near extinction and Internet surveys are only slightly more believable than Internet polls.

    There was a heyday for election polling spanning the 20th Century, when willing participants were more common and more easily reached. But barring a shift away from outdated, inadequate or underfunded research models, polling seems to be in crisis and simply cannot be relied upon as a trustworthy means of taking the electorate’s temperature.

    Pollsters well recognize3 this reality as more people use unlisted cellphones and increasingly don’t take the time to answer surveys, meaning that conducting quality research has become more difficult and expensive. This narrows the demographics of respondents to those older and less technologically connected, which is increasingly unrepresentative of the electorate.

    I had hoped media organizations would have been given more pause when Gallup, which has tracked horse races since it accurately predicted Franklin Roosevelt’s 1936 landslide, announced it wouldn’t be engaging in primary polling4 this election cycle due to significant mismatches between reporting and outcomes.

    There is a difference between a poll being accurate and one being predictive. Did the 600 people polled tell the truth about their candidate preferences? Probably, so the poll could be fairly accurate within the scope of the sample. But where the wheels fall off that bus is when media pundits and analysts try using such a snapshot to predict a race’s outcome when people start voting, and I would argue there simply isn’t enough reliable information to make that leap, especially in cases where actual voter turnout5 isn’t being accounted for.

    Many data journalists and statisticians, including the aforementioned Nate Silver of FiveThirtyEight, have been maligned for suggesting Donald Trump is unlikely to win the GOP nomination and that his soaring national poll numbers are likely inflated. This may or may not prove to be true. But Iowa was the first salvo showing Silver is probably more right than not, since Trump and Ted Cruz’s fortunes were reversed from expectations and Marco Rubio’s totals were significantly higher than polling aggregates suggested.

    Source: HuffPost Pollster

    I would argue any resemblance between the media’s polling and actual vote counts in primaries and caucuses is purely incidental. Even if we argue that aggregate polls for the Democratic race in Iowa showed a clear, narrowing trend line between the top two candidates over time, none suggested the series of coin tosses Hillary Clinton and Bernie Sanders found themselves engaged in. It was never supposed to be this close according to what the constant flow of media polls suggested.

    Source: HuffPost Pollster

    We only have Iowa under our belt thus far this primary season, so sure, every other state could somehow fall into line with horse race trends reported by various media outlets. But I’m not holding my breath.

    Ann Seltzer – hailed as the best pollster in Iowa for the Des Moines Register, who has a very good track record6 – showed a comfortable Trump caucus win in her final survey result (though Cruz has led the pack at various points over the last few weeks and the drill-down suggested he was more popular). Rather than a knock on a very good pollster who has done well in a very hard to gauge state7, the Register’s poll uncharacteristically missing the mark simply adds to growing doubts about the practice of horse race polling overall and further suggests that Gallup knows something many media outlets aren’t really widely acknowledging: primary polling is rather inaccurate.

    Do I have a solution to close the gap between primary polls and voting outcomes, a golden measurement to predict political futures? No. And there are pretty good polls and surveys out there outside of the realm of early presidential races. Plus, as the campaigns wear on and the field narrows into the general election, polls become slightly more valuable and even more predictive – though even then aren’t beyond suspicion.8

    Rather than tossing election polls out as utterly worthless just yet, I instead simply append a mental asterisk to new numbers and don’t let them guide my expectations. Because beyond the realm of data there are other, better indicators to determine how elections, nominations, primaries and caucuses will turn out – like waiting for people to vote.

    1. Amanda Cox, Alicia Parlapiano and Derek Watkins. “Iowa’s Democratic Caucus Results, Precinct by Precinct” The New York Times. February 2, 2016. Link 

    2. Taylor, Chris. “Triumph of the Nerds: Nate Silver Wins in 50 States” Mashable. November 7, 2012. Link 

    3. Zukin, Cliff. “What’s the matter with polling?” The New York Times. June 20, 2015. Link 

    4. Shepard, Steven. “Gallup gives up the horse race” Politico. October 7, 2015. Link 

    5. Phillip, Abby. “ 

    6. Enten, Harry. “The Best Pollster In Iowa Just Released Its Final Survey — How Accurate Has It Been?” FiveThirtyEight. Janury 30, 2016. Link 

    7. Silver, Nate. “Iowa Is The Hardest State To Poll” FiveThirtyEight. February 1, 2016. Link 

    8. Clement, Scott. “Gallup explains what went wrong in 2012” The Washington Post. June 4, 2013. Link 


  • London Eyes

    To paraphrase Alan Moore, his “V for Vendetta” depicted a world under constant audio and visual survelliance – and obviously, someone in London thought that was a fantastic idea.

    London has been the scene for many works of dystopian fiction, from George’s Orwell’s “Nineteen Eighty-Four,” Alan Moore’s “V for Vendetta” and P.D. James’ “The Children of Men.” Each has centralized government surveillance as hallmarks of future societies. It would seem London has already crossed the threshold of establishing omnipresent eyes tracking the activities of average citizens.

    A vast network

    The cameras really are everywhere and have simply become part of public life, peppered throughout the Underground and every London borough. Most paradoxical are signs prohibiting photography in certain public buildings, posted right next to the CCTV advisory. They can record us, but we must not record them.

    About three years ago now, the BBC plotted a density map1 of CCTV cameras in England. The Borough of Wandsworth had the highest number of cameras per thousand people, totaling about 1,113 individual devices, outnumbering the surveillance networks in Boston, Dublin and Johannesburg. Islington has about one camera per every 1,000 sets of feet walking the streets. The City of London has about 619 cameras to cover a population of 9,000, a staggering 68.7 cameras per 1,000 people.

    Creeping feelings of being watched extend beyond London’s vast limits too. The Shetland Islands and Corby –  two of the smallest local authorities in the United Kingdom – both have more CCTV cameras than are run by the San Francisco Police Department.

    Security and transparency

    CCTV recordings are subject to the Data Protection Act2, so people can request the footage of themselves going about their business, dancing with a sign or whatever. Similar to U.S. Freedom of Information Act requests, these can be frustrating to file and wait for.

    Police have argued that to provide full maps of CCTV placements would be useful to terrorists, so few complete listings exist. Some boroughs, like Lambeth, do have local maps available.3 Artist Manu Luksch mapped the cameras around Whitehall4, though not without encountering significant police resistance.

    The paranoia is somewhat understandable, since the British seem genetically predisposed to being blown up after years of battling the IRA, surviving the Nazi Blitz and more recently, the July 7, 2005 London Tube bombings. In that latter case, CCTV did nothing to prevent the attacks. For the July 21, 2005 bomb attempts to follow, authorities could merely hit rewind and see what happened, which is useful5 from the standpoint of prosecuting those responsible.6 But the constant drone of overhead voices throughout the Underground is how CCTV cameras hang from the ceiling “for your safety and security.”

    Privacy advocates view CCTV as an affront to personal liberties. Though do people really have a right to privacy in public spaces? A photojournalist would claim snapping pictures of someone on the streets for publishing is fair and legal practice. Is it really any different when the government does it?

    Debate rages over whether CCTV impacts crime statistics, with some reports showing little effect7 save for on car-related infractions, even while three quarters of anti-crime spending is allocated to the networks. Critics claim it’s used to spy on legal protests8, classrooms9 and pubs.9 Studies have argued the impact is difficult to measure10 because camera presence can lead to increases in reported criminal activity, which can skew numbers of actual crimes committed, prevented or punished.

    Senior police officials have both supported CCTV as an investigative tool while others have questioned whether it’s being used effectively.11

    Some Brits seem convinced it’s meant only as a deterrent, explaining why signs and voices are constantly warning people whenever CCTV is in use. If police really wanted to capture criminals in the act and prove the system’s effectiveness, wouldn’t they be more shifty about it? The cameras aren’t always turned on either, and the sheer number of devices on the network make it inconceivable they can all be individually and constantly monitored anyway, as demonstrated by the Tube bombings.

    Wandsworth, with its very high concentration of cameras, had a 8.69 percent crime rate in May 2012.12 Neighboring Richmond had 7.07 percent and is in the bottom ten boroughs for number of cameras. Comparing the Metropolitan Police’s London crime map13 and City of London statistics14 against the BBC’s camera density findings1 and ONS population statistics15 doesn’t seem to yield a direct connection between CCTV presence and criminal incidents. There are varying socioeconomic factors contributing to crime rates, but can data be produced demonstrating CCTV’s direct impact on those numbers? Or is it merely a data collecting system to better prosecute criminals after the fact?

    1. “The statistics of CCTV” BBC News. July 20, 2009. Link  2

    2. “Data Protection Act 1998” The National Archives. Link 

    3. “Cameras in Lambeth, London” Lambeth. Retrieved July 2012. Link 

    4. “Mapping CCTV around Whitehall” AmbientTV. Link 

    5. “Keep an eye on our growing surveillance culture” The Guardian. February 8, 2009. Link 

    6. “21 July 2005 London bombings” Wikipedia. Link 

    7. Travis, Alan. “CCTV schemes in city and town centres have little effect on crime, says report” The Guardian May 17, 2009. Link 

    8. Doctorow, Cory. “How British cops are criminalising peaceful protest” BoingBoing. March 27, 2009. Link 

    9. Doctorow, Cory. “English schoolkids go on strike until CCTVs are removed from classes” BoingBoing. May 22, 2009. Link  2

    10. Doctorow, Cory. “London police demand that new pub owner installs a CCTV as a licensing condition” BoingBoing. February 9, 2009. Link 

    11. “The Effectiveness of Public Space CCTV: A Review of Recent Published Evidence Regarding the Impact of CCTV on Crime” Scottish Government. December 2009. Link 

    12. “Police ‘not using CCTV properly’” BBC News. July 20, 2009. Link 

    13. “Metropolitan Police Crime Mapping” Metropolitan Police Department. Retrieved July 2012. Link 

    14. “Crime map” Metropolitan Police Department. Retrieved July 2012. Link 

    15. “2010 Population Statistics” London Datastore. Retrieved June 2012. Link 


Return to Top

Datamancy is the work of Frey Hargarten, a data journalist and artist.

Don't take my word for anything -- do lots of research instead. My work here is offered under the Creative Commons license.

Opinions are mine and don't reflect the stances of my employer.