Posts tagged fact checking

And so, to our year of bungles: the New Jersey waitress who received a homophobic comment on the receipt from a party she had served; comedian Kyle Kinane’s Twitter beef with Pace Salsa; the Chinese husband who sued his wife for birthing ugly children after he learned she’d had plastic surgery; Samsung paying Apple $1 billion in nickels…

…These all had one thing in common: They seemed too tidily packaged, too neat, “too good to check,” as they used to say, to actually be true. Any number of reporters or editors at any of the hundreds of sites that posted these Platonic ideals of shareability could’ve told you that they smelled, but in the ongoing decimation of the publishing industry, fact-checking has been outsourced to the readers. Not surprisingly—as we saw with the erroneous Reddit-spawned witch-hunt around the Boston Marathon bombing—readers are terrible at fact-checking. And this, as it happens, is good for business because it means more shares, more clicks.

This is not a glitch in the system. It is the system. Readers are gullible, the media is feckless, garbage is circulated around, and everyone goes to bed happy and fed. BuzzFeed’s Jonah Peretti admitted as much when explaining, that, when he’s hiring, he looks for “people who really understand how information is shared on Twitter and Facebook and Instagram and other emerging platforms, because that is in some cases as important as, you know, having traditional reporting talent.” Upworthy editorial director Sara Critchfield seconded the notion. “We reject the idea that the media elite or people who have been trained in a certain way somehow have the monopoly on editorial judgment.”

Luke O’Neil, Esquire. The Year We Broke The Internet.

Or as Ryan Grim, Huffington Post Washington Bureau Chief, told The New York Times, “If you throw something up without fact-checking it, and you’re the first one to put it up, and you get millions and millions of views, and later it’s proved false, you still got those views. That’s a problem. The incentives are all wrong.”

The FJP’s thoughts on fact checking? You can find those here.

Although real-time corrections are modestly more effective than delayed corrections overall, closer inspection reveals that this is only true among individuals predisposed to reject the false claim. In contrast, individuals whose attitudes are supported by the inaccurate information distrust the source more when corrections are presented in real time, yielding beliefs comparable to those never exposed to a correction.

R. Kelly Garrett and Brian E. Weeks, The Promise and Peril of Real-Time Corrections to Political Misperceptions (PDF).

Yesterday I published an article, Can Robots Tell the Truth?, that explores the Washington Post’s attempt to harness an algorithm that could conduct real-time fact checking on political speeches.

Today, Kelly and Brian forwarded this paper of theirs. It’s part of a larger project out of Ohio State University’s School of Communications called “Misperceptions in an Internet Era”. Their Twitter handle is @FalseBeliefNews.

So, if you take their findings and rewrite my headline, you’d end up with something along the lines of, “Who Cares if Robots Can Tell the Truth Because it’s not Going to Change Anyone’s Mind Anyway”.

Which is discouraging. — Michael.

Can Robots Tell the Truth?
Hi, I am a student in journalism and am preparing an article about robots (like the Washington Post’s Truth Teller) validating facts instead of journalists. I am curious to know the Future Journalism Project’s point of view of about this. What are the consequences for journalists, journalism and for democracy? — Melanié Robert
Hi Melanié,
Many thanks for this fascinating question and my apologies for the delay in getting back to you. Here’s what happened:
I started thinking about this, and then I started writing about it. And then I started thinking that what I really needed to do was some reporting. You know, journalism.
I didn’t know much about the Washington Post’s Truth Teller project. For others that don’t, it’s an attempt to create an algorithm that can fact check political speeches in real time.
Since I didn’t know much about it about I got in touch and interviewed the two project leads: Steven Ginsberg, the Post’s National Political Editor, and Cory Haik, the Post’s Executive Producer for Digital News.
They gave me background on Truth Teller and how it came about, and then where they hope it leads. 
But that doesn’t really get to the sociocultural and philosophical questions you pose. So I called upon someone else. His name is Damon Horowitz.
Damon’s spent his career in both artificial intelligence and philosophy. He’s currently Google’s In-House Philosopher (seriously, it’s on his business card) and Director of Engineering. He also teaches philosophy at Columbia University.
So, after talking to these people, and thinking about it some more, I wrote a fair bit. 
You can find your answer at theFJP.org and I hope it answers some of what you’re looking for. — Michael
Have a question? Ask Away.
Image: Marvin the Paranoid Android, Hitchhiker’s Guide to the Galaxy.

Can Robots Tell the Truth?

Hi, I am a student in journalism and am preparing an article about robots (like the Washington Post’s Truth Teller) validating facts instead of journalists. I am curious to know the Future Journalism Project’s point of view of about this. What are the consequences for journalists, journalism and for democracy? — Melanié Robert

Hi Melanié,

Many thanks for this fascinating question and my apologies for the delay in getting back to you. Here’s what happened:

I started thinking about this, and then I started writing about it. And then I started thinking that what I really needed to do was some reporting. You know, journalism.

I didn’t know much about the Washington Post’s Truth Teller project. For others that don’t, it’s an attempt to create an algorithm that can fact check political speeches in real time.

Since I didn’t know much about it about I got in touch and interviewed the two project leads: Steven Ginsberg, the Post’s National Political Editor, and Cory Haik, the Post’s Executive Producer for Digital News.

They gave me background on Truth Teller and how it came about, and then where they hope it leads. 

But that doesn’t really get to the sociocultural and philosophical questions you pose. So I called upon someone else. His name is Damon Horowitz.

Damon’s spent his career in both artificial intelligence and philosophy. He’s currently Google’s In-House Philosopher (seriously, it’s on his business card) and Director of Engineering. He also teaches philosophy at Columbia University.

So, after talking to these people, and thinking about it some more, I wrote a fair bit

You can find your answer at theFJP.org and I hope it answers some of what you’re looking for. — Michael

Have a question? Ask Away.

Image: Marvin the Paranoid Android, Hitchhiker’s Guide to the Galaxy.

So Cute it Has to Be True

Last fall this 30 second video clip of a pig saving a baby goat from a pond made the rounds, getting play on Time magazine’s and Ellen DeGeneres’ Twitter feeds, and broadcast time on the Today Show, NBC Nightly News, Good Morning America and Fox News.

Thing is, the video’s a stunt. Via the New York Times:

[The video] was created for a new Comedy Central series, “Nathan for You,” with the help of some 20 crew members, including animal trainers, scuba divers and humane officers, and required the fabrication of a plastic track to guide the pig to the goat (which was never in jeopardy).

That a faked video had been so rapidly disseminated by unskeptical news outlets was both surprising and dispiritingly familiar to professional experts on the news media.

“It really is embarrassing for the journalists who stumbled upon this and decided to promote it or share it with their audience,” said Kelly McBride, the senior faculty for ethics, reporting and writing at the Poynter Institute. “It’s almost a form of malpractice.”

But to the creators of the video — which has since been viewed more than seven million times — the news reports are the unexpected if felicitous results of a social experiment in which they say they were not aspiring to this level of deceit.

As the Times points out, when the video was shown on NBC’s Nightly News, Brian Williams said the network was “duty-bound to pass this on.” He did offer the caveat that they weren’t quite sure whether the video was true.

Unfortunate Pro Tip: Some things are too cute to be true.

How Did Sports Media Whiff on Manti Te'o's Fake Girlfriend?

Over at Slate, Josh Levin tries to suss out how and why the Sports Illustrated’s and ESPN’s of the world bought into the hoax that Manti Te’o had a girlfriend who died of leukemia at the beginning of the 2012 college football season.

Confirmation bias plays a role, he writes. This is the notion that people are prone to believe that which reinforces what they already believe, whether true or not. In this case, a national, fact-checked (most of the time?) magazine — Sports Illustrated — published a glowing feature on Te’o, complete with his religious, Boy Scout upbringing and personal triumph over tragedy. As his on field reputation grew, future reporters referenced biographical features found in the original SI feature without much thought.

Which leads Levin to explore the basic hagiography we use to report on celebrities. We apply well-worn, paint by numbers, templates — in this case, the heroic athlete overcoming adversity — that do both subjects and readers a disservice.

No matter what we learn about Te’o in the coming days, this black-and-white narrative—good man fixes bad things—enlightens no one and does the athlete no favors…

…Sports Illustrated looked at the linebacker and saw a classic template, not a human being who demanded the scantest thought or scrutiny. In the end, they got back the exact amount of effort they put in. This was journalism as fill-in-the-blank exercise, the creation of a simple story that tells you what you already know. In this case, what we already knew happened not to be the truth. If only Manti Te’o hadn’t been such a boy scout. Then we might have known how interesting he was all along.

Josh Levin, Slate. The Fake Girlfriend Experience.

There was no Lennay Kekua.

Lennay Kekua did not meet Manti Te’o after the Stanford game in 2009. Lennay Kekua did not attend Stanford. Lennay Kekua never visited Manti Te’o in Hawaii. Lennay Kekua was not in a car accident. Lennay Kekua did not talk to Manti Te’o every night on the telephone. She was not diagnosed with cancer, did not spend time in the hospital, did not engage in a lengthy battle with leukemia. She never had a bone marrow transplant. She was not released from the hospital on Sept. 10, nor did Brian Te’o congratulate her for this over the telephone. She did not insist that Manti Te’o play in the Michigan State or Michigan games, and did not request he send white flowers to her funeral. Her favorite color was not white. Her brother, Koa, did not inform Manti Te’o that she was dead.

Koa did not exist. Her funeral did not take place in Carson, Calif., and her casket was not closed at 9 a.m. exactly. She was not laid to rest.

In a bizarre investigative report, Deadspin discovers that the dead girlfriend of Notre Dame football star Manti Te’o was a fabrication.

As (mis)reported by ESPN, Sports Illustrated, Fox Sports, CBS, The New York Post, The Los Angeles Times and The South Bend Tribune among others, the Notre Dame football star carried a heavy heart throughout the 2012 football season when both his grandmother and Lennay Kekua, his girlfriend, died hours apart on the same September day.

Kekua, supposedly, due to complications from leukemia.

Turns out, Lennay Kekua is a fiction. Te’o, who says he met Kekua online, claims he is the victim of an Internet scam.

For the news organizations reporting on — and building up — the sad tale, let’s bring you back to newsroom 101 and the importance of fact checking: Just because one outlet reports it doesn’t make it true.

Read the piece though. A bizarre tale.

For more, ESPN does a follow-up.

Preventing errors from appearing in the magazine is not a simple process. For openers, you need to know that in addition to the basic reporting pieces, we also check “The Talk of the Town,” the critics, fiction, poetry, cartoons, art, captions, the table of contents, certain of the several-paragraph-long essays in the “Goings On” section. We also fact-check the contributors page, the cover wrap, the letters column, all the press releases, and a good deal of the recently mounted Web site.

To start checking a nonfiction piece, you begin by consulting the writer about how the piece was put together and using the writer’s sources as well as our own departmental sources. We then essentially take the piece apart and put it back together again. You make sure that the names and dates are right, but then if it is a John McPhee piece, you make sure that the USGS report that he read, he read correctly; or if it is a John le Carré piece, when he says his con man father ran for Parliament in 1950, you make sure that it wasn’t 1949 or 1951.

Or if we describe the basis on which the FDA approved or disapproved the medical tests that ImClone used for Erbitux, then you need to find out what the complexities of that whole situation were. And of course, this kind of thing has consequences, because if you get it wrong, it matters. We also work on complicated pieces such as the ones we’ve been running this fall about the Pentagon’s top-secret team that is trained to snatch nukes away from belligerent countries, or the piece about the Predator drone that had a clear shot at Mullah Omar, for better or for worse, and didn’t take the shot because the CENTCOM attorneys were not clear on the legality of that operation.

Back in 2002, New Yorker fact-checking director Peter Canby gave a lecture at the Columbia University Graduate School of Journalism about… fact-checking The New Yorker. The lecture is now a chapter in The Art of Making Magazines: On Being an Editor and Other Views from the Industry, an anthology released this September from this and similar Columbia lectures given over the years.

Peter Canby, Columbia Journalism Review. Fact-checking at The New Yorker.

There’s Even a Study for This

thenoobyorker:

By the way if CNN has taught us anything with their coverage of a hormone study about women and their voting habits, it’s that all this talk about “politically correct” science and science funding is bullshit.

CNN and news organizations are quicker to cover this shit than they are covering studies a few months from now that say that the initial study is “flawed” or not true.

FJP: May we introduce you to Why Most Biomedical Findings Echoed by Newspapers Turn Out to be False, summed up nicely by the CBC?

The researchers, lead by neurobiologist François Gonon, examined the way newspapers reported on a number of high profile studies on attention-deficit hyperactivity disorder (ADHD). They asked the question: do scientific claims reported in the media end up being proven true over time? Their answer: in most cases, no. Then they asked: do the media go back and set the record straight? No again.

In other words, we, in the media, make a big deal over a new research finding, but when it turns out to be less exciting, or even wrong after future research, we don’t tend to report that. ‘Never mind’ doesn’t usually make it into the news.

Misinformation and Its Correction is a nice follow-up when your depression about the above dissipates and you want to hunker back down into it again.

Hi FJP, I'm interested in polishing my fact-checking and research skills. What tools and/or websites would you recommend? Since I do not have any formal fact-checking or research experience outside of academia and cannot afford an unpaid internship, how can build a portfolio to demonstrate those skills? Thank you in advance! — Asked by polidigitalaffairs

Hi there,

Serendipity’s a beautiful thing. I say this because I was on a train this morning thinking about this question when I started listening to a Radiolab program that tackles this very issue.

So I’d start there. In particular, the segment about Errol Morris’ obsession with a famous Crimean War photograph that Morris first wrote about at the New York Times. Morris later turned this three-part, 25,000 word musing into a book called Believing is Seeing. The book explores how we perceive truth in documentary photography.

Next, I’d check out Nieman Reports. This summer they released “Truth in the Age of Social Media,” a look at how “the BBC, the AP, CNN, and other news organizations are addressing questions of truth and verification.”

And then I’d check out the FJP’s Fact Checking Tag which has a number of ideas about — and links to — fact checking issues and resources.

How to build a portfolio without an internship or some other entry into the field?

Just start doing it. Pick an event and start fact checking it. It’s high political season so there are plenty of things to choose from. I’ve written before about how we need to make our own opportunities by spending the time and effort to build our portfolios. So start a fact checking blog and show the world what you can do.

And remember to show us too. — Michael

Have a question? Ask away. Our inbox is open.

Misinformation is even more likely to travel and be amplified by the ongoing diversification of news sources and the rapid news cycle. Today, publishing news is as simple as clicking “send.” This, combined with people’s tendency to seek out information that confirms their beliefs, tends to magnify the effects of misinformation. Nyhan says that although a good dose of skepticism doesn’t hurt while reading news stories, the onus to prevent misinformation should be on political pundits and journalists rather than readers. “If we all had to research every factual claim we were exposed to, we’d do nothing else,” Nyhan says. “We have to address the supply side of misinformation, not just the demand side.”

Correcting misinformation, however, isn’t as simple as presenting people with true facts. When someone reads views from the other side, they will create counterarguments that support their initial viewpoint, bolstering their belief of the misinformation. Retracting information does not appear to be very effective either. Lewandowsky and colleagues published two papers in 2011 that showed a retraction, at best, halved the number of individuals who believed misinformation.

Fourteenth in a series of endless, tireless, exhaustive, hairsplitting, obsessive, resounding, never-before-attempted and conclusive posts on the fact-checking industry.

In which the Washington Post’s Eric Wemple investigates things so others don’t have to. For example:

In today’s installment, we learn that the AP had a loose quota on checking Michelle Bachman statements during the GOP primary.

Writes Wemple:

After the session, [AP fact-checking editor Jim] Drinkard said that there wasn’t an actual numerical quota on Bachmann at the AP. It’s just that if the AP had gone back and vetted all her claims that looked dicey, the result would “overload” the debate story. “Often she was just more prone to statements that just didn’t add up,” said Drinkard.

[D]emands for after-the-fact quote approval by sources and their press aides have gone too far.”

The practice risks giving readers a mistaken impression that we are ceding too much control over a story to our sources,” it says. “In its most extreme form, it invites meddling by press aides and others that goes far beyond the traditional negotiations between reporter and source over the terms of an interview.

Memorandum from New York TImes Executive Editor Jill Abramson to staff.

According to Margaret Sullivan, the Times Public Editor, a new policy is now in place that prohibits “after the fact” quote approval. 

The issue has gained attention since a July story by Jeremy Peters outlined how reporters often submit quotes to political campaign aides for approval before running a story.

New York Times, In New Policy, The Times Forbids After-the-Fact ‘Quote Approval’.

Diary of a Mad Fact-Checker

sasquatchmedia:

I work on and off as a fact-checker at the most accurate magazine in America. I think so, at least. The checker assigned to this piece may come up with a list of competitors for that title—and in that case I’ll say that, having either been fact-checked by or been a fact-checker at most of them, she can count this fact as my own original reporting. My editor will probably agree and, if she pushes it, tell her that anyway “most accurate” is a qualitative evaluation, like “best defensive shortstop,” or “hottest freshman.” He won’t say, though it’ll be implicit, that the whole idea of The Oxford American assigning an essay about fact-checking works better if the guy they got to write it works as part of the best research department in the country—which makes me seem like an authority—and that it’d be a shame to lose the superlative when the magazine in question isn’t even going to be named. Superlatives, if you pay attention, are how magazines make stories seem worth reading, and not even the checkers at the most accurate magazine in America can fight off all the spurious ones.

Bringing this to my news reporting class Tuesday. 

FJP: Genius. Read On.

This Photo Isn't Real

This morning we posted a photo from the RNC that shows a “We Built This” sign underneath the national debt clock. 

Good ironic humor that. Unfortunately, as BuzzFeed points out, the photo is a fake. 

We’ve updated the original post to indicate the error. 

With 2,400 notes attached to it and counting, we’re confronted with the social media conundrum of how to reel it back in.

Craig Silverman’s one of the best thinkers around in this regard and recently wrote up some ideas at Poynter:

We will always make mistakes. The process of gathering, packaging, editing and publishing/broadcasting news is rife with opportunities for things to go wrong. Every part of the process has potential points of failure.

Preventing mistakes is of huge importance, but so too is setting the stage to correct them quickly and fully by taking advantage of the networked news environment. Doing so not only meets our obligations to the public, but can in fact build trust and help us feel better about our work as journalists. Bottom line: corrections are important.

Read through for his advice on what to do when the errors get you.