Posts tagged algorithms
Can Robots Tell the Truth?
Hi, I am a student in journalism and am preparing an article about robots (like the Washington Post’s Truth Teller) validating facts instead of journalists. I am curious to know the Future Journalism Project’s point of view of about this. What are the consequences for journalists, journalism and for democracy? — Melanié Robert
Hi Melanié,
Many thanks for this fascinating question and my apologies for the delay in getting back to you. Here’s what happened:
I started thinking about this, and then I started writing about it. And then I started thinking that what I really needed to do was some reporting. You know, journalism.
I didn’t know much about the Washington Post’s Truth Teller project. For others that don’t, it’s an attempt to create an algorithm that can fact check political speeches in real time.
Since I didn’t know much about it about I got in touch and interviewed the two project leads: Steven Ginsberg, the Post’s National Political Editor, and Cory Haik, the Post’s Executive Producer for Digital News.
They gave me background on Truth Teller and how it came about, and then where they hope it leads.
But that doesn’t really get to the sociocultural and philosophical questions you pose. So I called upon someone else. His name is Damon Horowitz.
Damon’s spent his career in both artificial intelligence and philosophy. He’s currently Google’s In-House Philosopher (seriously, it’s on his business card) and Director of Engineering. He also teaches philosophy at Columbia University.
So, after talking to these people, and thinking about it some more, I wrote a fair bit.
You can find your answer at theFJP.org and I hope it answers some of what you’re looking for. — Michael
Have a question? Ask Away.
Image: Marvin the Paranoid Android, Hitchhiker’s Guide to the Galaxy.
Miami Herald
The Problem With Contextual Advertising
Image: Screenshot, Miami Herald Home Page, December 16, 2012.
Even robots have biases.
Any decision process, whether human or algorithm, about what to include, exclude, or emphasize — processes of which Google News has many — has the potential to introduce bias. What’s interesting in terms of algorithms though is that the decision criteria available to the algorithm may appear innocuous while at the same time resulting in output that is perceived as biased.
Nick Diakopoulos, Nieman Lab. Understanding bias in computational news media.
Whether the cause is ideological or systematic, the outcome is, for now, the same: algorithms appear to be as biased as editors in their sorting of news. Click the link to read on, and see more about its author here.
We need, in short, to pay attention to the materiality of algorithmic processes. By that, I do not simply mean the materiality of the algorithmic processing (the circuits, server farms, internet cables, super-computers, and so on) but to the materiality of the procedural inputs. To the stuff that the algorithm mashes up, rearranges, and spits out.
CW Anderson, Culture Daily. The Materiality of Algorithms.
In what reads like a starting point for more posts on the subject, CUNY Prof Chris Anderson discusses what documents journalists may want to design algorithms for, and just how hard that task will be.
Algorithms doing magic inside massive data sets and search engines, while not mathematically simple, are generally easy to conceptualize — algorithms and their data are sitting in the computer, the algorithm sifts through the excel sheet in the background and bam! you have something.
But if you’re working with poorly organized documents, it’s difficult to simply plug them in.
Chris writes that the work required to include any document in a set will shape the algorithm that makes sense of the whole bunch. This will be a problem for journalists who want to examine any documents made without much forethought, which is to say: government documents, phone records from different companies and countries, eye witness reports, police sketches, mugshots, bank statements, tax forms, and hundreds of other things worth investigating.
Chris quotes Jonathan Stray’s trouble preparing 4500 docs on Iraqi security contractors:
The recovered text [from these documents] is a mess, because these documents are just about the worse possible case for OCR [optical character recognition]: many of these documents are forms with a complex layout, and the pages have been photocopied multiple times, redacted, scribbled on, stamped and smudged. But large blocks of text come through pretty well, and this command extracts what text there is into one file per page.
To read the rest of Stray’s account, see his Overview Project.
And to see more with Chris Anderson, see our recent video interviews with him.
Google News at 10: How the Algorithm Won Over the News Industry
There is, on the one hand, an incredibly simple explanation for the shift in news organizations’ attitude toward Google: clicks. Google News was founded 10 years ago — September 22, 2002 — and has since functioned not merely as an aggregator of news, but also as a source of traffic to news sites. Google News, its executives tell me, now “algorithmically harvests” articles from more than 50,000 news sources across 72 editions and 30 languages. And Google News-powered results, Google says, are viewed by about 1 billion unique users a week. (Yep, that’s billion with a b.) Which translates, for news outlets overall, to more than 4 billion clicks each month: 1 billion from Google News itself and an additional 3 billion from web search.
As a Google representative put it, “That’s about 100,000 business opportunities we provide publishers every minute.”
Google News automation fail of the day week month ever.
For the unfortunate story, see here.
That the presidency ages people quickly is well documented. In a recent CNN article, Dr. Michael Roizen of the Cleveland Clinic says presidents age twice as fast while in office.
What’s new, as Google’s automated news algorithm illustrates here, is that the presidency also appears to be able to turn a black man into a white man.
Learning something new every day.
For content farms, the Panda doesn’t play nice.
You can’t mess with Google forever. In February, the corporation concocted what it concocts best: an algorithm. The algorithm, called Panda, affects some 12 percent of searches, and it has — slowly and imperfectly — been improving things. Just a short time ago, the Web seemed ungovernable; bad content was driving out good. But Google asserted itself, and credit is due: Panda represents good cyber-governance. It has allowed Google to send untrustworthy, repetitive and unsatisfying content to the back of the class. No more A’s for cheaters.
The $23,698,655.93 Book
A post-doctoral student at UC Berkeley wanted to buy a copy of Peter Lawrence’s The Making of a Fly so did like a lot of people do and went to Amazon.
There he found it on sale for $35.54 to $1,730,045.91 (plus shipping).
What gives?
Michael Eisen explains:
On the day we discovered the million dollar prices, the copy offered by bordeebook was 1.270589 times the price of the copy offered by profnath. And now the bordeebook copy was 1.270589 times profnath again. So clearly at least one of the sellers was setting their price algorithmically in response to changes in the other’s price. I continued to watch carefully and the full pattern emerged.
Once a day profnath set their price to be 0.9983 times bordeebook’s price. The prices would remain close for several hours, until bordeebook “noticed” profnath’s change and elevated their price to 1.270589 times profnath’s higher price. The pattern continued perfectly for the next week…
…As I amusedly watched the price rise every day, I learned that Amazon retailers are increasingly using algorithmic pricing (something Amazon itself does on a large scale), with a number of companies offering pricing algorithms/services to retailers. Both profnath and bordeebook were clearly using automatic pricing – employing algorithms that didn’t have a built-in sanity check on the prices they produced. But the two retailers were clearly employing different strategies.
Closing price before the “mistake” was found: $23,698,655.93.
Bloomberg: Personalizing the News for 20 Million People - GigaOm
The approach that most traditional news services take, Krim said — in which editors select and present the news that they think matters most to a generic reader — “doesn’t really scale very well.” But by using analytical tools on the data about those web visitors and their reading patterns and usage, Krim said that Bloomberg can “present 20 million different views of that information.” The company is also trying to take into account the differences in how users want to receive their news during the day, including whether they want content as text they can read on their laptop or mobile, or video they can watch, and so on.
The company now collects over 100 data points for every page a reader loads, based on what they interact with, what time of day it is, etc. — more than a terabyte of data every day in aggregate, Krim said — and the team has 15 different algorithms running in parallel to make recommendations for what that reader might want to see next.
“We started studying the behavior of decision makers who come to our site,” Krim said, “and we noticed that there are a number of different usage curves of news… TV is kind of a U-shape during the day, web usage is like an arc, mobile is like an oscillating curve, magazines ramp up during the day, and newspapers obviously ramp down during the day.” What Bloomberg Digital is trying to do, he said, is to understand how people move from one to the other, and then present information to them in the way that they want.
LinkedIn Gets Newsy
LinkedIn launched a new product yesterday called LinkedIn Today.
Much like Paper.li and The Tweeted Times, the service extracts links submitted to it (in this case via Twitter and LinkedIn) and lays out articles in an eye-pleasing manner.
The benefit for users is great: you can drill down deeply into the news of the day of industries that interest you.
For LinkedIn? They do have an IPO coming up. Looks like they’re adding layers of new tools to make their offering more attractive. If they nail it, the hope is that the site becomes a daily stop for business users.
Via Fast Company:
At the moment, LinkedIn Today only has 22 industries to choose from, but the product will expand to include more of the service’s 115 listed industries. One example: “The agriculture industry is not sharing enough content to build a compelling product, but we hope over time it will,” explains Liz Walker, Product Manager of LinkedIn Today. Eventually, users will also have the option of searching by different cuts of data—i.e. what CEOs in the Bay Area are reading, or what Product Managers at LinkedIn are looking at.
