One of the fun parts of working on Newsbot has been seeing what is possible when you combine news and computer technology. Last night, we shipped Newsbot's version of "The Daily Me" the latest in a long line of efforts to create a more personalized news experience. One of the things that has made this fun, is that it brings together two even more specific interests: personalization and political theory.
This probably sounds astounding - what does Newsbot have to do with political theory? That is a bit how I felt when I first saw Cass Sunstein's book Republic.com in 2001. Sunstein, a well-respected legal professor and theorist had written a book dedicated to the disection of the dangers posed by personalized news and Nicholas Negroponte's idea of "The Daily Me".
When I came to work at Amazon.com in 1996, I was fresh out of the University of Cambridge where I'd done graduate work in the history of political thought. Not the most direct connection to a new internet career, though I found training in conceptual frameworks good preparation for working in the nascent field of ecommerce. In 1998, as we started the personalization group at Amazon, my theoretical background helped me work with a diverse group of engineers, statisticians, designers and PMs. I can't claim I got to use more than an occassional Aristotelian principle or Machiavelli quotation in knocking out our work, but with today's project, Newsbot's version of "The Daily Me", I have an interlocutor in Sunstein questioning what I'm doing in a direct and relevant way.
In Republic.com, Sunstein argues that we need a more republican understanding of character formation informing our journalism, rather than the libertarian "customer is always right" approach to filtering the news around a reader's interest. In a Time magazine op-ed piece titled: Boycott the Daily Me Sunstein summarizes his argument: "For democracy to work, people must be exposed to ideas they would not have chosen in advance. Democracy depends on unanticipated encounters. It is also important for diverse citizens to have common experiences, which provide a kind of social glue and help them to see they are engaged in a common endeavor. A world where people only read news they preselect creates a risk of social fragmentation."
His view of "The Daily Me" is a sort of Rawlsian original position, where all choices about the news are made prior to finding out what's actually happening in the world. I was amazed to see a political theorist writing about these issues, but disheartened so little thought was given to what a personalized news system requires to be effective. Had Sunstein investigated more deeply, he'd have found his instincts for what a democratic order requires are similar to what makes a personalized news site succeed as well.
Alarmist warnings from Sunstein in Republic.com such as "There can be no assurance of freedom in a system committed to the 'Daily Me.'" or "For countless people, the Internet is producing a substantial decrease in unanticipated, unchosen encounters" drew the deserved attention of critics. James Fallows in The New York Review of Books wrote: "Sunstein's warnings last spring about the ominously perfect info-filtering technology did not, to put it mildly, have the easy authority shown in his discussions of the First Amendment. They were more like suburban fretting about the bad things that must be happening on the other side of town. . . Sunstein can't have spent much time using any of these sites if he thinks their filtering is effective enough to pose a threat. To see for yourself, go to one of the main news sites that offers a personalized compendium of information . . . and see how "me"-like you can make it. You can set it to display your city's weather, and the stock quotes you care about . . . But the rest of the information you see has a high chance of being "unexpected.""
That was three years ago. Several improvements have been made from the primitive state of personalized news - but when done well - I think it leads us even further away from Sunstein's dystopic future. Readers want to know not only what thay already know they are interested in, but also what is happening now, what stories are other people interested in, whether they be publishers or readers. One of the advances with Newsbot that is lacking in previous approaches to personalization is news aggregation. Sunstein fears an overly narrowed world where users filter the news to the extreme. But before you can recommend stories a reader' is uniquely interested in, you have to aggregate news from thousands of sources. When you aggregate news, not only do you have the broadest possible selection from thousands of publishers, you get a clearer sense of what stories are being covered around the world, what topics are readers searching for, and what other articles (published by different sources) might a reader also want. "Clustering the news" is a required step of making internet scale selection digestable and useful, but it also allows readers to compare and contrast coverage. On top of that, usage of a site like newsbot allows us to say what stories are popular, what stories did other users read that might be "unexpectedly" interesting to you.
This system is very different than the scenario Sunstein feared. Rather than narrow the news world, we've grown it to include all publishers across markets and even languages. Rather than pre-select stories, we recommend dynamically based on what you search out, discover or browse. Rather than filtering toward a private prison view of the world, you not only find and discover what you want, you see the foot paths of interested readers who came before. Rather than indiscriminately personalizing stories throughout the site (a mistake we made in our first release, now fixed) readers want to know what is unique for them but also what is popular, why was something recommended and what can they do to undo it. On Newsbot, in tribute to the original effort at MIT, we call these sections "The Daily Me" and "The Daily We". The ideal news reading experience is not one that limits what you see, so much as unlocks the news reading going on all over the internet and makes it discoverable and uniquely relevant.
These innovations aren't out of some altruistic effort to save democracy. News recommendations will be of poor quality if the sources of news are limited. So you need universal selection. Reading 5,000 news papers would be overwhelming and repetitive, so you need clustering. Users don't want to preselect keywords and topics that they are interested in, they want a system that recognizes, remembers and learns "how they read the news". Users don't want a news site that recommends willy nilly - it has to balance breaking news, important stories, and specific interests if it is to be useful. And users don't always already know what they want, so they want editors, other readers, and their demonstrated interests to drive the news they are shown.
But Fallows also takes Sunstein to task on a larger point:
"The filtering available on Internet sites is primitive compared to the filters, cushions, and blinders that surround us the rest of the time. The patterns Sunstein warns about—a lack of shared experience and the balkanization of Americans according to class, region, religion, and ethnicity—are real and worrisome enough. But the Internet is a trivial source of the problem— let's say one thousandth as important as the educational system, from school districts with their unequal funding to the faulty system of college admissions. Or residential patterns. Or who marries whom. Or tax policy. Or the existing broadcast media, which let you drive coast to coast listening to nothing but right-wing talk radio or NPR. Or cable TV, with one channel showing only bass fishermen and another showing only success-motivation seminars. Or patterns of commuting, which have evolved from buses to cars, and remove people from accidental contact with others. You could un-invent the Internet and still have every problem Sunstein fears."
Fallows interestingly turns to David Weinberger to further refute the idea that the Internet has a narrowing effect on people's minds:
there is more and more to distract us—more sites to visit, more arguments to jump into, more dirty pictures to download, more pure wastes of time. The fact that the Web is distracting is not an accident. It is the Web's hyperlinked nature to pull our attention here and there. But it is not clear that this represents a weakening of our culture's intellectual powers, a lack of focus.... Maybe set free in a field of abundance, our hunger moves us from three meals a day to day-long grazing.... Perhaps the Web isn't shortening our attention span. Perhaps the world is just getting more interesting. - Small Pieces Loosely Joined: A Unified Theory of the Web
All hail the promise and perils of the Daily Me! 20 years new and never far from the Daily We.
Recent Comments