The (open) future of news

First, we must start by defining news. At least I’ll explain what I consider it to be.

News is not something solely created by a high priesthood of journalists in non-smoky back rooms of some media organization. They do create news there. Lots of it.

Nor is it merely news organizations coupled with the fast and loose, breaking posts from a talented squadron of bloggers. They create news too. But wait, there is more.

The fact is, there is no strong line between who can create news and what is news.

The Rodney King video was an act of journalism. My wedding video was probably not (it could be, you don’t know my family).

My recent tweet (see Twitter if you don’t know what that is) saying “Good Night, all” is not journalism, but my next one about Google is definitely breaking news.

It’s not the creator or the medium that makes something journalism, it’s the substance. No doubt that professional journalists create more news than your average MySpace user. It’s their job. No doubt, they are more diligent about trying to be objective and telling both sides. But they make mistakes. They are even, dare I say, biased, even if they won’t admit it.

Okay, enough about that. The point is that we get our news from a variety of growing sources, fueled by syndication methods like RSS and Social Networks like Twitter and Facebook. Cheap devices and bandwidth have made it possible for countless people to contribute to the fabric of news flow than ever before. Nuff said.

Back when I studied Mass Communications at the University of Connecticut, we used to lament about the shrinking number of news sources. We used to say things like, “the number of media organizations has gone from over one hundred during WWII to less than 25 now,” or “you can fit the number of executive board members of all the media organizations in the country into one room.”

Well, those days are over. And with all the sources I mention above, it should just be one big party. . . right? Well, look out. There is a new editor in town; it’s the algorithm.

You see, we once criticized that the back room editorial policies weren’t transparent enough, and we heralded the transparency of the blogging world as the answer to that.

The only problem was that there are now too many sources to manage. We simply can’t handle it all. Yes, we rely on our digital social networks to do some moderating for us, but these grow as well, and contribute even greater amounts of noise. It’s not a self healing problem.

As well, we sometimes like to get a broader picture of what’s happening than our social network might afford. After all, that’s the beauty of greater sources. Greater perspective.

This has led to the rise of aggregators like Gabe Rivera’s Memeorandum and Techmeme. As well, it is a predominant way in which a whole bunch of mainstream web users consume their news with services like Google News and MyYahoo. Yes, you can personalize many of the systems, but an algorithm is often deciding what stories bubble up.

Perhaps you can see my concern. If we begin to rely on algorithms to filter our news consumption, if only partially, have we not discarded one opaque system for another.

It’s not a simple issue, but let’s try to use a simple example.

You may have heard that Google is going to archive the print editions of many newspapers, going back hundreds of years.

Now imagine that at a crucial point in history, many people want to know what happened in World War II . They search for it and Google displays the results. The black box algorithm that decides what articles rise to the top of that search will influence what people’s perception of that historical event was.

How about a more current example. People voting in the Ohio primaries on Tuesday may go to Google News and search for “Obama” and/or “Hillary.” Good for them. We need to know the issues.

The top results for articles could have a profound effect on the vote, and therefore the world. Did the results favor views that might lead voters on the edge to settle on one side or the other?

You might find the example extreme, but the fact is that an opaque system decided what people read, not a transparent one. It’s no longer the editor that sways public opinion, it’s the programmer.

Even Digg, a site that feigns democracy, has an algorithm that apparantly only a small monority of it’s users have have figured out and exploited. Or are they being exploited by the algorithm?

What we need is transparent algorithms that aggregate news, so that we are aware of what decided the results, even if we not in complete control.

In an ideal world we could mix and match plugins that controlled how the news is aggregated and sorted. Some will want to rely on a broader network, some on a less broad. Some will want to give greater weight to certain individuals and organizations.

The possibilities for forming our news consumption decisions are endless, and may never be perfected. We should strive to keep those decisions under our control, not the editor’s, not the programmer’s, and not the algorithm’s.

SIDENOTE: Prior to publishing this, I broke news about Google archiving old newspaper editions, on Twitter, just to prove a point. Most of the first readers of this post followed a link from there to here, so necessarily had the background knowledge I mention above.

P.S. Jeff Jarvis and some contributors are talking about the future of news organizations, a different but equally meaningful topic I’ll try to explore in another post.

New Paradigm Emerging

Steve Gillmor’s latest show Newsgang Live has two conversations going on. One is the presidential race, in particular the rise of Barack Obama, and the other revolves around what Bruce Lerner observes is the growing pervasiveness of “propellorhead technology” as it stretches into the mainstream.

This is not just “Web 2.0″ gaining wider acceptance, as one might be tempted to characterize it.

The platforms have emerged already, Steve contends, though they might still be in their infancy and subject to maturation. These platforms are RSS, Twitter, Facebook , iPhone and the like. They are all disruptive in their own way because they become part of our DNA.

Like the web before them, it becomes difficult to imagine how we would cope without the tools, despite the ironical fashion in which they simultaneously cause our overload.

But the emerging concept of a unified web identity(e.g. OpenID and inames) coupled with the Web OS (e.g. Silverlight, Ajax, Google Office) and the looming VRM revolution, leads one to believe that the coming years will be more innovative than even Cluetrain imagined.

If it sounds a bit nebulous, well, that makes sense. I think it’s no accident that the concepts of non-particular Social Change and non-particular Social Technology have intersected so beautifully in the Newsgang Live episodes. In fact, one has to wonder if Gillmor knew exactly what he was doing.

Obama is just a symbol of the emerging desires of a nation. Twitter is the web corollary.

It fulfilled the blogosphere’s desire for an central identity system. We just couldn’t wait for protocols and open-standards to catch up with what we needed now.

What’s missing is the decentralization that we love and still sorely want. That’s why the periodic downtime of Twitter causes the Blogosphere to erupt with negative passion. The same thing happens in the enterprise when the Exchange server goes down. It’s just unacceptable.

Dave Winer wants a backup. The XMPP advocates claim that they’ve had the system in place for years, but Jabber has never produced a (really popular) way to manage the social system beyond IM buddies, which tend to be more personal contacts than Bloggers or Twitter friends.

We need to have a “squelch,” or else we’d all still be using Usenet or email lists, which both still maintain active users. Before that, there was Ham Radio.

Perhaps more importantly, we need unity across all these wonderful services.

It doesn’t take a lot of imagination to envision unified communication. Achieving it has been more elusive.

One of the more interesting projects I’ve come across is Social.IM. Yanda Erlich gave a nice description of where the service wants to go on a recent Newgang Live. Of particular importance is the desire for Social.IM to work across other social networks, not only Facebook, which is now the case.

In that way, it’s similar to the aspirations for the humble designs of Gangbuster. Abstract out the services we love into one unified client, not to replace them, to but to increase their pervasiveness.

For a service to do this, it needs to be open source, to eliminate silo lock-in, as well as retain a carbon-copy of our gestures and “prosumption,” a term Doc Searl’s came up with to identify the fact that we produce and consume. It is the very nature of the web.

The need for open source is two-fold. One, that we might never lose our data, which becomes our real identity. Two, is to eliminate our reliance on any one service, in the case that it is down, or changes it’s nature in ways we did not anticipate(e.g. Google reader).

I may have recently coined a term (at least Google implies that), when I implied to Steve that the “cloudosphere” is the one service we really want. And every service needs a client.

Hello cloud!

Welcome to Gangbuster. The aim here is to create an open source multi-platform client that speaks to numerous cloud-based services all from one easy to use interface.

Tired of bouncing between your feed-reader, Twitter, email and your IM client. Gangbuster may be for you. Stay tuned.