The Long Tail

You ever see a phrase pop up overnight — suddenly it’s everywhere, you have no idea what it means? For me it was last week and it was “the long tail”. I saw it in blogs, I saw it in at least one news item, and someone even referenced it on a phone call.

“The Long Tail” refers to a phenomenal article by Chris Anderson in October’s Wired Magazine. The entire article is worth a read, (and re-read, for that matter), but here’s the key assertion that Anderson fleshes out in the article:

[The Long Tail] is an example of an entirely new economic model for the media and entertainment industries, one that is just beginning to show its power. Unlimited selection is revealing truths about what consumers want and how they want to get it in service after service, from DVDs at Netflix to music videos on Yahoo! Launch to songs in the iTunes Music Store and Rhapsody. People are going deep into the catalog, down the long, long list of available titles, far past what’s available at Blockbuster Video, Tower Records, and Barnes & Noble. And the more they find, the more they like. As they wander further from the beaten path, they discover their taste is not as mainstream as they thought (or as they had been led to believe by marketing, a lack of alternatives, and a hit-driven culture).

The Long Tail is counterintuitive, but fascinating for its consequences. Remember the Pareto principle (aka the 80-20 rule)? The Long Tail suggests that’s almost comically wrong when it comes to the services Anderson looked at (iTunes, Rhapsody, Netflix, Amazon.com):

With no shelf space to pay for and, in the case of purely digital services like iTunes, no manufacturing costs and hardly any distribution fees, a miss sold is just another sale, with the same margins as a hit. A hit and a miss are on equal economic footing, both just entries in a database called up on demand, both equally worthy of being carried. Suddenly, popularity no longer has a monopoly on profitability.

The second reason for the wrong answer is that the industry has a poor sense of what people want. Indeed, we have a poor sense of what we want. We assume, for instance, that there is little demand for the stuff that isn’t carried by Wal-Mart and other major retailers; if people wanted it, surely it would be sold. The rest, the bottom 80 percent, must be subcommercial at best. (emphasis mine)

The implication of all of this? Look at bookstores:

The average Barnes & Noble carries 130,000 titles. Yet more than half of Amazon’s book sales come from outside its top 130,000 titles. Consider the implication: If the Amazon statistics are any guide, the market for books that are not even sold in the average bookstore is larger than the market for those that are.

There’s more:

The average Blockbuster carries fewer than 3,000 DVDs. Yet a fifth of Netflix rentals are outside its top 3,000 titles. Rhapsody streams more songs each month beyond its top 10,000 than it does its top 10,000. In each case, the market that lies outside the reach of the physical retailer is big and getting bigger.

Now, there are implications for this way of thinking well outside of the media/entertainment empires Anderson looked at. If I understand Anderson, the premise of The Long Tail is that traditional market analysis is built around the premise that it is hard to get products to market, therefore you invest more resources in getting fewer items to market. (The 80-20 rule; 20% of the market’s products accounts for 80% of the market’s revenues.)

The reason companies like Amazon.com, Netflix, etc. can excel is that IT has revolutionized their ability to streamline the delivery of products to market. (For more on this, see my notes from William Janeway’s presentation at last year’s Red Herring Spring conference.)

I’ve had fun over the past week (since reading the article in Wired) thinking through how this line of analysis can be applied in different scenarios. As it applies to my job, I think the traditional approach to technology (top-down, centralized) yields a traditional 80-20 breakdown: 20% of the users (power users) get 80% (or more) of the benefit.

With Socialtext, we reduce the barriers to contributing and finding information, much as Amazon.com reduces the barriers to finding and buying products. By decentralizing the system, we eliminate the 80-20 barrier, ensuring more people can take advantage of the information within.

Anderson has a blog focused on the concept, and is working towards making this all into a book. I particularly like this observation made a few days ago:

In all these instances, data and metadata can structure a flat hierarchy and bring order out of chaos. The variety boom may have created the Long Tail, but it takes information about that variety to entice people down it.

More required reading on that concept (of flat hierarchies and order out of chaos): Meta Filter thread on hierarchies, del.ici.ous, flickr, and Wikipedia on folksonomy. More to come…