Category: The Long Tail

This is another part of my critical reader’s companion to The Long Tail, and it discusses Chapter 8 – Long Tail Economics. Part 0 is here. You can find a complete list of the Long Tail pieces here.

By this time in the book we’ve seen variants on the long tail graph a lot of times.
 

The chapter is a collection of observations about this shape. I don’t have a whole lot to say about this chapter, so this is a short post.

The chapter starts by talking about the Pareto 80/20 rule (that 20% of the people in populations Pareto studied owned 80% of the wealth) and Zipf’s Law (that the frequencies of use of words in the language follow a similar kind of fall-off). These and others are examples of power law distributions that crop up in a bunch of different places. Power laws are called long-tailed curves because "the amplitude … approaches but never reaches zero as the curve stretches out to infinity" [126]. This is true of many kinds of distributions, of course, including exponential and log-normal distributions. But even within the family of power-law curves, of the form

How Distribution Bottlenecks Distort Markets [127-130]. If you plot a power law on a log-log scale you get a straight line with slope -k. Anderson looks at some markets (movies on movie screens) and notices that they fall off below the line at a small value. He interprets this as a failure of markets to meet demand because of the cost structure of movie theatres. If there’s not enough demand to fill a theatre, films won’t get distribution. There’s some truth to this – there are costs to producing and distributing movies that demand a return on investment, and movies that don’t make the cut will not be distributed at all. In non-theatrical channels, where cost is less, more movies are available and watched. As the title of the section suggests, Anderson sees only the untruncated power law as somehow "natural" [127] and a reflection of consumer demand, and the truncated ones a measure of artificial scarcity caused by distribution bottlenecks, to be removed in the world of abundance. Yet even in the online world there are truncated distributions, they just occur at different places in the distribution chain. The small number of search engines or online auction houses are two examples. The idea that consumer demand is "naturally" a power law shape is unsubstantiated, at least in the pages of the book.

The strangeness of the comparison between theatres and rentals is
exemplified by this sentence: "demand keeps on going into niches that
were never even considered before — instructional videos, karaoke,
Turkish TV, you name it" [130]. I don’t know what he means by "never
even considered before" but I’m sure the reason instructional videos
were not made available on movie theatre screens was not so much "an
artifact of the traditional costs of offering them" but the fact that
the number of people who want to watch instructional videos in movie
theatres is probably vanishingly small no matter how large a geography
you spread them over. It’s like comparing the market for banquets to the market for snacks – they’re both food, but the demand pattern is different for one than for the other, and the limited demand for banquets is not "unnatural".

This section is one of those that come occasionally in the book,
containing a scattered set of only loosely related data points, with
sentences like "In books, Barnes and Noble found that the bottom 1.2
million titles represent just 1.7 percent of its in-store sales, but a
full 10 percent of its online (bn.com) sales. PRX, which licenses a
huge library of public radio programming online, reports that the
bottom 80 percent of its content now accounts for half of its
sales."[130] The "statistics" come from "direct personal correspondence
with their executives". Is this trustworthy data or is it picked to
illustrate a pre-conceived idea? What does PRX do and what is the
nature of its business? Who knows – that sentence is the sole reference
to them in the book. How does the Barnes and Noble statement fit with
Lee Gomes’ claim
in the Wall Street Journal that "The head of a major New York
publishing operation says that the
distribution of his titles is essentially the same in both online and
"bricks and mortar" channels." These isolated, context-free sentences
from executives seem to be the best we can do — actual company data
being tightly-held secrets.

I have little to say about the remainder of the chapter, The 80/20 Rule [130-135] is a meandering take on whether or not the Rule still applies. The other sections are Does a Longer Tail Mean a Shorter Head? [135-137], Does The Long Tail Increase Demand or Just Shift It? [137-138], Should Prices Rise or Fall Down the Long Tail? [133-139], "Microstructure" in the Long Tail? [139-142], The Long Tail of Time [142-143], and The Tragically Neglected Economics of Abundance [143-146]. As the titles suggest, this is a miscellany of observations about the shape of the graph. There’s not a whole lot to disagree with, but there’s not a whole lot of substance.

I could stop here… or I could just throw in a few things that irritate me, just to be grouchy. So don’t take this too seriously, but:

In the section on "Microstructure" [140] Anderson shows a graph of the
average sales for various of the many subgenres that Rhapsody divides
music into and claims that this selection shows "a Long Tail". Yet it
doesn’t. The graph does fall off left to right, as it cannot help but
do, but it is as close to a linear fall-off as a power law.

"One of the features of powerlaws is that they are ‘fractal’, which is to say that no matter how far you zoom in they still look like powerlaws" [138] Wrong. Straight lines still look like straight lines as you zoom in too, but that’s not enough to make a fractal. A fractal reveals new structure as you zoom in, and that structure is similar to the structure seen from further away.

"Einstein described time as the fourth dimension of space, you can think of it equally as the fourth dimension of the Long Tail" [142]. Wrong. It’s not the "fourth dimension of space", and the way he plots it is actually a third dimension of the Long Tail graph. Plus, it’s name dropping. This section, by the way, is devoted to the realization that "If you think about it, today’s hit is tomorrow’s niche", which is hardly shocking, but "Both hits and niches see their sales slow over time; hits may start higher, but they all end up down the Tail eventually. The research to quantify this conclusion is continuing" [142] Count me underwhelmed.

On abundance: "Abundance, like growth itself, is a force that is changing our world in ways that we experience every day, whether we have an equation to describe it or not" [146]. This is stuff that should have been thrown out during editing.

That’s all. More tomorrow, I hope.

This is another part of my critical reader’s companion to The Long Tail, and it discusses Chapter 7 – The New Tastemakers. Part 0 is here. You can find a complete list of the Long Tail pieces here.

In a comment on Chapter 6, John points us to an article which mentions Ranganathan’s Five Laws of Library Science.These laws are:

1.      Books are for use.   
2.      Every person his or her book.   
3.      Every book its reader.   
4.      Save the time of the reader.   
5.      The library is a growing organism.   

Laws 2 and 3 remind us that the whole publishing process is about guidance. Guiding readers to books they like, and guiding books to readers who like them. The various stages of writing, submitting to publishers, editing, publishing, publicising, reviewing, discovering, recommending, locating, browsing, requesting, buying/borrowing, delivering are one path through the maze that is needed to guide books to their readers and readers to their books. Guidance is the subject of Chapter 7.

I want to leap ahead some pages into the chapter this time. Because on at last, after a hundred and nine pages, we get some real evidence of a specific industry that is showing a big Long Tail effect – and it’s our old friend Netflix.

Filters rule [108-110] is a reflection of Anderson’s belief in the power of filters to "level the playing field, offering free marketing for films that can’t otherwise afford it,  and this spreading demand more evenly between hits and niches" [110]. The main effect, he says, is "to help people move from the world they know (‘hits’) to the world they don’t (‘niches’)" [109].

The evidence that he gives in this section in favour of the claim comes Reed Hastings, CEO of Netflix (and provider of a blurb for the book):

Historically BlockBuster has reported that about 90% of the movies they rent are new theatrical releases. Online they’re more niche: about 70% of what they rent from their website is new releases and about 30% is back catalog. That’s not true for Netflix. About 30% of what we rent is new releases and about 70% is back catalog and it’s not because we have a different subscriber. It’s because we create demand for content and we help you find great movies that you’ll really like. And we do it algorithmically, with recommendations and ratings.

In fact, since the book was published, Netflix have set up the Netflix Prize for people to suggest better algorithms. The first submission to improve the accuracy of the algorithm by 10% will win $1 million. So Netflix are taking this seriously. Finally, there appears to be some substance behind the shifting generalities of the book.

Or is there? Lee Gomes of the Wall Street Journal begs to differ.

Netflix defines "back catalog" expansively. A spokesman says it’s anything outside of the 50 or so DVDs getting heavy studio promotion at any given time. So even recent megahits like "Spiderman II" are in the back catalog.

What’s more, since Netflix rents 60,000 titles, it follows that those 50 titles — eight-tenths of 1% of inventory — generate 30% of all rentals.

This was the second of two Gomes articles on the Long Tail and part of a debate that focused on the extent of the effect that Anderson is describing. The first article was a review of the book, and in that review he referred to work by Harvard Business School’s Anita Elberse that only "shows a ‘slight shift’ toward the tail. But she also noted ‘a rapidly increasing number of titles that never, or very rarely, sell,’ which suggests ‘it is difficult for content providers to profit from the ‘tail.’ ". Again, a "slight shift" is not an "epochal shift". Anderson responded to that review, arguing that

As Professor Elberse told Gomes, she was only describing Nielsen VideoScan data, which is almost entirely taken from bricks-and-mortar sources. The Netflix data, which was the basis of the Long Tail analysis that she and I worked on together, tells a very different story (Elberse’s terms of data access don’t allow her to share that data; my terms allowed me to share what I published in the book). We both urged Gomes to make clear that the "slight shift" measured didn’t refer to the Netflix data that was at the core of the book’s conclusions. But he chose to make the point he wanted to make.

In comments to that post, Elberse herself stepped in to say that

You [Anderson] say "Nielsen VideoScan data (…) is almost entirely taken from bricks-and-mortar sources." I don’t think this is entirely correct. The VideoScan data reflect both offline and online sales, and actually break them down by channel. The breakdown is not as detailed as one might wish in an ideal world, but they do allow one to track whether, say, the share of offline sales go down over time. Therefore, I do think the fact that my colleague and I only observe a "slight" shift is meaningful.

In a letter published on Nicholas Carr’s weblog, Gomes said.

I’d like to correct an extremely serious misrepresentation Chris made at the end of his blog posting, to the effect that Anita Elberse of Harvard "urged" me not to characterize her work the way I did. This is manifestly false. Chris is either misremembering or deliberately conflating two separate issues. Prof. Elberse did indeed in an email remind me that the data she had for Netflix was under NDA, and I could thus not report it. But the comment had nothing to do with what Chris says it does. Let Prof. Elberse herself describe whether I got it right; below is the full text of an email she sent me after the story ran:

"I just read your article, and just wanted to thank you for being so careful in quoting me. I wish all journalists stayed this close to what was actually said! 🙂

"You did beat me ‘to the market’ with your article, but I hope our academic article (which should be ready in a few weeks) will further clarify the long tail phenomenon (or lack thereof)."

A lot of heat there. For anyone interested in more depth, I recommend you to Anita Elberse and Felix Oberholzer-Gee’s working paper "Superstars and Underdogs: An Examination of the Long Tail Phenomenon in Video Sales" (50 page PDF). Here is a paragraph from the abstract:

To shed light on this debate, we study the distribution of revenues across products in the
context of the U.S. home video industry for the 2000 to 2005 period. We find superstar and long-tail
effects in home video sales, but each effect comes with a twist. There is a long-tail effect in that the
number of titles that sell only a few copies every week increases almost twofold during our study
period. But at the same time, the number of non-selling titles rises rapidly; it is now four times as
high as in 2000. Many underdogs thus in fact appear to be losers. We also find evidence of a
superstar effect. Among the best-performing titles, an ever-smaller number of titles accounts for the
bulk of sales. The caveat here is that today’s superstars lack the punch of earlier generations: video
sales generally decrease over time across all quantiles of the sales distribution, but this effect is most
pronounced among best-selling titles. Our findings have important implications for entertainment
companies. Exploiting the tail might prove unprofitable if many titles do not sell at all. At the same
time, producing superstars is more difficult than ever. The trends we uncover thus point to
significant challenges for the entertainment industry.

In short, this most significant, most specific piece of information regarding an actual shift to long-tail behaviour, prompted and guided by Internet recommendation algorithms, turns out to be as insubstantial as other pieces of evidence. Filters don’t rule.

Well, with that over, let’s go back to the beginning. There is, we all know, a lot of stuff on the Internet, which is why the picture is so appealing. But as John pointed out in his Chapter 6 comment (and as I’ll say in more detail in Chapter 8), the tail has to be fat as well as long for the "theory" to make any sense. We also know that increasing returns and large fixed costs coupled with small marginal costs and freedom from geographical limitations gives the aggregators of the Internet (Amazon, Netflix, iTunes and the other handful of examples that Anderson returns to over and over again) the potential to be globe-straddling colossi, or oligonomies, to use Steve Hannaford’s word, who are oligopolies as far as customers go and oligopsonies as far as suppliers go. So the Internet is likely to have fewer vendors selling larger numbers of products than the brick & mortar world – One Big Virtual Tent rather than Many Small Tents. But where does this leave us when it comes to demand? It all depends on whether guidance on the Internet can help people find niche products in the Big Tent better than guidance in the physical world can help people find what they want in the Many Small Tents. The issue of guidance, then, is central.

Not only is it central, but it’s a particularly difficult problem for niche items. The problem of finding quality in "experience goods" – such as books, movies, and to some extent music – is a problem of asymmetric information. As such, it is prone to a form of market failure that goes under the name of the market for lemons. In short, the incentives are set up to encourage various forms of false reporting and gaming of the system (I have a strong incentive to give my own book a five-star rating on Amazon, for example). Knowing this, customers avoid those parts of the market where the asymmetry in information is strongest – the niches – and flee to those parts where information is most reliable – the hits. Predictability can drive out quality when information is scarce and unreliable. The market for lemons has been one of the most influential ideas in economics over the last several decades, its ramifications are ubiquitous, and it earned its inventor George Akerlof a Nobel Prize. But even though Anderson describes his book as "partly an economic research project" [11] there is no evidence this idea has troubled him, and it is not mentioned in his book.

Let’s look at how Anderson says guidance happens in the Long Tail world. Here’s the big picture, optimistic as ever:

Faith in advertising and the institutions that pay for it is waning, while faith in individuals is on the rise. Peers trust peers. Top-down messaging is losing traction, while bottom-up buzz is gaining power. Dell spends hundreds of millions each year on promoting its quality and customer service, but if you Google "dell hell" you’ll get 55,000 pages of results [click here to see why this number is wildly wrong]. Even the word "dell" returns customer complaints by the second page of results. The same inversion of power is now changing the marketing game for everything from individual products to people. The collective now controls the message….The new tastemakers are us… The ants have megaphones. [98-99]

But as Oligopoly Watch reminded us just   yesterday,   "while the myth is that the Internet represents an infinite array of shelves   (I think of   Borges’s library   of Babel) with everything democratically and randomly available, the real   world has a way of organizing things up front or way back, even when it’s all   cyberspace." Aggregators are One Big Virtual Tent, and vendors will scrabble   to get a place on the tables by the entrance. Oligopoly Watch quotes a Wall   Street Journal article on Apple’s iTunes (‘Music’s New Gatekeeper’) as saying   "Every day, the   roughly one million people who visit the iTunes Store home page are presented   with several dozen albums, TV shows and movie downloads to consider buying —   out of the four million such goods the Apple site offers. This prime promotion   is analogous to a CD being displayed at the checkout stands of all 940 Best   Buy stores or featured on the front page of Target’s ad   circular." Here is a way that Internet commerce, with its tendency to produce oligopolies, promotes uniformity rather than promoting diversity. Instead of many different store fronts, we have One Big Virtual Storefront. A high proportion of regular bookstores’ sales come from the highly-promoted items, and there is no reason to believe that aggregators’ sales will be different.

Bonnie McKee [98-103], My Chemical Romance [103-104] and BirdMonster [104-106] are three stories about different bands and their mixed experiences with online recommendation systems, social networking sites, and blogs to promote their music. While Anderson says that the stories show "how the three forces of the Long Tail are overturning the status quo in the music industry" [104]. A big statement, with little behind it. My Chemical Romance is a success, Bonnie McKee is struggling, and BirdMonster is a local band in San Fransisco. This is new? No.

The Power of Collective Intelligence [106-108] introduces us to the filter, which is "the catch-all phrase for recommendations and other tools that help you find quality in the Long Tail" [106]. Anderson is very enthusiastic about recommendation systems, saying that "the trend-watchers at Frog Design" see the rise of recommendations as "nothing less than an epochal shift" [107].

The adoption of recommendation systems on all kinds of web sites has been a boon to help promote worthwhile content and demote non-worthwhile content, and as they get more sophisticated they are a continuing innovation of great worth. But they set out to rectify a problem that is peculiar to the Internet, after all, which is the problem of anonymity and lack of trust. How do you establish trust in the online world? The existence of recommendations is a reflection of the fact that the Internet is handicapped when it comes to reliable and trustworthy communication. Recommendation systems are a great effort to overcome an obstacle that the physical world (which has its own problems) faces to a much smaller degree. When a friend recommends a book or movie to me, I have a reasonable idea of how to take that recommendation because I know my friend. When a book has a 3.5 rating on Amazon.com with three reviews, what am I to make of that? Was it friends of the author? Quite possibly.

The next section, "Filters Rule" was discussed at the top of the article.

One Size Filter Doesn’t Fit All [110-112] looks at various kinds of filter in more depth. It shows that filters are an increasingly sophisticated set of tools for aggregators to use to attract customers. But it does not show whether or not these filters are anything close to the power of our own offline networks of friends.

Not All Top Ten Lists are Created Equal [112-115] is more in a similar vein. Some online sites have developed filters that use an increasingly fine granularity and set of classifications of online lists. But can any algorithmic granularity capture our quirky tastes?

Is the Long Tail Full of Crap? [115-119] is a venture into explanation. It contains references to information theory, to zero sum games, to non-rivalrous goods, and to wide dynamic ranges. But they are name dropping, inserted to hint at a greater intellectual behind the Long Tail. It isn’t there. The point he is making is that, as Theodore Sturgeon [I always thought he was a creation of Kurt Vonnegut, but Wikipedia tells me that Vonnegut’s Kilgore Trout was based on the real-life Theodore Sturgeon] apparently said "ninety percent of everything is crud". But as long as you have unlimited shelf space, it doesn’t matter, because filters can help you find the good. There is an odd unsubstantiated claim that the material "in the tail" ranges from worse than that in the hits to better but, he goes on, "averages don’t matter. Diamonds can be found anywhere."[118].

The Tail That Wags Everything Else [119-122] is another slight section which reiterates the previous point: "As the Tail gets longer, the signal-to-noise ratio gets worse. Thus, the only way a consumer can maintain a consistently good enough signal to find what he or she wants is if the filters get increasingly powerful" [119]. It is out here in the tail, where comments on Amazon, links, and other filters are rare per item, that problems of asymmetric information are greatest, raising a further undiscussed barrier to those good quality items in the tail achieving recognition. So filters have a greater job to do, and are fighting an uphill battle. How do they do? Well I could use the Anderson method and point out that typing "ebay fraud" into Google gives [about] 4,160,000 hits – but in fact, Google estimates being what they are, the number is actually 780, so that tells us little. But see here, here, and here for articles about manipulating reputation systems – the last two, ironically, from the Anderson-edited Wired Magazine. The first article of the three, by Princeton University Computer Science Professor Ed Felten, says this:

There’s a myth floating around that such systems distill an uncannily accurate folk judgment from the votes submitted by millions of ordinary citizens. The wisdom of crowds, and all that. In fact, reputation systems are fraught with problems, and the most important systems survive because companies expend great effort to supplement the algorithms by investigating abuse and trying to compensate for it. eBay, for example, reportedly works very hard to fight abuse of its reputation system.

"The wisdom of crowds, and all that" just about summarizes Anderson’s sunny outlook on this promising but still-flawed, and perhaps unavoidably flawed, method for directing demand.

I should also point out that Anderson also briefly quotes[120] Nassim Taleb’s engaging book Fooled By Randomness, on the unpredictability of hits.

Pre-Filters and Post-Filters [122-124] distinguishes the "pre-filters" of the offline world which "filter before things get to market" and the recommendation and search technologies, or "post-filters" of the Internet. Here, then, are two models of guidance in an attempt to achieve that "every book its reader" ideal. Given the challenges faced by post-filters (the increasing "signal to noise" ratio in the tail discussed by Anderson, the potential for gaming the system, the market-for-lemons problem that particularly targets niche products in the face of that potential), it is good to see Anderson does acknowledge the difficulties: "Because post-filters tend to be amateurs, oftentimes that means less critical independence and more random malice." But in general he is remarkably sunny about the prospect for Internet recommendations to direct people reliably to even the deepest darkest corners of the One Big Virtual Tent. Filters are great, but they have a big job to do. Overall, I don’t share his optimism.

Here are links to each post in my critical reader’s companion to The Long Tail by Chris Anderson.

0- Cards on the Table
0.1 – The Cover
0.2 – The Introduction
1 – The Long Tail
2 – The Rise and Fall of the Hit
3 – A Short History of the Long Tail
4 – The Three Forces of the Long Tail
5 – The New Producers
5.1 – One Big Virtual Tent
6 – The New Markets
7.0 – A Teaser
7 – The New Tastemakers
8 – Long Tail Economics
8.1 – A Note on Networks
9.1 – The Short Head Part 1
9.2 – The Short Head is Alive and Well
9.3 – The Short Head Part 2
10 – The Paradise of Choice
10.1 – A Short Note
11.1 – Living in a Niche Culture (1)
11.2 – Living in a Niche Culture (2)
12 – The Infinite Screen
13 – Beyond Entertainment
14 – Long Tail Rules
15 – Coda

Or you can download the whole thing as a 150-page PDF here.

First, welcome to readers from progressivebloggers.ca, where I am now an affiliate (just).

If you are coming here for the first time, let me catch you up. I’m in the middle of a chapter-by-chapter, nearly page-by-page critique of Chris Anderson’s book about "the future of business" called "The Long Tail". I’m doing it because the book is widely read, influential, enthusiastically pro-market, and also sloppy and wrong wrong wrong. If you want to see the beginning of the series, go here. If you want to see them all in descending order, go here.

I usually try for about three chapters a week, but I only got two done over the last week, so here’s a bonus teaser for the next chapter, which is Chapter 7. It’s a little thing – one sentence in the middle of the book – but it says something about the author’s cavalier attitude to facts.

Chris Anderson says:

Dell spends hundreds of millions each year on promoting its quality and
customer service, but if you Google "dell hell" you’ll get 55,000 pages
of results.[99]  (I put page numbers in square brackets)

At ten results per page (the default) that’s 550,000 results. Let’s see if we can reproduce this number.

First let’s try "dell hell" in quotes, as he shows it. Over at the right of the page it says "results 1-10 of about 69,600". That’s 696 pages, not 55,000.

But Google’s "about" numbers are notoriously inaccurate. To get a more precise number, first I recommend changing the preferences to return 100 results per page (less clicking), redo the search, and then click through the pages. How many results do you end up with? A total of 693. That’s 70 pages of results, not 55,000.

Suppose he meant without the quotes. Let’s try that. The "about" number at the top right of the page is now 1,620,000 – or 162,000 pages of regular results. That’s about three times the number Anderson quotes – perhaps that’s what’s changed in the last two years. So this is where he gets his number from, I guess.

But as I said, that "about" number is notoriously unreliable. Click through the pages and how many results do you get? 753. That’s 76 pages of results at the default ten-per-page.

So Anderson claims 55,000 pages of results because it’s a big number, and a big number illustrates the point he is making, but a five minute test shows that he is overestimating by about 54,924.

So much for that claim. And unfortunately (in case you were wondering what I think) it’s typical of the book.

This is another part of my critical reader’s companion to The Long Tail, and it discusses Chapter 6 – The New Markets. Part 0 is here. You can find a complete list of the Long Tail pieces here. You have to read upwards, in usual blog fashion. When I get to the end I hope to put these all together in one big file for download.

Bored of this yet? Well that’s just too bad, because this blog is nothing but The Long Tail until I’ve gone through the whole thing, repetition and tedium be damned. So if you don’t like that, come back in about three or four weeks if things go according to plan.

Still here? Great. Glad to have you. It is time to cast your mind back and recall that Chapter 4 spelled out the Three Forces of the Long Tail and that Chapter 5 was about the first one: Production. Chapter 6, today’s subject, is about the second force: distribution. And whereas Chapter 5 was bolted on to the side of the Long Tail hypothesis, Chapter 6 is right in the middle of it. It’s about Internet aggregators, in particular Amazon, and how they go about filling their long shelves.

The first section of the chapter is one of the more interesting stories in the book. It’s about the success, at the third attempt, of Alibris, a company that seeks to "deliver the growing power of e-commerce to independent                booksellers while delivering extraordinary selection to businesses                and book lovers" (from the Alibris web site).  The used book market (other than university textbooks) is fine if you are a browser hoping to stumble across a quirky find, but problematic for people looking for a particular book because there are "not enough sellers and buyers of an unbounded set of commodities" [87]. As a result, the odds of finding your book are slim. "Thus, most buyers simply never consider a used-book store when they’re shopping for something specific" [87].

Alibris takes the inventory of used book stores, gathers them together, and puts this big database online, making it available to Amazon and other online outlets such as bn.com, where used books appear alongside new books when customers search for a book. The result is a fast-growing used-book industry (11% in the US in 2004) [88]. This story shows what the Internet is good at: bringing unconnected people together. It sounds like, and maybe is, a hopeful story, "creating a liquid market where there was an illiquid market before" [88]. (Aside – it is odd that Anderson talks about the American Alibris and completely neglects its bigger Canadian competitor Abebooks, who now own the fine librarything web site, but perhaps he is subject to the tyranny of geography in the physical world).

It is likely that a move to used book buying and selling is a pretty diverse market compared to new books, but it may be of interest to note that the Alibris founder says, in discussing a new Alibris pricing initiative that " It’s true that the Alibris Pricing Service really is oriented mainly around common books – but that’s more than 80% of the demand out there, so it tackles the vast majority of the market". Alibris is a private company and does not make its detailed sales figures available, so it is difficult to know for sure. But used books are one of those areas where the Long Tail hypothesis has the strongest ring of truth.

Enter the Aggregators [88-89] picks up from the Alibris example and uses it as an example of a "Long Tail ‘aggregator’ – a company or service that collects a huge variety of goods and makes them available and easy to find" [88]. And the Alibris story shows that there are ways in which the Internet can pull people together. But while aggregation can be a fine thing, it does have a dark side, and that side is market power. Canny aggregators know this. Alibris, for example, does not put its customers in touch with the used book stores that actually sell the books. Instead, the book store owner sends their book to Alibris, who then sends it on to the customer. In this way, Alibris becomes the hub of the market: the central repository of knowledge, while the individual book stores who it relies on are kept in the dark. This appears to have led to some resentment amongst some librarians and book store owners, although I have no idea how widespread this is.

What this means is that online aggregators become the new powerhouse in the industries they move into. Amazon in the world of new books, Abebooks and Alibris in the world of used books, and so on. Google, of course, is seeking to aggregate everything and so become the hub for "the world’s information".

This is a mixed development for those whose information is being aggregated. Newspapers, for example, are not thrilled at the idea of Google News presenting their information freely to the world, while the newspapers themselves struggle over lost income. With Google making scads of advertising money off presenting information to people, it looks like the balance of power has shifted from the content originators to aggregators.

Aggregators are here to stay. What remains to be seen, however, is if they bring with them any real move towards a more diverse pattern of demand and consumption.

So aggregators are the Long Tail enterprises that represent "the future of business", and Anderson says "there are literally thousands of them" [88]. Are we now going to hear about the wave of the future, and see what shape it is taking?

No. Frustratingly, the only examples he lists after that teasing sentence are our old friends Google, Rhapsody, iTunes, Netflix and eBay — the same handful of companies he’s been going on about in the first few chapters. He does also mention Bloglines, an RSS feed aggregator, and Wikipedia, which I’ve argued is not an "aggregator of the Long Tail of knowledge" but those are the only ones of those "literally thousands" that he actually lists.

What he means by his "literally thousands" becomes clear when he says that "a single blog that collects all the information that it can about a topic, let’s say needlework, is an aggregator" [89]. Well, if this is where his "literally thousands" of aggregators are coming from, count me underwhelmed. Amazon is one thing, and someone collecting links on needlework is another. I mean, Wal-Mart puts things on shelves and I put things on shelves, but that doesn’t make me the same kind of thing as Wal-Mart. And  although he mentioned needlework or "SEC filings or techno music" he does not actually list real sites. The wave of the future, it seems, is still intangible.

Anderson does tease us again by claiming that "in this chapter, I’ll focus on the business aggregators. They fall mostly into five categories:

1. Physical goods (eg Amazon, eBay)
2. Digital goods (eg iTunes, iFilm)
3. Advertising/services (eg Google, Craigslist)
4. Information (eg Google, Wikipedia)
5. Communities/user-created content (eg MySpace, Bloglines)" [89]

The rest of the chapter, however, is devoted to a discussion of just one aggregator — Amazon (again!) with occasional mentions of old chums Netflix and Rhapsody. The fact that we are a third of the way into the book and, when it comes to specifics, Anderson keeps going back to the well to show us Amazon or Netflix one more time suggests that he is a little short of real examples, and that perhaps the aggregator model, while a powerful one for some people, is not something that is going to spawn large numbers of companies.

We’ve already seen (earlier chapters) that while Amazon does make more money off its "Long Tail" than individual brick and mortar stores, its figures are not spectacularly different after the downward revision between the original article and the published book, and that Rhapsody seems just as hit-dependent as HMV. When it comes down to it, we have here a short, limited list of examples which we’ve already seen provide little evidence for a significant Long Tail effect.

Hybrid versus Pure Digital [89-91] This section is devoted to a single rather obvious idea, which is that while it’s pretty cheap to sell physical goods from digital shelves ("hybrid retailer"), it’s even cheaper to sell digital goods from digital shelves ("pure digital retailer") because such a good is just a row in a database table until someone orders it. This latter form of business is, he says "the holy grail of retail – near-zero marginal costs of manufacturing and distribution" [91]. Here, he actually mentions (unlike in the previous chapter) that it is marginal cost that is being minimized. He does not go on to point out that the industries with large fixed costs and tiny marginal costs tend to form oligopolies, but they do. It’s the main reason why there really aren’t "literally thousands" of aggregators and it’s why he has to go back to the same old examples over and over again. Fixed costs and other forms of increasing returns mean that there just aren’t that many of them.

Tripping Down the Tail [91-94] is also about Amazon (yawn), and shows how it is getting "closer and closer to breaking the tyranny of the shelf entirely" [94] by introducing Amazon Marketplace – a program in which the Amazon web site is a storefront for many small vendors (rather like eBay). "Retailers and distributors of any size, from specialty shops to individuals, could have their goods listed on Amazon.com just like the products in Amazon’s own warehouses – and the customers could buy either just as easily." [93]

Amazon Marketplace shows the real appeal of the aggregator business model, and it’s not about Long Tail, it’s about One Big Virtual Tent (see yesterday’s post). Why? Because the Amazon Marketplace model gets Amazon itself out of the physical shelf business, even though as Anderson writes, apparently without noticing the implications (he really is an incurable optimist) "With the Amazon Marketplace form of distributed inventory, the products are still on shelves around the country, but they are collectively catalogued and offered in one central place – Amazon’s Web site. Then, when people order them, the products are boxed up and shipped directly to the customer by the small merchants who have held the inventory all along" [94 – my emphasis] and if the goods don’t sell "Amazon bears none of the cost — the surplus stock simply depreciates on the shelves of a third party" [94].

It is easy to see why this is a great deal for Amazon: they are using their market power to put the squeeze on the individual retailers whose very physical shelves are holding very physical products. Like Alibris, Amazon have put measures in place to prevent customers and sellers communicating directly. They are the ones in charge of this relationship.

So Amazon Marketplace is yet another story with at most a tenuous relationship to the Long Tail thesis. It provides Amazon with another source of revenue, but it is not clear if or how it moves overall customer demand to a Long Tail in any significant way. Sales of small items are gravy for a site with the huge web site resources of Amazon,  but Anderson does not say how much business has been done on Amazon Marketplace, whether the small suppliers (for whom it is one more outlet) are gaining or losing overall from any move online that has happened, and so on. If, for example, there is a move to online purchase of the kind of goods found on Amazon Marketplace, then it could make sense for individual retailers to put their items there (because any sale is better than no sale) and yet be worse off than they were before Internet selling came along. It’s just shoddy writing. And it doesn’t get better.

Inventory on Demand [94-96] is yet more Amazon smoke and mirrors. By this time it is really getting silly. Anderson talks about Amazon’s big commitment to print on demand publishing as a way of selling all those niche books. But while he notes that "the potential of print-on-demand is extraordinary" [96] he doesn’t give any real-world numbers. How many print-on-demand books does Amazon sell? Not a clue. Will it help publishers get past the problem of returns from physical stores or does POD have its own problems? Who knows? Not Anderson, that’s for sure. And then at the very end of the section he says that print-on-demand may also be useful for big sellers (hits) as well as niche products, so it may not actually "push demand down the long tail" at all. Remarkable.

The End of Inventory Altogether [96-97] is the final section of this chapter. It finally moves away from Amazon to talk about iTunes, Netflix and other pure-digital outlets as the wave of the future – which we have already seen are pretty hit-based. There is no doubt that the digital world is getting bigger, but this is not news. I for one am no cellulose sentimentalist and can’t wait for flexible electronic paper, but when it gets here I hope there is something more solid than The Long Tail to read on it.

So there we are. Another chapter, another lack of real substance to Anderson’s thesis. It is clear that there is money to be made by aggregators, but not at all clear that the coming of the aggregators is the coming of "infinite choice" [180].

This is a part of my critical reader’s companion to The Long Tail, although it doesn’t discuss any one chapter. Part 0 is here. You can find a complete list of the Long Tail pieces here. You have to read upwards, in usual blog fashion. When I get to the end I hope to put these all together in one big file for download.

It’s time to take a breather and stop turning the pages of The Long Tail. I hope I’ve made the point now that there are a lot of problems with the book, a lot of places where (as John said) "It looks like his theory has become so obvious and inevitable to him that he sees the ‘revolution’ coming even when the facts seem to point in the other direction." It’s time to step back a bit. After all, even if much of The Long Tail is fluff, what about the idea at the heart of the book? Even if it takes only a few sentences to state instead of 240 pages, it could still be a penetrating insight that holds up despite the sloppiness of the supporting material. So the question is, if I don’t think the world of the Internet is Long Tail, what do I think it is like? Criticism can only take us so far without something constructive to put in its place.

The short answer to these questions are that I believe The Long Tail idea has merit only to the extent that the physical world is trapped in a digital vice of IT-driven big-box stores on the one hand and online stores on the other. Second, that to the extent a single image can be used to describe the world of the Internet, we could say that online commerce replaces a world of Many Little Tents by One Big Virtual Tent.
There is a lot of stuff under the One Big Virtual Tent, but that stuff is hard to find, there’s only one of it,  and it’s probably based in California.

The core of the Long Tail hypothesis is that (i) Internet-based businesses, unencumbered by the tyranny of physical space, are able to make far more different products available than "bricks and mortar" stores, (ii) Internet-driven increased access to these niche products leads to greater demand for them – demand is driven down into the Long Tail, with the end result that (iii) we move from a market driven by hits or blockbusters to a market of niches. Finally (iv) this coming together of a diversity of tastes with a democracy of production is a think to be celebrated as a natural, unfiltered, and diverse outcome. The Internet brings an "explosion of variety" where  "On the infinite aisle, everything is possible" [226].

The first thing to say is that (the title of this entry aside) to stretch one simple image over the entire Internet phenomenon – which Anderson does sometimes – cannot work and is of little use. It’s as if we tried to stretch a single image over the invention and development of the internal combustion engine. If you wanted to write a book in the early years of motorized transport talking about the future of business and of culture what would you say? Just that there were many futures to business. There were futures of families scattering across the miles as travel became easier, and of families overcoming distance to stay in touch. There were futures of homogeneity as economies of scale took hold, and futures of variety as people became exposed to influences from outside their own neighbourhood. There were futures of war and of peace, of great depressions, general strikes, longer lives, wonderful sights that could never be reached before. You get the idea. You just can’t take a phrase like The Long Tail and stick in on a development with such far-reaching implications  as computerization and the Internet and apply it to everything without it losing all meaning. So the Long Tail in its broadest use is meaningless, and needs no competing theory to replace it.

Narrowing the scope of the idea to something more manageable, a reasonable question is "will the shift to online commerce (Amazon, Netflix, iTunes) lead to a more diverse environment for cultural products?" I predict no with some qualifications. Such businesses are freed, as Anderson points out, from the constraints of geography. But geography, as Anderson admits from time to time, not only limits diversity, it is also a source of diversity. When we move our shopping online, the first physical stores to feel the pinch are not the Borders, Blockbuster, and Wal-Marts but are the smaller specialist sellers within physical communities. These specialist sellers are almost completely ignored by Anderson, as I’ve pointed out in the last several posts. Some specialist sellers will manage to thrive in the online world, but others (a computer book store in my town, for example) will be driven to the wall by online competition. To the extent that we abandon local businesses for online shopping, we lose rather than gain diversity.

If we are to compare the online world to the physical world we must do the job properly and compare all the online world to all the physical world. This is a difficult job, because the physical world is much more uneven than the online world: the economies of scale in the online world tend to create oligopolistic or even monopolistic markets in a way that outdoes even the Wal-Marts of the physical world, and Amazon is as accessible to people in small towns as it is to people in major cities. When looking at the availability of jazz music, for example, piefuchs’ comment on Chapter 2 captures the difficulty of measuring the change:

Growing up in small-town NS I used to love to come to Boston to buy CDs – especially jazz. Now however, anyone can buy any CD on the web, but living here in Boston I can no longer go to a store see the same magnitude of selection, since every record store with a substantial new jazz department has closed. So are more people actually buying jazz instead of the blockbuster or are more people in small towns buying jazz on the web rather than when they travel?

The physical world is a world of heterogeneous variety, while the online world is a world of homogeneous variety. It is not clear that averages help very much when comparing physical apples to online oranges, but we can at least say that one thing you don’t do is compare Borders (or Chapters/Indigo, or Waterstones) to Amazon and claim you are comparing the physical and online worlds. The measurement biases of the book are pervasive and ignore the shift from a relatively heterogeneous retail environment in the physical world to a homogeneous retail environment in the online world. These biases are a big part of what make the book fundamentally wrong.

The other reason for my disbelief in the Long Tail is that it is not only the niches that benefit from being freed from "the tyranny of geography". Hits benefit too. The phenomenon of movies opening simultaneously worldwide is possible only in a digital age. The possibility of such global rewards, especially when complemented by supply-side economies of scale and multiplied by synergistic products (the action figure and video game of the movie of the book) and given the fixed costs of producing digital content pushes markets towards the Short Head, not the Long Tail. The phenomenon of world-wide bestseller book releases is likewise a recent phenomenon (I believe, but confess I don’t have figures to back it up) and as John commented, there is evidence via Anderson’s weblog itself that midlist titles are struggling relative to hits in the bookselling market.

On the demand side too, there are forces that push us towards the Short Head. As Dipper pointed out in the comments to Chapter 1:

One reason people buy arts and cultural products such as books, dvd’s, games, clothes, cars is to participate in cultural life. That means buying what other people are buying

The weakening of geographical constraints means that the definition of "other people" broadens out.

What is the net result of these competing forces? The nearest I can see to a simple statement is that the online world is One Big Virtual Tent whereas the physical world is Many Little Tents. There’s little doubt that there is more variety under the One Big Tent than in any of the Many Little Tents (even the biggest of them). But is there more variety in the One Big Tent than in all the Many Little Tents together? Well, if you live close to only one little tent, the answer is undoubtedly Yes. If you live closer to a field crammed with Little Tents, then the answer may be No. And can you find what you want in the One Big Tent? Not yet would be my answer. Finding things online is a topic of Chapter 7 in the book, so there will be more to say there, but let’s just say that if you know what you are looking for you can find it on Amazon, but you are unlikely to come across, as I did in my public library, the strange and compelling stories of Agota Kristof by chance.

In comparing Many Little Tents with One Big Virtual Tent also need to look at geography at the level of nations as a source of diversity. As in so much else on this topic, my favourite reference is Blockbusters and Trade Wars, Popular Culture in a Globalized World by Peter Grant and Chris Wood. The theme of the book is the breaking down of trade and cultural barriers by commerce, including but not exclusively online commerce, and the attempts by smaller countries to preserve and enhance their own cultural output in the face of this challenge. They make a convincing case that a state-driven "cultural toolkit" including tools such as public broadcasting, quotas, subsidies, and more have been essential in helping a diversity of culture to survive and even, on occasion, thrive.

Needless to say, Anderson does not even begin to address issues of international cultural diversity beyond the fondness of online teenagers for Japanese anime films. Sitting in San Francisco, he does not seem to even notice the fact that there is a distinction to be made between American cultural consumption and that in other parts of the world. I wish I thought he had even heard of Grant and Wood, but they come from Canada so I would guess that he hasn’t. Google tells me that the only page outside this little corner of the Internet that mentions both "long tail" and "blockbusters and trade wars" is a bibliography page for a course on the Structure of the Book Publishing Industry in Canada from Simon Fraser University in Vancouver. The ability of online vendors to cross borders is a double-edged sword when it comes to diversity. The ability to keep diversity in One Big Virtual Tent will depend to a large extent on the ability of governments to regulate portions of that tent.

There is one scenario in which the Long Tail does lead to an increase in diversity compared to the physical world, which is the scenario of the Digital Vice (I’ve renamed it since Chapter 1). The variety of physical stores that unevenly drive an uneven amount of uneven variety may be replaced by a combination of homogeneous big-box stores that sell mainly best-sellers and online stores that sell best-sellers and everything else. In such a world, there is no doubt that the niche purchases will take place online, so that the online world will represent a Long Tail compared to the physical world. But it’s a grim outcome nonetheless.

The second question here is, what about the Long Tail of production represented by YouTube, Facebook, MySpace, Flickr, and the world of blogs and Wikipedia? To some extent I tackled this in the discussion of Chapter 5. There are, I would argue, two things going on here, neither of which is Long Tail. One is the moving online of social conversations, play, and pastimes. Writing a diary online does not make it more "production" than writing a diary at home. The shift online loosens some ties and builds others. We tend to end up with (as Yochai Benkler points out in The Wealth of Networks) a looser but wider set of connections than in the physical world – we inhabit networks rather than communities, with different networks for different parts of our lives. Again, Anderson has eyes only for the building of online networks, not the loss of those other contacts that they replace. He repeatedly welcomes the online world in terms of its potential for a democratic future, but such a future is far from assured. To take one example, while the Internet is a great source of information for those who oppose the US war in Iraq, it has failed to coalesce into a significant political movement in the way that opposition to Vietnam did. The opposition to globalization that crystallized around 1999 did so at a physical place (Seattle): the effects of geography are strong and not easily replaced. While the Internet may provide new opportunities for political networking, to the extent that it replaces physical world actions it will not promote a broadening of democracy.

The most interesting and promising aspect of Internet technology is the large-scale collaborative efforts represented by the peer production models of Wikipedia, open-source software, SETI@home and others. These are remarkable, although again they tend to be operations that replace Many Small Tents by One Big Tent – there are fewer of them, but each one is bigger – than in the physical world. The endless comparisons of Encyclopedia Britannica to Wikipedia are somewhat misleading, because it again neglects the complementary physical books (specialist reference material that doesn’t get into Britannica) that do not exist in the online world. Is there a big online reference for birdwatchers? I’m not sure, but my guess is that the ease of use of Wikipedia will make it the equivalent of many offline reference works, rather than a single general-purpose encyclopedia. In the offline world, there is no longer a need for Small Tent production – just put your material into the One Big Tent. Again, does this represent greater variety? In some senses yes, in some senses no.

So if there is a single image I’d use instead of the Long Tail, it’s the One Big Virtual Tent. It’s a tent that holds more than any of the Many Small Tents of the physical world, but which does not guarantee an overall increase in variety. And if you’re going to build One Big Virtual Tent, the chances are you won’t be building it from an out of the way place – to that extent, the revenue from the homogeneous variety of the online world is going to be finding its way to Silicon Valley rather than to your local community.

This is a part of my critical reader’s companion to The Long Tail and discusses "Chapter 5 – The New Producers". Part 0 is here. The previous part in this series is here.

At the beginning of this chapter [58] Anderson is back in storytelling mode, taking us to the Kamiokande II observatory in Japan to tell us how "one of the greatest astronomical discoveries of the twentieth century unfolded. A key theory explaining how the universe works was confirmed thanks to amateurs in New Zealand and Australia, a former amateur trying to turn professional in Chile, and professional physicists in the United States and Japan." [60] The story shows, according both to British Think-Tank Demos and to astronomy author Timothy Ferris, that astronomy has shifted from "the old days of solitary professionals at their telescopes to a worldwide web linking professionals and amateurs" [60].

Like the story of Touching the Void in Chapter 1, it is a story that is intriguing by itself, but which has very little relationship to the actual Long Tail thesis. The world of astronomy is a different world from that of Amazon.com. The book is about "The Future of Business" and yet astronomy is not a business. It’s about "a market of multitudes" [5], but there is nothing in this story about markets. It’s about the changes being brought about by the World Wide Web, but although Demos referred to a "worldwide web linking professionals" this story took place in 1987, before the World Wide Web was invented.

That is the weak point of this whole chapter: it tells us that amateur producers and volunteers have taken on a new importance in the world of the Internet but fails to show how this links to the rest of the book. In his original article Anderson did not talk about the production side of business at all. In short, this chapter is only tangentially related to the rest of the book. It looks as if Anderson needed material to fill out his 240 pages (this is one of the two longest chapters in the book), and so he bolted this on to the side. But it doesn’t fit.

So when, as in this first section of the chapter, the book talks [61-62] about SETI@home (the use of spare cycles on home computers to search through large amounts of data in the search for extraterrestrial life) or "clickworkers" identifying craters on Mars, or even open source software — all interesting subjects — I’m going to be relentlessly negative and ask "what does this have to do with the rest of the book?"

Democratizing the Tools of Production [62-65] is all about how people can use their PC’s at home to engage in "production". Just like I’m doing now, in fact (I have a day off work). Anderson points us to the world of music – "Just as the electric guitar and the garage democratized pop forty years ago, desktop creation and production tools are democratizing the studio" [63], to the written word "aways the leading edge of egalitarianism" [63] and to photo editing and printing. He includes in his reach "video games that let people create and share their own alternative levels" and print-on-demand book publishing. Put together all these examples and it looks as if "we’re starting to shift from being passive consumers to active producers" [63]. It’s an inspiring image and a common one these days. But again, and as in previous chapters, Anderson is being selective in his selection of examples as he seeks to bolster his thesis. There are two questions that we have to ask when confronted by this plethora of anecdotes.

The first is "What were these new producers (or their equivalents) doing before this new kind of activity came along?" They must have been doing something, after all. We know the answer, I suspect. Some of the time we were indeed watching TV (passive consumers). At other times, as Robert Putnam describes in "Bowling Alone", we were engaged in other activities in or outside the home – local and community activities, whether it be a drink at the local bar or playing second clarinet in a Little Theatre production. A generation ago, the equivalent of those kids spending hours at their desktop digital music studio were probably producing music some other way. I don’t know of any study that shows that teenage participation in music-making has increased over the last decade. Maybe it has, maybe it hasn’t, but Anderson doesn’t even begin to make the case that it has, and pointing to PC software does not make the case.

Another question is about the definition of "production". The Internet is still a shiny new thing, and the things it gets used for are changing rapidly – and will continue to change. But just because something happens on the Internet doesn’t make it important, and doesn’t make it "production". When I was young I used to make Airfix models of planes and tanks and so on, and I spent hours painting small metal figures of Napoleonic soldiers and creatures from Lord of the Rings. It was fun, but was it "production"? Only in a very distorted view of the world. So what about Chris Anderson’s kids "who are, as I write, into machinima — short computer-animated movies made with video game software…. The first reaction of the kids was to watch and enjoy the machinima movies as entertainment. Their second was to express curiosity as to how they’re made. And their third was to ask if they could make on themselves (The answer, of course, is yes)." [64] Do the young Andersons’ game-playing activities count as "production" any more than mine did? I suggest not. The fact that they are using a computer is not in and of itself very important.

Or consider home photography. Twenty years ago we may have been sticking photos in scrapbooks rather than putting them on Flickr, but we should ask whether posting on Flickr is any more "production" than sticking pictures in scrapbooks. It is a more visible activity for researchers, for sure. It is easier to count the number of photos on Flickr than to count the number of photos in scrapbooks around the globe, and as a result it is tempting to identify this as new activity or an expanding one. But like the "production" of music it may well not be. It’s a shift of an activity from one medium, where it remains fairly private, to another where researchers can more easily access it. But that does not make it significantly different.

The end result is that, whereas Anderson sees the rise of a new expanded class of creative people: "millions of ordinary people have the tools and the role models to become amateur producers" [65], I suggest that those creative people have always been there, and have always been creating. It’s just that the Internet makes it more easy to count their creations.

But, I conveniently hear you say, what about blogging, where "millions of people publish daily for an audience that is collectively larger than any single mainstream media outlet can claim" [63]?  Isn’t that production? Or what about Wikipedia, which competes with the Encylopedia Britannica? Surely that is production? Patience. That’s the next section.

The Wikipedia Phenomenon [65-67] is the first of four consecutive sections (ten pages) that talks about Wikipedia, the remarkable online encyclopedia. And remarkable it is. Even over the last few months, I’ve found myself using it more and more as a first reference source for a wide variety of questions, from wireless network technologies (where it is very strong, unsurprisingly) to historical questions. Sometimes it’s great, sometimes not so much. But it is remarkable.

What’s more, Wikipedia counts as production. If it competes with the Encylopedia Britannica, which it does, then I’m happy to count it as bona fide production.

But I’m still going to say that this ten-page section on Wikipedia does little for Anderson’s Long Tail thesis, for a few reasons. First, a glance at the index confirms that this is the first place in the book where Wikipedia has been mentioned, and it is mentioned in only scattered places later on. We haven’t seen it in the history of the long tail, or in the introductory chapter. Why not? Because like the astronomy story it isn’t an example of a market and it’s not a business. There has been a lot written about Wikipedia and other open-source content production; the best I’ve read is The Wealth of Networks, the exhaustive if sometimes heavy-going book by Yochai Benkler. Benkler describes Wikipedia as part of "the networked public sphere" – a public place that is provided by a network of loose collaborators. A public place is different from a commercial venture such as Amazon.com, and there is reason to believe that Wikipedia and Amazon are not part of the same thing.

Anderson links Wikipedia into his Long Tail thesis by pointing out that, while the Encyclopedia Britannica has to draw a line at which "the priests… decide ‘This is not worthy’" [72], Wikipedia "just keeps going". While Britannica has 80,000 entries, Wikipedia has well over a million on all kinds of obscure topics. These entries, from 80,000 to 1 million and beyond, are "the Tail" of Wikipedia [72]. This does look a little like a Long Tail. After all, Britannica is put on a physical shelf and Wikipedia isn’t. Britannica is pre-Internet and Wikipedia is Internet. So am I splitting hairs here when I say that Wikipedia is not "Long Tail"? Aren’t these things enough to tie them together as part of "the same thing"? I’ll argue not, precisely because the Internet is so important. Its impact is so big, so multifarious, that it does little good to link up all Internet-related activities as if they are a single thing.

Here’s an analogy. In Chapter 3 Anderson told us about the expansion of supermarkets in post-World War America, and explained how this was a precursor of the Internet Long Tail, driven instead by a technologically-enhanced infrastructure of better transportation and communication. At the same time a huge collective endeavour was thriving – scientific research, which expanded manyfold from its pre-war scale. This too was enabled by new technologies of communication and transport, enabling people to move around and share information, and providing the ability to collaborate across the miles. And guess what? There is a long tail structure to scientific research too. It goes all the way from the hits (DNA structure, the discovery of the laser, and so on) down to those niche items that are of interest to only a few (improved helium-acetylene potential energy functions, as a non-random example). Whereas pre-war research focused on the hits, the new post-war institution "just keeps going", exploring ever-more obscure backwaters of knowledge, some of which turn out to be surprisingly important. But does it make any sense to talk about scientific R&D and supermarkets as two parts of the post-war Long Tail? No. There’s nothing to stop you from doing it, but it doesn’t really add anything to our understanding. In the same way, to see Wikipedia as a production-equivalent of Amazon and Netflix does not really help us.

Wikipedia and other large-scale collaborative ventures among both amateurs and professionals are best viewed as networked public goods, as Benkler describes them. The fact that they are big does not make them Long Tail. After all, take any large enterprise, break it into its constituent pieces, plot them along down an x axis in order of decreasing demand, and you are bound to see something like the long tail graph.

Self-Publishing Without Shame [75-78] follows Anderson’s discussion of Wikipedia (some of which I am skipping over, having said quite enough I think) and presents a different view of what lots of little-looked-at content can be on the Internet. There are, you will be shocked to hear, things I agree with in this section. He writes about blogs, other forms of self-publishing, and "citizen journalism" (especially South Korea’s "OhmyNews" [78]) and points to these areas as a "crucible of creativity, a place where ideas form and grow before evolving into commercial form". This makes sense. Barnes & Noble CEO says it this way: "Over the next few years, the traditional definition of what a ‘published’ book is will have less meaning. Individuals will increasingly use the Internet as a first stage to publish their work… The best of this work will turn into physical books."  [77] This view of blogs and other web-based forms of expression as "first-step publishing" or pre-commercial publishing seems realistic.

But I don’t want to get carried away with this constructive and positive view of the Long Tail, so let me revert to negativism and point out that this interpretation of what is happening in the world of blogs and other self-publication is not what he has been talking about in the earlier part of the book. First, to see these DIY areas of self-publishing and social networks as
a pool from which hits will be picked is quite different from saying
that demand is shifting to niche products. Second, he says "Blogs are a
Long Tail" [69] but blogs are a non-commercial space for unmediated discussion and opinion. That’s great, but the non-commercial nature of it sets it apart from most of what he is talking about as the Long Tail (an infinite shelf of commercial items for sale). The border between non-commercial and commercial parts of the Internet are blurred and changing, but there is still a border. It is not clear, for example, that Wikipedia contributors would continue to contribute if a company "owned" their product and made money off it. It is not clear that bloggers would contribute if others took their content and re-used it for money. Cases are appearing where photographers are complaining about the commercial use of their freely-produced images. The issues of rights and ownership are thorny and there will be years of debate to sort them out.

Case Study: Lonely Island [78-82] reverts to story-telling mode for the case of comedy troupe Lonely Island. It starts with media mogul Barry Diller scoffing at the idea that peer production, or "18 million people producing stuff they think will have appeal" could rival Hollywood and then tells how Lonely Island took their act online and found success after being picked up by Saturday Night Live. The lesson of the story is "on one hand, the existing entertainment industry filters did recognize the appeal of the Lonely Island and found a way to tap it. In that sense, maybe the system works. Yet if three kids with a video camera doing goofy raps and putting them on their web site isn’t ’18 million people producing stuff they think will have appeal’… I really don’t know what is." [81] It is a weak story, of the same kind as the earlier Touching the Void – the Internet as source for successful acts. I could tell the story of unsigned singer Billy Bragg hearing that DJ John Peel was hungry and taking a curry to the recording studio, getting a few tracks played on the show in return, and finding himself on the way to stardom. It’s not the Internet, but if it’s not someone producing stuff they think will have appeal, well I don’t know what is. The point is, there is no point.

The Architecture of Participation [82-84] finishes the chapter in bizarre fashion by acknowledging that "we’ve seen parts of this story before" and talking about punk rock and its "anyone can do it" attitude. I do wish he would turn back a few pages to where he told us that the ’70s and ’80s were the years of blockbuster culture: now it’s the start of an architecture of participation. And the punk rock story is a good reminder that, for all the mashups and videos on the Internet, there is not a single artistic movement to have come from the Internet culture that rivals punk in its impact. Lonely Island comedy troupe just doesn’t do it.

So that’s the end of the chapter. For anyone still reading, I want to add one thing about production of digital goods that Anderson does not mention, which is that while it does make certain kinds of production cheaper, it also tends to create "winner take all" markets as described by Robert Frank and Philip Cook, also explained in detail in Blockbusters and Trade Wars by Peter Grant and Chris Wood. The fact that the marginal cost of additional copies of a digital good are basically free to create, whether the first copy cost $1 or $100 million, has a number of consequences. One is that "cultural products that are attractive to consumers in a large geographical market have a lower risk and a much greater potential reward than do those that are produced for a smaller market" [Grant and Wood, p. 55] Here is a force that, as they describe, tend to constrain choice rather than promote it. I’m not going to go into this in detail here – just to say that (as John pointed out in a great comment on Chapter 3) – there are winner take all forces at work in cultural industries that push demand away from the "Long Tail", which are ignored by Anderson.

[Just to finish in a really picky way, Anderson says of Wikipedia "There are the popular top 1,000 [articles], which can be found in any encyclopedia: Julius Caesar, World War II, Statistics, etc." [72]. He is being selective again: according to a list linked to from Wikipedia itself, the most viewed articles in the English language version in February 2007 were (ignoring home page and other non-article pages): Anna Nicole Smith, Wiki, Valentine’s Day, Wikipedia, United States, Naruto (a Japanese anime series), Chinese New Year, World War II, Sex, Deaths in 2007, List of Pokemon, List of Sexual Positions, Sexual Intercourse, Category: Female Porn Stars, and Barak Obama. How many of these top articles have entries in Britannica?]