Author: Tom Slee

This is another part of my critical reader’s companion to The Long Tail, and it discusses Chapter 10 – The Paradise of Choice. Part 0 is here. You can find a complete list of the Long Tail pieces here.

The purpose of this chapter is to counter "the notion that ‘too much choice’ is overwhelming, a belief so common and ill-founded that it deserves its own chapter" [167].

The last 50 years have seen "an explosion of variety" [169], which Anderson ascribes to three factors. The first is "globalization and the hyperefficient supply chains it brings"; the second is the change in the population – in the US, he mentions ethnic diversity and increasing affluence, leading to a cultural shift from "I want to be normal" to "I want to be special"; the third is The Long Tail – or the proliferation of variety brought about by iTunes, Netflix, Amazon, and eBay.

But there’s variety and then there’s variety. Grocery shopping is often a focus for this kind of discussion (and the book uses it in the next section) so let’s use supermarkets as a way into the topic. For me, the proliferation of options at the cheese counter is a great thing: the range of options there seems to me a real variety. On the other hand, the toothpaste section (over forty different types of toothpaste) does not seem to be real variety. Now this could just be me, who likes cheese and can’t really be bothered about toothpaste as long as it does the job, but I think it’s more than that. There is another difference between the two, and it’s the issue of manufacturers or suppliers. The cheese counter contains products from a variety of producers, while the toothpaste shelves contain products mainly from two producers – Colgate-Palmolive has 37% of the American market; Procter & Gamble (makers of Crest and now merged with Gillette) follow with 34% – with a few "smaller" players like GlaxoSmithKline (Aquafresh) and Unilever filling out the remainder. If anything, the toothpaste market has become more dominated over the years by this small number of companies; meanwhile, the profusion of items on the shelf is growing. Oligopoly Watch (from a few years ago)  describes these twin trends of corporate consolidation and product profusion in the world of toothpaste. Is the combination an increase or a decrease in real "choice" and "variety"?

Anderson is as ebulliently optimistic as ever that the increase in number of products is simply a reflection of what we consumers want – "More choice really is better" [174], but very well-known research by Richard Schmalensee showed that the proliferation of toothpaste varieties may have little to do with responding to consumer demand, and be more about preventing other companies from getting a foothold in the market. Back in 1978 he looked at the ready-to-eat breakfast cereal market, where six companies dominated the market and introduced eighty different brands between 1950 and 1972 – more choice, for sure. Now Schmalensee is no radical – he was on Microsoft’s team in their anti-trust suit, arguing that Microsoft is not a monopoly – but he concluded that this particular case of product proliferation was all about trying to deter entry by other companies. The strategy is to "pack the product space and leave no profitable niche unfilled" (this from the standard graduate text by Tirole, p. 346). Schmalensee showed "how a cartel (a group of firms that act as a single monopolist) crowds a product space". The cartel in this case needs no explicit collusion, but the strategies of the incumbent firms complement each other to produce the same effect as a monopoly. The result – higher prices for consumers, as the incumbent firms can raise prices without new competitors coming to undercut them.

How much of the "explosion of variety" that we have experienced is real variety, and how much is entry deterrence and the ability to raise prices? It requires a lot more space and expertise than I have to pick apart what’s what in the myriad of different retail spaces, but it is something to think about when standing in front of a wall of shampoo, deodorant, mustard, breakfast cereal, or whatever. One would think that if The Long Tail is, among other things, an "economics research project" [11] and "a preview of 21st century economics" [11] then its author might have looked a little harder at the work of actual economists, however 20th century they might be.

Too Much Choice? [170-172] is a quick summary of one of the points made in Barry Schwartz’s excellent book The Paradox of Choice. It describes one well-known experiment that Schwartz talks about on pages 19-20, carried out originally by Sheena Iyengar of Columbia University. The experimenters set up a table at a specialty food store and offered customers a taste of a range of jams, and a $1 coupon to use against the purchase of a jam. When only 6 jams were put on the table, 30 percent of customers went on to buy a jar of jam; when 24 jams were put on the table, only 3 percent did. Faced with extra options, people shied away from choosing any of them. After presenting the experiment, Anderson goes on to say:

Schwartz describes the conclusion this way:

As the number of choices keeps growing, negative aspects of having a multitude of options begin to appear. As the number of choices grows further, the negatives escalate until we become overloaded. At this point, choice no longer liberates, but debilitates. It might even be said to tyrannize.

I found those sentences on page 2 of the book (The Long Tail does not indicate where they came from – it’s endnotes are lousy). They are separate from the jam experiment (pp 19-20). Schwartz’s book addresses a broad range of questions about the proliferation of choices that we are faced with, and while the sentences do reflect an argument that Schwartz makes, it oversimplifies his reaction to this particular experiment.

Anderson’s response to Schwarts is to say:

I’m skeptical. The alternative to letting people choose is choosing for them. The lessons of a century of retail science (along with the history of Soviet department stores) are that this is not what most consumers want [171].

His rebuttal does not do justice to The Paradox of Choice. Two sentences before the convenient quotation above, Schwartz says "When people have no choice, life is almost unbearable", and a few sentences after them he says "Choice is essential to autonomy, which is absolutely fundamental to well-being. Healthy people want and need to direct their own lives". So Schwartz is clearly not advocating Soviet department stores here, or the elimination of personal choice. The suggestion that the only options are "letting people choose" or "choosing for them" is a silly false dichotomy that I got accused of with my book too. There are many forms of choice, from individual consumer choice to our choices as citizens to collective choices – not just "choosing for them". Anderson is substituting a quick dig for real argument. It’s weak. But then, Schwartz’s book is detailed, subtle, and packed full of references to work by other people…

In fact, Schwartz’s book is more about how to cope with the profusion of choice than advocating a removal of it. One of eleven recommendations he makes is to "satisfice", but Anderson ignores the other ten and says: "Schwartz recommends that consumers ‘satisfice,’ in the jargon of social science, not ‘maximize’. In other words, they’d be happier if they just settled for what was in front of them rather than obsessing over whether something else might be even better" [171]. Contrast this with Schwartz’s actual definition of "satisfice" (p78):

To satisfice is to settle for something that is good enough and not worry about the possibility that there might be something better. A satisficer has criteria and standards. She searches until she finds an item that meets those standards, and at that point, she stops.

I don’t think Anderson’s paraphrase is a fair one, and I don’t think his arguments refute much of The Paradox of Choice. (They also don’t even start addressing most of the problems that are the subject of No One Makes You Shop at Wal-Mart, but that’s another story).

Variety is Not Enough [172-175] starts out by refuting Iyengar’s jam experiment. Anderson wrote to Iyengar and asked "why the people who should actually know the most about consumer choice in a supermarket were ignoring their [Iyengar and coworkers’] conclusions" [172]. He summarizes her reply (and a work in progress [PDF download]) as "The solution, they found, is to order the choice in ways that actually help the consumers" [173]. And with this, Anderson’s faith in limitless choice restored and he is off to find out how online resellers manage to guide people through their choices.

But just as Schwartz is more nuanced than Anderson gives him credit for, so Iyengar and coworkers are more nuanced in their conclusions. They distinguish "Preference Constructors" from "Preference Matchers". Preference Matchers are often experienced with the particular choice they are faced with (choosing a cup of Starbucks coffee is one they discuss); they are "someone who enters the decision making task looking for either a specific option or an option that possesses a number of pre-defined attributes. This type of chooser restricts the multitude of available options in a purchasing situation by creating “consideration sets,” or sets of options with a high probability of containing his or her optimal choice". Meanwhile "Preference Constructors" are those approaching the choice for the first time. Faced with a dizzying array (24 jams) such a chooser may shy away. They can be helped by providing some guidance: "To enable Preference Constructors to distinguish a hazelnut steamer from a vanilla chai and thus successfully form their preference criteria before identifying their preferred option, these decision makers may require more than a limitless, unstructured choice set." Here is the basis for Anderson’s optimism.

His contention is that online retailers are ideally placed to guide you through the choice in a comfortable way, so that us consumers are not faced with the 24-jam problem. But I’m just an old cynic. Whereas Chris Anderson is thankful for all this guidance he’s getting, I’m just suspicious. As George Akerlof wrote (and he knows about people guiding you through choices), the problem goes back as far as horse trading: "“if he wants   to sell that horse, do I really want to buy   it?” Such questioning is fundamental to the market for   horses and used cars, but it is also at least minimally present   in every market transaction."

Here’s Anderson’s take on how online retailers can guide people through a proliferation of choices.

There are a nearly infinite number [nitpick alert: "nearly infinite"??? – TS] of techniques to tap the latent information in a marketplace and make that selection process easier. You can sort by price, by ratings, by date, and by genre. You can read customer reviews. You can compare prices across products and, if you want, head off to Google to find out as much about the product as you can imagine. Recommendations suggest products that ‘people like you’ have been buying, and surprisingly enough, they’re often on-target. Even if you know nothing about the category, ranking best-sellers will reveal the most popular choice, which both makes selection easier and also tends to minimize post-sale regret. After all, if everyone else picked a given product, it can’t be that bad. [173]

I confess I did a double-take when I read that paragraph. The first half is confusing: is the goal of "supplementing an unstructured choice set" really accomplished by adding to that bewildering array a second bewildering array of different techniques to find out bits and pieces about the merits of different choices? I don’t think so. And the second half is a concise argument as to why online retailers so often fail to drive demand down to The Long Tail – in direct contrast to what he claims elsewhere. Anything that gets people to choose an item just because other people have chosen it leads to increasing returns. Once you’ve chosen it, it will be recommended to other people because you bought it, and so on. Note that without recommendations, you can’t even tell if they liked it or not once they bought it. And pointing people to best sellers because "it can’t be all that bad"? This is the logic of the blockbuster. Nevertheless, Anderson is so pleased with his conclusion that he reiterates it three times: "More choice really is better… The paradox of choice turned out to be more about the poverty of help in making that choice than a rejection of plenty. Order it wrong and choice is oppressive; order it right and it’s liberating" [174].

The other argument that Anderson has in favour of unrestricted choice is taken from libertarian journalist Virginia Postrel, and is again about help. It is true that, around most choices that are both expensive and complex, an industry of brokers rises up. Real estate agents, wedding planners, financial planners and so on, all "help us be ourselves" [174]. Another way of looking at such an industry is that it represents the cost of choosing (wedding planners are not free, after all): the rise of such an industry indicates that there is a cost to choice that we are prepared to pay substantially for, but this is in itself neither an argument for or against the proliferation of variety and pseudo-variety that we are immersed in.

The Economics of Variety [175-176] finishes the chapter by asking "Does more choice encourage consumers to buy more?" [175], and concluding that, by making choices easier, digital distribution "widens the field of possible customers and shortens the search time. Over time, this should increase sales and grow the overall market. As we saw in Chapter 8, longer tails can be thicker too" [176].

I’m not entirely sure what he means here by "the overall market", but if he is claiming – as I think he is – that the overall economy will grow because people will buy more stuff, then this is a claim that needs a lot of substantiation. The money, after all, has to come from somewhere. Unless he is advocating a further increase in consumer debt as a way of boosting the overall economy, then I’m not sure what his argument is.

But then that’s how I feel about the whole book, of course. The Long Tail sounds nice, and it would be great if online retail could not only break the tyranny of the blockbuster and promote a diverse niche culture, but also boost the economy at the same time – but just because it would be nice doesn’t make it so, and the book continues to fail to convince.

This is another part of my critical reader’s companion to The Long Tail, and it discusses the second half of Chapter 9 – The Short Head. Part 0 is here. You can find a complete list of the Long Tail pieces here.

The second half of Chapter 9 is a meandering stroll around the topic of "finding things". It’s an odd place for it in the book – this reads like a part of Chapter 7 – The New Tastemakers that has wandered into the wrong part of the book by mistake. Warning – this is even more rambling than the other posts in the series: it is more of a set of half-baked thoughts than a coherent thesis. But hey, this is a blog.

In the Library of Misshelved Books [156-160] starts off this way: "One of the most vexing problems with physical goods is that they force us into crude categorization and static taxonomies, as we saw with Wal-Mart. That means a windbreaker can be in the ‘Jackets’ section or the ‘Sports’ section, but not in a ‘Blue’ section or ‘Nylon’ section" [156]. A thing has to be in one place or another but not both, and so can be difficult to find. In the online world, things can be in many places at once, and this opens new doors: "The efficiency and success of online retail have illuminated the cost of traditional retail’s inflexibility and taxonomical oversimplification. It’s one thing to have high prices or limited selection; it’s quite another to simply be able to help people find what they want". [157]

Despite this start, the section is mainly about libraries. It describes the Dewey Decimal System for organizing books and its cultural biases and limitations. The numbers from 200 to 300, for example, are "Religion", but all religions apart from Christianity are lumped into the interval from 290 to 300. But although the cultural bias of the Dewey Decimal System is clear, its practical consequences are small because, as any library aficionado knows, the Dewey Decimal System goes on several places past the decimal point. As a result you can fit as many books between 290 and 300 as in any other interval, because  The Long Tail, for example, is "658.802 And"; No One Makes You Shop at Wal-Mart is "306.123 Sle". Anyone who has spent time in a library knows there are some areas where the numbers fly by and others where there are whole shelves that have the same number up to the second decimal place. But that’s OK, because additional numbers are free: you could say that the Dewey Decimal System exploits the free shelf space of numbers to extend into the Long Tail of digits. (Aaargh! I have to give Anderson this – the phrase is very infectious.) But anyway, once you have a number for a book, finding it on a shelf is rarely difficult and that’s the point.

Shopping in the Miscellaneous Aisle [160-162] starts off by admitting this: "In libraries, at least there is a standard categorization scheme — the card catalog is there to be searched, and librarians tend to know their stuff" [160]. Haven’t seen a card catalog in a while, but it is good to know that Chris Anderson and I agree there is little problem with libraries. So where is there a problem?

Well, it’s retail stores: those nylon jackets. And it’s true that it can be difficult finding things from time to time (wandering around Toys-R-Us with two pre-school kids was my worst experience, I think). But while this is a section I actually have more agreement with than most in the book, Anderson sets us up again. He complains how difficult it was to look for the Japanese anime classic "Akira" in a Blockbuster store, but "As it turned out, it didn’t matter – they didn’t have the film" [161]. He contrasts this experience with Amazon, where he types the title into the search bar and gets right to the film immediately, along with recommendations for other films. He also says all films were "in stock and cheaper than Blockbuster. The experience I had with these two stores couldn’t have been more different" [161] . He is exaggerating again. This section is about locating things, and if you are searching for something not in stock, then it’s not surprising you don’t find it.

But let me give credit where credit is due. The access methods that Amazon has built into its site are very impressive and increasingly sophisticated. As examples I looked for two obscure songs: Ken Boothe’s 1972 "Freedom Street" and Fontella Bass’s "To Be Free" (I couldn’t remember the title and had to browse) and found both quickly, complete with sound sample; I also tried to cheat by looking up "The The" – a band with the worst possible name in a Google-dominated world (try finding them) – but Amazon tracked them down.

If the thing you are looking for is a digital product (a category that is growing rapidly, including songs, beginning to include movies and books) then Amazon can take you to it. But if the thing you are looking for is not digital, as in Anderson’s example in Chapter 10 of looking for jam, then Amazon can’t take you to the product itself. It can take you to an image of it, and written material about it, but sometimes these are not enough. The virtual shelf is somewhere in between an actual shelf and a card catalogue. Some of the items on the virtual shelf are actual items, and others are cards that tell you "we can send you what’s described here".

An area not covered by Anderson is browsing – which is what many of us do when looking through music stores, bookstores, or video stores. Amazon is doing wonders improving its browsing experience (the ability to look inside the book being an obvious example), but it is still struggling uphill, I would say.

The Tyranny of Geography [162-164] returns to one of the book’s original claims, that geography limits variety: "in the tyranny of physical space, an audience too thinly spread is the same as no audience at all. Thus, local demand must be a a high enough concentration to compensate for the high costs of physical distribution. In other, more obvious words, not enough local demand equals no store" [163]. The shortfalls in this argument – the various workarounds that we employ, the role of cities in providing that variety and of communities in passing niche products around – were already highlighted by piefuchs in a comment on Chapter 2. It’s not that there’s no truth to Anderson’s claims, but it’s just irritating that he stacks the deck in his own argument’s favour even here. When looking at the restrictions on physical stores, he sets out the calculation of sales they can make this way [163]:

But for some specialist stores in big cities, it could be like this:

Variety in the physical world is, as discussed earlier, much more uneven than in the online world. This heterogeneous nature of the physical world does have some other benefits. A place that is a centre for jazz music stores is also likely to be a centre for jazz clubs in the evening, and to be a place where jazz musicians as well as jazz audiences tend to aggregate. The benefits of such cultural centres spill over from the consumption of culture into the production of culture, and then into the invention of new forms of culture. So while it is true that, just like catalogue shopping increased variety in a convenient way to small-town residents in years past, so online retailers "can reach all those many low-density towns as efficiently as the high-density ones, they can tap the Long Tail of distributed demand" [163-164], it is also true that the homogeneity of the online retail world may erode the vitality of those cultural centres that create the culture they sell.

The same is true – even more so – when it comes to international culture. The online world reaches small countries with the same ease that it reaches big countries, and promises to even out consumer access to cultural goods around the world. But this will do little to even out the production of cultural variety, because cultural variety is a product of the lumpiness of our world. It is the uneven concentrations of people, the barriers between one place and another — the fact that we are all isolated, to some degree, from people elsewhere in the world and so have to grow within our own communities  — that lead to cultural diversity, whether its the brass bands of coal mining towns in the North of England or cajun cooking or the Milan opera. Various forms of cultural protection – subsidies to cultural industries, restrictions on imports, national content regulations – have been successfully used all around the world to promote local culture, and so to maintain diversity in the face of economies of scale.

Cultural protection has something of a bad reputation these days. Opponents argue that any culture worth its salt will prosper in a global market, but such an argument neglects the economics of cultural goods, where marginal cost is zero. Any revenue that an American TV show sold to a smaller market gains is gravy for its producers, and as a result American shows will typically be far cheaper for TV stations to broadcast than local fare. Again, cultural production is caught in a vice, with one jaw being those economies of scale driving production geared "for an international audience" and the other jaw being the online world, where endless variety is present in principle but not always in practice.

Skipping ahead a little, Anderson complains that "the Long Tail doesn’t have a lobby, so all too often only the Short Head is heard" [167] in legislation of the Internet. But the Long Tail does have a lobby – all those national cultural industries lobbying for the promotion of local culture are part of a struggling Long Tail – but it’s not a lobby that gets much support from the Silicon Valley entrepreneurs who praise Anderson’s book. This should remind us that if we are looking for a route to a "niche driven culture" then Amazon and Netflix and Rhapsody and Apple’s iTunes are not the people we should be looking to as guides to take us there.

Scarce Air [164-166] is a return to another earlier topic: broadcast radio and TV. It adds little to the discussion in Chapter 2. Only one paragraph demands comment, and it’s the final one where Anderson laments ad clutter on television. In the US, apparently, "following deregulation in the mid-1980s, network TV ad time per hour increased from six minutes and forty-eight seconds in 1982 to twelve minutes and four seconds in 2001" [165] Why? Because viewers had nowhere to go.

The implicit claim is that audiences are now "starting to take back their attention" [166] and refusing to put up with this advertising overload. And yet it seems clear that the main way of making money from online activity has become, thanks to Google, the insertion of pervasive advertising content alongside and interspersed with content that we are actually interested in. To suggest that we are moving away from an advertising-driven culture is possible only with the blinkered view of cultural industries that characterise the whole book. But that’s part of his irrepressible ability to see the online world through rose-tinted glasses, a trait continued in the final section, which is…

The Dangers of Hitism [166-167] Here Anderson celebrates those "Kids who started using the Internet as twelve-year-olds in 1995 [and who] turned eighteen (the beginning of Nielsen’s 18-34 demographic that is highly coveted by advertisers). The males of the species, in particular, were watching less television. Given a choice between the infinite variety and easy ad-dodging online versus network TV, they were choosing the former… The audience is migrating away from broadcast to the Internet, where niche economics rule" [166].

It takes only a few moments of reflection to realise that the "infinite variety" of the online world was not the only place they were going. They were also spending many hours in the hit-driven world of video games, where endless variations on the theme of war games are aimed at precisely this 18-34 male audience. But Anderson is blind to all manifestations of hit driven culture that don’t fit his thesis.

It takes only a few more moments to realise that an environment funded by advertising in which ads are easily avoided will not last for long. Something has to change – either companies whose commercial model is based on advertising will fail or advertising will become more intrusive so we can’t avoid it.

The chapter as a whole does not, I would say, advance the Long Tail thesis. Instead, by returning to some of the topics from earlier in the book, it inadvertently opens our eyes to the complexities and subtleties of what it takes to make a world of variety. The Long Tail model of internet commerce is not up to the task of building such a world.

Hits really are as big as ever. Link: BBC NEWS | Entertainment | Record print run for final Potter.

A record 12 million copies of the final Harry Potter book are to be released in its first US print run.
Publisher Scholastic said July’s release of Harry Potter and the Deathly Hallows would be supported by a multi-million dollar marketing drive.

Meanwhile, Waterstone’s is getting caught in the big vice between the online retailers and the big grocery chains, (see previous post for how this has affeected CDs)

HMV is planning to close up to 30 of its Waterstone’s book shops, give
more space to higher margin items and reduce the number of high brow
books, as part of an overhaul to restore the fortunes of the struggling
business. Details of the restructuring were announced yesterday as HMV
issued yet another profits warning, sending its shares almost 16% lower
to 128.5p…

The books and music retailer is facing an onslaught from all sides; the
growth of digital music through the likes of iTunes, online retailers
led by Amazon and the big grocery chains such as Tesco and Asda [Wal-Mart] selling
discounted books and CDs.

This is another part of my critical reader’s companion to The Long Tail, and it discusses the first half of Chapter 9 – The Short Head. Part 0 is here. You can find a complete list of the Long Tail pieces here.

Before I start, a tip of the hat to Aaron Swartz for his kind review of No One Makes You Shop at Wal-Mart – even if he does think my politics are odd – and for pointing to this series of posts on Chris Anderson’s The Long Tail. And welcome to all those readers who have followed the link. Hope you can stick around – I’m going to be posting two or three things a week until I get to the end of the book, which will probably take a week or two.

The first half of this chapter compares shelves with online stores in a little more detail than earlier chapters, looking at "the virtues of shelves and their costs" [147] as well as the benefits of "the Internet, where niche economics rule" [166]. The second half (coming later this week) looks at how you find things. Let’s get to it.

The introductory section puts forward an idea that hits and niches can be complementary. The Long Tail is not against hits. Hits, it says, will always play an important role: they "serve as a source of common culture around which more narrowly targeted markets can form" [148].  Anderson goes on: "Successful Long Tail aggregators need to have both hits and niches. They need to span the full range of variety, from the broadest appeal to the narrowest, to be able to make the connections that can illuminate a path down the Long Tail that makes sense for everyone" [148].

"Long Tail" here has two separate meanings, not well distinguished. One is cultural – the claim that we are moving from "mass markets into millions of niches" [12]. The second is a more narrow business meaning – that you can build a successful online business if you do what Amazon has done so well. The book rarely mentions when it is moving back and forth between these meanings, but it’s worth remembering that they are not the same thing at all. We might get successful Long Tail businesses but not a Long Tail culture if online aggregators drive sources of variety out of business.

Let’s stick with the narrow business meaning for now. Pages 148-149 document the failure of one of the early online music ventures, MP3.com, because it had only niche music. "The reason MP3.com’s model didn’t succeed and the iTunes model — which is less oriented towards independent musicians – did is that iTunes began by making deals with major record labels, which gave it a critical mass of mainstream music. Then it added more and more niche content, as "rights aggregators" [not sure what that means – TS] shipped it hard drives full of hundreds of thousands of independent musicians [presumably their songs, not the musicians themselves – TS]. Thus, iTunes customers were able to dive into an already working market where the categories were defined by known commercial acts, which served as a natural leaping off point for the discover of niche music". [149]

There is an assumption by this stage in the book that iTunes is a "Long Tail aggregator". But is it? iTunes is mentioned on 30 separate occasions in the book, but the only actual figures about their sales are the following:

• "Apple said that every one of the then 1 million tracks in iTunes had sold at least once" [8]
• "As of early 2006, Apple had sold 42 million iPods and 1 billion tracks on iTunes" [175]

And that’s it. This deserves to be emphasized. The book never demonstrates that iTunes is a "Long Tail aggregator", so its repeated references to iTunes’ ability to (for example) "democratize distribution"[57 – table] and drive demand down the Long Tail are nowhere backed up.

iTunes has a lot of stuff, but does it sell it?

With Apple’s own figures being — like so many others on which the thesis of the book relies — secret, I can only stand on the outside and listen to those who claim to know. Here’s what I could find.
We already saw in the notes to Chapter 7 that, according to the Wall Street Journal, iTunes is a hits-driven business: "Every day, the roughly one million people who visit the iTunes Store home page are presented with several dozen albums, TV shows and movie downloads to consider buying —  out of the four million such goods the Apple site offers. This prime promotion is analogous to a CD being displayed at the checkout stands of all 940 Best Buy stores or featured on the front page of Target’s ad circular."
As I quoted in the notes to the Introduction, Lee Gomes wrote that "At Apple’s iTunes, one person who has seen the data — which Apple doesn’t disclose — said sales "closely track Billboard. It’s a hits business. The data tend to refute ‘The Long Tail.’ " One of the things about infinite shelves is that there’s no harm in saying "sure, bring in those hard drives with all those songs. Throw them on the shelf." That doesn’t mean they get sold.
Another claim, although also circumstantial, that iTunes is just as hit driven as physical stores was reported in the Guardian in August 2006:

At a recent debate hosted by digital music consultancy firm MusicAlly, eMusic’s chief executive David Pakman came up with a startling claim. He said that 10% of product on iTunes accounts for 90% of the store’s total sales. Rather than smashing the 80/20 rule, the world’s most popular download store appeared to be exacerbating it.

He goes on to say:

"For the most part, stores that are stocked with music from the majors tend to focus on mainstream music," he says. "It doesn’t mean they don’t carry a more diverse selection, but when you have Beyonce and the Pussycat Dolls you put them on the home page, and you market and promote that and use it on your advertising. By definition, that tends to attract youth and mainstream music buyers. They come to your site looking for the hits."

As a competitor who focuses on niche music, Pakman has a vested interest in pointing fingers at Apple. This should make us treat his statement with caution, but it also reminds us that we should be suspicious of the vested interests on the other side – those CEO’s who blurb Anderson’s books and who are looking to promote their businesses as revolutionary and shiny new. It is interesting to note that Pakman and his eMusic does believe in the Long Tail, but with a different slant from the one Anderson is giving it in these pages:

Consequently, 70% of eMusic’s catalogue sells more than once every quarter. It sells 5m tracks a month, from a subscriber base of only 200,000 users. "Our observation is that the Long Tail does exist," says Pakman, "but it doesn’t happen naturally. You have to focus on selling it, almost at the expense of promoting the popular stuff."

iTunes’ hits might be "a natural leaping off point for the discover of niche music", but there is little evidence that people are making the jump.

The Urban Tail [149-151] is one of those odd sections Anderson puts in from time to time where he wanders off to almost unrelated topics, seeing power laws everywhere. This doesn’t have much to do with the remainder of the book. Here it’s cities. "Another sort of ‘hit’ is major cities" [149] he says, pointing out that the distribution of city populations follows a power law distribution over many orders of magnitude. This is well known, of course. "In a sense, you can think of cities as the Long Tail of urban space in the same way the Internet is the Long Tail of idea space or cultural space" [150]. Hmmm. Not sure I know what that sentence means, but I’m sure it doesn’t add to the basic thesis of the book, so I’ll move on.

In Defense of Shelves [151-152] has a clear purpose. "Before we bury the shelf, let us first praise it" [151]. It provides some lovely little facts about just how detailed our knowledge of shelf space and how it works is. A nice couple of pages.

Rent by the Half Inch [152-154] starts the attempt to bury the shelf. And shelves do have their problems. Despite everything it’s pretty expensive to stock things on shelves, and you can only put an item in one place – a contrast to the digital world, where you can find this very page you are reading by searching for any number of the distinctive and elegant turns of phrase that grace its pixels. This section is a lead-in to the substance of the argument, so there’s not much more to say except to point out that Anderson here does his usual trick of comparing the digital world to only the most hit-driven part of the physical world and presenting the comparison as if it was a complete comparison of physical and digital worlds: "Today, the average cost of carrying a single DVD in a movie rental store is $22 a year. Only the most popular titles rent often enough to make that back (there’s a reason why they call it ‘Blockbuster’)." It is likely, of course, that any independent video store (hello Generation X) will have cheaper rent than a Blockbuster and so is more likely to stock minority interest content.

The Wal-Mart Effect [154-156], named after the excellent Charles Fishman book, continues this slanted comparison by focusing on the company with more shelves than anyone else in the world, and looking at its CD aisle. Wal-Mart stocks about 4,500 unique CD titles, as was noted in Chapter 1, compared to 800,000 for Amazon [155].  As Anderson points out, "Wal-Mart, which accounts for about one-fifth of all music sales in America, is by far the nation’s largest music retailer" [155]. "Today, the number of large independent music stores like the one I worked at has dropped dramatically, the classical music listening room is now an endangered species. There are, needless to say, few import aisles left" [155].

There is no doubt that Wal-Mart’s efforts have directly cut into the diversity of music available on those physical shelves. A 2004 article in Rolling Stone by Warren Cohen charts Wal-Mart’s growth and impact: "Unlike a typical Tower store, which stocks 60,000 titles, an average Wal-Mart carries about 5,000 CDs". (Tower has apparently gone bust since then – I’m writing from outside the US, so American readers please excuse my unfamiliarity with Tower). Even some well known albums are missing, Cohen points out in the same article: "At a Wal-Mart Supercenter in Thorton, Colorado, for example, there were no copies of the Rolling Stones’ Exile on Main Street or Nirvana’s Nevermind." Anderson makes a similar point, describing a walk he took around a Wal-Mart in Oakland, where "There are no copies of the Rolling Stones’ Exile on Main Street or Nirvana’s Nevermind". Oops! I guess Mr. Anderson has been reading Rolling Stone. Should have given Mr. Cohen some credit there.

But enough with the scurrilous suggestions – I do have a serious point to make about this section and it’s about that old Wal-Mart/digital comparison. Way back in Chapter 1 Anderson compared Wal-Mart’s 4500 CDs (60,000 tracks, he estimated) to Rhapsody, where "Not only is every one of Rhapsody’s top 60,000 tracks streamed at least once each month, but the same is true for its top 100,000, top 200,000, and top 400,000 — even its top 600,000, top 900,000, and beyond" [22]. The listing of increasing hundreds of thousands of tracks is clearly intended to contrast the paucity of choices available on Main Street with the digital Nirvana. It’s an intent made even clearer in his diagram on page 23 where he categorizes stuff you can’t buy at Wal-Mart as "Products you can’t find anywhere but online". But this description is false. The Rolling Stone article suggests that a specialist music store would stock 60,000 titles which, at the same tracks per CD ratio used by Anderson, is 800,000 tracks. Rhapsody’s selling is not, perhaps, so niche driven or abundant as we would think.

The big point about The Long Tail is a cultural one, and the moral of the story according to Anderson is that we are moving away from a world of hits to a world of niche interests. But that’s not the story this comparison tells. Instead, the story is of a retail model with a fair amount of effective diversity (the specialist record store) being squeezed in a technology-driven vice. One jaw of the vice is the technology-driven Wal-Mart, which uses its IT expertise and economies of scale along with other strategies to take the hits away from the specialists, driving them out of business. The other jaw is the online stores, which also benefit from economies of scale and are left to mop up any demand for diversity that is left, because it is no longer attainable in the physical world.

From what we can tell, iTunes is doing a poor job of selling niche products and, as the Guardian quotes Greg Eden, general manager of the trade organisation for the UK’s independent music sector:

unlike the high street, online distribution means one supplier can dominate the mass market. "What you see on the internet is very much the ‘Highlander Theory’," says Eden. "’There can be only one’. There’s only one search engine, there’s only one big book retailer, one big online auction house, and so on. .

The driving force of technology is not having effect Anderson would like it to have. There may be ways of generating more demand for out-of-the-mainstream tastes and a challenging, diverse culture, but Wal-Mart + iTunes isn’t it.

Next time, the second half of Chapter 9, which turns the topic to finding things on shelves.

This is another part of my critical reader’s companion to The Long Tail, and it discusses a piece of Chapter 8 – Long Tail Economics that I missed yesterday. Part 0 is here. You can find a complete list of the Long Tail pieces here.

I meant to write about page 141, but I didn’t.

What happens on page 141 of The Long Tail is that Chris Anderson acknowledges, for the first and only time in the book, how some word of mouth effects can run counter to the long tail forces he talks about. Dipper asked the question, back in comments on Chapter 1, "One reason people buy arts and cultural products such as books, dvd’s, games, clothes, cars is to participate in cultural life. that means buying what other people are buying". Anderson phrases the question here, in a section on music and how the one-big-powerlaw is composed of many little powerlaws:

The characteristic steep falloff shape of a popularity powerlaw comes from the effect of powerful word-of-mouth feedback loops that amplify consumer preference, making the reputation-rich even richer and the reputation-poor relatively poorer. Success breeds success. In network theory such positive feedback loops tend to create winner-take-all phenomena, which is another way of saying that they’re awesome hit-making machines.
Compounding matters, today’s filters make word-of-mouth even more powerful by measuring so much more of it from so many more people and for so many more products. Shouldn’t that then have the effect of making the powerlaw even steeper, increasing the gap between hits and niches rather than having a leveling effect?

I did wonder what Anderson was going to address this obvious counter-argument somewhere in his book, and this is the place he does so. It’s a very plausible argument – and one that I personally believe has some merit, especially given the shaky empirical support the book musters for an actual Long Tail shift. He addresses it in half a page. He argues that

"filters and other recommendation systems actually work most strongly at the niche level, within a genre and subgenre. But between genres their effect is more muted… Thus the most popular "ambient dub" artist at the very head of the ambient dub popularity curve can hugely outsell the others in that category, but that doesn’t mean the artist will snowball and tear up the charts to knock 50 Cent out of the top ten. The lesson from this microstructure analysis is that popularity exists at multiple scales, and ruling a clique doesn’t necessarily make you the homecoming queen."

And that’s it. There is no back-up in the end notes, not cases, no references to more detailed theoretical work, nothing. It is a tiny sketch of the beginnings of an argument, but it’s nothing more than that. Here are some obvious questions that come to my mind:

• If the effects of filters are muted between genres, why did you tell us about the great success of Touching the Void at the beginning of the book? Is that somehow different? Is there not a mountaineering genre in the book market? How does that fit with your argument here?
• If the effects of filters are muted between genres, where does that leave the mechanism for "pushing demand down the long tail"? Haven’t you just argued against the mechanism you promoted in the previous chapter?
• Do you have any data from any industry to back up these speculations?

There is no answer to any of these questions in this book. It reads as if Anderson has found a plausibility argument strong enough for himself, and has then dismissed the question and moved on. Too bad – I thought we were going to get some real argument here.

This is another part of my critical reader’s companion to The Long Tail, and it discusses Chapter 8 – Long Tail Economics. Part 0 is here. You can find a complete list of the Long Tail pieces here.

By this time in the book we’ve seen variants on the long tail graph a lot of times.
 

The chapter is a collection of observations about this shape. I don’t have a whole lot to say about this chapter, so this is a short post.

The chapter starts by talking about the Pareto 80/20 rule (that 20% of the people in populations Pareto studied owned 80% of the wealth) and Zipf’s Law (that the frequencies of use of words in the language follow a similar kind of fall-off). These and others are examples of power law distributions that crop up in a bunch of different places. Power laws are called long-tailed curves because "the amplitude … approaches but never reaches zero as the curve stretches out to infinity" [126]. This is true of many kinds of distributions, of course, including exponential and log-normal distributions. But even within the family of power-law curves, of the form

How Distribution Bottlenecks Distort Markets [127-130]. If you plot a power law on a log-log scale you get a straight line with slope -k. Anderson looks at some markets (movies on movie screens) and notices that they fall off below the line at a small value. He interprets this as a failure of markets to meet demand because of the cost structure of movie theatres. If there’s not enough demand to fill a theatre, films won’t get distribution. There’s some truth to this – there are costs to producing and distributing movies that demand a return on investment, and movies that don’t make the cut will not be distributed at all. In non-theatrical channels, where cost is less, more movies are available and watched. As the title of the section suggests, Anderson sees only the untruncated power law as somehow "natural" [127] and a reflection of consumer demand, and the truncated ones a measure of artificial scarcity caused by distribution bottlenecks, to be removed in the world of abundance. Yet even in the online world there are truncated distributions, they just occur at different places in the distribution chain. The small number of search engines or online auction houses are two examples. The idea that consumer demand is "naturally" a power law shape is unsubstantiated, at least in the pages of the book.

The strangeness of the comparison between theatres and rentals is
exemplified by this sentence: "demand keeps on going into niches that
were never even considered before — instructional videos, karaoke,
Turkish TV, you name it" [130]. I don’t know what he means by "never
even considered before" but I’m sure the reason instructional videos
were not made available on movie theatre screens was not so much "an
artifact of the traditional costs of offering them" but the fact that
the number of people who want to watch instructional videos in movie
theatres is probably vanishingly small no matter how large a geography
you spread them over. It’s like comparing the market for banquets to the market for snacks – they’re both food, but the demand pattern is different for one than for the other, and the limited demand for banquets is not "unnatural".

This section is one of those that come occasionally in the book,
containing a scattered set of only loosely related data points, with
sentences like "In books, Barnes and Noble found that the bottom 1.2
million titles represent just 1.7 percent of its in-store sales, but a
full 10 percent of its online (bn.com) sales. PRX, which licenses a
huge library of public radio programming online, reports that the
bottom 80 percent of its content now accounts for half of its
sales."[130] The "statistics" come from "direct personal correspondence
with their executives". Is this trustworthy data or is it picked to
illustrate a pre-conceived idea? What does PRX do and what is the
nature of its business? Who knows – that sentence is the sole reference
to them in the book. How does the Barnes and Noble statement fit with
Lee Gomes’ claim
in the Wall Street Journal that "The head of a major New York
publishing operation says that the
distribution of his titles is essentially the same in both online and
"bricks and mortar" channels." These isolated, context-free sentences
from executives seem to be the best we can do — actual company data
being tightly-held secrets.

I have little to say about the remainder of the chapter, The 80/20 Rule [130-135] is a meandering take on whether or not the Rule still applies. The other sections are Does a Longer Tail Mean a Shorter Head? [135-137], Does The Long Tail Increase Demand or Just Shift It? [137-138], Should Prices Rise or Fall Down the Long Tail? [133-139], "Microstructure" in the Long Tail? [139-142], The Long Tail of Time [142-143], and The Tragically Neglected Economics of Abundance [143-146]. As the titles suggest, this is a miscellany of observations about the shape of the graph. There’s not a whole lot to disagree with, but there’s not a whole lot of substance.

I could stop here… or I could just throw in a few things that irritate me, just to be grouchy. So don’t take this too seriously, but:

In the section on "Microstructure" [140] Anderson shows a graph of the
average sales for various of the many subgenres that Rhapsody divides
music into and claims that this selection shows "a Long Tail". Yet it
doesn’t. The graph does fall off left to right, as it cannot help but
do, but it is as close to a linear fall-off as a power law.

"One of the features of powerlaws is that they are ‘fractal’, which is to say that no matter how far you zoom in they still look like powerlaws" [138] Wrong. Straight lines still look like straight lines as you zoom in too, but that’s not enough to make a fractal. A fractal reveals new structure as you zoom in, and that structure is similar to the structure seen from further away.

"Einstein described time as the fourth dimension of space, you can think of it equally as the fourth dimension of the Long Tail" [142]. Wrong. It’s not the "fourth dimension of space", and the way he plots it is actually a third dimension of the Long Tail graph. Plus, it’s name dropping. This section, by the way, is devoted to the realization that "If you think about it, today’s hit is tomorrow’s niche", which is hardly shocking, but "Both hits and niches see their sales slow over time; hits may start higher, but they all end up down the Tail eventually. The research to quantify this conclusion is continuing" [142] Count me underwhelmed.

On abundance: "Abundance, like growth itself, is a force that is changing our world in ways that we experience every day, whether we have an equation to describe it or not" [146]. This is stuff that should have been thrown out during editing.

That’s all. More tomorrow, I hope.

This is another part of my critical reader’s companion to The Long Tail, and it discusses Chapter 7 – The New Tastemakers. Part 0 is here. You can find a complete list of the Long Tail pieces here.

In a comment on Chapter 6, John points us to an article which mentions Ranganathan’s Five Laws of Library Science.These laws are:

1.      Books are for use.   
2.      Every person his or her book.   
3.      Every book its reader.   
4.      Save the time of the reader.   
5.      The library is a growing organism.   

Laws 2 and 3 remind us that the whole publishing process is about guidance. Guiding readers to books they like, and guiding books to readers who like them. The various stages of writing, submitting to publishers, editing, publishing, publicising, reviewing, discovering, recommending, locating, browsing, requesting, buying/borrowing, delivering are one path through the maze that is needed to guide books to their readers and readers to their books. Guidance is the subject of Chapter 7.

I want to leap ahead some pages into the chapter this time. Because on at last, after a hundred and nine pages, we get some real evidence of a specific industry that is showing a big Long Tail effect – and it’s our old friend Netflix.

Filters rule [108-110] is a reflection of Anderson’s belief in the power of filters to "level the playing field, offering free marketing for films that can’t otherwise afford it,  and this spreading demand more evenly between hits and niches" [110]. The main effect, he says, is "to help people move from the world they know (‘hits’) to the world they don’t (‘niches’)" [109].

The evidence that he gives in this section in favour of the claim comes Reed Hastings, CEO of Netflix (and provider of a blurb for the book):

Historically BlockBuster has reported that about 90% of the movies they rent are new theatrical releases. Online they’re more niche: about 70% of what they rent from their website is new releases and about 30% is back catalog. That’s not true for Netflix. About 30% of what we rent is new releases and about 70% is back catalog and it’s not because we have a different subscriber. It’s because we create demand for content and we help you find great movies that you’ll really like. And we do it algorithmically, with recommendations and ratings.

In fact, since the book was published, Netflix have set up the Netflix Prize for people to suggest better algorithms. The first submission to improve the accuracy of the algorithm by 10% will win $1 million. So Netflix are taking this seriously. Finally, there appears to be some substance behind the shifting generalities of the book.

Or is there? Lee Gomes of the Wall Street Journal begs to differ.

Netflix defines "back catalog" expansively. A spokesman says it’s anything outside of the 50 or so DVDs getting heavy studio promotion at any given time. So even recent megahits like "Spiderman II" are in the back catalog.

What’s more, since Netflix rents 60,000 titles, it follows that those 50 titles — eight-tenths of 1% of inventory — generate 30% of all rentals.

This was the second of two Gomes articles on the Long Tail and part of a debate that focused on the extent of the effect that Anderson is describing. The first article was a review of the book, and in that review he referred to work by Harvard Business School’s Anita Elberse that only "shows a ‘slight shift’ toward the tail. But she also noted ‘a rapidly increasing number of titles that never, or very rarely, sell,’ which suggests ‘it is difficult for content providers to profit from the ‘tail.’ ". Again, a "slight shift" is not an "epochal shift". Anderson responded to that review, arguing that

As Professor Elberse told Gomes, she was only describing Nielsen VideoScan data, which is almost entirely taken from bricks-and-mortar sources. The Netflix data, which was the basis of the Long Tail analysis that she and I worked on together, tells a very different story (Elberse’s terms of data access don’t allow her to share that data; my terms allowed me to share what I published in the book). We both urged Gomes to make clear that the "slight shift" measured didn’t refer to the Netflix data that was at the core of the book’s conclusions. But he chose to make the point he wanted to make.

In comments to that post, Elberse herself stepped in to say that

You [Anderson] say "Nielsen VideoScan data (…) is almost entirely taken from bricks-and-mortar sources." I don’t think this is entirely correct. The VideoScan data reflect both offline and online sales, and actually break them down by channel. The breakdown is not as detailed as one might wish in an ideal world, but they do allow one to track whether, say, the share of offline sales go down over time. Therefore, I do think the fact that my colleague and I only observe a "slight" shift is meaningful.

In a letter published on Nicholas Carr’s weblog, Gomes said.

I’d like to correct an extremely serious misrepresentation Chris made at the end of his blog posting, to the effect that Anita Elberse of Harvard "urged" me not to characterize her work the way I did. This is manifestly false. Chris is either misremembering or deliberately conflating two separate issues. Prof. Elberse did indeed in an email remind me that the data she had for Netflix was under NDA, and I could thus not report it. But the comment had nothing to do with what Chris says it does. Let Prof. Elberse herself describe whether I got it right; below is the full text of an email she sent me after the story ran:

"I just read your article, and just wanted to thank you for being so careful in quoting me. I wish all journalists stayed this close to what was actually said! 🙂

"You did beat me ‘to the market’ with your article, but I hope our academic article (which should be ready in a few weeks) will further clarify the long tail phenomenon (or lack thereof)."

A lot of heat there. For anyone interested in more depth, I recommend you to Anita Elberse and Felix Oberholzer-Gee’s working paper "Superstars and Underdogs: An Examination of the Long Tail Phenomenon in Video Sales" (50 page PDF). Here is a paragraph from the abstract:

To shed light on this debate, we study the distribution of revenues across products in the
context of the U.S. home video industry for the 2000 to 2005 period. We find superstar and long-tail
effects in home video sales, but each effect comes with a twist. There is a long-tail effect in that the
number of titles that sell only a few copies every week increases almost twofold during our study
period. But at the same time, the number of non-selling titles rises rapidly; it is now four times as
high as in 2000. Many underdogs thus in fact appear to be losers. We also find evidence of a
superstar effect. Among the best-performing titles, an ever-smaller number of titles accounts for the
bulk of sales. The caveat here is that today’s superstars lack the punch of earlier generations: video
sales generally decrease over time across all quantiles of the sales distribution, but this effect is most
pronounced among best-selling titles. Our findings have important implications for entertainment
companies. Exploiting the tail might prove unprofitable if many titles do not sell at all. At the same
time, producing superstars is more difficult than ever. The trends we uncover thus point to
significant challenges for the entertainment industry.

In short, this most significant, most specific piece of information regarding an actual shift to long-tail behaviour, prompted and guided by Internet recommendation algorithms, turns out to be as insubstantial as other pieces of evidence. Filters don’t rule.

Well, with that over, let’s go back to the beginning. There is, we all know, a lot of stuff on the Internet, which is why the picture is so appealing. But as John pointed out in his Chapter 6 comment (and as I’ll say in more detail in Chapter 8), the tail has to be fat as well as long for the "theory" to make any sense. We also know that increasing returns and large fixed costs coupled with small marginal costs and freedom from geographical limitations gives the aggregators of the Internet (Amazon, Netflix, iTunes and the other handful of examples that Anderson returns to over and over again) the potential to be globe-straddling colossi, or oligonomies, to use Steve Hannaford’s word, who are oligopolies as far as customers go and oligopsonies as far as suppliers go. So the Internet is likely to have fewer vendors selling larger numbers of products than the brick & mortar world – One Big Virtual Tent rather than Many Small Tents. But where does this leave us when it comes to demand? It all depends on whether guidance on the Internet can help people find niche products in the Big Tent better than guidance in the physical world can help people find what they want in the Many Small Tents. The issue of guidance, then, is central.

Not only is it central, but it’s a particularly difficult problem for niche items. The problem of finding quality in "experience goods" – such as books, movies, and to some extent music – is a problem of asymmetric information. As such, it is prone to a form of market failure that goes under the name of the market for lemons. In short, the incentives are set up to encourage various forms of false reporting and gaming of the system (I have a strong incentive to give my own book a five-star rating on Amazon, for example). Knowing this, customers avoid those parts of the market where the asymmetry in information is strongest – the niches – and flee to those parts where information is most reliable – the hits. Predictability can drive out quality when information is scarce and unreliable. The market for lemons has been one of the most influential ideas in economics over the last several decades, its ramifications are ubiquitous, and it earned its inventor George Akerlof a Nobel Prize. But even though Anderson describes his book as "partly an economic research project" [11] there is no evidence this idea has troubled him, and it is not mentioned in his book.

Let’s look at how Anderson says guidance happens in the Long Tail world. Here’s the big picture, optimistic as ever:

Faith in advertising and the institutions that pay for it is waning, while faith in individuals is on the rise. Peers trust peers. Top-down messaging is losing traction, while bottom-up buzz is gaining power. Dell spends hundreds of millions each year on promoting its quality and customer service, but if you Google "dell hell" you’ll get 55,000 pages of results [click here to see why this number is wildly wrong]. Even the word "dell" returns customer complaints by the second page of results. The same inversion of power is now changing the marketing game for everything from individual products to people. The collective now controls the message….The new tastemakers are us… The ants have megaphones. [98-99]

But as Oligopoly Watch reminded us just   yesterday,   "while the myth is that the Internet represents an infinite array of shelves   (I think of   Borges’s library   of Babel) with everything democratically and randomly available, the real   world has a way of organizing things up front or way back, even when it’s all   cyberspace." Aggregators are One Big Virtual Tent, and vendors will scrabble   to get a place on the tables by the entrance. Oligopoly Watch quotes a Wall   Street Journal article on Apple’s iTunes (‘Music’s New Gatekeeper’) as saying   "Every day, the   roughly one million people who visit the iTunes Store home page are presented   with several dozen albums, TV shows and movie downloads to consider buying —   out of the four million such goods the Apple site offers. This prime promotion   is analogous to a CD being displayed at the checkout stands of all 940 Best   Buy stores or featured on the front page of Target’s ad   circular." Here is a way that Internet commerce, with its tendency to produce oligopolies, promotes uniformity rather than promoting diversity. Instead of many different store fronts, we have One Big Virtual Storefront. A high proportion of regular bookstores’ sales come from the highly-promoted items, and there is no reason to believe that aggregators’ sales will be different.

Bonnie McKee [98-103], My Chemical Romance [103-104] and BirdMonster [104-106] are three stories about different bands and their mixed experiences with online recommendation systems, social networking sites, and blogs to promote their music. While Anderson says that the stories show "how the three forces of the Long Tail are overturning the status quo in the music industry" [104]. A big statement, with little behind it. My Chemical Romance is a success, Bonnie McKee is struggling, and BirdMonster is a local band in San Fransisco. This is new? No.

The Power of Collective Intelligence [106-108] introduces us to the filter, which is "the catch-all phrase for recommendations and other tools that help you find quality in the Long Tail" [106]. Anderson is very enthusiastic about recommendation systems, saying that "the trend-watchers at Frog Design" see the rise of recommendations as "nothing less than an epochal shift" [107].

The adoption of recommendation systems on all kinds of web sites has been a boon to help promote worthwhile content and demote non-worthwhile content, and as they get more sophisticated they are a continuing innovation of great worth. But they set out to rectify a problem that is peculiar to the Internet, after all, which is the problem of anonymity and lack of trust. How do you establish trust in the online world? The existence of recommendations is a reflection of the fact that the Internet is handicapped when it comes to reliable and trustworthy communication. Recommendation systems are a great effort to overcome an obstacle that the physical world (which has its own problems) faces to a much smaller degree. When a friend recommends a book or movie to me, I have a reasonable idea of how to take that recommendation because I know my friend. When a book has a 3.5 rating on Amazon.com with three reviews, what am I to make of that? Was it friends of the author? Quite possibly.

The next section, "Filters Rule" was discussed at the top of the article.

One Size Filter Doesn’t Fit All [110-112] looks at various kinds of filter in more depth. It shows that filters are an increasingly sophisticated set of tools for aggregators to use to attract customers. But it does not show whether or not these filters are anything close to the power of our own offline networks of friends.

Not All Top Ten Lists are Created Equal [112-115] is more in a similar vein. Some online sites have developed filters that use an increasingly fine granularity and set of classifications of online lists. But can any algorithmic granularity capture our quirky tastes?

Is the Long Tail Full of Crap? [115-119] is a venture into explanation. It contains references to information theory, to zero sum games, to non-rivalrous goods, and to wide dynamic ranges. But they are name dropping, inserted to hint at a greater intellectual behind the Long Tail. It isn’t there. The point he is making is that, as Theodore Sturgeon [I always thought he was a creation of Kurt Vonnegut, but Wikipedia tells me that Vonnegut’s Kilgore Trout was based on the real-life Theodore Sturgeon] apparently said "ninety percent of everything is crud". But as long as you have unlimited shelf space, it doesn’t matter, because filters can help you find the good. There is an odd unsubstantiated claim that the material "in the tail" ranges from worse than that in the hits to better but, he goes on, "averages don’t matter. Diamonds can be found anywhere."[118].

The Tail That Wags Everything Else [119-122] is another slight section which reiterates the previous point: "As the Tail gets longer, the signal-to-noise ratio gets worse. Thus, the only way a consumer can maintain a consistently good enough signal to find what he or she wants is if the filters get increasingly powerful" [119]. It is out here in the tail, where comments on Amazon, links, and other filters are rare per item, that problems of asymmetric information are greatest, raising a further undiscussed barrier to those good quality items in the tail achieving recognition. So filters have a greater job to do, and are fighting an uphill battle. How do they do? Well I could use the Anderson method and point out that typing "ebay fraud" into Google gives [about] 4,160,000 hits – but in fact, Google estimates being what they are, the number is actually 780, so that tells us little. But see here, here, and here for articles about manipulating reputation systems – the last two, ironically, from the Anderson-edited Wired Magazine. The first article of the three, by Princeton University Computer Science Professor Ed Felten, says this:

There’s a myth floating around that such systems distill an uncannily accurate folk judgment from the votes submitted by millions of ordinary citizens. The wisdom of crowds, and all that. In fact, reputation systems are fraught with problems, and the most important systems survive because companies expend great effort to supplement the algorithms by investigating abuse and trying to compensate for it. eBay, for example, reportedly works very hard to fight abuse of its reputation system.

"The wisdom of crowds, and all that" just about summarizes Anderson’s sunny outlook on this promising but still-flawed, and perhaps unavoidably flawed, method for directing demand.

I should also point out that Anderson also briefly quotes[120] Nassim Taleb’s engaging book Fooled By Randomness, on the unpredictability of hits.

Pre-Filters and Post-Filters [122-124] distinguishes the "pre-filters" of the offline world which "filter before things get to market" and the recommendation and search technologies, or "post-filters" of the Internet. Here, then, are two models of guidance in an attempt to achieve that "every book its reader" ideal. Given the challenges faced by post-filters (the increasing "signal to noise" ratio in the tail discussed by Anderson, the potential for gaming the system, the market-for-lemons problem that particularly targets niche products in the face of that potential), it is good to see Anderson does acknowledge the difficulties: "Because post-filters tend to be amateurs, oftentimes that means less critical independence and more random malice." But in general he is remarkably sunny about the prospect for Internet recommendations to direct people reliably to even the deepest darkest corners of the One Big Virtual Tent. Filters are great, but they have a big job to do. Overall, I don’t share his optimism.