Category: The Long Tail

This is another part of my critical reader’s companion to The Long Tail, and it discusses the first part of Chapter 11 – Living in a Niche Culture. Part 0 is here. You can find a complete list of the Long Tail pieces here.

We’re getting towards the end of
the book now, and it is moving on from the core material to some
speculative asides about broader issues. The material in this chapter
illustrates the reasons for the book’s success and also one of its most
pervasive weaknesses – which is all about the meaning of the title.
Throughout the book, Chris Anderson uses the phrase "The Long Tail" to
mean several different things.

1.     It’s a business model (create a profitable business by "selling less of more")
    
2. It’s a commentary on the benefits of Internet commerce (online commerce is more diverse than those old physical shelves)
    
3.     It’s a trendspotting book about changes to contemporary culture (the end of the hit parade, the fall of the blockbuster)

The shift back and forth between these different meanings is what makes
the book so frustrating. Anderson wants to eat his cake and have it too
– which is the theme for this posting.

When it suits, Anderson can be very cautious about what he means by
"The Long Tail". Witness two entries in his blog. First, here’s part of
his response to Lee Gomes’s article, which I’ve mentioned a few times:

First, the book doesn’t claim that there are any cases where sales of products not
available in the dominant bricks-and-mortar retailer in a sector (my
definition of "tail") are larger than the sales of products that are
available in that retailer ("head")… Which is why the language Gomes
cites from the book jacket is actually all phrased in the future
conditional tense ("What happens when the combined value of all the
millions of items that may sell only a few copies equals or exceeds the
value of a few items that sell millions each?").

That "future conditional" thing is really a bit precious isn’t it?

Or here, in a post he calls What The Long Tail Isn’t:

There are many distortions of the term, but the most common
one is to use it as a newly-positive synonym for "fringe"… for Long
Tail effects to work, you need both a head of relatively few hits and a tail of many niches, so that recommendations and other filters can lead consumers from one to the other.

A tail without a head is too noisy and apparently random to get
consumer traction; people need to start with the familiar and then
move, via trusted recommendations, to the unfamiliar. Likewise a head
without a tail is too limited in choice; the odds of finding a niche
you want are too low to bother exploring much beyond what you already
know.

Thus the two big Long Tail opportunities are:

1.     Aggregating hits and niches into a one big curve, from head to tail.
    
2.     Creating content and products that can plug into someone else’s aggregated curve.
    

He goes on to list some other things that "The Long Tail Isn’t":

•    Commodification – The LT is about nicheification, which is different.
     
   
•    Simple variety
  – Offering a few different choices or a bit of customization (like the
  sandwich filling options in the risible example above) is not enough.
  Long Tail effects kick in when you’re expanding variety and choice by
  orders of magnitude–from 10x to infinity.
     
   
•     The case for an all amateur, self-published future
  – The LT will probably have as much commercial content as ever. It will
  just be joined by far more amateur fare, forming a relatively seamless
  continuum from pros to ams.
     
   
•    The actual end of hits
  – The LT ends the tyranny of hits, shifting the market equally to
  niches. But it certainly acknowledges that some things will continue to
  be a lot more popular than others. Powerlaw distributions are as
  natural as diversity itself.
   
•    A focus on small markets at the exclusion of large ones – Again, you need both hits and niches to allow the filters and recommendation engines to work, driving demand down the curve from the known to the unknown.
     
   
•    Just any powerlaw
  – Powerlaws are ubiquitous. Long Tails are not. The first shows up
  anywhere you have variety, inequality, and network effects (word of
  mouth). The second requires massive variety and a wide range between
  the hits and niches. After all, many short tails are simply truncated
  powerlaw distributions. They just aren’t, er, long.
   

Mr. Anderson wants to have it both ways.

He wants to disown
the wilder uses of his catch phrase, ignoring the fact that the
catchiness is the core of its popularity and influence. Also, he is not
immune to misusing the phrase himself. He wants to say that "offering a
few choices or a bit of customization" is not Long Tail – yet he writes
about twenty varieties of flour as the "Long Tail of flour" [11], or
customized T-shirts as the "Long Tail of fashion " [50]. He disowns "a
focus on small markets at the exclusion of large ones", and yet writes
that microbreweries are the "Long Tail of beer" [50]. He says that
"just any powerlaw" is not a long tail and yet describes cities as "The
Long Tail of urban space" [150] and Al Qaeda as "The Long Tail of
national security" [50]? If it wasn’t for the aura of universality that
such misuses create then the thesis of the book would have attracted
much less attention than it has.

The first section of Chapter 11 is about The History of House Sound of Chicago,
a book by Stuart Cosgrove. The emergence of this form of music in the
early 1980s, from the embers of disco, is the story of a "proto-Long
Tail music culture" [177]. Mr. Anderson backs up the claim by reference
to the three forces of the Long Tail from Chapter 4:

• "It
  started with the spread of  affordable technology, from mixing decks to
  multitrack recorders. That’s the first force of the Long Tail, the
  democratization of the tools of production." [178]
• An explosion
  of records created the need for a way for people to get to them: "Which
  is exactly what clubs and warehouse parties offered, thus providing the
  necessary second force — democratized distribution" [179]
• The "hyperspecialized genres" of house music were identified by a
  proliferation of distinct record labels, sometimes owned by the same
  company: "Indie record labels are like tags, providing clues to which
  microgenre a track is likely to be" [179]. This "connects supply and
  demand" which is his third force.

Does Chicago House music have anything to do with the Long Tail? Well,
what I know about House music could fit on a single 45rpm label, but
the answer is clearly No.

The
story of the House Sound of Chicago is a familiar story about the
emergence of a subculture in a particular physical place and time – it
is an example of variety and minority-interest culture emerging from
the very offline, physical proximity whose limitations he decries in
the remainder of the book.

Looking back at the criteria for
something to be Long Tail, there is nothing here about either
"Aggregating hits and niches into a one big curve, from head to tail"
or "Creating content and products that can plug into someone else’s
aggregated curve." Whereas Anderson claims that The Long Tail is not "a
focus on small markets at the exclusion of large ones", the indie music
he is invoking did just that. As for his "three forces": folk music in
the broadest sense has always been produced by cheap technology, from
harmonicas to fiddles to guitars to whistles, so there is nothing new
there. Using clubs and parties as a way to share and provide access to
music? Been done for hundreds or thousands of years. And the idea of
labels as being "like tags" is simply a way of relating the culture of
music to the culture his core audience is familiar with, which is the
online computer world.

Elsewhere, Anderson laments that the physical world limits cultural
diversity because of what we might call the Bollywood Problem: culture
that appeals to a specific group of widely dispersed people has to
reach a certain threshold of appeal in a single locale before
being viable, and so might be viable nowhere (well, in North America
anyway). One of the many things he neglects about the cultural
phenomena at the core of his book is the fact that much of culture is
not a "market" with distinct supply and demand. Culture as a
participatory activity involves an intermingling of supply and demand,
and it is precisely the specific nature of physical environments that
leads to the development of new genres and flavours of music, whether
it’s Northern Soul or the collectivist music ethic that started in Montreal with Godspeed You! Black Emperor and has now given us the snow-influenced best band in the world and has also travelled down the 401 to Toronto to produce the almost-great Broken Social Scene.

From "Or" Culture to "And" Culture
[180-182] continues with the big statements that Anderson disowns
elsewhere, starting off with "The Long Tail is nothing more than
infinite choice" [180]. Once people get on the internet "they don’t
just go from one media outlet to another — they simply scatter.
Infinite choice equals ultimate fragmentation" [181]. Yet he
simultaneously wants to say that this is "simply a rebalancing of the
equation, an evolution from an ‘Or’ era of hits or niches (mainstream culture vs. subcultures) to an ‘And’ era… Mass culture will not fall, it will simply get less mass." [182]

So
if "Mass culture will not fall" and the Long Tail is not "The actual
end of hits" then why does he entitle a chapter "The Rise and Fall of
the Hit", or title sections "Who Killed the Hit Album?" [33] or "The
End of the Hit Parade" [31]? or write "We are turning from a mass
market back into a niche nation" [40]. Parsing "The End of the Hit
Parade" carefully (there is no tense, not even future conditional), you
could argue that ending the Hit Parade is different from Hits
themselves, but I think it’s fair to say, in a book as rhetorical as
this one, the author is out to create an impression that we are on the
edge of a bigger transition than a mere rebalancing of the equation.

I’d
like to say that Mr. Anderson needs to make up his mind, but plainly he
doesn’t need to at all. The ability to switch back and forth between
eschatology and business models is what gives the book its appeal, and
he’s done very well by it. The fact that it is inconsistent to the
point of foolishness appears not to matter. Too bad.

This set of posts has been a lot of fun. Thanks to those who have given me feedback along the way, and to those (like Dave of How To Furnish a Room and RAD) who have linked to it and said nice things about it. If you’re looking for a book to read, by the way, How To Furnish a Room is a great source of ideas.

I had some notes on each chapter and an idea of what I wanted to do with them once I finished reading The Long Tail, but it really is true that writing is a way of finding out what you think, so my views on the book have expanded along the way.

When I started out, I thought it would be enough to just go through the book as I have been doing, but now I don’t think that will quite do the trick. The book is really a Big Picture book, so once I get to the end (four more chapters) I think I’ll post a couple of Big Picture posts to wrap it up. Then I’ll really have to return the book to the library and read something else.

This is another part of my critical reader’s companion to The Long Tail, and it discusses Chapter 10 – The Paradise of Choice. Part 0 is here. You can find a complete list of the Long Tail pieces here.

The purpose of this chapter is to counter "the notion that ‘too much choice’ is overwhelming, a belief so common and ill-founded that it deserves its own chapter" [167].

The last 50 years have seen "an explosion of variety" [169], which Anderson ascribes to three factors. The first is "globalization and the hyperefficient supply chains it brings"; the second is the change in the population – in the US, he mentions ethnic diversity and increasing affluence, leading to a cultural shift from "I want to be normal" to "I want to be special"; the third is The Long Tail – or the proliferation of variety brought about by iTunes, Netflix, Amazon, and eBay.

But there’s variety and then there’s variety. Grocery shopping is often a focus for this kind of discussion (and the book uses it in the next section) so let’s use supermarkets as a way into the topic. For me, the proliferation of options at the cheese counter is a great thing: the range of options there seems to me a real variety. On the other hand, the toothpaste section (over forty different types of toothpaste) does not seem to be real variety. Now this could just be me, who likes cheese and can’t really be bothered about toothpaste as long as it does the job, but I think it’s more than that. There is another difference between the two, and it’s the issue of manufacturers or suppliers. The cheese counter contains products from a variety of producers, while the toothpaste shelves contain products mainly from two producers – Colgate-Palmolive has 37% of the American market; Procter & Gamble (makers of Crest and now merged with Gillette) follow with 34% – with a few "smaller" players like GlaxoSmithKline (Aquafresh) and Unilever filling out the remainder. If anything, the toothpaste market has become more dominated over the years by this small number of companies; meanwhile, the profusion of items on the shelf is growing. Oligopoly Watch (from a few years ago)  describes these twin trends of corporate consolidation and product profusion in the world of toothpaste. Is the combination an increase or a decrease in real "choice" and "variety"?

Anderson is as ebulliently optimistic as ever that the increase in number of products is simply a reflection of what we consumers want – "More choice really is better" [174], but very well-known research by Richard Schmalensee showed that the proliferation of toothpaste varieties may have little to do with responding to consumer demand, and be more about preventing other companies from getting a foothold in the market. Back in 1978 he looked at the ready-to-eat breakfast cereal market, where six companies dominated the market and introduced eighty different brands between 1950 and 1972 – more choice, for sure. Now Schmalensee is no radical – he was on Microsoft’s team in their anti-trust suit, arguing that Microsoft is not a monopoly – but he concluded that this particular case of product proliferation was all about trying to deter entry by other companies. The strategy is to "pack the product space and leave no profitable niche unfilled" (this from the standard graduate text by Tirole, p. 346). Schmalensee showed "how a cartel (a group of firms that act as a single monopolist) crowds a product space". The cartel in this case needs no explicit collusion, but the strategies of the incumbent firms complement each other to produce the same effect as a monopoly. The result – higher prices for consumers, as the incumbent firms can raise prices without new competitors coming to undercut them.

How much of the "explosion of variety" that we have experienced is real variety, and how much is entry deterrence and the ability to raise prices? It requires a lot more space and expertise than I have to pick apart what’s what in the myriad of different retail spaces, but it is something to think about when standing in front of a wall of shampoo, deodorant, mustard, breakfast cereal, or whatever. One would think that if The Long Tail is, among other things, an "economics research project" [11] and "a preview of 21st century economics" [11] then its author might have looked a little harder at the work of actual economists, however 20th century they might be.

Too Much Choice? [170-172] is a quick summary of one of the points made in Barry Schwartz’s excellent book The Paradox of Choice. It describes one well-known experiment that Schwartz talks about on pages 19-20, carried out originally by Sheena Iyengar of Columbia University. The experimenters set up a table at a specialty food store and offered customers a taste of a range of jams, and a $1 coupon to use against the purchase of a jam. When only 6 jams were put on the table, 30 percent of customers went on to buy a jar of jam; when 24 jams were put on the table, only 3 percent did. Faced with extra options, people shied away from choosing any of them. After presenting the experiment, Anderson goes on to say:

Schwartz describes the conclusion this way:

As the number of choices keeps growing, negative aspects of having a multitude of options begin to appear. As the number of choices grows further, the negatives escalate until we become overloaded. At this point, choice no longer liberates, but debilitates. It might even be said to tyrannize.

I found those sentences on page 2 of the book (The Long Tail does not indicate where they came from – it’s endnotes are lousy). They are separate from the jam experiment (pp 19-20). Schwartz’s book addresses a broad range of questions about the proliferation of choices that we are faced with, and while the sentences do reflect an argument that Schwartz makes, it oversimplifies his reaction to this particular experiment.

Anderson’s response to Schwarts is to say:

I’m skeptical. The alternative to letting people choose is choosing for them. The lessons of a century of retail science (along with the history of Soviet department stores) are that this is not what most consumers want [171].

His rebuttal does not do justice to The Paradox of Choice. Two sentences before the convenient quotation above, Schwartz says "When people have no choice, life is almost unbearable", and a few sentences after them he says "Choice is essential to autonomy, which is absolutely fundamental to well-being. Healthy people want and need to direct their own lives". So Schwartz is clearly not advocating Soviet department stores here, or the elimination of personal choice. The suggestion that the only options are "letting people choose" or "choosing for them" is a silly false dichotomy that I got accused of with my book too. There are many forms of choice, from individual consumer choice to our choices as citizens to collective choices – not just "choosing for them". Anderson is substituting a quick dig for real argument. It’s weak. But then, Schwartz’s book is detailed, subtle, and packed full of references to work by other people…

In fact, Schwartz’s book is more about how to cope with the profusion of choice than advocating a removal of it. One of eleven recommendations he makes is to "satisfice", but Anderson ignores the other ten and says: "Schwartz recommends that consumers ‘satisfice,’ in the jargon of social science, not ‘maximize’. In other words, they’d be happier if they just settled for what was in front of them rather than obsessing over whether something else might be even better" [171]. Contrast this with Schwartz’s actual definition of "satisfice" (p78):

To satisfice is to settle for something that is good enough and not worry about the possibility that there might be something better. A satisficer has criteria and standards. She searches until she finds an item that meets those standards, and at that point, she stops.

I don’t think Anderson’s paraphrase is a fair one, and I don’t think his arguments refute much of The Paradox of Choice. (They also don’t even start addressing most of the problems that are the subject of No One Makes You Shop at Wal-Mart, but that’s another story).

Variety is Not Enough [172-175] starts out by refuting Iyengar’s jam experiment. Anderson wrote to Iyengar and asked "why the people who should actually know the most about consumer choice in a supermarket were ignoring their [Iyengar and coworkers’] conclusions" [172]. He summarizes her reply (and a work in progress [PDF download]) as "The solution, they found, is to order the choice in ways that actually help the consumers" [173]. And with this, Anderson’s faith in limitless choice restored and he is off to find out how online resellers manage to guide people through their choices.

But just as Schwartz is more nuanced than Anderson gives him credit for, so Iyengar and coworkers are more nuanced in their conclusions. They distinguish "Preference Constructors" from "Preference Matchers". Preference Matchers are often experienced with the particular choice they are faced with (choosing a cup of Starbucks coffee is one they discuss); they are "someone who enters the decision making task looking for either a specific option or an option that possesses a number of pre-defined attributes. This type of chooser restricts the multitude of available options in a purchasing situation by creating “consideration sets,” or sets of options with a high probability of containing his or her optimal choice". Meanwhile "Preference Constructors" are those approaching the choice for the first time. Faced with a dizzying array (24 jams) such a chooser may shy away. They can be helped by providing some guidance: "To enable Preference Constructors to distinguish a hazelnut steamer from a vanilla chai and thus successfully form their preference criteria before identifying their preferred option, these decision makers may require more than a limitless, unstructured choice set." Here is the basis for Anderson’s optimism.

His contention is that online retailers are ideally placed to guide you through the choice in a comfortable way, so that us consumers are not faced with the 24-jam problem. But I’m just an old cynic. Whereas Chris Anderson is thankful for all this guidance he’s getting, I’m just suspicious. As George Akerlof wrote (and he knows about people guiding you through choices), the problem goes back as far as horse trading: "“if he wants   to sell that horse, do I really want to buy   it?” Such questioning is fundamental to the market for   horses and used cars, but it is also at least minimally present   in every market transaction."

Here’s Anderson’s take on how online retailers can guide people through a proliferation of choices.

There are a nearly infinite number [nitpick alert: "nearly infinite"??? – TS] of techniques to tap the latent information in a marketplace and make that selection process easier. You can sort by price, by ratings, by date, and by genre. You can read customer reviews. You can compare prices across products and, if you want, head off to Google to find out as much about the product as you can imagine. Recommendations suggest products that ‘people like you’ have been buying, and surprisingly enough, they’re often on-target. Even if you know nothing about the category, ranking best-sellers will reveal the most popular choice, which both makes selection easier and also tends to minimize post-sale regret. After all, if everyone else picked a given product, it can’t be that bad. [173]

I confess I did a double-take when I read that paragraph. The first half is confusing: is the goal of "supplementing an unstructured choice set" really accomplished by adding to that bewildering array a second bewildering array of different techniques to find out bits and pieces about the merits of different choices? I don’t think so. And the second half is a concise argument as to why online retailers so often fail to drive demand down to The Long Tail – in direct contrast to what he claims elsewhere. Anything that gets people to choose an item just because other people have chosen it leads to increasing returns. Once you’ve chosen it, it will be recommended to other people because you bought it, and so on. Note that without recommendations, you can’t even tell if they liked it or not once they bought it. And pointing people to best sellers because "it can’t be all that bad"? This is the logic of the blockbuster. Nevertheless, Anderson is so pleased with his conclusion that he reiterates it three times: "More choice really is better… The paradox of choice turned out to be more about the poverty of help in making that choice than a rejection of plenty. Order it wrong and choice is oppressive; order it right and it’s liberating" [174].

The other argument that Anderson has in favour of unrestricted choice is taken from libertarian journalist Virginia Postrel, and is again about help. It is true that, around most choices that are both expensive and complex, an industry of brokers rises up. Real estate agents, wedding planners, financial planners and so on, all "help us be ourselves" [174]. Another way of looking at such an industry is that it represents the cost of choosing (wedding planners are not free, after all): the rise of such an industry indicates that there is a cost to choice that we are prepared to pay substantially for, but this is in itself neither an argument for or against the proliferation of variety and pseudo-variety that we are immersed in.

The Economics of Variety [175-176] finishes the chapter by asking "Does more choice encourage consumers to buy more?" [175], and concluding that, by making choices easier, digital distribution "widens the field of possible customers and shortens the search time. Over time, this should increase sales and grow the overall market. As we saw in Chapter 8, longer tails can be thicker too" [176].

I’m not entirely sure what he means here by "the overall market", but if he is claiming – as I think he is – that the overall economy will grow because people will buy more stuff, then this is a claim that needs a lot of substantiation. The money, after all, has to come from somewhere. Unless he is advocating a further increase in consumer debt as a way of boosting the overall economy, then I’m not sure what his argument is.

But then that’s how I feel about the whole book, of course. The Long Tail sounds nice, and it would be great if online retail could not only break the tyranny of the blockbuster and promote a diverse niche culture, but also boost the economy at the same time – but just because it would be nice doesn’t make it so, and the book continues to fail to convince.

This is another part of my critical reader’s companion to The Long Tail, and it discusses the second half of Chapter 9 – The Short Head. Part 0 is here. You can find a complete list of the Long Tail pieces here.

The second half of Chapter 9 is a meandering stroll around the topic of "finding things". It’s an odd place for it in the book – this reads like a part of Chapter 7 – The New Tastemakers that has wandered into the wrong part of the book by mistake. Warning – this is even more rambling than the other posts in the series: it is more of a set of half-baked thoughts than a coherent thesis. But hey, this is a blog.

In the Library of Misshelved Books [156-160] starts off this way: "One of the most vexing problems with physical goods is that they force us into crude categorization and static taxonomies, as we saw with Wal-Mart. That means a windbreaker can be in the ‘Jackets’ section or the ‘Sports’ section, but not in a ‘Blue’ section or ‘Nylon’ section" [156]. A thing has to be in one place or another but not both, and so can be difficult to find. In the online world, things can be in many places at once, and this opens new doors: "The efficiency and success of online retail have illuminated the cost of traditional retail’s inflexibility and taxonomical oversimplification. It’s one thing to have high prices or limited selection; it’s quite another to simply be able to help people find what they want". [157]

Despite this start, the section is mainly about libraries. It describes the Dewey Decimal System for organizing books and its cultural biases and limitations. The numbers from 200 to 300, for example, are "Religion", but all religions apart from Christianity are lumped into the interval from 290 to 300. But although the cultural bias of the Dewey Decimal System is clear, its practical consequences are small because, as any library aficionado knows, the Dewey Decimal System goes on several places past the decimal point. As a result you can fit as many books between 290 and 300 as in any other interval, because  The Long Tail, for example, is "658.802 And"; No One Makes You Shop at Wal-Mart is "306.123 Sle". Anyone who has spent time in a library knows there are some areas where the numbers fly by and others where there are whole shelves that have the same number up to the second decimal place. But that’s OK, because additional numbers are free: you could say that the Dewey Decimal System exploits the free shelf space of numbers to extend into the Long Tail of digits. (Aaargh! I have to give Anderson this – the phrase is very infectious.) But anyway, once you have a number for a book, finding it on a shelf is rarely difficult and that’s the point.

Shopping in the Miscellaneous Aisle [160-162] starts off by admitting this: "In libraries, at least there is a standard categorization scheme — the card catalog is there to be searched, and librarians tend to know their stuff" [160]. Haven’t seen a card catalog in a while, but it is good to know that Chris Anderson and I agree there is little problem with libraries. So where is there a problem?

Well, it’s retail stores: those nylon jackets. And it’s true that it can be difficult finding things from time to time (wandering around Toys-R-Us with two pre-school kids was my worst experience, I think). But while this is a section I actually have more agreement with than most in the book, Anderson sets us up again. He complains how difficult it was to look for the Japanese anime classic "Akira" in a Blockbuster store, but "As it turned out, it didn’t matter – they didn’t have the film" [161]. He contrasts this experience with Amazon, where he types the title into the search bar and gets right to the film immediately, along with recommendations for other films. He also says all films were "in stock and cheaper than Blockbuster. The experience I had with these two stores couldn’t have been more different" [161] . He is exaggerating again. This section is about locating things, and if you are searching for something not in stock, then it’s not surprising you don’t find it.

But let me give credit where credit is due. The access methods that Amazon has built into its site are very impressive and increasingly sophisticated. As examples I looked for two obscure songs: Ken Boothe’s 1972 "Freedom Street" and Fontella Bass’s "To Be Free" (I couldn’t remember the title and had to browse) and found both quickly, complete with sound sample; I also tried to cheat by looking up "The The" – a band with the worst possible name in a Google-dominated world (try finding them) – but Amazon tracked them down.

If the thing you are looking for is a digital product (a category that is growing rapidly, including songs, beginning to include movies and books) then Amazon can take you to it. But if the thing you are looking for is not digital, as in Anderson’s example in Chapter 10 of looking for jam, then Amazon can’t take you to the product itself. It can take you to an image of it, and written material about it, but sometimes these are not enough. The virtual shelf is somewhere in between an actual shelf and a card catalogue. Some of the items on the virtual shelf are actual items, and others are cards that tell you "we can send you what’s described here".

An area not covered by Anderson is browsing – which is what many of us do when looking through music stores, bookstores, or video stores. Amazon is doing wonders improving its browsing experience (the ability to look inside the book being an obvious example), but it is still struggling uphill, I would say.

The Tyranny of Geography [162-164] returns to one of the book’s original claims, that geography limits variety: "in the tyranny of physical space, an audience too thinly spread is the same as no audience at all. Thus, local demand must be a a high enough concentration to compensate for the high costs of physical distribution. In other, more obvious words, not enough local demand equals no store" [163]. The shortfalls in this argument – the various workarounds that we employ, the role of cities in providing that variety and of communities in passing niche products around – were already highlighted by piefuchs in a comment on Chapter 2. It’s not that there’s no truth to Anderson’s claims, but it’s just irritating that he stacks the deck in his own argument’s favour even here. When looking at the restrictions on physical stores, he sets out the calculation of sales they can make this way [163]:

But for some specialist stores in big cities, it could be like this:

Variety in the physical world is, as discussed earlier, much more uneven than in the online world. This heterogeneous nature of the physical world does have some other benefits. A place that is a centre for jazz music stores is also likely to be a centre for jazz clubs in the evening, and to be a place where jazz musicians as well as jazz audiences tend to aggregate. The benefits of such cultural centres spill over from the consumption of culture into the production of culture, and then into the invention of new forms of culture. So while it is true that, just like catalogue shopping increased variety in a convenient way to small-town residents in years past, so online retailers "can reach all those many low-density towns as efficiently as the high-density ones, they can tap the Long Tail of distributed demand" [163-164], it is also true that the homogeneity of the online retail world may erode the vitality of those cultural centres that create the culture they sell.

The same is true – even more so – when it comes to international culture. The online world reaches small countries with the same ease that it reaches big countries, and promises to even out consumer access to cultural goods around the world. But this will do little to even out the production of cultural variety, because cultural variety is a product of the lumpiness of our world. It is the uneven concentrations of people, the barriers between one place and another — the fact that we are all isolated, to some degree, from people elsewhere in the world and so have to grow within our own communities  — that lead to cultural diversity, whether its the brass bands of coal mining towns in the North of England or cajun cooking or the Milan opera. Various forms of cultural protection – subsidies to cultural industries, restrictions on imports, national content regulations – have been successfully used all around the world to promote local culture, and so to maintain diversity in the face of economies of scale.

Cultural protection has something of a bad reputation these days. Opponents argue that any culture worth its salt will prosper in a global market, but such an argument neglects the economics of cultural goods, where marginal cost is zero. Any revenue that an American TV show sold to a smaller market gains is gravy for its producers, and as a result American shows will typically be far cheaper for TV stations to broadcast than local fare. Again, cultural production is caught in a vice, with one jaw being those economies of scale driving production geared "for an international audience" and the other jaw being the online world, where endless variety is present in principle but not always in practice.

Skipping ahead a little, Anderson complains that "the Long Tail doesn’t have a lobby, so all too often only the Short Head is heard" [167] in legislation of the Internet. But the Long Tail does have a lobby – all those national cultural industries lobbying for the promotion of local culture are part of a struggling Long Tail – but it’s not a lobby that gets much support from the Silicon Valley entrepreneurs who praise Anderson’s book. This should remind us that if we are looking for a route to a "niche driven culture" then Amazon and Netflix and Rhapsody and Apple’s iTunes are not the people we should be looking to as guides to take us there.

Scarce Air [164-166] is a return to another earlier topic: broadcast radio and TV. It adds little to the discussion in Chapter 2. Only one paragraph demands comment, and it’s the final one where Anderson laments ad clutter on television. In the US, apparently, "following deregulation in the mid-1980s, network TV ad time per hour increased from six minutes and forty-eight seconds in 1982 to twelve minutes and four seconds in 2001" [165] Why? Because viewers had nowhere to go.

The implicit claim is that audiences are now "starting to take back their attention" [166] and refusing to put up with this advertising overload. And yet it seems clear that the main way of making money from online activity has become, thanks to Google, the insertion of pervasive advertising content alongside and interspersed with content that we are actually interested in. To suggest that we are moving away from an advertising-driven culture is possible only with the blinkered view of cultural industries that characterise the whole book. But that’s part of his irrepressible ability to see the online world through rose-tinted glasses, a trait continued in the final section, which is…

The Dangers of Hitism [166-167] Here Anderson celebrates those "Kids who started using the Internet as twelve-year-olds in 1995 [and who] turned eighteen (the beginning of Nielsen’s 18-34 demographic that is highly coveted by advertisers). The males of the species, in particular, were watching less television. Given a choice between the infinite variety and easy ad-dodging online versus network TV, they were choosing the former… The audience is migrating away from broadcast to the Internet, where niche economics rule" [166].

It takes only a few moments of reflection to realise that the "infinite variety" of the online world was not the only place they were going. They were also spending many hours in the hit-driven world of video games, where endless variations on the theme of war games are aimed at precisely this 18-34 male audience. But Anderson is blind to all manifestations of hit driven culture that don’t fit his thesis.

It takes only a few more moments to realise that an environment funded by advertising in which ads are easily avoided will not last for long. Something has to change – either companies whose commercial model is based on advertising will fail or advertising will become more intrusive so we can’t avoid it.

The chapter as a whole does not, I would say, advance the Long Tail thesis. Instead, by returning to some of the topics from earlier in the book, it inadvertently opens our eyes to the complexities and subtleties of what it takes to make a world of variety. The Long Tail model of internet commerce is not up to the task of building such a world.

Hits really are as big as ever. Link: BBC NEWS | Entertainment | Record print run for final Potter.

A record 12 million copies of the final Harry Potter book are to be released in its first US print run.
Publisher Scholastic said July’s release of Harry Potter and the Deathly Hallows would be supported by a multi-million dollar marketing drive.

Meanwhile, Waterstone’s is getting caught in the big vice between the online retailers and the big grocery chains, (see previous post for how this has affeected CDs)

HMV is planning to close up to 30 of its Waterstone’s book shops, give
more space to higher margin items and reduce the number of high brow
books, as part of an overhaul to restore the fortunes of the struggling
business. Details of the restructuring were announced yesterday as HMV
issued yet another profits warning, sending its shares almost 16% lower
to 128.5p…

The books and music retailer is facing an onslaught from all sides; the
growth of digital music through the likes of iTunes, online retailers
led by Amazon and the big grocery chains such as Tesco and Asda [Wal-Mart] selling
discounted books and CDs.

This is another part of my critical reader’s companion to The Long Tail, and it discusses the first half of Chapter 9 – The Short Head. Part 0 is here. You can find a complete list of the Long Tail pieces here.

Before I start, a tip of the hat to Aaron Swartz for his kind review of No One Makes You Shop at Wal-Mart – even if he does think my politics are odd – and for pointing to this series of posts on Chris Anderson’s The Long Tail. And welcome to all those readers who have followed the link. Hope you can stick around – I’m going to be posting two or three things a week until I get to the end of the book, which will probably take a week or two.

The first half of this chapter compares shelves with online stores in a little more detail than earlier chapters, looking at "the virtues of shelves and their costs" [147] as well as the benefits of "the Internet, where niche economics rule" [166]. The second half (coming later this week) looks at how you find things. Let’s get to it.

The introductory section puts forward an idea that hits and niches can be complementary. The Long Tail is not against hits. Hits, it says, will always play an important role: they "serve as a source of common culture around which more narrowly targeted markets can form" [148].  Anderson goes on: "Successful Long Tail aggregators need to have both hits and niches. They need to span the full range of variety, from the broadest appeal to the narrowest, to be able to make the connections that can illuminate a path down the Long Tail that makes sense for everyone" [148].

"Long Tail" here has two separate meanings, not well distinguished. One is cultural – the claim that we are moving from "mass markets into millions of niches" [12]. The second is a more narrow business meaning – that you can build a successful online business if you do what Amazon has done so well. The book rarely mentions when it is moving back and forth between these meanings, but it’s worth remembering that they are not the same thing at all. We might get successful Long Tail businesses but not a Long Tail culture if online aggregators drive sources of variety out of business.

Let’s stick with the narrow business meaning for now. Pages 148-149 document the failure of one of the early online music ventures, MP3.com, because it had only niche music. "The reason MP3.com’s model didn’t succeed and the iTunes model — which is less oriented towards independent musicians – did is that iTunes began by making deals with major record labels, which gave it a critical mass of mainstream music. Then it added more and more niche content, as "rights aggregators" [not sure what that means – TS] shipped it hard drives full of hundreds of thousands of independent musicians [presumably their songs, not the musicians themselves – TS]. Thus, iTunes customers were able to dive into an already working market where the categories were defined by known commercial acts, which served as a natural leaping off point for the discover of niche music". [149]

There is an assumption by this stage in the book that iTunes is a "Long Tail aggregator". But is it? iTunes is mentioned on 30 separate occasions in the book, but the only actual figures about their sales are the following:

• "Apple said that every one of the then 1 million tracks in iTunes had sold at least once" [8]
• "As of early 2006, Apple had sold 42 million iPods and 1 billion tracks on iTunes" [175]

And that’s it. This deserves to be emphasized. The book never demonstrates that iTunes is a "Long Tail aggregator", so its repeated references to iTunes’ ability to (for example) "democratize distribution"[57 – table] and drive demand down the Long Tail are nowhere backed up.

iTunes has a lot of stuff, but does it sell it?

With Apple’s own figures being — like so many others on which the thesis of the book relies — secret, I can only stand on the outside and listen to those who claim to know. Here’s what I could find.
We already saw in the notes to Chapter 7 that, according to the Wall Street Journal, iTunes is a hits-driven business: "Every day, the roughly one million people who visit the iTunes Store home page are presented with several dozen albums, TV shows and movie downloads to consider buying —  out of the four million such goods the Apple site offers. This prime promotion is analogous to a CD being displayed at the checkout stands of all 940 Best Buy stores or featured on the front page of Target’s ad circular."
As I quoted in the notes to the Introduction, Lee Gomes wrote that "At Apple’s iTunes, one person who has seen the data — which Apple doesn’t disclose — said sales "closely track Billboard. It’s a hits business. The data tend to refute ‘The Long Tail.’ " One of the things about infinite shelves is that there’s no harm in saying "sure, bring in those hard drives with all those songs. Throw them on the shelf." That doesn’t mean they get sold.
Another claim, although also circumstantial, that iTunes is just as hit driven as physical stores was reported in the Guardian in August 2006:

At a recent debate hosted by digital music consultancy firm MusicAlly, eMusic’s chief executive David Pakman came up with a startling claim. He said that 10% of product on iTunes accounts for 90% of the store’s total sales. Rather than smashing the 80/20 rule, the world’s most popular download store appeared to be exacerbating it.

He goes on to say:

"For the most part, stores that are stocked with music from the majors tend to focus on mainstream music," he says. "It doesn’t mean they don’t carry a more diverse selection, but when you have Beyonce and the Pussycat Dolls you put them on the home page, and you market and promote that and use it on your advertising. By definition, that tends to attract youth and mainstream music buyers. They come to your site looking for the hits."

As a competitor who focuses on niche music, Pakman has a vested interest in pointing fingers at Apple. This should make us treat his statement with caution, but it also reminds us that we should be suspicious of the vested interests on the other side – those CEO’s who blurb Anderson’s books and who are looking to promote their businesses as revolutionary and shiny new. It is interesting to note that Pakman and his eMusic does believe in the Long Tail, but with a different slant from the one Anderson is giving it in these pages:

Consequently, 70% of eMusic’s catalogue sells more than once every quarter. It sells 5m tracks a month, from a subscriber base of only 200,000 users. "Our observation is that the Long Tail does exist," says Pakman, "but it doesn’t happen naturally. You have to focus on selling it, almost at the expense of promoting the popular stuff."

iTunes’ hits might be "a natural leaping off point for the discover of niche music", but there is little evidence that people are making the jump.

The Urban Tail [149-151] is one of those odd sections Anderson puts in from time to time where he wanders off to almost unrelated topics, seeing power laws everywhere. This doesn’t have much to do with the remainder of the book. Here it’s cities. "Another sort of ‘hit’ is major cities" [149] he says, pointing out that the distribution of city populations follows a power law distribution over many orders of magnitude. This is well known, of course. "In a sense, you can think of cities as the Long Tail of urban space in the same way the Internet is the Long Tail of idea space or cultural space" [150]. Hmmm. Not sure I know what that sentence means, but I’m sure it doesn’t add to the basic thesis of the book, so I’ll move on.

In Defense of Shelves [151-152] has a clear purpose. "Before we bury the shelf, let us first praise it" [151]. It provides some lovely little facts about just how detailed our knowledge of shelf space and how it works is. A nice couple of pages.

Rent by the Half Inch [152-154] starts the attempt to bury the shelf. And shelves do have their problems. Despite everything it’s pretty expensive to stock things on shelves, and you can only put an item in one place – a contrast to the digital world, where you can find this very page you are reading by searching for any number of the distinctive and elegant turns of phrase that grace its pixels. This section is a lead-in to the substance of the argument, so there’s not much more to say except to point out that Anderson here does his usual trick of comparing the digital world to only the most hit-driven part of the physical world and presenting the comparison as if it was a complete comparison of physical and digital worlds: "Today, the average cost of carrying a single DVD in a movie rental store is $22 a year. Only the most popular titles rent often enough to make that back (there’s a reason why they call it ‘Blockbuster’)." It is likely, of course, that any independent video store (hello Generation X) will have cheaper rent than a Blockbuster and so is more likely to stock minority interest content.

The Wal-Mart Effect [154-156], named after the excellent Charles Fishman book, continues this slanted comparison by focusing on the company with more shelves than anyone else in the world, and looking at its CD aisle. Wal-Mart stocks about 4,500 unique CD titles, as was noted in Chapter 1, compared to 800,000 for Amazon [155].  As Anderson points out, "Wal-Mart, which accounts for about one-fifth of all music sales in America, is by far the nation’s largest music retailer" [155]. "Today, the number of large independent music stores like the one I worked at has dropped dramatically, the classical music listening room is now an endangered species. There are, needless to say, few import aisles left" [155].

There is no doubt that Wal-Mart’s efforts have directly cut into the diversity of music available on those physical shelves. A 2004 article in Rolling Stone by Warren Cohen charts Wal-Mart’s growth and impact: "Unlike a typical Tower store, which stocks 60,000 titles, an average Wal-Mart carries about 5,000 CDs". (Tower has apparently gone bust since then – I’m writing from outside the US, so American readers please excuse my unfamiliarity with Tower). Even some well known albums are missing, Cohen points out in the same article: "At a Wal-Mart Supercenter in Thorton, Colorado, for example, there were no copies of the Rolling Stones’ Exile on Main Street or Nirvana’s Nevermind." Anderson makes a similar point, describing a walk he took around a Wal-Mart in Oakland, where "There are no copies of the Rolling Stones’ Exile on Main Street or Nirvana’s Nevermind". Oops! I guess Mr. Anderson has been reading Rolling Stone. Should have given Mr. Cohen some credit there.

But enough with the scurrilous suggestions – I do have a serious point to make about this section and it’s about that old Wal-Mart/digital comparison. Way back in Chapter 1 Anderson compared Wal-Mart’s 4500 CDs (60,000 tracks, he estimated) to Rhapsody, where "Not only is every one of Rhapsody’s top 60,000 tracks streamed at least once each month, but the same is true for its top 100,000, top 200,000, and top 400,000 — even its top 600,000, top 900,000, and beyond" [22]. The listing of increasing hundreds of thousands of tracks is clearly intended to contrast the paucity of choices available on Main Street with the digital Nirvana. It’s an intent made even clearer in his diagram on page 23 where he categorizes stuff you can’t buy at Wal-Mart as "Products you can’t find anywhere but online". But this description is false. The Rolling Stone article suggests that a specialist music store would stock 60,000 titles which, at the same tracks per CD ratio used by Anderson, is 800,000 tracks. Rhapsody’s selling is not, perhaps, so niche driven or abundant as we would think.

The big point about The Long Tail is a cultural one, and the moral of the story according to Anderson is that we are moving away from a world of hits to a world of niche interests. But that’s not the story this comparison tells. Instead, the story is of a retail model with a fair amount of effective diversity (the specialist record store) being squeezed in a technology-driven vice. One jaw of the vice is the technology-driven Wal-Mart, which uses its IT expertise and economies of scale along with other strategies to take the hits away from the specialists, driving them out of business. The other jaw is the online stores, which also benefit from economies of scale and are left to mop up any demand for diversity that is left, because it is no longer attainable in the physical world.

From what we can tell, iTunes is doing a poor job of selling niche products and, as the Guardian quotes Greg Eden, general manager of the trade organisation for the UK’s independent music sector:

unlike the high street, online distribution means one supplier can dominate the mass market. "What you see on the internet is very much the ‘Highlander Theory’," says Eden. "’There can be only one’. There’s only one search engine, there’s only one big book retailer, one big online auction house, and so on. .

The driving force of technology is not having effect Anderson would like it to have. There may be ways of generating more demand for out-of-the-mainstream tastes and a challenging, diverse culture, but Wal-Mart + iTunes isn’t it.

Next time, the second half of Chapter 9, which turns the topic to finding things on shelves.

This is another part of my critical reader’s companion to The Long Tail, and it discusses a piece of Chapter 8 – Long Tail Economics that I missed yesterday. Part 0 is here. You can find a complete list of the Long Tail pieces here.

I meant to write about page 141, but I didn’t.

What happens on page 141 of The Long Tail is that Chris Anderson acknowledges, for the first and only time in the book, how some word of mouth effects can run counter to the long tail forces he talks about. Dipper asked the question, back in comments on Chapter 1, "One reason people buy arts and cultural products such as books, dvd’s, games, clothes, cars is to participate in cultural life. that means buying what other people are buying". Anderson phrases the question here, in a section on music and how the one-big-powerlaw is composed of many little powerlaws:

The characteristic steep falloff shape of a popularity powerlaw comes from the effect of powerful word-of-mouth feedback loops that amplify consumer preference, making the reputation-rich even richer and the reputation-poor relatively poorer. Success breeds success. In network theory such positive feedback loops tend to create winner-take-all phenomena, which is another way of saying that they’re awesome hit-making machines.
Compounding matters, today’s filters make word-of-mouth even more powerful by measuring so much more of it from so many more people and for so many more products. Shouldn’t that then have the effect of making the powerlaw even steeper, increasing the gap between hits and niches rather than having a leveling effect?

I did wonder what Anderson was going to address this obvious counter-argument somewhere in his book, and this is the place he does so. It’s a very plausible argument – and one that I personally believe has some merit, especially given the shaky empirical support the book musters for an actual Long Tail shift. He addresses it in half a page. He argues that

"filters and other recommendation systems actually work most strongly at the niche level, within a genre and subgenre. But between genres their effect is more muted… Thus the most popular "ambient dub" artist at the very head of the ambient dub popularity curve can hugely outsell the others in that category, but that doesn’t mean the artist will snowball and tear up the charts to knock 50 Cent out of the top ten. The lesson from this microstructure analysis is that popularity exists at multiple scales, and ruling a clique doesn’t necessarily make you the homecoming queen."

And that’s it. There is no back-up in the end notes, not cases, no references to more detailed theoretical work, nothing. It is a tiny sketch of the beginnings of an argument, but it’s nothing more than that. Here are some obvious questions that come to my mind:

• If the effects of filters are muted between genres, why did you tell us about the great success of Touching the Void at the beginning of the book? Is that somehow different? Is there not a mountaineering genre in the book market? How does that fit with your argument here?
• If the effects of filters are muted between genres, where does that leave the mechanism for "pushing demand down the long tail"? Haven’t you just argued against the mechanism you promoted in the previous chapter?
• Do you have any data from any industry to back up these speculations?

There is no answer to any of these questions in this book. It reads as if Anderson has found a plausibility argument strong enough for himself, and has then dismissed the question and moved on. Too bad – I thought we were going to get some real argument here.