Before leaving for a holiday (it was lovely; thanks for asking) I was going through a trio of books on the topical topic of “prediction is difficult, especially of the future”. I decreed that Dan Gardner’s Future Babble was limited, but otherwise OK, and then deemed Tim Harford’s Adapt a failed attempt to justify free-market thinking in the aftermath of its biggest failure in decades.
So now it’s onto the final book of the trio, and my favourite by some distance. Australian Duncan Watts is a physicist-turned sociologist who now works! at! Yahoo! (I called him American a few posts ago. Thanks to Kevin Horgan for correcting me.) You may know Watts from such hits as his explication of network science Six Degrees of Freedom; his Music Lab experiments with Matthew Salganik and Peter Dodds (pdf) showing that social influence can overwhelm any special quality of particular songs in separating hits from misses; and his arguments against Malcolm Gladwell’s The Tipping Point. Each of these reappears in his latest book, Everything is Obvious (Once You Know the Answer) – henceforth EIO (home page).
(Attention conservation notice: for some reason this took for ever to write and I still don’t like it, but I said I’d post it so I’m damn well going to, and then I can move on.)
If I were writing for a real publication I’d try to be dispassionate and objective, but the fact is that EIO‘s goals are ones that I am hugely sympathetic to, and so I found myself cheering Watts along rather than reading him critically. I try not to do this, but between you and me and the wall we all do it to some extent. Certainly the reviewers of Adapt could have used a little less enthusiasm and a little more critical thinking before putting their praise in print. Still, I do think that EIO is the book that Adapt and Future Babble were trying to be.
According to its subtitle, EIO is about “how common sense fails us”, but this is misleading. “Common sense” has become a shorthand for anti-intellectual conservative approaches to government and social problems based on down-home practical experience, as in Common Sense Revolution, but Watts has a more ambitious target than that; “common sense” is the need (among intellectuals and others alike) to impose patterns on current events and on history, and our belief that once we see these patterns we will have a better grasp of the future.
On the one hand, the achievements of common sense are easily overlooked, but remarkable. Our socially-honed and culturally-specific intuitions are remarkably good at helping us to navigate the maze of implicit rules and norms that help society function. Common sense has proven subtle enough to derail whole avenues of artificial intelligence research.
On the other hand, the strengths of common sense in navigating the particulars of travelling the subway or getting by in the workplace are weaknesses when it comes to understanding anything other than everyday life. Common sense is “not so much a worldview as a grab bag of logically inconsistent, often contradictory beliefs, each of which seems right at the time but carries no guarantee of being right at any other time” (17). When it comes to understanding society, we appeal to it at our peril.
Watts’s most obvious target is the kind of thinking that is commonplace in business, in politics, and in punditry – analogies, stories, and overextended but half-baked theories. But as a sociologist, Watts also has in his sights the economists’ belief that sociology is plagued by woolly thinking, and that economists can do sociologists’ work more rigorously and with more insight than sociologists themselves because economists understand incentives. He quotes from Freakonomics: “The typical economist believes that the world has not yet invented a problem that he [sic] cannot fix if given a free hand to design the proper incentive scheme” (51).
The usual modern take against homo economicus theories of society is that we are not rational beings. But the problem is not just that “our mental model of individual behavior is systematically flawed”, although Watts does run through failures of framing, priming and other foibles of the Predictably Irrational kind. The problem I’ve always had with such psychological explanations of our failures is that, once we are made aware of our systematic flaws, we can surely work on overcoming them. Would that solve the problem?
Well no. And it’s not just that we sumble into fallacies of composition either, although he does show why intuitions and theories based on “representative individuals” whose “actions stand in for the actions and interactions of the many” are both tempting and doomed to fail. The focus on “special individuals” as the sparks that light the fires of viral success, Malcolm Gladwell’s “Law of the Few”, also comes in for debunking, including some convincing experiments based on Twitter cascades. And expertise as a whole is not of much use: Chapter 6 covers much of what’s in Future Babble. But the smartest and most self-confident will still see themselves, plausibly, as able to surmount these barriers, cutting through the deadwood to find real causes, real incentives, and real mechanisms.
Watts continues though. It is in his treatment of successes and failures that he makes his strongest arguments. Watts convincingly shows many suggested explanations of successes, from the Mona Lisa to Facebook, to be exercises in circular thinking, where “Harry Potter was successful because it had exactly the attributes of Harry Potter, and not something else” (60). It is tempting to think that there is something special about the Mona Lisa that causes it to be the most famous painting in the world, some quality that it has that no other painting has. But there isn’t.
In the Music Lab experiments, different “worlds” of listeners were given a set of songs to listen to and, in some cases, information on the listening history of others in their world. The experiment showed that songs that were hits in some worlds were unpopular in others, and that this variation came about specifically because of the information about others’ listening histories. Not only is it a mistake to look for Gladwell-like “special people”, it’s also a mistake to look for special qualities in hit songs as the “cause” of their success. As the fictional Mark Zuckerberg says to the Winklevoss twins in The Social Network, “if you had invented Facebook, you would have invented Facebook”. There was no crucial idea that separated Facebook from the pack; it just separated from the pack because social networks are governed by cumulative advantage, and that’s how cumulative advantage works.
And he doesn’t stop there. We’ve perhaps read before that “history is only run once”, causing us to mistakenly “perceive what actually happened as having been inevitable” (112), but Watts goes further in explaining why we misunderstand the causes of events: events themselves are identifiable only in hindsight. Was “the US financial crisis of 2008” a blip in the recent history of capitalism, or was it just the first part of “the international financial crisis of 2008 to 2015”, including the splitup of the euro zone and who knows what else? We can only know in hindsight.
In summary, Watts is saying not only that the “right lever” is difficult to find when it comes to understanding society, but that in many social phenomena there never was a lever at all; “what appear to us to be causal explanations are in fact just stories — descriptions of what happened that tell us little, if anything, about the mechanisms at work” (27).
There’s a lot here, and by two-thirds of the way through the book I was happily convinced that I would never again need to reach for a business strategy or populist social science book. It’s a thorough and convincing debunking of attempts to understand the chaotic path of social progress.
The difficulty, of course, is that once the Emperor’s nudity has been pointed out, the man still needs a set of clothes. What to put in the place of mistaken theories? Part II is an attempt to address this question, and it’s less successful than part I. There are three main chapters (Chapter 7 belongs, to my mind, in Part I rather than Part II). Chapter 8 covers much the same ground as Adapt, with some of the same suggestions: experiment, measure and react, rely on prediction as little as possible. Condensed into one chapter I didn’t mind this as much as I did in Adapt, but it’s still the weakest chapter in the book. It’s a smattering of ideas, some stronger than others, but still searching for that magic recipe which he has just convinced us doesn’t exist.
Chapter 9 is far more interesting though. If success and failure are governed largely by luck, how does that affect our views of society? Watts looks at theories of justice in the light of what Part I has told us, and comes out firmly in favour of an egalitarian, or at least Rawlsian, point of view. It’s just one chapter, and Watts would (I would guess) be the first to admit it’s just an introduction, but it’s a bold step for a book on popular social science to take, and a valuable one.
It’s enough that I forgive Watts his final chapter, which is really a set of reflections on physics and sociology that somehow forgets to mention that of all the natural sciences to base social theories on, especially when you have convinced us that specifics of particular situations are crucial, physics is surely the least plausible. It’s easy to forget how little predictive success physics has. Sure it does a good job with the tides, but even the best physics theories of complex phenomena (say the BCS theory of superconductivity for example, or the Kondo model) have little success when it comes to prediction. BCS did not guide or anticipate the discovery of high-temperature superconductors, for example.
After reading these three books I am left with several unanswered questions about how to proceed once we know that large parts of the future are hidden from view? When it comes to decisions, well we still need to make them (including big and irreversible decisions) even in the face of uncertainty, but can we still put heart and soul into new initiatives when we know that their outcome is, at least in part, out of our control? How do you motivate teams without misrepresenting the role of luck? How do we approach questions of reward and punishment when the consequences of our actions are unforeseeable? I’m sure there’s lots out there on these questions, and I think I’ll be looking for it with a new appreciation of their importance. Any suggestions on places to start?
In the case of the Mona Lisa It is not that the common “causal explanations” that account for the fame of the painting are “just stories”. In my opinion, STORIES are the causal explanation. The attributes of the Mona Lisa that account for the success are real and measurable, they just have nothing to do with the aesthetic attributes. The stories surrounding the painting involving the artist, the subject, and the history of the artwork itself are the important attributes. You could survey people who took a tour of the Louvre and determine what stories they remember most, what paintings resulted in conversations afterwards, how many social reminders (think Cadbury commercial) do people encounter.
It’s not a matter of luck, it’s a matter of picking the critical variables when dealing with a non-linear system. Checking your assumptions, constantly measuring, and all the tricks to remain agile (i.e. Adapt) are really ways of ensuring that the linear model devised to solve the non-linear system still holds true.
The tensile strength of steel is linear until temperatures get low enough that brittleness becomes the critical factor (as early steel ship builders learned the hard way). The first aluminum motorcycle frames were designed for tensile strength but failed due to fatigue (vibrations) so had to be redesigned with fatigue as the critical calculation. Engineering doesn’t demand an equation to work in every situation, it only demands to know the conditions under which it will work.
I think you can reframe any popular heuristic like unintended consequences and unknown unknowns into a framework of solving non-linear systems. One of the key aspects of reductionist science is holding most variables constant so that the experiment can be reproduced. Test tubes and vacuums are tricks to turn non-linear systems into linear ones.
Watts spends some time on the Mona Lisa. Turns out it only became the world’s most famous painting in the 20th century. In 1911 it was stolen from the Louvre by an Italian who wanted to repatriate it, and the French and Italians were both captivated by the theft. So the story catapulted it to fame, but not many of us know the story now (I certainly didn’t) but we all know the painting. Yes it’s a story, but it’s a 20th century story, and his main point is that it’s a fluke of history. I think that fits with what you say too.
I don’t think the Mona Lisa is a fluke of history. I certainly don’t think Leonardo da Vinci is a fluke of history. The smile is the story that most of us think about. But it doesn’t have to get that complicated, it really comes down to a Sesame Street “One of these things is not like the other” game. Take a bunch of da Vinci paintings and the Mona Lisa stands out as different. Take the most well known portraits of women in any format and I think the Mona Lisa will stand out as completely different and leave you scratching your head wondering “whats up with that?”.
The 1911 theft is certainly a fluke of history but I’m not sure it was a prerequisite for the smile meme taking off in the age of mass media. The Mona Lisa is an enigmatic painting by an extremely well known and productive artist whose other paintings are anything but enigmatic. This quality captures the imagination.
… but apparently this distinctive, enigmatic quality did not make Mona Lisa stand out from other Da Vinci paintings until the 20th century.
True enough, the Mona Lisa did not stand out until the 20th century… the number of reproductions did not take off until the 20th century either. I’m guessing photographic/printing technology had something to do with the “stand out” number of reproductions. 20th century mass media and popular culture are no different. I don’t think it is fair to compare a 20th century metric of success prior to the 20th century.
What was the metric used prior to the 1911 theft? Price, literary references?
I like the way you approach this subject, so I’ll just jump in:
While you cannot uphold a deterministic view where all your actions have foreseeable consequences when facing complexity I think that it is still very much viable to attempt the approximation of predictability. Even if a model is reduced to a probabilistic heuristic it is still better than no model (or random guessing) if it gives you better predictive power than chance. More importantly it gives you explanatory power that would be lacking otherwise.
The critical problem though is that while we see models that succeed in producing “better than nothing” results we are failing to discard models that don’t succeed. There is an epistemic reason to continue using the weather forecast. To continue using last centuries rational choice models that fail time and time again the reason is not that they are useful so much as that too many people have vested interests in preserving the institutionalized knowledge and the time and resources they poured into acquiring it. When stories are abused as an ex post rationalization they are detrimental to understanding the world. But that is not due to the properties of stories as such but rather to the inability of people to use them as helpful tools.
Stories by themselves are merely filters to manage the complexity of information, much like filtering algorithms, but it is their saliency in human communication that makes them a rather helpful tool for humans to understand issues (unlike mathematical algorithms). If they are used to describe the most relevant variables (much like RAD suggested) they are great to help people understand the world.
I recently wrote a blog entry that expands on this line of argument if you are interested:
http://blog.jochmann.me/post/7529461950/filter-bubble-complexity-psychology-internet-technodeter
(sorry for double posting, looks like the send action is messing with my feedreader)
There is, behind all this, the question of what is predictable and what is not (and Watts does address this). But the limits of expertise, whether cast as stories or not, are pretty severe when it comes to prediction. It is one thing to “understand” an event, as in “see how it could have happened”, but another thing to claim that the story tells us a cause-and-effect logic at work. I very much like Watts’s take on the difference, but didn’t explain it well here.
I agree that you can’t have a mass phenomenon without a mass media. The argument is that before its 1911 theft, “even when it was moved to the Louvre, after the French revolution, it did not attract as much attention as the works of other artists, like Esteban Murillo, Antonio do Corregio, Paolo Veronese, Jean-Baptiste Greuce, and Pierre Paul Prud’hon, names that for the most part are virtually unheard of today outside art history classes. And admired as he was, up until the 1850’s da Vinci was consideed no match for the true greats of painting, like Titian and Rafael, some of whose works were worth almost ten times as much as the Mona Lisa. [p 56]
Sure there are limits, but there are some frameworks that work better than monkeys throwing darts. The greater the scale of those predictions and the more clear the relevant starting variables, the better the predictive model. Black Swan events nonwithstanding some general trends can quite successfully be applied to predict future developments.
I think that the quality of solid models (or rather lack thereof) and research is the real culprit here. Prediction also is rarely done outside of the influence of policy makers and other particularist interests – which is why the pressure to develop more reliable models comes second to the pressure to develop models that support a certain agenda (even if just to preserve the reputation of institutionalized knowledge). Also for prediction to be useful it should fit in a timeframe in which action can be undertaken – again leading to less than optimal results, because the shorter the timeframe the less reliable the prediction. All in all I’d say that experts generally suck, because our society does not pose a favorable environment for better experts to appear, if you excuse the evolutionary analogy. Evolutionary theories by the way are yet another example of rather well deserved trust in their power – and yet you can hardly witness when the predictions turn out right given your life span.
RAD, part of the point is that there are many ways to be “not like the other”. And the one which becomes world-famous often has as much to do with having a good press agent as anything else (note, this is not saying that good press is all that matters, but instead, from a pool of possible choices, the winner may simply be one of a set of peers who happens to garner the most publicity or an initial boost which then snowballs).
There’s something of a logical paradox here, in that it’s very easy to say “The winner won because its qualities are the absolute best – just looks at how fantastic it is, that’s why it’s the winner”. The problem is that argument is tautological to the extent we’re conditioned to see those qualities as best because we already know it’s the winner.
Seth, I think you are inadvertently making part of my point. I claim that story attributes have as much, if not more, to do with the success of visual art than the artistic qualities do. The good press agent is all about the story.
The Mona Lisa is a story machine. Now if you and Tom want to claim that I only believe that the Mona Lisa is a story machine because it is well established as a story machine then at least we’re talking about the same thing.
Iconic images like the Mona Lisa, the Che Guevara photo, Marilyn Monroe’s blowing dress, and the Iwo Jima photo do have aesthetic qualities that set them apart. These qualities, however, have more to do with story enablement than composition/lighting/color qualities. In my opinion, our bias is in discounting the story attributes in comparison to the “artistic” attributes. I think the story qualities can be measured objectively by all kinds of different tests/surveys/experiments.
I hear what Watts/Slee/Finkelstein are trying to say to me. What I’m saying is that I don’t buy it unless an analysis is done on the “storiness” which I think is the critical variable in the case of the Mona Lisa.
I don’t think we’re quite talking about the same thing. Maybe what you call “story machine” might be “cultural symbol”. And the point being made is that potentially any of a range of cultural artifacts might serve the role – which specific one ends up doing so is more a product of luck and chance rather than it being somehow the best symbol(“story machine”?). This is not provable in an absolute sense. But there’s a lot of evidence that is suggestive, such as the experiments noted in the post above.
We are talking about the same thing, I’m just not articulating my point well. My general claim is that what we often attribute to randomness and chance is actually caused by a shift in the critical variables of a non-linear system. Non-linear systems are not perfectly predictable (non-deterministic) but if we pay careful attention to critical variables and the ranges/states in which they are valid then the misses add to our ability to predict moving forward.
The reason I’m harping about the Mona Lisa is that I don’t think most evaluations of the success of works of (visual) art consider the story attributes of the artwork while I think they are often critical.
The claim I often hear that VHS won the video format war despite being an inferior technology is another example. This claim is based on the fact that Betamax supports higher resolution images yet lost in the market. My claim is that tape length and/or price was the critical variable. A proper analysis, in my opinion, carefully states why a variable is assumed to be critical and this assumption is constantly re-evaluated for correctness.
Unless an analysis takes a non-linear systems view I am skeptical of the conclusions. The evidence and experiments mentioned do not take this approach. I think a non-linear systems approach is one possible solution to Tom’s quest to find predictability techniques that produce better than random outcomes.
I appreciate this comment is slightly (completely) off topic, so feel free to delete it.
I’d just like to suggest, based on your apparent interests and general approach, a book called ‘The Skeptical Economist’ by Jonathan Aldred. It has a similar approach to this book, regarding ‘incentives’ and the general worldview that economists have, and is a fantastic read.
Just a note “Evolutionary theories by the way are yet another example of rather well deserved trust in their power – and yet you can hardly witness when the predictions turn out right given your life span.”
Not at all. Drug-resistant bacteria are a spectacular human lifespan scale proof of evolution and an obvious prediction.
And arguably, the modern media system, properly understood (sensation and controversy over accuracy), is a pretty good metaphorical evolution proof of incentives producing pundit species to feed off them.
Just a quick and somewhat orthogonal point:
the Mona Lisa was already being singled out as singularly brilliant masterpiece by Walter Pater in his 1869 essay on DaVinci:
“She is older than the rocks among which she sits; like the vampire, she has been dead many times, and learned the secrets of the grave; and has been a diver in deep seas, and keeps their fallen day about her; and trafficked for strange webs with Eastern merchants: and, as Leda, was the mother of Helen of Troy, and, as Saint Anne, the mother of Mary; and all this has been to her but as the sound of lyres and flutes, and lives only in the delicacy with which it has moulded the changing lineaments, and tinged the eyelids and the hands. The fancy of a perpetual life, sweeping together ten thousand experiences, is an old one; and modern philosophy has conceived the idea of humanity as wrought upon by, and summing up in itself all modes of thought and life. Certainly Lady Lisa might stand as the embodiment of the old fancy, the symbol of the modern idea.”
So you are getting the history of the painting’s celebrity wrong if you think it only achieved superstardom in the 20th century.