What Cascade Theories Don’t Tell Us

Attention conservation notice: 3,000 words of amateur rumination on the problems with agent-based theories of uprisings. Part of a series about identity, institutions, and uprisings.

In the torrent of debate over the causes and dynamics of the "Arab Spring" uprisings, one of the strongest currents has its source in cascade models of uprisings. The starting points for these  models are Mark Granovetter's simple and abstract Threshold Models of Collective Behaviour and Thomas Schelling's similar ideas in "Macromotives and Microbehaviour". If a population of individuals is presented with a choice between A and B, and if each will choose A only if some number of other people (their "threshold") also choose A, then very small differences in the distribution of thresholds can lead to very different results. In some cases, everyone chooses A, in others everyone chooses B, in others, the population is split. One spark sputters; an identical spark starts a prairie fire.

Timur Kuran and Susanne Lohmann used these ideas to explain the sudden and surprising uprisings of 1989 in Eastern Europe. Both drew attention to the paucity of information in an authoritarian state, about what others believe and about the nature of the state itself. Highly motivated protesters with a low threshold engage in dissent or protest and thereby reveal information to other disgruntled people about the breadth of disenchantment, or about the nature of the state, or about the experience of other individuals in relation to the state. Others join in and by doing so reveal more. And so it goes. With the right distribution of thresholds, a single person's action can light a fire that sweeps across a continent. Sudden, dramatic uprisings in authoritarian states are a switch from one equilibrium to another as dissent draws back the veil of silence and people see each other, see the nature of the regime under which they live, and realize that everyone else sees it too.

Digital cascades

A new twist has been added to the cascade mix by the 2009 revolt in Iran and the "Arab Spring" uprisings that erupted in Tunisia two years ago. The highly-publicised role of social media posed new challenges for theories of contentious politics, and three related facets of the events of Spring 2011 made cascade theories appealing:

  • The sudden, unexpected nature of the uprisings and their swift spread from country to country bring to mind the Eastern European regimes in 1989-1990.
  • The lack of a strong, organized opposition during the early stages of the uprisings is a challenge for theories of social movements, and plays to the strengths of cascade models.
  • The prominent role of digital technology, and particularly social media, lends itself to network-based models of society.

Cascade models suggest that contentious politics is an assurance game that can be solved so long as the transaction costs associated with information exchange are low enough. According to a popular narrative, social media networks lower those costs, rendering newly vulnerable those states that rely for their stability on information scarcity and on preference falsification. In an information-rich world, the low-information equilibrium is no longer tenable.

Cascade models have, well, cascaded out from academia and into the mainstream world as the primary way to understand the Arab Spring uprisings. For one influential statement, see Clay Shirky in Foreign Affairs, but a quick look through a bibliography of Arab Spring literature (herehere) will reveal many more.

As the current gained strength, it absorbed a number of tributaries. One was the Habermasian idea that a rich public sphere or a strong civil society is the fertile ground in which discontent can take root. Another stream focuses on the role of information and the media, bringing communication studies and media studies scholars into the cascade current. The end result is a current that has brought to the fore the role of social media in the Arab Spring, flowing into the sea of mainstream opinion. If you follow the River Nile of commentary about "Twitter Revolution" and "Facebook Revolution" and "Revolution 2.0" upstream far enough you reaach the theoretical source: Granovetter and Schelling, Kuran and Lohmann.

But it's a long way from Granovetter to the Arab Spring, and along the way important things have been lost, and Mistakes Have Been Made.

Caught in the Net

One of the lovely things about cascade theories is that they lend themselves to simulations. Take that idea of a threshold, for example. Doesn't it make sense that your threshold for action depends more on the actions of people in your social networks than people in some distant city that you've never met? So you can start doing computer simulations of cascades, and how they depend on network structure, and how they depend on the density of connections, and… well, the possibilities are almost endless. So network models have become a common way to extend the original simple ideas in more sophisticated ways. New hypotheses get expressed as network models, and the conclusions feed back into the theory.

But all the simulations in the world don't change the fact that adopting a network model is an input to the theory, not an output, and that the network society perspective brings with it a whole set of assumptions and priorities that need examination. For example, adopting a network model means relegating organizations and institutions to the periphery, and moving ideas of "self-organization", connectivity, and peer-to-peer communication to the centre — not because of any factual conclusion but because some concepts can be expressed naturally within a network model and others can't. Concepts of symbolism, identity, institutions, and the difficulty of establishing trust are hard to express and so get pushed aside or ignored completely. The popularity of agent-based models doesn't disprove the importance of such concepts, it just makes us blind to them.

Information and symbolism

Just as the cascade point of view has taken on the "network society", so it has highlighted the role of information in authoritarian states, and hence the role of media and technology in fostering change. It's true that information in authoritarian states is limited, almost by definition, but it's a long way from that to the idea that the paucity of common knowledge is what is holding back uprisings. Again, it's easy to express ideas of information flow across networks within agent-based models, but that doesn't actually tell us that information flow in authoritarian societies is the barrier to social change. Yes, there is something of interest in the lens that information provides, but treating it as the only lens of interest is too narrow.

Once you start swimming in the cascade current, you find yourself surrounded by like-minded individuals. It's no surprise that agent-based theories have found fertile ground among those interested in technology. After all, technophiles are typically happy with a focus on the role of information, tend to have a mathematical cast of mind, but are less happy with sociological concepts or social-historical investigations of social movements. Economic concepts and techniques find a welcome home (transaction costs and utility functions) while talk of collective identity is more difficult to translate into agent models. To be blunt, adopting the agent-based outlook saves you a lot of work, because you don't have to read all those historical studies of movements and organizations and make up your mind about the issues they raise.

That there is something of a cultural divide between the agent-based modelers and other currents of the debate, such as those who study social movements, is not new to me of course: here is Andrew Walder:

The field of social movements and contentious politics has been a prolonged effort to establish a sociological alternative to the more parsimonious theories of economics. The increasing insistence on the subjective dimensions of mobilization—collective action frames, the formation of collective identities, the role of emotions—is essentially motivated by a feeling that the initial emphasis on organizations, networks, and political opportunity structures were not sufficiently different from rational social choice models to offer a fully sociological alternative.

Protests and other acts of dissent share many common features. For one, they are often deeply symbolic acts. Protesters choose symbolic days on which to make their point (the 40th anniversary of the GDR), they focus their actions on national symbols (the storming of the National Palace by the Sandinistas). Presenting protest as information-revelation ignores the symbolic nature of protests, the importance of demonstrating "worthiness, unity, numbers, and commitment", the practice of drawing from a shared repertoire of actions, which other currents of thought have shown to be important. At least, my reading of the literature suggests that there is a lot of verbiage around the basic theory, and sometimes in this verbiage there is reference to the symbolic nature of the protests, but there's little to actually tie the words to the theory.

If scholars of social movements have demonstrated that identity and its formation are important motivators for political action, and if these factors appear nowhere in the cascade models that seek to explain sudden political change, then there's a gap in the theory.

Preference falsification, or obedience?

In Timur Kuran's work, the central concept of "preference falsification" is grounded in the distinction between an internal, private viewpoint and an externally expressed opinion. There is, however, a difficulty with ascribing political function to an internal, psychological state of mind. A diagnosis of "preference falsification" is only possible after the fact of an uprising. Before the fact, from a political point of view there is no observable difference between a nation of preference falsifiers and a nation of contented citizens, and to the extent that there is any observable difference between the two, preference falsification loses its explanatory power. Similarly, what is the observable difference before the uprising between Lohmann's East German citizens, who each have bad experiences with their government but are unaware of its overall bad performance, and an East Germany of citizens who have had good experiences with their government?

The concept of preference falsification makes it dangerously easy to read one's own assumptions and views into the lives of others. The danger becomes apparent in Kuran's book-length exploration of preference falsification. He treats affirmative action in American society as a case of preference falsification, in which the "real" feelings of the silent majority were (are?) opposed to affirmative action, but were not voiced because "to voice misgivings is to invite censure. Conscious of the risks, Americans have tended to hide their reservations behind a veneer of public consent" (p 222), a claim that many would consider exaggerated, at least. He is quick to interpret an incorrect poll on the eve of Nicaragua's 1990 election as evidence of oppression by the Sandinistas over the previous decade. And while Kuran spends much time on preference falsification under Soviet Bloc communism, he has nothing to say about 1930s Fascist Europe. What does the concept of preference falsification have to say about lack of successful internal uprisings against the governments of Nazi Germany and Fascist Italy? Were these cases of falsified preferences that did not have a chance to be unfalsified, or did the populace "really believe" in the official ideology? And is the distinction between the two politically meaningful or did Hanna Arendt have it right when she wrote that "politics is not like the nursery; in politics obedience and support are the same".

The Arendt quotation comes from a persuasive essay by Xavier Marquez, "On the Meaning of Political Support". Marquez goes on to quote Robert F. Worth of the New York Times writing about the fall of Gaddafi: "Everyone in Tripoli, it seemed, had been with Qaddafi, at least for show; and now everyone was against him." Here is Marquez at length:

Were these people deceiving themselves or others? Did the soldiers really support Gaddafi in the past but now do not? Do some of these people support Gaddafi still? The question makes less sense to me than it once did. It is clear that they once obeyed Gaddafi and now do not… but to attempt to determine if, in their heart of hearts, these people supported Gaddafi then (net of all of these forces) and now do not seems slightly absurd. Their obedience and disobedience, support and lack of support are nothing but the vector product of all the forces (threats of coercion, positive incentives, beliefs about Gaddafi, idiosyncratic likes and dislikes, moral convictions, obscure and half-formed ideas about the future, etc.) operating through them. It may make sense to attempt to disentangle these forces if we are interested in legal or moral responsibility, or in the private tragedies of everyday life in Libya, but it does not make sense to me to attempt to figure out if Gaddafi enjoyed some "genuine" level of support (independent of coercion, money, etc.) as a separate explanatory factor.

It's not that there is nothing to preference falsification, but it has come to occupy a status of orthodoxy, and it has pushed other mechanisms into undeserved obscurity. In Steven Pfaff's excellent "Exit-Voice Dynamics and the Collapse of East Germany", which presents a rich and multifaceted account of the events of 1989, I could not help but see a tension: it seemed to me that he falls back on preference falsification as explanation more often than he would like to (although he does point to other mechanisms) because of the absence of alternative frameworks in which to think.

Safety in numbers?

Finally, somewhere in the middle of most cascade models there is a "safety in numbers" assumption. Lohmann (on p92 of her major article) writes that the cost of protest "is assumed to be decreasing in the turnout… This assumption can be motivated with the 'safety in numbers' characteristic of the technology of suppression: given the amout of resources a regime devotes to suppressing mass protest, it is plausible that a higher number of activists is associated with a lower likelihood that any one activist will experience injury, death, or imprisonment." Kuran writes "The external payoff to siding with the opposition… is apt to become increasingly favorable… with S (the size of public opposition). The larger S, the smaller the individual dissenter's chances of being persecuted for his identification with the opposition."

This assumption is weak. An action that the government could afford to ignore during a period of political calm cannot be ignored if it gets big enough. Tienanmen Square did not become a safer place the more people gathered, and neither does Tahrir Square: perhaps quite the opposite. Many protests are deliberately risk-seeking, actively looking to provoke a response from the government. As Gandhi said, "The function of a civil resistance is to provoke response and we will continue to provoke until they respond or change the law."

Do people join large protests because it is safer than joining small protests? Or do they join them because, for example, large protests simply matter more? When a country polarizes, the central issues become deeper dividing lines between supporters of the status quo and those in opposition. The question "which side are you on?" becomes one that has to be answered.

Wrapping Up

What I've tried to say is that there are underexplored weaknesses in the cascade theory description of uprisings which have continuing impacts on current debates, not only in the academic world, but in the wider world – even including the decisions and actions of dissidents in perilous circumstances. The weaknesses lie in the notions of preference falsification, in assumptions about safety in numbers, in assumptions built into the network models constructed on top of cascade models, and in blind spots regarding the forms of political action, the importance of symbols, the roles of institutions, the formation of identity, and other factors that shape the events around sudden uprisings. The effects of these weaknesses are amplified by the enthusiastic adoption of cascade models, supplemented by loose analogies and anecdotes as if they form part of the theory itself, and have a distorting effect on our understanding of these events, making it easy to see some patterns and difficult to see others. Of course, it would be easier to counter the influence of information-driven cascade theories if there were alternative approaches that also reproduced the dramatic "cascade" results. More on that next time.

 Org version 7.8.10 with Emacs version 23

When Theories Matter: Uprisings in Authoritarian States

(Second in a series of hopefully accessible posts about this hard-to-read paper).

From time to time, sitting in a comfortable chair with a cup of coffee reading disputes about Twitter Revolts and Facebook Revolutions, it is easy to think that The Argument is the Thing. But it isn't, of course. The public profile of these debates about how digital technologies intertwine with dissent in authoritarian states, sprawling from the pages of the New Yorker to Foreign Policy, from specialized academic journals to urgent pamphlets, means that the arguments may influence the choices of dissidents operating in perilous environments; may sway them one way or another as they make life-changing decisions. So the least we can do, even those of us on the fringes of the debates, is to try for the truth.

Particularly strange, perhaps, is that these disputes are unavoidably theoretical. Of course, it matters greatly to tell a coherent and accurate story of how events played out in each particular case, but the implications of the debates are most urgent for uprisings that have not yet happened and for protests that have not yet been organized. No matter how exhaustively one recounts the unfolding of events in Tunisia in 2011 and 2012, such a telling alone cannot provide guidance to dissidents in Azerbaijan, in Russia, or in Canada for that matter. We cannot avoid theory: we point to a set of mechanisms and say "this happened here because of these conditions and for these reasons. In other conditions in different places, here are the possible outcomes". Talk turns inevitably to what is primary and what is mere epiphenomenon. We tell stories that highlight what seems essential and draw attention away from the factors that we deem unimportant. We theorize, and we tell stories based on those theories. We retell anecdotes that encapsulate those theories to prove our points.

When it comes to questions of digital technologies and their roles in political change, and particularly dissent in authoritarian states, there is a whole vocabulary that has come into being that carries along a set of narratives. There are assumptions behind terms such as "digital activist", "internet freedom", and "network society" that carry over into the stories and the habitual grooves of thought that we take with us as we try to understand new developments.

One current that has been particularly influential in the debates around the "Arab Spring" uprisings is based on the idea of informational cascades, and this current provides the theory behind much of the optimistic talk about Facebook revolutions and the potential for digital technologies to undermine authoritarian regimes. It's not the only theory of uprisings, but it is a compelling one. So if you're going to tell a different story about digital technologies, you need a different theory. I went looking for what that theory should be, and didn't find one, so I put one together myself, and that's what these posts are about.

Next: a look at information cascades and what they leave out.

Writing Towards Zero

Five months have passed since I posted here because I found myself drawn to finishing an earlier project. I thought it would take a couple of weeks but it ended up taking five months. The end result is called "Identity, Institutions, and Uprisings", is available at SSRN, and over the next few weeks I'm going to provide a more accessible, more political, and less academic version of the project here. I suspect the potential audience is vanishingly small but hey, I'm pleased with it. I'll start tomorrow.

Open Data at Crooked Timber

Prompted by the flurry of activity around here, Henry Farrell of the highly-regarded Crooked Timber blog has organized a seminar on Open Data. In Timber-speak a seminar is a series of posts over the course of a week or so by a variety of guest bloggers, together with comments from the CT crowd, who are a very smart crowd indeed. So I'm thrilled at the seminar, and even more thrilled to be the first contributor; my contribution is Seeing Like a Geek.

Open Data Movement Redux: Tribes and Contradictions

1 Introduction

I have two things to say to those who responded to Why the ‘Open Data Movement’ is a Joke:

  1. Thank you for putting so much effort into providing such thoughtful, reflective, articulate affirmations of your point of view. You gave me (and others, I hope) a lot to think about, and a lot to read over the last several days.
  2. Unfortunately, you’re still wrong.

The original post was written in the heat of the moment, so here is a more detailed and considered, and therefore almost certainly less-likely-to-be-read, argument about the contradictions and problems of the “Open Data Movement”.

2 The Open Government Data Landscape

First, here is a map of Open-Government-Dataland (click for a larger popup).

Ogdlandscape

The longitude, marked across the x axis, indicates the impact of the data itself. The line x=0 is the Yu-Robinson Meridian1 and separates Open-Government-Dataland into an eastern and a western hemisphere, each populated by a spectrum of data. Harlan Yu and David G. Robinson explain the hemispheres this way:

“A machine-readable bus schedule aims to promote convenience, commerce and a higher quality of life—it enhances service delivery. Disclosures of public contracting opportunities play a dual role, potentially enhancing both economic opportunity and public integrity. And core civic data, such as legislative or campaign finance information, serves a more purely civic role, enhancing transparency.”

Tom Lee describes these two hemispheres as “‘open-as-in-data.gov’ and ‘open-as-in-FOIA'”.

The latitude, marked along the y axis, marks the incentive of the data users. Travelling from south to north along a meridian takes us from the purely commercial activities in the south to the purely non-commercial in the relatively poor northern lands. At y=0 we cross the Bates Parallel, named after Jo Bates who has this to say about the conflict between the commercial and non-commercial hemispheres:2

“Interviews and observations suggest that some at the periphery of the OGD [Open Government Data] initiative have tended to conceptualise OGD as being about small start-ups, voluntary ‘civic hackers’, and other micro/small enterprises. This is unsurprising given the heavy weighting towards micro/small businesses in the UK’s IT sector and the large number of ‘civic hackers’ active in the OGD community; however, the potential re-user industry for OGD is broader than this. The PSI [Public Sector Information] re-use industry comprises a range of industries and includes multinational corporations (MNCs) such as Google and LexisNexis, conglomerates such as Daily Mail and General Trust whose DMG information division is the parent company of the UK based Landmark Information Group, as well as an array of SMEs, micro enterprises, independent developers, and voluntary ‘civic hackers’.”

While this landscape is rich, the inhabitants of different quadrants of the map can be distinguished easily:

  • In the harsh mountains of the North-West3 live the hardy non-commercial “civic hackers” whose diet consists of transit timetables, weather forecasts, and other pragmatic, useful data obtained, often piece-by-piece, from municipal governments. The culture of the civic hackers displays a combination of civic goals and enjoyment in the intrinsic interest of programming, with a twist of start-up mentality. It is a tribe described vividly by David Eaves4: “geek, technically inclined, leaning left, and socially minded. There are many who don’t fit that profile, but that is probably the average.” This tribe includes organizations such as Code for America (“A new kind of public service”) as well as small and loosely-coordinated groups of individuals working at a local level and who identify with their role as citizens over their role as consumers.
  • The North-Eastern tundra is the home of civil liberties activists, a resourceful tribe bent on promoting government transparency. It is driven by a desire to make information such as lobbying activities, campaign funding, government operations, and legal statutes open and accessible. In the USA you will find organizations such as Sunlight Foundation (“Making Government Transparent and Accountable”) and Public.Resource.Org (“Making Government Information More Accessible”) living here. Tom Lee’s response5 and Carl Malamud’s comments hail from these lands.
  • In the warmer lowlands of the South-East, food is a little more plentiful. Here you can find a long-nosed tribe of commercial organizations and individuals who get paid to hold government accountable: the tribe of “data-driven journalists” which focuses on working with data to carry out its function.
  • Finally, by far the most comfortable quadrant in the land is the lush pastures of the South-West, where a tribe of comfortable and well-nourished commercial organizations lives. Some members of this tribe are small and fast moving, but others have grown to giant size and sport odd names such as Google, Microsoft, or ESRI. One member of this tribe, Socrata, wrote another retort to my post of last week.

Not everyone lives in a single spot; some organizations are nomadic, wandering from quadrant to quadrant. As just one example, mySociety in the UK has both a commercial and a charitable wing; it runs both transparency-oriented projects such as They Work For You, and service-oriented projects such as Mapumental, which is a commuter-mapping service offered as a commercial product. O’Reilly Media lives in the South-West, but Alex Howard (a member of that tribe) spends considerable time in the South East.6

3 Open Data: Is it a Movement?

I hope the geography tour is pretty uncontroversial, and that it helps to orient ourselves with respect to the three claims I made last week about the “Open Data Movement”:

  1. It’s not a movement in a political or cultural sense of the word.
  2. It’s doing nothing for transparency and accountability in government.
  3. It’s co-opting the language of progressive change in pursuit of a small-government-focused subsidy for industry.

Of these three, I stand by about two and a half, although I do agree that the wording is sloppy and could be misleading. In my defense, I wrote quickly, expecting attention from my usual handful of readers (you know who you are; thanks for hanging around.) and not the much bigger audience that the post ended up attracting.7

It would take too long to engage in a defence of each of these claims8 so instead I’ll set out what I see as the contradictions and confusions that come out of labelling all four tribes that inhabit Open-Government-Dataland as a single movement, and distinguishing them from tribes who do not appear on this map: those share a similar interest (eg, Civil Liberties groups and journalists) but who do not focus on data. I see little coherence in the interests or priorities of the Open-Government-Dataland tribes; in particular the giants of the south-west cast a dark shadow over the other quadrants, and the other tribes may have to drive them out of Open-Government-Dataland or succumb to their hegemony.

4 Do Civil Liberties and Privatization Belong Together?

Encyclopedia Britannica says that a social movement is a “loosely organized but sustained campaign in support of a social goal” and that’s the definition I’ll stick with. So what’s the social goal of the Open Data Movement? There is a technological goal, spelled out a few years ago by some of its leading lights in terms of the formats, timeliness, completeness and licensing of the data,9 but what is its social goal? Pretty much any description I’ve seen gives two separate goals: improved government efficiency and transparency, corresponding to the west and east hemispheres of Open-Government-Dataland.

Being in favour of efficiency and transparency is a bit like being in favour of chocolate and cheese: both are good, but it’s not clear that they have very much to do with each other. But the problem is deeper than this: Open Data advocates argue not just for efficiency, but for a particular vision of “efficiency” captured by Tim O’Reilly’s phrases “Government as a Platform” and “Gov 2.0”.10 This vision places the interests of “the public” or “the people” on the same side as corporations and in conflict with those of the state. The thinking of Open Data advocates is open to the same kind of critique that Jodi Dean makes today about Adbusters’ Kalle Lasn: that an apparent populist leftism disguises (intentionally or otherwise) an economically neoliberal agenda.

“Lasn misrepresents the economic problem of neoliberal capitalism as a division between neoclassical economics and the “new ecological or bionomic or psychonomic discipline that is bubbling underneath the surface.” Now maybe I just don’t know what he’s talking about, but it looks to me like the sort of stuff that is usually wrapped up as complexity theory, with all its talk about emergence and swarms and self-organization and criticality (I talk about this in the first chapter of Blog Theory). It’s the same set of ideas part of New Economy thinking, which isn’t opposed to neoliberalism at all but was a primary carr[ier] of it, especially insofar as regulation is bad and free flow is good. Thomas Friedman, after all, is like the poster boy of horizontality–The World is Flat”

It does seem to me that the ideologically neoliberal aspects of “Gov 2.0” have not been absorbed by some of those in the civil liberties tribe. Tim O’Reilly, for example, is both ambitious in his small-government vision (“Government 2.0… is government stripped down to its core, rediscovered and reimagined as if for the first time”) and explicit in his commitment to market-based delivery of services. He approvingly quotes David G. Robinson’s Government Data and the Invisible Hand:11 “Private actors, either nonprofit or commercial, are better suited to deliver government information to citizens”. When it comes to healthcare he writes “[government] should not [take part] by competing with the private sector to deliver health services, but by investing in infrastructure (and ‘rules of the road’) that will lead to a more robust private sector ecosystem”.

The “Government as Platform” vision is even more market-driven than that of the “Cambridge Study” reported by Jo Bates (link), and to which Rufus Pollock of the Open Knowledge Foundation contributed. As Bates says, the Cambridge Study argued for “unrefined digital data to be available for re-use at marginal cost (general zero for digital resources), whilst the charging regime on refined PSI products should remain intact. These refined products, it is argued, would then be in fair competition with other suppliers, since there would be equal access to unrefined data inputs… In a further paper, Pollock goes on to argue that the optimal charging model would be direct state subsidy or, in some cases, charges to update the database. These economic arguments thus draw on a liberal economic paradigm with strong emphasis on supply-side policies based on removing constraints on commercial production through liberalisation and marketisation, combined with taxpayer subsidisation of infrastructural resources such as data.”

Jo Bates’s paper This is what modern deregulation looks like (link) explores the contradictions between the efficiency and transparency hemispheres in a thorough and lucid way and really you should just read that if you want a better-informed version of my own views. Here is one of the more abstract and general sections, that sums up her claims:

“the current ‘transparency agenda’ [of the UK government, supported by prominent Open Data advocates] should be recognised as an initiative that also aims to enable the marketisation of public services, and this is something that is not readily apparent to the general observer. Further, whilst democratic ends are claimed in the desire to enable “the public” to hold “the state” to account via these measures, there is an issue in utilising a dichotomy between the state and a notion of ‘the public’ which does not differentiate between citizens and commercial interests… The construction… encourages those attracted to civic engagement into an embrace of solidarity with profit seeking interests, distanced from the ever suspect notion of the state.”

The “Government as Platform” vision widely accepted among Open Data advocates12 thus overlaps significantly with the views of the UK government quoted by Jo Bates, culminating in Francis Maude’s statement that Open Government Data is “what modern deregulation looks like”. Is this neoliberal deregulation a vision that Tom Lee and David Eaves support?

The transparency agenda has been used by the PSI Reuse industry and by right-wing governments as a camouflage for other, economically neoliberal goals. Tom Lee describes the Open Data Movement as a “self-described nonpartisan activist movement” but while I accept his argument that the Civil Liberties tribe, including the Sunlight Foundation, are non-partisan (and, yes, are a movement), I do not think his characterisation can carry over to other tribes. The support, tacit or otherwise, of the Civil Liberties groups for the “Government as Platform” agenda, means that the Sunlight Foundation is promoting a neoliberal economic position with which its members may not agree.13 I don’t dismiss the views of Kevin Merrit, CEO of Socrata, as “self-serving and profit-motivated” when he argues that the Open Data Movement has promoted transparency, but I do believe there is a conflict of interest (which is a structural fact, not a personal quality) between arguing for an Open Data policy and then making money by providing software to implement that policy. It’s a conflict that makes episodes such as New York City’s unfortunate release of individual teacher assessments more likely.

5 Civic and Commercial Interests: Complement or Conflict?

Most Open Data advocates don’t phrase the issue in terms of private-sector provision of services, but instead phrase it in terms of civic engagement, non-profit groups, and “people”. Tim O’Reilly often phrases his arguments purely in terms of a civic public (and may see it that way himself), as in “This is the right way to frame the question of Government 2.0. How does government become an open platform that allows people inside and outside government to innovate?”

Carl Malamud goes further, arguing that the Open Data Movement is a replacement for a regime in which “the commercial sector is raping and pillaging the public treasury, getting exclusive deals on data that not only keeps out other companies, but researchers, public interest groups, and everybody else who make up ‘the public.’ In many cases, the government data is so tightly behind a cash register that even government workers enforcing the law can’t afford to buy copies of the data they produce or the rules they promulgated.” Others see no conflict between commerce and civic activity in this area: Tom Lee writes “I think it’s flatly wrong to consider private actors’ interest in public data to be uniformly problematic.”

David Eaves makes a strong argument for the vitality of the civic hacker tribe, and points out that Open Data has been largely ignored by Canadian corporations (although US companies such as Socrata have gained contracts for providing municipal “open data platforms”). The Canadian environment may be like that of the UK, where there is a “heavy weighting towards micro/small businesses in the … IT sector” to quote Jo Bates. Similarly, there are just are not that many Canadian companies deeply involved in government operations or in the use of public data.

(There have been positive statements from Open Text, and Desire2Learn has sponsored an “Edge Challenge” that has attracted app developers using open transit data [disclaimer: in my day job I have had some tangential involvement in that competition], but I can see what he means.)

So why would I focus on the private-sector, market-based actors of the south west quadrant when the civic hackers are perhaps more prevalent? Because of an argument made a year or two ago by Michael Gurstein, who asks “who is in a position to make ‘effective use’ of this newly available data?” and answers himself:

“‘open data’ empowers those with access to the basic infrastructure and the background knowledge and skills to make use of the data for specific ends. Given in fact, that these above mentioned resources are more likely to be found among those who already overall have access to and the resources for making effective use of digitally available information one could suggest that a primary impact of “open data” may be to further empower and enrich the already empowered and the well provided for rather than those most in need of the benefits of such new developments.”

Data’s value is combinatorial. It is most powerfully used by those who can combine it with other sources of data and who have the scale and resources to use it effectively. I think it’s fine that “civic hackers” are developing transit apps, but in the end that market is likely to be won by a single company under the current licensing and standards approaches.14

While Open Data advocates appear “open” to many new ideas, everything I’ve read suggests that they are near-united on the principle of “non-discriminatory” licensing, meaning making data available to commercial enterprises (of any size) on the same terms as to the Civic Hackers. The economy of data-driven products is similar to the economics of cultural industries: it tends to end in winner-take-all outcomes and favours large-scale enterprises. In cultural markets, this tendency has led many countries to adopt a toolbox of techniques to maintain domestic cultural industries in the face of the scale of the American cultural industry, from quotas to subsidies to non-market providers.15 Such measures have much in their favour, yet the Open Data Movement is apparently united in opposing them.

Economically, Silicon Valley is likely to be the major winner in the Open Government Data game. It is difficult to see how to justify a subsidy to Silicon Valley companies as a priority for cash-strapped governments of smaller countries.

An example: Jo Bates (again) describes the interest in weather data. “In the context of the UK there has been significant lobbying by the financial industry to get better access to UK weather data so that it is able to compete in this [weather risk management] market. Groups such as the Lighthill Risk Network, of which Lloyds of London are a member, have lobbied government for better weather data so that they can develop risk based weather products. Similarly, the insurance industry has requested real time information on the pretext that they might respond more quickly to extreme weather events such as flooding. My own research and the recent announcement suggest that these demands have been met enthusiastically by well placed policy makers in national government who are keen to develop a UK weather derivatives market.” Weather risk management might seem like an odd duck, but Bates reports that “This weather risk management market far outweighs the USA’s commercial weather products market which in 2000 was estimated at approximately $500 million a year”, touching over $45 billion in 2005-06.

The rhetoric of civic engagement is appealing, but blurring the boundary between small-scale civic “hackathons” and the major financial institutions is a position that simply ignores major economic and political issues.

The benefits of standards-driven formats are, for municipal activities, not obvious unless you want to attract global interest. I continue to believe that licensing and formats are an area where there is still room for innovation, and where a premature focus on standardization may shorten the lifespan of civic-hacker use of municipal data before the big players get to pull it into their own systems. I’d argue, as I have before, for some form of charging to be enabled, at least on large-scale commercial use of data. I’d also argue that standardization should not be high on the agenda for municipal governments looking to build and collaborate with a local community of hackers.

6 Summary

Let me return to my three claims:

  1. It’s not a movement in a political or cultural sense of the word.
  2. It’s doing nothing for transparency and accountability in government.
  3. It’s co-opting the language of progressive change in pursuit of a small-government-focused subsidy for industry.

I’d argue that (1) holds: there is simply too much incoherence, too much in the way of conflicting interests and non-overlapping goals, for the “Open Data Movement” to be a movement. And see also the footnote.14

Item (2) is harsh. There are many within the “Open Data” sphere who live in the Civil Liberties area who have made significant contributions to transparency and accountability. But as a net effect, I’d stand by the claim – the overall impact of open data initiatives could well be to promote a kind of government that is prone to secrecy, as “small government” parties have often been.

Item (3): the co-option is being done by a vocal and influential section of Open Data advocates, but I’d definitely hold to the claim that the language of progressive change is being used, and the actions of civil liberties activists used, by some whose agenda is closer to neoliberal than egalitarian.

If you are still with me after all that; thanks for reading.

Footnotes:

1 Harlan Yu and David G. Robinson, /The New Ambiguity of “Open Government”/, Working Paper, draft of Feb 28, 2012. Retrieved from SSRN.

2 Jo Bates, /”This is what modern deregulation looks like”: co-optation and contestation in the shaping of the UK’s Open Government Initiative/, The Journal of Community Informatics, 8 (2). Retrieved from ci-journal.net.

3 This article adopts a northern-hemispherical hegemonic worldview.

4 David Eaves, Open Data Movement is a Joke?, May 2, 2012. Retrieved from eaves.ca.

5 Tom Lee, Defending the Big Tent: Open Data, Inclusivity and Activism, May 2, 2012. Retrieved from sunlightfoundation.com

6 Alex Howard, No joke: Open data fuels transparency, civic utility and economic activity, May 2, 2012. Retrieved from govfresh.com.

7 If I had know the audience was to be so large, I would have written more cautiously, and then the audience would not have been so large.

8 While I don’t want to trespass on everyone’s attention for that length of time, if you are interested in discussing these do send me an email (tslee at web dot ca) and I’d be happy to respond.

9 Open Government Working Group, 8 Principles of Open Government Data, 8 December 2007. Retrieved from opengovdata.org.

10 Tim O’Reilly, Government as Platform, Chapter 1 of /Open Government: Collaboration, Transparency, and Participation in Practice/, Retrieved from oreilly.com.

11 David Robinson, Harlan Yu, William P. Zeller, and Edward W. Felten, Government Data and the Invisible Hand, 11 Yale J.L. & Tech. 160 (2009). Retrieved from SSRN.

12 The only dissenting view I’ve seen that tackles it explicitly, apart from my own, is that of Andrea DiMaio, who also has some smart things to say here.

13 On this I disagree with Catherine Fitzpatrick, who shares some of my views about the Open Data movement and argues forcefully for them here. I appreciate Catherine’s robust arguments in the debate, but she does come at this from a very different political point of view to my own, as her comment on Tom Lee’s post makes clear.

14 Aside: David Eaves’ description of municipal-level civic-hackers (North-West quadrant) in Canada is compelling, but this group of people is also Not a Movement. Now some people claim not to care about the word “movement”, and if you don’t then skip back to the main text, but I think it matters.

There are many admirable socially and civically beneficial activities that are not movements. Many people coach children’s sports teams (I’ve done it myself); millions of people take part in such activities, they are committed, involved, and do a lot of civic good, but they do not form a movement: I was a soccer coach, not a public sports activist. Similarly, birdwatching has a long and distinguished history of contributing to social goals (protection of birds and their habitats) and of sharing their observations in socially and scientifically useful ways (my brother Dorset Dipper contributes to the Hertfordshire Bird Atlas) but birdwatching is a hobby, not a movement. This thing about technologists claiming to be a movement is something that, perhaps irrationally, irritates me. Calling “Open Data” a movement is not quite as daft as calling “NoSQL” a movement, despite the arguments of O’Reilly’s Mike Loukides, but to my mind invoking a “movement” is a way to give added weight and significance to activities that may be admirable and useful, but that are ultimately uncontentious: it smacks of self-importance and that rubs me the wrong way.

15 For a fine description of the economics of cultural products, and the toolboxes that smaller economies have used to maintain cultural diversity, see Blockbusters and Trade Wars by Peter S. Grand and Chris Wood, Douglas & McIntyre 2005. Link.

Date: 2012-05-08 21:40:38

Org version 7.8.06 with Emacs version 23

Is philosophy a joke?

No, not a continuation of the theme of the previous two posts.

My son, who has been pursuing a degree in philosophy over the last couple of years with steadily mounting frustration, has decided to "drop out", work for a while, and consider where he goes next. He signs off his blog here:

Although it is unfortunate to quit before I can figure out whether (the vast majority of) philosophers are actually as staggeringly incompetent as they appear to be, or are simply playing an elaborate practical joke, I simply couldn’t stand it either way.

Yes, he's disenchanted. I'm proud of him and his efforts to find a way to apply reason to important questions about life, and I've been dismayed to watch the discipline of philosophy lose someone as motivated as he has been, punishing originality rather than encouraging it, pandering to intuition, and giving up on reason while spending its time on issues such as whether proper names are rigid designators.

I'm no philosopher, but while I look forward to seeing what my son does next, I have no inclination whatsoever to read Naming and Necessity.

Reactions to ‘Why the “Open Data Movement” is a Joke’

[Update: In the light of morning I dislike this post. I'll leave it up, but it is too defensive, explains too much, and is too conciliatory given what was thrown at me yesterday.]

This morning's post, Why the "Open Data Movement" is a Joke, attracted more attention than most of what I've written here. Largely this was a result of a Twitter debate between Evgeny Morozov (@evgenymorozov) and O'Reilly Media's Alex Howard (@digiphile). Thanks also to Lorenz Matzat (@lorz) and Ryan Shaw (@rybesh) for arguing broadly in favour of the post.

Alex Howard hated the piece, calling it "ill-informed", "lazy, ignorant writing" that "didn't even bother to cite the relevant scholarship", "demonstrably incorrect", "laughable" and more. He also writes that "The author has a habit of writing polemics that include errors or omissions of fact." I am terrible at expressing anything in 140 chars so I'll respond here.

First, it should be obvious that the post was prompted by events here in Canada – and yet no one has actually mentioned any of the Canadian content in any of the comments about the post. This is frustrating. The last five years have been terrible ones for accountability and transparency in this country and yet Canada has just joined and endorsed the Open Government Partnership (link). Does this conjunction say anything about "open data" as a goal? To me it says that a technological "open data" agenda does not indicate a political "open data" agenda, and that — as I wrote about the Wikileaks cables a year ago (link) –the fault lines of political beliefs run perpendicular to attitudes about technology, not parallel. So it made me wonder if the idea of open data as a goal for a coherent movement holds water, and whether "opening government via technology" make sense. To Alex Howard – well I don't really know, because for all his outrage he doesn't actually say anything about the first half of the post or about the events that obviously moved me to write it and led to the frustrated tone I wrote it in.

Beyond that, he seems to confuse my contention that the idea of open data as a "movement" is a joke with a broader claim that "open data" is a waste of time or that people working on making data open are all dupes. No such thing! Open data can be a fine thing, but I'd much rather have a fully-staffed StatsCan charging for data than a half-staffed StatsCan providing it for free. Which would he choose? Obviously a fully-staffed StatsCan providing data for free would be ideal, but it doesn't look like we're getting that any time soon. The UK's Francis Maude, the incoming co-chair of the Open Government Partnership, says that "we want to create an army of armchair auditors who can hold government to account". This would be nice, but not at the cost of a real independent auditor.

I suspect that Alex Howard and I just see the world from different points of view. Que sera, sera. But from what I know the Sunlight Foundation is on the side of the angels so I was disappointed to see that Tom Lee of the Sunlight Foundation considered my post "a jumbled mess". I do realize that coalition politics makes strange bedfellows, and that broad coalitions can still be worthwhile, but some tents can be so big that they collapse in a shapeless pile of canvas. I worry that the open data tent is one such, and that the apparent common goals of some people under the canvas hide bigger differences.  I share the concerns of Alex Howard's colleague Nat Torkington when he writes this:

Obama and his staff, coming from the investment mindset, are building a Gov 2.0 infrastructure that creates a space for economic opportunity, informed citizens, and wider involvement in decision making so the government better reflects the community's will. Cameron and his staff, coming from a cost mindset, are building a Gov 2.0 infrastructure that suggests it will be more about turning government-provided services over to the private sector.

For me, the gap between the two visions is fundamental and makes the idea that these two goals are part of the same movement, well, a joke. The tension/contradiction between commercial and civic interests that these sentences highlight is one division that seems unresolved and yet fundamental. So having read the responses, I stand by what I wrote.