1 Introduction
I have two things to say to those who responded to Why the ‘Open Data Movement’ is a Joke:
- Thank you for putting so much effort into providing such thoughtful, reflective, articulate affirmations of your point of view. You gave me (and others, I hope) a lot to think about, and a lot to read over the last several days.
- Unfortunately, you’re still wrong.
The original post was written in the heat of the moment, so here is a more detailed and considered, and therefore almost certainly less-likely-to-be-read, argument about the contradictions and problems of the “Open Data Movement”.
2 The Open Government Data Landscape
First, here is a map of Open-Government-Dataland (click for a larger popup).
The longitude, marked across the x axis, indicates the impact of the data itself. The line x=0 is the Yu-Robinson Meridian1 and separates Open-Government-Dataland into an eastern and a western hemisphere, each populated by a spectrum of data. Harlan Yu and David G. Robinson explain the hemispheres this way:
“A machine-readable bus schedule aims to promote convenience, commerce and a higher quality of life—it enhances service delivery. Disclosures of public contracting opportunities play a dual role, potentially enhancing both economic opportunity and public integrity. And core civic data, such as legislative or campaign finance information, serves a more purely civic role, enhancing transparency.”
Tom Lee describes these two hemispheres as “‘open-as-in-data.gov’ and ‘open-as-in-FOIA'”.
The latitude, marked along the y axis, marks the incentive of the data users. Travelling from south to north along a meridian takes us from the purely commercial activities in the south to the purely non-commercial in the relatively poor northern lands. At y=0 we cross the Bates Parallel, named after Jo Bates who has this to say about the conflict between the commercial and non-commercial hemispheres:2
“Interviews and observations suggest that some at the periphery of the OGD [Open Government Data] initiative have tended to conceptualise OGD as being about small start-ups, voluntary ‘civic hackers’, and other micro/small enterprises. This is unsurprising given the heavy weighting towards micro/small businesses in the UK’s IT sector and the large number of ‘civic hackers’ active in the OGD community; however, the potential re-user industry for OGD is broader than this. The PSI [Public Sector Information] re-use industry comprises a range of industries and includes multinational corporations (MNCs) such as Google and LexisNexis, conglomerates such as Daily Mail and General Trust whose DMG information division is the parent company of the UK based Landmark Information Group, as well as an array of SMEs, micro enterprises, independent developers, and voluntary ‘civic hackers’.”
While this landscape is rich, the inhabitants of different quadrants of the map can be distinguished easily:
- In the harsh mountains of the North-West3 live the hardy non-commercial “civic hackers” whose diet consists of transit timetables, weather forecasts, and other pragmatic, useful data obtained, often piece-by-piece, from municipal governments. The culture of the civic hackers displays a combination of civic goals and enjoyment in the intrinsic interest of programming, with a twist of start-up mentality. It is a tribe described vividly by David Eaves4: “geek, technically inclined, leaning left, and socially minded. There are many who don’t fit that profile, but that is probably the average.” This tribe includes organizations such as Code for America (“A new kind of public service”) as well as small and loosely-coordinated groups of individuals working at a local level and who identify with their role as citizens over their role as consumers.
- The North-Eastern tundra is the home of civil liberties activists, a resourceful tribe bent on promoting government transparency. It is driven by a desire to make information such as lobbying activities, campaign funding, government operations, and legal statutes open and accessible. In the USA you will find organizations such as Sunlight Foundation (“Making Government Transparent and Accountable”) and Public.Resource.Org (“Making Government Information More Accessible”) living here. Tom Lee’s response5 and Carl Malamud’s comments hail from these lands.
- In the warmer lowlands of the South-East, food is a little more plentiful. Here you can find a long-nosed tribe of commercial organizations and individuals who get paid to hold government accountable: the tribe of “data-driven journalists” which focuses on working with data to carry out its function.
- Finally, by far the most comfortable quadrant in the land is the lush pastures of the South-West, where a tribe of comfortable and well-nourished commercial organizations lives. Some members of this tribe are small and fast moving, but others have grown to giant size and sport odd names such as Google, Microsoft, or ESRI. One member of this tribe, Socrata, wrote another retort to my post of last week.
Not everyone lives in a single spot; some organizations are nomadic, wandering from quadrant to quadrant. As just one example, mySociety in the UK has both a commercial and a charitable wing; it runs both transparency-oriented projects such as They Work For You, and service-oriented projects such as Mapumental, which is a commuter-mapping service offered as a commercial product. O’Reilly Media lives in the South-West, but Alex Howard (a member of that tribe) spends considerable time in the South East.6
3 Open Data: Is it a Movement?
I hope the geography tour is pretty uncontroversial, and that it helps to orient ourselves with respect to the three claims I made last week about the “Open Data Movement”:
- It’s not a movement in a political or cultural sense of the word.
- It’s doing nothing for transparency and accountability in government.
- It’s co-opting the language of progressive change in pursuit of a small-government-focused subsidy for industry.
Of these three, I stand by about two and a half, although I do agree that the wording is sloppy and could be misleading. In my defense, I wrote quickly, expecting attention from my usual handful of readers (you know who you are; thanks for hanging around.) and not the much bigger audience that the post ended up attracting.7
It would take too long to engage in a defence of each of these claims8 so instead I’ll set out what I see as the contradictions and confusions that come out of labelling all four tribes that inhabit Open-Government-Dataland as a single movement, and distinguishing them from tribes who do not appear on this map: those share a similar interest (eg, Civil Liberties groups and journalists) but who do not focus on data. I see little coherence in the interests or priorities of the Open-Government-Dataland tribes; in particular the giants of the south-west cast a dark shadow over the other quadrants, and the other tribes may have to drive them out of Open-Government-Dataland or succumb to their hegemony.
4 Do Civil Liberties and Privatization Belong Together?
Encyclopedia Britannica says that a social movement is a “loosely organized but sustained campaign in support of a social goal” and that’s the definition I’ll stick with. So what’s the social goal of the Open Data Movement? There is a technological goal, spelled out a few years ago by some of its leading lights in terms of the formats, timeliness, completeness and licensing of the data,9 but what is its social goal? Pretty much any description I’ve seen gives two separate goals: improved government efficiency and transparency, corresponding to the west and east hemispheres of Open-Government-Dataland.
Being in favour of efficiency and transparency is a bit like being in favour of chocolate and cheese: both are good, but it’s not clear that they have very much to do with each other. But the problem is deeper than this: Open Data advocates argue not just for efficiency, but for a particular vision of “efficiency” captured by Tim O’Reilly’s phrases “Government as a Platform” and “Gov 2.0”.10 This vision places the interests of “the public” or “the people” on the same side as corporations and in conflict with those of the state. The thinking of Open Data advocates is open to the same kind of critique that Jodi Dean makes today about Adbusters’ Kalle Lasn: that an apparent populist leftism disguises (intentionally or otherwise) an economically neoliberal agenda.
“Lasn misrepresents the economic problem of neoliberal capitalism as a division between neoclassical economics and the “new ecological or bionomic or psychonomic discipline that is bubbling underneath the surface.” Now maybe I just don’t know what he’s talking about, but it looks to me like the sort of stuff that is usually wrapped up as complexity theory, with all its talk about emergence and swarms and self-organization and criticality (I talk about this in the first chapter of Blog Theory). It’s the same set of ideas part of New Economy thinking, which isn’t opposed to neoliberalism at all but was a primary carr[ier] of it, especially insofar as regulation is bad and free flow is good. Thomas Friedman, after all, is like the poster boy of horizontality–The World is Flat”
It does seem to me that the ideologically neoliberal aspects of “Gov 2.0” have not been absorbed by some of those in the civil liberties tribe. Tim O’Reilly, for example, is both ambitious in his small-government vision (“Government 2.0… is government stripped down to its core, rediscovered and reimagined as if for the first time”) and explicit in his commitment to market-based delivery of services. He approvingly quotes David G. Robinson’s Government Data and the Invisible Hand:11 “Private actors, either nonprofit or commercial, are better suited to deliver government information to citizens”. When it comes to healthcare he writes “[government] should not [take part] by competing with the private sector to deliver health services, but by investing in infrastructure (and ‘rules of the road’) that will lead to a more robust private sector ecosystem”.
The “Government as Platform” vision is even more market-driven than that of the “Cambridge Study” reported by Jo Bates (link), and to which Rufus Pollock of the Open Knowledge Foundation contributed. As Bates says, the Cambridge Study argued for “unrefined digital data to be available for re-use at marginal cost (general zero for digital resources), whilst the charging regime on refined PSI products should remain intact. These refined products, it is argued, would then be in fair competition with other suppliers, since there would be equal access to unrefined data inputs… In a further paper, Pollock goes on to argue that the optimal charging model would be direct state subsidy or, in some cases, charges to update the database. These economic arguments thus draw on a liberal economic paradigm with strong emphasis on supply-side policies based on removing constraints on commercial production through liberalisation and marketisation, combined with taxpayer subsidisation of infrastructural resources such as data.”
Jo Bates’s paper This is what modern deregulation looks like (link) explores the contradictions between the efficiency and transparency hemispheres in a thorough and lucid way and really you should just read that if you want a better-informed version of my own views. Here is one of the more abstract and general sections, that sums up her claims:
“the current ‘transparency agenda’ [of the UK government, supported by prominent Open Data advocates] should be recognised as an initiative that also aims to enable the marketisation of public services, and this is something that is not readily apparent to the general observer. Further, whilst democratic ends are claimed in the desire to enable “the public” to hold “the state” to account via these measures, there is an issue in utilising a dichotomy between the state and a notion of ‘the public’ which does not differentiate between citizens and commercial interests… The construction… encourages those attracted to civic engagement into an embrace of solidarity with profit seeking interests, distanced from the ever suspect notion of the state.”
The “Government as Platform” vision widely accepted among Open Data advocates12 thus overlaps significantly with the views of the UK government quoted by Jo Bates, culminating in Francis Maude’s statement that Open Government Data is “what modern deregulation looks like”. Is this neoliberal deregulation a vision that Tom Lee and David Eaves support?
The transparency agenda has been used by the PSI Reuse industry and by right-wing governments as a camouflage for other, economically neoliberal goals. Tom Lee describes the Open Data Movement as a “self-described nonpartisan activist movement” but while I accept his argument that the Civil Liberties tribe, including the Sunlight Foundation, are non-partisan (and, yes, are a movement), I do not think his characterisation can carry over to other tribes. The support, tacit or otherwise, of the Civil Liberties groups for the “Government as Platform” agenda, means that the Sunlight Foundation is promoting a neoliberal economic position with which its members may not agree.13 I don’t dismiss the views of Kevin Merrit, CEO of Socrata, as “self-serving and profit-motivated” when he argues that the Open Data Movement has promoted transparency, but I do believe there is a conflict of interest (which is a structural fact, not a personal quality) between arguing for an Open Data policy and then making money by providing software to implement that policy. It’s a conflict that makes episodes such as New York City’s unfortunate release of individual teacher assessments more likely.
5 Civic and Commercial Interests: Complement or Conflict?
Most Open Data advocates don’t phrase the issue in terms of private-sector provision of services, but instead phrase it in terms of civic engagement, non-profit groups, and “people”. Tim O’Reilly often phrases his arguments purely in terms of a civic public (and may see it that way himself), as in “This is the right way to frame the question of Government 2.0. How does government become an open platform that allows people inside and outside government to innovate?”
Carl Malamud goes further, arguing that the Open Data Movement is a replacement for a regime in which “the commercial sector is raping and pillaging the public treasury, getting exclusive deals on data that not only keeps out other companies, but researchers, public interest groups, and everybody else who make up ‘the public.’ In many cases, the government data is so tightly behind a cash register that even government workers enforcing the law can’t afford to buy copies of the data they produce or the rules they promulgated.” Others see no conflict between commerce and civic activity in this area: Tom Lee writes “I think it’s flatly wrong to consider private actors’ interest in public data to be uniformly problematic.”
David Eaves makes a strong argument for the vitality of the civic hacker tribe, and points out that Open Data has been largely ignored by Canadian corporations (although US companies such as Socrata have gained contracts for providing municipal “open data platforms”). The Canadian environment may be like that of the UK, where there is a “heavy weighting towards micro/small businesses in the … IT sector” to quote Jo Bates. Similarly, there are just are not that many Canadian companies deeply involved in government operations or in the use of public data.
(There have been positive statements from Open Text, and Desire2Learn has sponsored an “Edge Challenge” that has attracted app developers using open transit data [disclaimer: in my day job I have had some tangential involvement in that competition], but I can see what he means.)
So why would I focus on the private-sector, market-based actors of the south west quadrant when the civic hackers are perhaps more prevalent? Because of an argument made a year or two ago by Michael Gurstein, who asks “who is in a position to make ‘effective use’ of this newly available data?” and answers himself:
“‘open data’ empowers those with access to the basic infrastructure and the background knowledge and skills to make use of the data for specific ends. Given in fact, that these above mentioned resources are more likely to be found among those who already overall have access to and the resources for making effective use of digitally available information one could suggest that a primary impact of “open data” may be to further empower and enrich the already empowered and the well provided for rather than those most in need of the benefits of such new developments.”
Data’s value is combinatorial. It is most powerfully used by those who can combine it with other sources of data and who have the scale and resources to use it effectively. I think it’s fine that “civic hackers” are developing transit apps, but in the end that market is likely to be won by a single company under the current licensing and standards approaches.14
While Open Data advocates appear “open” to many new ideas, everything I’ve read suggests that they are near-united on the principle of “non-discriminatory” licensing, meaning making data available to commercial enterprises (of any size) on the same terms as to the Civic Hackers. The economy of data-driven products is similar to the economics of cultural industries: it tends to end in winner-take-all outcomes and favours large-scale enterprises. In cultural markets, this tendency has led many countries to adopt a toolbox of techniques to maintain domestic cultural industries in the face of the scale of the American cultural industry, from quotas to subsidies to non-market providers.15 Such measures have much in their favour, yet the Open Data Movement is apparently united in opposing them.
Economically, Silicon Valley is likely to be the major winner in the Open Government Data game. It is difficult to see how to justify a subsidy to Silicon Valley companies as a priority for cash-strapped governments of smaller countries.
An example: Jo Bates (again) describes the interest in weather data. “In the context of the UK there has been significant lobbying by the financial industry to get better access to UK weather data so that it is able to compete in this [weather risk management] market. Groups such as the Lighthill Risk Network, of which Lloyds of London are a member, have lobbied government for better weather data so that they can develop risk based weather products. Similarly, the insurance industry has requested real time information on the pretext that they might respond more quickly to extreme weather events such as flooding. My own research and the recent announcement suggest that these demands have been met enthusiastically by well placed policy makers in national government who are keen to develop a UK weather derivatives market.” Weather risk management might seem like an odd duck, but Bates reports that “This weather risk management market far outweighs the USA’s commercial weather products market which in 2000 was estimated at approximately $500 million a year”, touching over $45 billion in 2005-06.
The rhetoric of civic engagement is appealing, but blurring the boundary between small-scale civic “hackathons” and the major financial institutions is a position that simply ignores major economic and political issues.
The benefits of standards-driven formats are, for municipal activities, not obvious unless you want to attract global interest. I continue to believe that licensing and formats are an area where there is still room for innovation, and where a premature focus on standardization may shorten the lifespan of civic-hacker use of municipal data before the big players get to pull it into their own systems. I’d argue, as I have before, for some form of charging to be enabled, at least on large-scale commercial use of data. I’d also argue that standardization should not be high on the agenda for municipal governments looking to build and collaborate with a local community of hackers.
6 Summary
Let me return to my three claims:
- It’s not a movement in a political or cultural sense of the word.
- It’s doing nothing for transparency and accountability in government.
- It’s co-opting the language of progressive change in pursuit of a small-government-focused subsidy for industry.
I’d argue that (1) holds: there is simply too much incoherence, too much in the way of conflicting interests and non-overlapping goals, for the “Open Data Movement” to be a movement. And see also the footnote.14
Item (2) is harsh. There are many within the “Open Data” sphere who live in the Civil Liberties area who have made significant contributions to transparency and accountability. But as a net effect, I’d stand by the claim – the overall impact of open data initiatives could well be to promote a kind of government that is prone to secrecy, as “small government” parties have often been.
Item (3): the co-option is being done by a vocal and influential section of Open Data advocates, but I’d definitely hold to the claim that the language of progressive change is being used, and the actions of civil liberties activists used, by some whose agenda is closer to neoliberal than egalitarian.
If you are still with me after all that; thanks for reading.
Footnotes:
1 Harlan Yu and David G. Robinson, /The New Ambiguity of “Open Government”/, Working Paper, draft of Feb 28, 2012. Retrieved from SSRN.
2 Jo Bates, /”This is what modern deregulation looks like”: co-optation and contestation in the shaping of the UK’s Open Government Initiative/, The Journal of Community Informatics, 8 (2). Retrieved from ci-journal.net.
3 This article adopts a northern-hemispherical hegemonic worldview.
4 David Eaves, Open Data Movement is a Joke?, May 2, 2012. Retrieved from eaves.ca.
5 Tom Lee, Defending the Big Tent: Open Data, Inclusivity and Activism, May 2, 2012. Retrieved from sunlightfoundation.com
6 Alex Howard, No joke: Open data fuels transparency, civic utility and economic activity, May 2, 2012. Retrieved from govfresh.com.
7 If I had know the audience was to be so large, I would have written more cautiously, and then the audience would not have been so large.
8 While I don’t want to trespass on everyone’s attention for that length of time, if you are interested in discussing these do send me an email (tslee at web dot ca) and I’d be happy to respond.
9 Open Government Working Group, 8 Principles of Open Government Data, 8 December 2007. Retrieved from opengovdata.org.
10 Tim O’Reilly, Government as Platform, Chapter 1 of /Open Government: Collaboration, Transparency, and Participation in Practice/, Retrieved from oreilly.com.
11 David Robinson, Harlan Yu, William P. Zeller, and Edward W. Felten, Government Data and the Invisible Hand, 11 Yale J.L. & Tech. 160 (2009). Retrieved from SSRN.
12 The only dissenting view I’ve seen that tackles it explicitly, apart from my own, is that of Andrea DiMaio, who also has some smart things to say here.
13 On this I disagree with Catherine Fitzpatrick, who shares some of my views about the Open Data movement and argues forcefully for them here. I appreciate Catherine’s robust arguments in the debate, but she does come at this from a very different political point of view to my own, as her comment on Tom Lee’s post makes clear.
14 Aside: David Eaves’ description of municipal-level civic-hackers (North-West quadrant) in Canada is compelling, but this group of people is also Not a Movement. Now some people claim not to care about the word “movement”, and if you don’t then skip back to the main text, but I think it matters.
There are many admirable socially and civically beneficial activities that are not movements. Many people coach children’s sports teams (I’ve done it myself); millions of people take part in such activities, they are committed, involved, and do a lot of civic good, but they do not form a movement: I was a soccer coach, not a public sports activist. Similarly, birdwatching has a long and distinguished history of contributing to social goals (protection of birds and their habitats) and of sharing their observations in socially and scientifically useful ways (my brother Dorset Dipper contributes to the Hertfordshire Bird Atlas) but birdwatching is a hobby, not a movement. This thing about technologists claiming to be a movement is something that, perhaps irrationally, irritates me. Calling “Open Data” a movement is not quite as daft as calling “NoSQL” a movement, despite the arguments of O’Reilly’s Mike Loukides, but to my mind invoking a “movement” is a way to give added weight and significance to activities that may be admirable and useful, but that are ultimately uncontentious: it smacks of self-importance and that rubs me the wrong way.
15 For a fine description of the economics of cultural products, and the toolboxes that smaller economies have used to maintain cultural diversity, see Blockbusters and Trade Wars by Peter S. Grand and Chris Wood, Douglas & McIntyre 2005. Link.
Date: 2012-05-08 21:40:38
Org version 7.8.06 with Emacs version 23
Tom,
I think your comment regarding your second claim, “But as a net effect, I’d stand by the claim – the overall impact of open data initiatives could well be to promote a kind of government that is prone to secrecy, as “small government” parties have often been.” is pretty accurate, at least based on UK experience.
If you look at what is going on in the UK, you can see explicit Prime Ministerial attacks on Freedom of Information legislation (it ‘furs up the arteries of government’) while promoting ‘open data’ as the ‘real freedom of information’.
http://www.bbc.co.uk/news/uk-politics-17286328
The UK is the incoming co-chair of the Open Government Partnership, and there is a lot of concern, particularly from the civil society/civil liberties quadrant of your map, concerning the government’s current lack of commitment to consultation, engagement, civic-participation and opening up government via the Freedom of Information Act. See, for example, http://www.opengovernment.org.uk/open-letter-to-francis-maude/ .
The UK governing coalition’s health minister has just used the veto provision in the FOI Act to block disclosure of a register of risks drawn up officials to map the possible problems with the government’s privatisation of the National Health Service. They’re not interested in making transparent anything which might cast doubt on their ‘small government’ agenda, but they are very interested in disclosure of data which will enable private health companies maximise their gains from the new environment of contracting and competition.
http://www.bbc.co.uk/news/health-17989929
Unsurprisingly, the person chosen to head up the UK government’s Transparency and Open Data work, Tim Kelsey, is an ex-McKinsey consultant who sold his company that analysed public health data company to the government. http://en.wikipedia.org/wiki/Tim_Kelsey
Whether Mr Kelsey and the UK government shift course to embrace an un-weakened FOI Act and the civic-participation side of open government, or continue on the “open data = real freedom of information” path they appear to be on at the moment, will be one of the key issues of the next 12 months in this arena.
Thanks for this, it’s helpful to see this conversation through a different perspective, and I find this productive.
First, you should know I’m Sunlight’s policy director, and agree with Tom that a big tent is useful, and with Ellen’s recent post, asserting that our mission is about power, accountability, and influence.
Sunlight sits very much in the “open data movement” (if there is one), and also at its intersection with a variety of other communities — where “open data” intersects with a whole constellation of other issues, like FOI and the right to know, contracting oversight, information policy, professional oversight communities, commercial publishers and the amateurs democratizing their business models, regulatory oversight and transparency experts, legal publishing, legislative transparency, etc.
I’m not commenting here to contradict your thesis, because I don’t particularly care whether “open data” is a movement or not.
But I think it might be helpful to remember that some of the neoliberal or libertarian assumptions behind some of the open data voices you reference don’t actually go unchallenged as much as it may seem.
If the “open data” movement is a thing with a coherent ideology and agenda (which I doubt), but if it is, it’s a thing that is growing up alongside a much broader, established set of issues. Those issues, like the ones I outlined above, often have an ideological identity — spending transparency enthusiasts currently come from the right, and money in politics enthusiasm comes more from the left (at least right now). New ideas, like those arising from self-identified “gov 2.0” or “open data” enthusiasts, aren’t threatening to other NGOs who work on transparency, because the issue we work on necessitates accommodating ideological differences.
That’s not to say bad ideas are accepted. The Invisible Hand paper was actually met with a lot of consternation:
https://groups.google.com/forum/#!topic/openhouseproject/GZEnZDz17Gk
…for a variety of reasons, and you could easily read the critique of its assumptions that happened on that listserve and elsewhere as a critique of neoliberal approaches to government reform. So the “release all the data, let the market sort it out” argument doesn’t really carry much water, and is actually only really useful to me as a demonstration of a sort of purist position.
I feel similarly about a number of other arguments — open data as economic stimulus is often overwrought, but largely harmless, I don’t see any policy (at least in the US) being shaped based on job-creation projections, although if more businesses grow like brightscope, then great.
I’d like to ask you about (feel free to respond via email, jwonderlich @ sunlightfoundation.com ) your suggestion that unrestricted data will favor large, established actors. That’s possible, I suppose, but when we’ve fought to have data released, it’s served to help undermine large commercial publishers who reinforce inappropriate disparities in power, by selling access.
(For example, this functionality we’ve just released only existed through expensive services:
http://scout.sunlightfoundation.com/search/federal_bills/section%20552%20of%20title%205
)
I could go into more examples, but I think our work at Sunlight almost always ends up democratizing access to data and information, breaking up rent-seeking commercial publishers interests (albeit very slowly), and bringing more information to more people online, for free. I’m bringing this up because if there’s a sense in which unrestricted data is harmful, I’d like to hear more about how.
Read “The Price of Access” for more thoughts on commercial publishers and transparency:
http://sunlightfoundation.com/blog/2010/10/16/the-price-of-access/
Finally, I’d like to say that the enthusiasm for data portals and data policies has been a huge boon for our efforts to bring rigor and reliability to information policies across the government. To pick just one recent example:
http://sunlightfoundation.com/blog/2012/03/22/the-missing-open-data-policy/
If “open data” rhetoric sometimes lacks rigor, then I see it primarily as an opportunity for that rigor to be injected. I don’t particularly care whether others see that value, but there’s certainly a robust debate going on about the issues you raise here, like appropriate licensing, whether some transparency policies subsidize already empowered actors, what the appropriate roles of government must be with respect to information, etc.
I find the “open data movement” debate to be a not particularly helpful frame, and I also reject the “transparency vs. services” / “commercial vs. noncommercial” landscape map, which strikes me as simplistic and not very helpful.
But whatever. I hope more of the thoughts at the heart of your critique, like this licensing question, get a broader airing, because they’re important questions.
I didn’t bother to respond to your first piece, since it was, as you admit yourself, a rant. This piece, however, is full of insight and food for thought.
I was completely with you through section 4, but at that point, I began to feel you were arguing with your own straw man. In particular, I found myself and my words shoehorned into categories of your devising, whether or not they fit. Because as it turns out, I have no problem supporting *both* the notion of open data for transparency and open data for utility, of promoting “government as a platform” as a way of increasing private sector provision of services without letting go of the notion that, as Lincoln said, government is a way of doing things together that we can’t do as well alone. In telling the government as platform story, I’ve used the example of the iPhone app store. There’s no question that Apple is singular in its goals of managing and creating a desired user experience, yet it also unleashed the creativity of third parties. That’s what I’m looking for in government – not the shabby abdication of responsibility of Cameron’s “Big Society.”
In fact, I am about as far from a libertarian pro-private-sector advocate as you can imagine. But neither am I the liberal that the libertarian commenters on my G+ postings seem to believe I am.
I’m instead trying to suggest that platform thinking, as learned from technology, can show us a third way – neither slavish adherence to a failed philosophy of the market’s wisdom when left to itself, nor to a failed philosophy of unlimited government intervention, but to a design in which we think about outcomes, and design the interfaces between government and society to promote them.
That being said, there’s no question that some of the things I’ve written can be as misinterpreted by advocates who think they are embracing my ideas as by critics like yourself who reject them. But that’s the way of the world.
In that regard, let me take up your three concluding assertions:
1. “there is simply too much incoherence, too much in the way of conflicting interests and non-overlapping goals, for the “Open Data Movement” to be a movement.”
I have some significant experience with technology movements, and both “open source” and “web 2.0” were also not “movements” by your definition, but in the light of distance, they appear to have been. Each was as rife with contradiction, conflict, misinterpretation, and co-optation as open government. In particular, the notion of “Web 2.0” was hijacked by those who thought the term referred to lightweight-startups with advertising business models, and it was only after the better part of a decade that my original intent – to recognize the shift in power and influence from software to data as the core of innovation in computer applications.
These technology movements certainly aren’t social movements on the scale of the civil rights movement, the women’s movement, but they are precisely “movements” because they are a migration of people and ideas, only loosely connected, in a common direction, that when it runs its course, will leave the world in a profoundly different place, or else, if it fails, will still be marked as one of those times when the world turned over in its sleep.
2. “It’s doing nothing for transparency and accountability in government.”
This assertion is simply wrong, and your argument for it, based on the notion that a co-optation or counter-attack by those who oppose transparency might succeed, is harmful. Your argument smacks of the notion that only movements that are ideologically pure can make change. I would argue instead that the most successful movements have many stakeholders with different goals. Your assertion is also refuted by the fact that the utilitarian wing and the transparency wing work together, support each other, and are learning from each other.
Overall, while I think you are asking good questions, and making a good distinction between the transparency wing of open government and the utilitarian wing, I am troubled by your strenuous attempt to render everything as “either-or.” That polarization is at the heart of so much that is wrong with politics today. I have always been an “inclusionist” – all of my “movement” stories are about how to make common cause between people who didn’t realize they belong together. I am less fond of movements that work very hard to drive out those who don’t fit the boundaries of their a-priori logic. The world is a messy place, and movements that change the world are also messy.
3. “It’s co-opting the language of progressive change in pursuit of a small-government-focused subsidy for industry.”
I think you misunderstand the notion of government as platform quite profoundly if you believe that there’s any subsidy involved. Take any great “government as platform” story – GPS, say. The government built that for its own use, and still has that use. The Gov as Platform element is that a policy change opened that for civilian use. No extra cost to government, lots of extra benefit to society.
Or more recently: When the city of Portland approached Google and other search engines with the notion that if they published their transit schedules in a standardized format (GTFS), would they use it, there was no subsidy by government. Instead, smart thinking by government led to a great deal of public utility at practically no expense.
Or consider now the great experiments in open data for healthcare. The notion isn’t that government shouldn’t provide healthcare, and should get out of the way of the private sector, as you twist my words to argue, but instead, that we should open up government data to build services that will save taxpayers money. (And in fact, the same people who are doing that are also those who would prefer a national healthcare system, but are trying to make the patchwork of public and private systems work better.)
Will some companies find ways to take advantage of open data and get a “subsidy”? Absolutely. But companies are doing that in the absence of open data. There is no logical connection between the two.
Overall, the notion that you can’t be “progressive” or concerned about the public good if you want to think differently about how government provides its services wrongs those of us who are doing both of those things.
Thank you for this thoughtful response, Tom. As I’m sure you could anticipate, I’m still not convinced. But I agree with others that this has been a useful discussion. Still, I find your continued suspicion frustrating, and can’t help responding to some elements of this latest post.
Perhaps it’s mostly a question of the frame through which you seem to view all of this. I have to admit that I heaved a sigh when you excerpted an accusation of neoliberal witchcraft aimed at someone from Adbusters(!). Certainly you should pursue those conversations if you enjoy them, but understand that many of us find it wearying to have this frame applied to absolutely every political project that expresses its argument in economic terms.
My colleague John is right to point out, for instance, that it’s not at all clear that the “winner-takes-all” frame borrowed from critiques of the global economy is at all applicable here. Certainly your own example of weather data seems to contradict it–the only evidence you’ve brought to bear is a new association with the admittedly disreputable financial industry (but then we might as well indict carpenters and the lumber yards they buy from, too). I can say that in my own case, Yahoo and the Weather Channel’s use and republication of meteorological data did absolutely nothing to stop me from incorporating it into a modest nonprofit SMS alert system I build many years ago (in fact Yahoo’s use of this data made my own substantially more feasible). Nor did it stop wunderground.com from evolving from a university project into a vibrant and much-used alternative to better-known corporate vendors.
Still, in some cases the implications of the (yes) movement’s economic arguments are important, and have substantial power: when Rufus argues against restrictions on supply of data, he’s doing so because in the digital realm — where marginal costs are basically zero — the deadweight losses imposed by such restrictions are potentially enormous. That’s a real argument, and one with real consequences for our advocacy (I happen to think Rufus is right, but certainly in other cases seemingly inescapable economic conclusions are being subject to reexamination).
In other cases, though, you’re reading too much into things. When Frances Maude says Open Data is the new face of deregulation, it means about as much as John Boehner saying welfare reform is about job creation, or Harry Reid saying a copyright bill is about protecting struggling artists. Politicians cast policies into the molds favored by their constituencies. Claiming that open data policies are deregulatory — despite their accompanying disclosure mandates, implicit enforcement of bureaucratic protocol, and threats to existing business models based on information scarcity — is at least open to debate, in my opinion. (As John has pointed out, there’s a vigorous conversation about these topics within our space that you might not fully appreciate.)
You seem to view open data initiatives as a huge new giveaway to private industry. They’re simply not. In some cases, there’s just not going to be a market for this stuff–if there was, it would already exist (note that advocates have an incentive to claim greater commercial potential for the data than might be justifiable). In other cases, as Carl has pointed out, freeing the data from restrictions will displace existing vendors. I assure you: Socrata — which is still quite a small business! — selling implementations of open data portals is costing us all much less than Lexis-Nexis gating and selling the data itself, as it has with patent data, for instance, and continues to do with various kinds of legal information.
The move to open data, open standards and open source doesn’t represent a shift of resources toward a new market, but rather a shift in norms within an existing world of contractors that is currently beset by lock-in and proprietary standards, both of which cost government resources that could be better spent elsewhere. For many of us, the view promoted in the Invisible Hand paper — that government should focus on data release and leave consumer-facing interfaces to the private sector — has less to do with pro-privatization ideology than with government’s miserable track record of producing substandard interfaces that focus on user experience at the expense of the data that would allow superior alternatives (commercial or otherwise) to exist at all.
Finally, let me address your skepticism of standards. To me, this is frankly bizarre — particularly since you acknowledge that, “Data’s value is combinatorial. It is most powerfully used by those who can combine it with other sources of data and who have the scale and resources to use it effectively.” I believe you’ve been involved in technical procurement, if not the engineering itself; to the extent that’s true, I certainly don’t mean to just shout you down. But, speaking as a and on behalf of software developers, open format skepticism is about as credible as arguing that the Earth is flat. A huge amount of expensive duplicative effort can be avoided through standardization, and not simply through the adoption of enormous vendors (I invite you to browse GitHub or the Drupal project’s repository for just a few open standards-enabled, open source modules that use GeoXML, Atom or Open 311). If a standard is proprietary (ESRI’s GeoDB), it can be harmful and should be avoided. If it’s insufficiently comprehensive, perhaps it will need to be extended (GTFS’s recent adoption of realtime capabilities).
But the idea that governments should try to “innovate” their own standards flies in the face of decades of practical experience, and is a surefire recipe for whatever you create needing to be torn out and wastefully rebuilt in just a few short years. I don’t think it’s too strong to say that, for a software engineer, creating bespoke standards when alternatives exist amounts to professional malpractice. The only justification on offer seems to be erecting hurdles to private use of data — since consumption of data is (unlike just about everything else) non-rivalrous, I still don’t see any rationale for introducing this eye-popping inefficiency other than a murky but strongly-felt anti-business orientation that’s being borrowed from other liberal intellectual projects and sloppily applied to this very different circumstance.
Tom Lee – I think you’re seriously misunderstanding the man you’re talking to here. Tom S. demonstrably has no problem with “absolutely every political project that expresses its argument in economic terms.” He has a genuinely excellent, and theoretically sound book applying game theory to questions of individual choice. See Alex Tabbarok, who is avowedly no friend to Tom’s politics.
I haven’t posted to this conversation because my main take-away is so small as to be probably lost in the conversation. I’m not going to weigh in on if it is a movement, I’ve got particular ideas about the economics of open data, but my real question, which gets lost in the above conversation, is
WHAT DATA ARE WE TALKING ABOUT?
I believe that government data is of three kinds: operational, administrative, and statistical. The open gov movement as I have experienced it almost completely disregards statistical data in favor of operational (snow plows) and administrative (staff salaries).
There is as yet no substitute for the federal statistical system, gathering data on the polity at a fine-as-possible geography. Poverty rates, unemployment, population density, etc.- as yet the open gov data movement has ignored how these questions will be answered with data exhaust created from government operations and administrative activities.
For the most part, the eco system of organizations that have worked with local non profits and in local settings to use and wedge out better data for understanding urban America has not been part of the open gov data movement, which is a loss.
@VL Carlson – couldn’t agree more. (I’m on the supply side of open data in UK local government) we are harried for political stuff, wages and spending data mainly, whilst the actual useful stuff, the stuff that’d lever in funding or might actually help drive an intelligent debate about a locality is ignored.
And I’m afraid I largely blame lazy “data journalists” (oxymoron if ever there was one) and hacks looking to make a cheap buck by getting their name out there for making every train in the country appear in a different colour on Google Maps or something. As it stands the open data movement is a market led, lowest-common-denominator approach which provides a short-term bit of excitement but offers limited long-term value.
That said, I really don’t mean to denigrate the excellent work done by those who are actually working to make government more open and I’m sure as the novelty of a newsroom hack being able to pull together a thematic map on Fusion tables wears off the quality overall will improve.
Tom,
This is a public service and great analysis, thank you.
But I disagree that the problem is that Tim O’Reilly’s business uses the language of civic engagement to hide big business ambitions. It does that, sure, and that wouldn’t be a bad thing, if it *didn’t also have its own notion of what civic-minded was, which didn’t just serve big business, but serves technocommunism*.
This isn’t a term you even agree with, I realize, but “neoliberalism” isn’t a term I’ll ever accept with a straight face, since it’s always used by people who give away their judgement of capitalism, with a lurking Marxist analysis not far from view.
Tim O’Reilly’s business cult isn’t just flim-flamming government under the guise of looking like an NGO movement. It’s flim-flamming government to collectivize data and use it for technocommunism — and undermine democratic government in the process.
The take-over is very reminiscent of how all real-life communist experiments end — with suppression of criticism and with wealth for a few oligarchs and state-capitalist loyalists.
The problem with those hardy civic folk in your rocky north quadrant is that they really don’t have a liberal civic notion of democracy, either. They are using the label “civic” to demand socialist and revolutionary changes to society, all while pretending to be “the people”. So there is bad faith on all sides here.
And then there’s the boorish culture of the open source cult. That’s why the code for America types can end up with apps paid for by taxpayers that route young men to all the free condom dispensers in the city, or route young men to data banks of designated drivers when they’re drunk, or route young men to entertainment, while sticking in a few “clean air” or “health food” apps for cover — without every creating a useful authentic public service apt that helps everybody find the clean public bathrooms. And I say this not as some aggrieved feminist but simply as a commentator on how these things turn out: self-serving, because they aren’t really liberal or democratic.
I think what’s actually happened is not that the old civil liberties crowd got gulled by a civil-liberty sheen from big IT posing with this mantra (at least as they exist in the US, which is a combination of Norman Thomas and Dorothy Day socialists and faith-based humanists as well as secular lawyers in the Democratic Party).
I think another, worse thing happened, which is big IT and its forward shocktroops in Electronic Frontier Foundation, Participatory Culture Foundation, etc. etc. are exploiting the civil liberties language and values to cover up their anti-copyright and anti-intellectual property battles as “free speech”. I find this the most abhorrent thing they do.
I wonder why it is that you, Tom, and others can’t see the bald-faced communistic imperative and Marxist scientism in a statement like this from O’ Reilly:
“I’m instead trying to suggest that platform thinking, as learned from technology, can show us a third way – neither slavish adherence to a failed philosophy of the market’s wisdom when left to itself, nor to a failed philosophy of unlimited government intervention, but to a design in which we think about outcomes, and design the interfaces between government and society to promote them.”
We, Tim? Who is We? Only the coders? Only the likeminded? Only those who can engage in “constructive” conversation? Only those with whom you deign to talk after they cease “ranting” and make a McMillan matrix for you that looks like an engineer’s diagram?
The market’s wisdom, when left to itself, is actually pretty wise. It stampedes away from the platform of Google+; it stampedes away even from Twitter, with its content-producers’ power curve, and stampedes into the Facebook platform, the sort of walled garden you hate. Good for the market!
A bit late but I wrote a response here: http://omniorthogonal.blogspot.com/2012/05/open-government-and-private-interests.html
On your is-it-a-movement footnote, here’s a bit of the appendix to my book:
Social movements can be defined as groups within society which combine three factors: an oppositional or reformist stance towards the political status quo; autonomy from the conventional political sphere; and a degree of ideological or cultural coherence, based on a shared set of fundamental concerns or values. Social groups which have two of these properties but lack the third are not social movements: examples include groups of hobbyists, parliamentary pressure groups or groups defined by an officially discouraged activity (e.g. cannabis smokers).
To unpack that last bit (I was editing pretty harshly towards the end):
“groups of hobbyists”: autonomous from conventional politics, shared values, not oppositional
“parliamentary pressure groups”: shared values, oppositional, not autonomous from conventional politics
“groups defined by an officially discouraged activity (e.g. cannabis smokers)”: oppositional, autonomous from conventional politics, no shared values
I agree that Open Data types don’t tick all three boxes, but what’s interesting is that they don’t fall cleanly into the first group; some look much more like the third. You could define being a hacker, in particular, either in terms of sharing a distinct culture (but not being particularly radical politically), or in terms of being politically oppositional (but not sharing any particular culture). Just not in terms of culture and political radicalism – it’d be neater but not true.