Volkswagen, IoT, the NSA and open source software: a quick note

(Attention conservation notice: the most interesting paragraph is the one about project NiFi, starting “Finley also writes…”)

According to Klint Finley in Wired, the lesson from the Volkswagen testing scandal is to use more open source software in more places. In particular, that Internet of Things (IoT) devices should be driven by open source software (even though the VW was not an IoT case). Here is Finley:

To protect consumers and realize its true promise, the Internet of Things must go the direction of the software and hardware that supports the Internet itself: it must open up…

Today, the vast majority of smart home gadgets, connected cars, wearable devices, and other Internet of Things inhabitants are profoundly closed… Ostensibly, this is for your own protection. If you can’t load your own software, you’re less likely to infect your car, burglar alarm, or heart monitor with a virus. But this opacity is also what helped Volkswagen get away with hiding the software it used to subvert emissions tests.

This seems wrong on two counts.

Finley writes about initiatives like the OpenWrt operating system for embedded devices as an alternative, but a lot of IoT devices already run on Linux. What stops individuals from being able to exert control over their gadgets is the use of Linux permissions structures, not the openness of the OS code. IoT security frameworks will be much like security frameworks on Android and other mobile operating systems: sandboxed applications running in their own user space, using the security features of the operating system. The open source/closed source distinction is essentially irrelevant to the problem.

Finley also writes that the closed nature of some IoT devices “makes it harder to trust that your thermostat isn’t selling your personal info to door-to-door salesmen or handing it out to the National Security Agency.” Which is ironic, because the software that might be handing out your personal info to the NSA is already open source. The NSA NiagaraFiles  project provides routes data among different computer networks and protocols. The NSA recently released this software as open source, and it is now hosted as an Apache project called NiFi. So that is the open source community (in the form of Apache) actively assisting the NSA in its data collection activities. The core developers on the project are all from the NSA and defense contractors (link). And NiFi is being touted as a big thing for IoT applications, so that your personal info can be more effectively routed to more destinations. The NiFi project is one more step in the active collaboration of Apache with the NSA, which I discussed back here and here, and tangentially in my FORTHCOMING BOOK.

The VW case is important and raises some big questions, but open-source vs closed source software is not  one of them. For a better take, see Zeynep Tufekci here.

(Full disclosure and openness: in my day job I have some involvement in IoT projects. My employer — FOR WHOM I DO NOT SPEAK –uses a mixture of open source and proprietary code in its work.)

Bookmark the permalink.

8 Comments

  1. Tom, I really don’t understand your position. Even the Tufekci piece you recommend identifies proprietary copyright claims as being one of the barriers to accountability for software-managed systems. Yes, it’s true that building a system on free software only grants audit rights to the owner of the system. If a user is not the owner (ie does not have root access), he or she will be unable to audit the integrity of the code. Device lockdown/DRM is one manifestation of this problem, but of course if Diebold owns its own voting machines and leases them to users then those users might not be granted root access and won’t be able to verify the integrity of the code running on the leased machines. GPLv3 includes provisions intended to defeat device lockdown, but there are of course ways to structure transactions to minimize software freedom. These are known issues from way back. Maybe I’m confused about who your audience is…?

    • Picador: I think we’ve had this discussion before 🙂

      For me, cloud computing (where the systems are obviously unauditable) makes the idea that open source software license are some form of protection against the surveillance state implausible.

      But worse is the fact that the surveillance state infrastructure is built on open source software maintained with the active assistance of Apache. Given a choice between limiting software freedom and assisting the surveillance state, Apache has chosen to assist the surveillance state, and Wired is now invoking FLOSS as protection against the surveillance state it (Apache) is enabling. Of course, many proprietary software companies also assist the operations of the NSA (and are frequently criticized by FLOSS advocates for doing so), and in a market economy one can expect this, but FLOSS has set itself (or I thought it had set itself) a higher standard.

      I do appreciate that I am presenting a complex collection of organizations and interests and points of view as a single entity. But I am arguing against a similarly blanket set of assertions on the part of Wired and others.

      I also admit that the Volkswagen case is different. But here, (some people at) VW set out to deceive. The kind of audit one would have to do to prevent such active attempts at deception seems to me more intrusive than code review, and real-world testing seems like a more realistic approach. Code review couldn’t hurt, and may help, but (and I admit this is just a matter of judgment from a distance) it seems far from the centre of the problem.

      • I guess my main problem is that the people and organizations who have pushed for software freedom since day one — the FSF, the SFLC, and so on — have ALWAYS been aware of these various collateral avenues of attack on user freedoms and have ALWAYS actively worked to address those as well. Stallman sets out a set of basic freedoms, and the GNU GPL is just one attempt to secure those freedoms, within a very specific social context. DRM has long been recognized as a potential end-run around free software, and GPLv3 was drafted in part to address that concern. Cloud computing has long been recognized as a threat to software freedom, and the Affero GPL was drafted explicitly to address that concern. But of course there are many other threats to the basic user freedoms that Stallman sets out in his early manifesto, and nobody thinks that free software is somehow magically capable of securing those freedoms in the absence of other supporting social factors. Free software doesn’t protect my privacy if the state stations a police officer in my bedroom to observe me 24 hours a day. Free software doesn’t promote transparency if a representative of the software company stands over my shoulder and threatens to slide a stiletto into my ear if I look at the source code. I am really very dubious that anyone is making these kinds of claims about free software being a universal panacea.

        Where free software DOES make a difference is in a social context whereby people are told that they have “purchased” a piece of software, and it is running on “their” device, but they are not allowed to inspect or modify the source code for that software (where “source code” is defined broadly enough, as it is in licenses like the GPL, to include things like runtime configuration parameters). THAT is a social context in which free software makes all the difference in the world. And while it’s not universal (and arguably not as common as it was in the 90s), it’s still pretty widely applicable.

        As for your beef with Apache: I’m aware that there are “open source” advocates who have adopted a depoliticized strain of free software advocacy (or rather, a strain politically aligned with the pseudo-libertarian ideology of Silicon Valley capitalism). They have long been counseled by the more principled members of the movement that their position was incoherent and would not be effective in advancing the goals they claimed to value. If the Apache community has been collaborating with the NSA, I would say that that’s a pretty solid vindication of the criticism coming from the more principled wing of the movement and a serious discredit to the “open source not free software” crowd.

        • Picador – I make no claim to be saying anything original here, and I know that the issues were debated decades ago by early FLOSS advocates. But, a few points:

          – One is that the context is now so different that the debates are surely not done once and for all. Old and settled debates come up again — ask the British Labour party!
          – I don’t think a movement based on openness can distinguish between “the more principled members of the movement” and those who follow the pseudo-libertarian ideology. The NSA approach to open software is explicitly FLOSS, and while they may have adopted an Apache license, I don’t think a more GPL-ish license would have made any real difference to what they are doing.
          – Even if the debate was over and done a couple of decades ago in your part of the openness movement, those of us outside that have a right to a say as well, especially in the light of the Snowden revelations. Stallman repeatedly advocated freedom over any limitations, for example, on military or other “for use” restrictions. That’s his choice, but while he deserves respect I’m sure you’d agree he is not beyond critique. The bad-corporate vs good-FLOSS distinction seems to me to have vanished now. My day job is in the enterprise software world, and that world now happily moves back and forth between openness and proprietary as it suits: the various Apache enterprise and big data projects being some of the more popular examples.

      • Real-world testing might be the best solution for the narrow case of preventing car companies from making cars that drive one way in tests, but another way on the road.

        But there are other exploits in which testing is of pretty much no use at all. You can’t expect to find a backdoor by just monitoring the outputs of a device–not unless you could test the device against every possible signal, which would take infinite time. You can’t test that the device would operate one way this year, while you’re testing it, then silently start operating a different way next year after the testing is complete.

        Testing is only useful for detering VW-like misbehavior because VW was trying to optimize for things that were immediately apparent–the car’s performance in emission’s test versus the car’s performance on the road at time of purchase. If you want to verify that devices are operating the way they’re supposed to in the general case, I don’t think you can do that without somehow looking inside at the hardware and software. “Looking inside” doesn’t have to mean open or free, but it has to mean something more than testing the external behavior if there is to be any hope of trustworthy devices at all.

  2. First of all, thanks for your reply. I agree that there’s more to solving these problems than just open source software—open source is just one piece of it. I certainly hope I didn’t imply that open source was a cure all. I also agree that Zeynep’s piece is indispensable and she makes a lot of important points. If you have to choose between reading one or the other, I would certainly recommend hers over mine.

    My point in the piece is that openness, not necessarily open source, is a key part of solving the problem here. I think part of what you’re tapping into here is the fact that just because something is built with open source software doesn’t make it open. All sorts of cloud services are built on open source software, and as you point out the NSA uses quite a bit of open source software (they’re also big users of Hadoop and OpenStack, which to me is the bigger ethical question for the open source community than the Apache software agreeing to host code that the NSA has decided to publish). But just because a cloud service is running open source software doesn’t mean you have any more control over it than you would a cloud that’s running a proprietary operating system. When it comes to the cloud, the important thing is knowing where your data is going and how it’s being used. (I think we’re in agreement on this point, I just wanted to clarify it.)

    Likewise, a gadget that’s running Linux but is locked down in such a way that you can’t replace its firmware is not really an open gadget. Plus it’s quite likely running some proprietary software on top of Linux, and communicating with a server somewhere that’s entirely outside of your control but is required to make the gadget function. Being able to replace or alter the software or firmware of gadget means that you can set it to communicate only with a server that you trust, or perhaps, if there’s enough brains on board, with no server at all. Even if a gadget is running nothing but FOSS but you don’t have root access, then it’s not really all that open of a product. I certainly could have made that more clear in the article.

    There are a bunch of other issues with open source in general, particularly its power dynamics and its reliance on free labor (see https://modelviewculture.com/pieces/the-hidden-power-dynamics-of-open-source and http://www.ashedryden.com/blog/the-ethics-of-unpaid-labor-and-the-oss-community for instance). But I stand by the claim that increased openness is a crucial part of getting our technologies to stop spying on us.

  3. Apologies to Picador and Klint for not picking up on your responses – it has been a hectic few days. If either of you are still checking, let me try to add a few things.

    First, I agree with you that I underestimated the degree to which closed software may have been a problem in this case: some people in VW clearly knew about their problems, crossed their fingers, and hoped they would get away with it, and they may have been less likely to do so if their code had to be open to scrutiny. Also with Consumatopia’s point that black-box testing after the fact is no broad substitute for a proper audit.

    I do think there has been confusion about the costs and benefits of openness in this case (open as in open source, and also – as Kint writes in the comment above – open in the “freedom to tinker” sense).

    There are three parties involved here: the manufacturer, the car owner, and third parties who may provide friendly or hostile modifications to the software in the car. And the issues that are at stake are “freedom to tinker” (ownership), and security (malware) but mainly, in this case, adherence to standards. And some difficult choices have to be made.

    I find David Golumbia’s post convincing, that if you permit openness in the “freedom to tinker” sense, then you may be less likely to have the VW problem, but you are opening up a whole other set of probably-worse problems: aftermarket products that let drivers break the standards, in addition to malware. Giving everyone the power to read and modify the emissions software in their cars will lead to more wrongdoing rather than less.

    You can, as Kint says, have open source without openness (I didn’t know until a colleague told me about some of the secure-boot technologies to check that no modifications have been made, but that’s the way these things would have to go). That would be using open source to build a secure system in the sense of unmodifiable by owners or third parties. But I don’t think you can securely allow owners to modify without allowing third parties, and you can’t allow either owners or third parties to modify if you want the standards to be enforceable (which is at the heard of the VW scandal of course) so security implies a closed system, no matter whether the algorithms and code is open or not. And Picador would not be happy because that would remove the freedom to tinker that is, for him, central.

    If the open-but-secured solution was pursued then, and this is a guess, I doubt that the kind of scrutiny that is needed would be forthcoming from the broader open source community. If you can’t modify the software, it’s not clear that there’s a lot of incentive to audit it. And if the code was posted somewhere for external scrutiny then you’d have to trust that the code is what’s on the box — which is assuming the good faith that you are trying to catch.

    So I do remain sceptical that openness is at the core of this problem. But thanks for making me think more carefully about it.

Comments are closed