Update 2: This essay is the starting point for Chapter 7: “Strangers Trusting Strangers”, in my book “What’s Yours is Mine” (2015 in the US and the UK; 2016 in Canada and elsewhere).
Update: now with added footnotes.
Debates around the “sharing economy” have been driven by personal stories and broad claims. In contrast, this is a dense and not-easy-to-read step-by-step look at the internet reputation systems on which the sharing economy claims are based, with some predictions about the neoliberal future of the sharing economy. References, statistics, and, yes, some personal stories are relegated to footnotes. If you’re really interested you can download a PDF.
Introduction
Internet reputation systems let individuals rate other individuals over the internet and provide recommendations based on those ratings. A new class of enterprise claims to use internet reputation systems to enable sharing of personal goods and services at unprecedented scale. Its rise has been announced by both Forbes, and by The Economist, according to which accommodation rental service Airbnb…
…is the most prominent example of a huge new “sharing economy”, in which people rent beds, cars, boats and other assets directly from each other, co-ordinated via the internet. …[T]echnology has reduced transaction costs, making sharing assets cheaper and easier than ever—and therefore possible on a much larger scale… social networks provide a way to check up on people and build trust; and online payment systems handle the billing.
The claim is that internet reputation systems solve two problems. One is coordination (can I find someone who has what I want, or wants what I have?) and the other is trust (can you trust the person on the other side of the exchange to keep their end of the bargain?).1 Sharing economy advocates claim, and I will return to this at the end of the essay, that it is both necessary and sufficient to solve these problems to unlock a large new economy of resource sharing.
Trust and Coordination
To understand the sharing economy it is necessary to understand trust.2
A truster must decide whether or not to make a loan to a potential trustee; if the truster does make the loan, then the trustee must decide whether or not to repay it. We say the truster trusts the trustee if she expects him to repay, and the trustee is trustworthy if he would repay a loan, should the truster make it.
Trust is a problem of asymmetric information: a truster cannot divine the trustee’s trustworthiness directly but must look instead for signs of trustworthiness.
An opportunist is someone who is not trustworthy but who seeks to mimic signs of trustworthiness in order to deceive potential trusters. Opportunists create what Bacharach and Gambetta call a “problem of secondary trust” which, they argue, “almost always accompanies, and is often the key to solving, problems of primary trust” (p158). Instead of just looking for signs of trustworthiness, the truster must decide whether she can trust those signs; instead of just displaying signs of trustworthiness, the trustee must convince the truster that he is not mimicking them.
Secondary trust is a signalling problem in the sense first spelled out by economist Michael Spence. An effective signal is an action or sign that is easy for a trustworthy person to display but costly for an untrustworthy person to display. If it’s not worth the effort for an opportunist to mimic the signal, we say that the signal separates trustworthy people from untrustworthy people or discriminates between them. If no discriminating signal is available, then there is no way to distinguish trustworthy people from opportunists—an outcome that is called pooling—and trust cannot be established between truster and trustee. In real life, of course, we deal with probabilities rather than certainties, but there is a spectrum from separating to pooling outcomes in problems of trust.
The Economist observed, above, that the internet has reduced the transaction costs of collaboration, enabling what Yochai Benkler calls a “new modality of organizing production: radically decentralized, collaborative, and non-proprietary… ‘commons-based peer production’” (The Wealth of Networks, p60). But the problem of secondary trust emphasizes that low transaction costs do not necessarily improve collaboration.
Other things being equal, and in the absence of opportunists, lower transaction costs (discovery and communication) should increase the amount of collaboration, but in the presence of opportunists—in what Bacharach and Gambetta call “mimic-beset trust games”—collaboration is possible only in the presence of an effective signalling mechanism, and lowering transaction costs can destroy trust-dependent collaboration by making it easier for opportunists to mimic trustworthiness.3
Reputation
In the sense used here, reputation is a sign of trustworthiness manifested as testimony by other people. When my neighbour says “Don’t hire John the Plumber: he came to fix my sink but it’s still blocked”, she is providing information that lets me decide whether to trust John to fix my drains.
When it works well, reputation is an effective discriminating signal that promotes trust and collaboration based on trust. In a community with strong word of mouth, it is easy for a good plumber to establish a reputation as reliable, punctual, and skilled simply by being reliable, punctual, and skilled; it is difficult for an incompetent or lazy plumber to do the same.
Reputation is not a perfect discriminating signal. Much of what is communicated in testimonies may be private and informal (“he fixed my sink and came on time, but there was something about him… I just didn’t like having him in my house”) and this privacy and informality can have both good and bad effects. It can transmit justified but nebulous suspicions, but it makes it difficult for John to gain a good reputation—no matter how trustworthy he is—if he is a black man trying to find work in a white community with a history of racism, or difficult for Jane the Plumber’s skills to be taken seriously if the community has traditional norms about women’s roles. “Old boys’ clubs” and other insider groups provide members with an inbuilt advantage when it comes to establishing a reputation.
Reputation is only one mechanism for solving the problem of trust. Others include reciprocity in long-term relationships, regulations (you can trust this restaurant because it has passed a food safety inspection), professional qualifications (you can trust this person to fix your leg because she is a doctor), voluntary industry certifications (you can trust this coffee to be fair trade because there is a fair trade label on the package), independent rating agencies, individual firm commitments (you can trust this retailer because they have invested heavily in their brand, and so must act accordingly), the common property regimes explored by Elinor Ostrom, and many others.
Reputation, in the sense used here, is peer-to-peer, informal, decentralized, community-driven, and non-commercial, and it is those alternative qualities that sharing economy advocates claim can be scaled up by using internet reputation systems. Airbnb and BlaBlaCar both describe themselves as “a trusted community marketplace”; Lyft’s one-million rides show “the power of community”.
The effectiveness of reputation depends on the motivations of those giving testimonies as well as on the actions of the trustee: the problem of secondary trust described above. Reputation is effective only if the testimonies are independent and free from the taint of collusion or retaliation. Testimony from John’s brother does not carry the same weight as that of someone who has no stake in John’s success or failure, and while John may not want my neighbour to tell me about his failure to fix their sink, there’s not a lot he can do about private conversations over a garden fence.
Market-based incentives erode the effectiveness of reputation, and in this respect reputation is a cultural commons (see here, and see also me here). In her TED talk, influential author Rachel Botsman says that in the new economy “reputation will be your most valuable asset”, but as reputation becomes an important asset, markets will grow around it and intermediaries will claim to help you boost your reputation, but these market-based incentives destroy the value of reputation as a mechanism for establishing trust. Mechanisms for buying and selling testimonies, for example, cause testimonies to lose their ability to discriminate between trustworthiness and opportunism because an opportunist with money could buy themselves a good reputation.
Internet Reputation Systems
Internet reputation systems promise to create a global village by scaling up informal word-of-mouth reputation mechanisms for sharing and for creating trust, and so solve both the coordination and the trust problem for a variety of services which could not previously be exchanged. For sharing economy advocates, reputation is an alternative to regulation: in the recent book The Reputation Economy, law professor Lior Strahilevitz asks us to “imagine if every plumber, manufactured product, cell phone provider, home builder, professor, hair stylist, accountant, attorney, golf pro, and taxi driver were rated… In such a world, there would be diminished need for regulatory oversight and legal remedies because consumers would police misconduct themselves.”
Do internet reputation systems act as an effective signal of trustworthiness?
Figure 1 is the distribution of ratings for the Netflix Prize data set. Netflix ratings are not a reputation system in the sense used here, in that they are not testimonials about people: the data set consists of ratings of movies and TV shows by Netflix customers. There is every reason to believe that the ratings are independent and honest: the rater can offer an opinion freely, having no reason to expect expect reward or punishment for any particular rating. The rater also has an incentive to give a rating that matches their actual opinion, as it enables Netflix to recommend movies that better match their tastes. So Figure 1 can take this as a reasonable distribution of independent ratings.
- Figure 1.
BlaBlaCar, a French sharing economy company that connects “drivers with people travelling the same way” throughout Europe, has over a million registered drivers, transports over half a million passengers every month, and is expanding rapidly. Also, it makes testimonial-based ratings available on its web site. Figure 2 is the distribution of a set of 190,000 ratings from the blablacar.com site.4
- Figure 2.
Of 190129 distinct ratings, 2152 were one-star, there was not a single two-star rating, there was one three-star rating, five four-star ratings, and 187971 five-star ratings. A BlaBlaCar rating means something different from a Netflix movie rating.
With over 98% of ratings being five stars, the reputation system does not meaningfully discriminate among drivers or riders. A reputation system that does not discriminate fails as a reputation system: it fails to solve the problem of trust.5
Collusion and fear of retaliation are the reasons why there are essentially no reviews less than five stars for rides that take place. If you give a less-than-five star review then, unlike in the case of offline community-based testimonials, it is visible to the reviewee, who can give you a harsh review in return and so affect your chance of getting future rides. Do you want to defend your opinion that the driver was a bit close to the car in front, or that the car was a bit dirty, or do you just want to give a five-star review and make a note to yourself not to ride with them again? Collusion is the other side of the retaliation coin: I know I turned up late and was eating smelly food in your car and you didn’t like it, but so long as you give me five stars I’ll give you a good positive rating and we’re both better off. Neither of these factors need to be explicit or even to be very important to produce large effects, because it makes no difference to me how I rate you. One seemingly tiny difference between word-of-mouth and the internet rating system makes all the difference, that testimonials are visible to everyone including the reviewee instead of everyone except the reviewee.
The problem is not unique to BlaBlaCar. Reciprocity and collusion in the eBay reputation system has been studied here and the authors also provide an estimate of how many dissatisfied people are not rating their trustee:
The fact that from 742,829 eBay users… who received at least one feedback, 67% have a percentage positive of 100%, and 80.5% have a percentage positive of greater than 99%, provides suggestive support for the bias. The observation is in line with Dellarocas and Wood (2008) who examine the information hidden in the cases where feedback is not given. They estimate, under some auxiliary assumptions, that buyers are at least mildly dissatisfied in about 21% of all eBay transactions, far higher than the levels suggested by the reported feedback. They argue that many buyers do not submit feedback at all because of the potential risk of retaliation.
Finally, on Airbnb, reviewing of hosts by guests and guests by hosts also happens in public and is reciprocal. The Airbnb web site does not display individual numerical reviews, although it does display individual text reviews; instead it displays the average rating that a room has received in each of several categories (cleanliness, location, communication,…) together with an overall average, rounded off to the nearest 0.5 out of five. The web site is less easy to traverse programmatically, but out of well over a hundred offerings in New York, Sydney, Berlin and Paris I have yet to see a single one that is not rated 4.5 or 5.6
So even in the absence of explicit gaming, peer-to-peer internet reputation systems do not solve the problem of trust. The BlaBlaCar site fails the basic test of discriminating among almost any of the 190,000 drives that took place—it fails to deliver any useful information beyond giving the occasional sign that a driver or rider may not turn up.
The “Growth” of the “Sharing Economy”
Rachel Botsman claims that “Even four years ago, letting strangers stay in your home seemed like a crazy idea”, and she describes the meteoric growth of the sharing economy. The picture she paints would seem to be incompatible with the idea that internet reputation systems fail to solve the problem of trust. What’s going on?
Some perspective is in order. The rapid growth of individual sharing economy companies does not represent the appearance of new social practices of sharing. The growth of sharing economy companies is, at least in part, a movement of already-existing social practices to online forums.
“Letting strangers stay in your home” has long been a common practice. Millions of people let strangers stay in their homes without the benefit of internet reputation systems: the overall vacation rental market, which includes cottages, apartments, second homes and other personal rentals, is much larger than Airbnb.7 , 8
Similarly, BlaBlaCar is a small fraction of the overall carpooling practice, which precedes internet reputation systems. Carpooling has been a widespread practice in many urban centres for decades, encouraged by local governments and transit authorities with “Park-and-Ride” facilities, especially in Europe where fuel costs are higher than North America. Student union noticeboards have long been a way to coordinate rides home for the weekend.9 , 10
The dispersed nature of tourism boards, local travel authorities, booking agencies, university notice boards, and so on make counting trips and visits difficult, while the centralized nature of sharing economy sites makes data collection trivial for those with the infrastructure, so it is easy to underestimate—or entirely neglect—the pre-existing economy.
Also, not all the activity on sharing economy websites is of the personal, informal kind that the site owners portray, so the growth of these companies overstates the growth of sharing. Multiple rentals are common, suggesting rentals of properties other than the host’s primary residence, such as investment properties.11 , 12
The Problem of Trust in the Sharing Economy
Still, sharing economy web sites are growing fast. How are they succeeding if the peer-to-peer reputation systems fail to solve the problem of trust?
One reason is that coordination is useful in itself. Classified ads, whether local newspapers or sites like Craigslist and Kijiji, solve coordination problems but do not even try to solve problems of trust beyond the most basic verification. It’s left to individuals to contact each other, make arrangements, decide on the terms of a sale, and complete the deal. In trading second-hand lawnmowers for cheap prices, the worst that can happen to a purchaser is that they overpay by a few dollars, and that’s a risk that many are prepared to take. One option for sharing economy companies would be to accept that they are solving only the simpler coordination problem, and adopt a business model that has no involvement in the transaction itself and which charges small listing fees.13 But such a business model will not provide the returns that venture capital is expecting from this industry. Sharing economy companies funded by venture capital have no option but to solve the problem of trust.
In some cases, community membership itself has provided an adequate signal of trustworthiness, particularly in communities that are prepared to accept some level of risk. Opportunists are screened out, to the extent that they need to be, by an implicit community selection process, so that matching within the community is reduced to a coordination problem. For example, travel site Couchsurfing built itself largely by word-of-mouth among young travellers. Couchsurfing members pre-selected themselves to be adventurous (so not looking for a high degree of assurance from the organization) and community-minded individuals with a common interest in travel. Simply being part of the Couchsurfing community was a sign that correlated with trustworthiness, and community members voluntarily undertook the additional risk that came with the program.
Unfortunately, community membership as a sign of trustworthiness does not survive large scale growth, for two reasons.
As a community grows, it attracts opportunists. In a small community, the benefit to an opportunist of mimicking a sign of trustworthiness is small, but as the community scales, the potential benefits for opportunists are larger, and the incentive to mimic trustworthiness is greater. In evolutionary terms, Bacharach and Gambetta describe the phenomenon as “model precedes mimic”. Sharing economy sites have benefited from community membership as a screening process, but as they become larger they will need new solutions.
Second, people who have a commitment to a community may be prepared to take on additional personal risk, either because of the nature of the community itself (a community of adventurers is not looking for a high level of security) or because they are prepared to overlook lapses to support the community. There is a reciprocity, not just between individuals, but between the members and the community-as-a-whole. However, the revenue and growth models of venture-capital funded companies are based on providing a service that can scale to people with no particular commitment to the community itself. As AllThingsD reporter Liz Gannes writes: “maintaining customer trust is paramount because, at any given moment, they are all one bad incident away from users turning back to more traditional arrangements”
The Future of the Sharing Economy
Venture capital demands for scale will produce changes in the nature of the sharing economy sites, changes that erode any community focus they have, and which turn them into far more traditional models. Such changes are already underway at the largest, most heavily funded sites.
As Gannes reports, a single bad incident has forced Airbnb to hire a 50-person “trust and safety team” headed by a former US Army intelligence office and a former government investigator. The use of a human team clearly doesn’t scale, so Airbnb is now turning to centralized analysis to solve its problems, saying “We want to apply data to every decision. We want to be a very data-driven company.” On April 30 2013, asserting that “Trust is the key to our community”, Airbnb introduced a “Verified ID program” which demands that you provide government-verified identification and permit the company to analyze your social networking presence or provide it with a video profile.
There is also a drive for more professionalism among hosts. Airbnb now lets hosts sell tours and activities, and here is Chip Conley, the new “Head of Global Hospitality” for Airbnb, hired from the hotel industry, in a September 2013 interview:
We’ll be introducing nine minimum standards around what we expect an Airbnb experience to be, whether it’s related to cleanliness or the basic amenities you expect, which is not currently the case. The idea that we create some amenities that you should expect—clean towels, clean sheets—that’s important.
In short, Airbnb is abandoning the idea that peer-to-peer reputation systems can solve the problem of trust, is moving away from the casual “air bed” mentality that gave it its name, and is resorting to traditional centralized systems of enforced minimum standards, documentary verification, and so on.
There is, however, one remaining difference between Airbnb and a traditional hospitality business. To go back to the beginning of this essay, sharing economy companies claim that it is both necessary and sufficient to solve problems of trust and coordination to unlock a large new economy of resource sharing. The “sufficient” part of this is valid only if there are no spill-over effects from the operations of the sharing economy, so sharing economies will campaign for freedom from those constraints that prevent them maximizing their returns: health and safety standards, employment standards, licensing laws, and so on.14
To be successful, the venture-capital-funded “sharing economy” will be forced to lose all those aspects of informal sharing that makes “sharing” attractive, and to keep those aspects that erode neighbourhoods, erode employment rights, and remove basic standards. And if they succeed, they will have used the language of sharing to bring about an unregulated, free-market, neoliberal economy.
Footnotes
10 Personal story: My own pre-digital ridesharing experiences include a year of commuting between Hamilton and Waterloo, driven by four different—and all wonderful—people, as well as several years of intermittent and luckily incident-free hitch-hiking in the UK, mainly between Leeds and London.Quiz for the reader: how would you rate the following drivers?
- The German ex-prisoner of war who stayed in the UK and who gave me a ride in a small lorry, spending much of the ride explaining in a friendly manner how Hitler had been misunderstood.
- The driver who spent 30 minutes complaining about how dark-skinned immigrants were wrecking the country.
- The driver of the empty lorry who explained that he was picking up my friend Lawrence and myself on a windy day just to provide some extra ballast.
- The driver who had cerebral palsy and whose hand shook increasingly as he reached for the gear lever, explaining that he drove better when he’d had a couple of drinks because he didn’t tremble as much.
- Women may experience many kinds of problem that I have never faced. What is the acceptable level of flirtatiousness and/or overt propositioning from a driver? And would you report over-the-line behaviour if the driver had your phone number?
I think it’s very hard to accurately measure online reputation for just some of the following reasons.
* Reputation is subjective, not objective. Objective facts can be measured in some standardized units in some way. Subjective opinions can’t really be done this way. Things like Klout scores are an attempt to make it a reality, but its just one interpretation.
* All of the major review sites, social networks, and commercial entities are just about completely gamed through review and other types of manipulation. Even if you could come up with the perfect formula, it’s not really accurate.
* Something like peer to peer would be ideal, where you could use something like Facebook or whatever to gauge trustworthiness from people you know in real life. There’s more than a few problems with that though. Forget about privacy issues for a second, I think it’s hard to squeeze honest opinions about people when you’re not anonymous. People are worried about not offending anybody. (As an aside, Google+ could have been a juggernaut if they just went with nicknames and anonymity rather than forcing you to use your real name everywhere. I think they really screwed up by trying to clone Facebook).
* Anonymity works well to get honest opinions from people. Back at its peak 4chan produced the most amazingly entertaining information about everything under the sun. But there’s a price to anonymity. Things like DirtyPhoneBook have a lot of potential in that regard for getting honest opinions about people, but there’s a lot of trolling there as well. I think nicknames that are tied to one account are close to ideal, but as mentioned Google+ really screwed up on that.
* This might be controversial depending on your political bent. Not only can they be abused (The government is currently trying to create some kind of regulatory structure for determining who is a real journalist or not to mess with the careers of anybody who doesn’t toe the line as official mouthpieces of the state media) but things like regulations sometimes have the opposite of their intended effect. In theory, regulations mandating food safety inspections sound like a great idea. In reality, they create a situation where people aren’t really cautious and looking out for potential issues and dull people’s senses and get them to turn off their brains. I rarely think regulations are the right answer for anything.
* I’m more than a bit distrustful of professional qualifications. In computers for instance, virtually all professional qualifications are worthless for determining skill. Every doctor must be legally certified and every teacher must be as well but there are bad teachers and bad doctors and you have to research anybody you get any healthcare from.
* Services like Angieslist are probably on the right track, but I think most people balk at paying money for anything that’s kind of available for free.
* Airbnb is one of the more competently run companies I’ve seen in recent years, despite their handful of relatively minor growing pains. I think watching them grow will provide a lot of insight into where the internet will be going.
Interesting thoughts that spurred a few ideas.
Those are interesting comments (and fast!) Just one response — I agree that Angieslist is an interesting case. It’s not available where I am, so I don’t know it well. I agree it has a better chance of being on the right track.
This is the kind of article I wished someone would write, Tom Slee. +1 reputation to you, in whatever form you wish to take it!
I don’t entirely agree with your last paragraphs, though. I think there are ways to solve trust problems online. You write “peer-to-peer internet reputation systems do not solve the problem of trust.” For one, that’s a too high bar to raise – other reputation systems don’t fully solve the problem of trust either. There are degrees. For another, I think these sites can do much, much better without selling their soul (those that had any to begin with).
One thing is that you cast your net a little narrowly. You focus on AirBnB, Lyft, and the most recent venture-capitalized hype, but your observations apply far wider, and there are also lessons to take from that wider domain. Take Kickstarter. It’s also very much about solving a trust problem – not quite as peer-to-peer, but still fairly decentralized. If you compare it with their main competitor IndieGoGo and their now defunct spiritual ancestor Fundable, I think there are happier lessons to be learned than in your last paragraph. Kickstarter has more integrity, both artistic and economically, and is more successful for it.
One very early effort to think critically about (more purely) online trust was Raph Levien’s work on attack-resistant trust metrics, and the site Advogato which rose from it. If you don’t know about it (and far too few people do) I suggest looking it up and maybe even talk with Levien.
Also, he didn’t publish much, but I know that Steve Randy Waldman of Interfluidity used to be very focused on trust-building systems. He still is, I suppose, although he’s more focused on macroeconomy than websites now.
You are right to say that reputation is critical. Communities form around shared experience which has to be genuine, and this is an area where larger brands or franchises struggle to connect.
In the sports world where my main interest lies, being a fan means sharing experiences with fellow fans that are driven by what the team does but are strongest and expressed best when growing outside of a controlled ‘team’ portal.
The trick for those teams is to let go of part of their urge to control and become part of rather than the owner of a community.
When that happens it can open up all sorts of opportunities for engagement, learning and even sponsorship where brands will want to associate themselves with the free flow of (positive or even negative) emotion that can be generated.
Really great article Tom, thank you for sharing.
Resnick and Zeckhauser looked at this back in 2000 using eBay as a case study, and came to similar conclusions. These problems are not new, but we have since learned that somehow they don’t seem to impinge on the growth of the sites that employ these kinds of systems.
It’s interesting that you cited Netflix as an example of a review mechanism that provides useful information about a customer-base. That isn’t really the case. Netflix’s head of engineering recently said that Netflix is not relying as heavily as they once were on ratings data to inform their recommendation system. Instead they are looking at other features of their customers’ behaviour (what they watch, when they watch etc.). I think platforms in the sharing economy could learn a lot from Netflix’s example and leverage information on how their users act and interact, rather than rely on what they say.
Although it’s outside the scope of your article, online reputation systems have been useful in other ways, notably as a form of incentive. StackExchange, Reddit and many others are using reputation to encourage deeper engagement to great effect. Rachel Botsman touched on this in her TED talk I believe.
Nice article. The problems mentioned with reciprocal ratings are not insoluble. They are caused by the ability of each rater to modify the rating they give based on the other person’s rating. If this is taken away, ratings will become more honest. One might ask whether websites cynically want to keep ratings artificially high, to make the service they provide seem more useful.
How to fix the problem:
1. The ratings users submit are not posted (made publically visible) until both parties to a transaction have submitted their ratings of each other.
2. Users are given a time limit, e.g. a week, to submit their ratings.
3. Once posted, ratings may not be changed.
Thanks for the comments everyone.
In general – I do agree that there may be other reputation systems, including ones that exist now, which don’t have the problems I talk about here. But these seem to be the things that social economy sites are making claims for, so I don’t feel too bad about that limitation.
Harald: I will follow up on your suggestions. I’ve followed Steve Randy Waldman’s work a bit, although macroeconomics is over my head, but didn’t know the others.
Rob – There seem to be a surprising number of parallels between sports culture and internet culture (as it was): the roots in a form of amateurism, and the uneasy relationship between community and ownership.
Kevin – I use StackOverflow all the time. I think the StackExchange use of reputation is a bit different: it’s definitely reputation, but it’s not really solving a problem of trust. Netflix’s choices are interesting – I knew they hadn’t taken up the prize winner algorithm, but didn’t realize they had moved so far away from rating based algorithms.
Vadim – I think the suggestion you make was spelled out in a paper I looked at, and I’ve added the link now. It’s here (PDF). I don’t use eBay and don’t know if it has followed that. Part of the issue, I think, is that sites like Airbnb need to have a much higher threshold of trust than eBay if they are to have longevity, so I think they’ll have to go past reputation anyway. At their best, reputation systems have to let bad things happen in order for the feedback to take place (there’s a reason we don’t use reputation systems for heart surgeons), and I don’t think they’ll be able to live with that.
Thanks again.
Pingback: Trust, reputation and regulation: Can the new sharing economy find a way to scale? — Tech News and Analysis
Hi Tom – An excellent article and very thought provoking. I am the founder of a nascent collaborative pet-sitting start-up in Australia (PetHomeStay.com) who have been tussling with this very issue. 95% of our feedback ratings are 5*, with one 1* to date, which to us, doesnt mean that much. The value of trust is a hard one because it is so subjective – as you point out in your notes – which truck driver (or pet sitter) would you trust?
In our mind, trust is about two things – the truster being confident about getting the service as promised before the transaction, and recommending the provider to others once the transaction has completed.
A good example is the eBay-of-NZ, TradeMe.co.nz. They do not have 5* ratings, just a simple ‘Happy’ or ‘Sad’ face per transaction. They then provide the ability for people to search based only 100% of Happy. This drives people to deliver the service as promised, to only get Happy ratings, and therefore maintain their high ratings.
More importantly, they provide a messaging system for feedback, per purchase; for the seller to then provide a written response to the feedback; and the ability for a potential purchaser to contact any of the previous purchasers of the product or service.
Its not perfect, and you can game the system – for example getting lots of Happy, but low value, transactions to build up reputation before offering a car – but it is better than other 5* systems.
I think a key part of reputation or trust in a system is the extent to which the purchaser can be sure of things being made right in the event of something going wrong. This is the comfort bubble that professional firms offer – Satisfaction guaranteed etc.
So I think peer to peer scores have to be rethought to be based on the behaviour they want to encourage – 100% happiness is the level, if a customer is anything less than that, the service provider has the opportunity to contact them, make it right in whatever way they see fit, in order for the rating to be readjusted. In extreme cases the website might have to arbitrate, but I would see it as rare.
On a final note – there are some trust providers out there who will aggregate your scores from all social sites – eBay, LinkedIn, Facebook, Mobile, etc such as TrustCloud. I guess these work on the assumption that its too much hassle to replicate an identity across all the sites just to game someone once?
thanks again for a great article, cheers
Tom – thanks for the comment and good luck with the venture. You’ve probably spent a lot more time thinking through the practical side of this than I have, so take this with a big pinch of salt.
I suspect that the biggest value of the reputation systems is to act as matching services (solve the coordination problem) with a little bit of extra assurance – basically the rating is a way to say “thank you” and it adds a layer of politeness to the exchange. I think there is a lot of benefit in that effort, where digital technologies can give a little nudge to human contacts. If it’s really a personal, informal exchange you are looking to promote, then maybe a model where you are not part of the transaction itself would work. I think where the venture-capital funded efforts go off the rails is that they want to be right in the middle of everything, including transactions, to justify their big revenue requirements. And while they would rather not take on liability, they are forced to just because they are taking such a big chunk of each exchange.
My own experience with pet exchanges is looking after a dog for a couple of friends, and I take our own dog occasionally to a professional place where we pay for her to stay a day. Thinking about it – if anything had gone wrong while I was looking after our friends’ dog, that would have been sad but would have had to be sorted out personally. But if something goes wrong at Country Paws I’d expect it to get sorted properly. Crossing that line between the informal and the formal comes with all kinds of issues, I think. So if you can promote the informal with a useful service, I’d say good luck.
PS I think Trustcloud, as well as Klout etc, are going to have big gaming problems as intermediaries like reputation.com start to manage people’s reputations.
Tom, interesting point regarding Klout. I’ve know Klout’s founder for many years (disclaimer).
I think there are interesting questions around how we view Klout scores. For an individual relying on a specific recommendation or mention by another individual, the reliability of that single score is of course very important.
But I wonder if for businesses who may use Klout scores as a means to identify / focus on a subset of users for a particular purpose (marketing, offers, engagement, etc.), should we perhaps evaluate much how we do marketing list response: the important aspect is does the tool (“Klout scores”) provide a % lift to the response rate or goal? If so, then its worthwhile even though its far from perfect.
In the game of big numbers, a few % points can = many % increased business success metric.
Pingback: 7 Stories to read this weekend — Tech News and Analysis
Pingback: Science behind making the best possible cup of coffee | Phat Dat Mai
There is an exciting start-up Traity which may provide a solution to some of the information asymmetry and “gaming of reputation” issues. Check out: Traity.com
Rama – With all due respect I think Traity is exactly the kind of effort that won’t fix the problem. Promoting reputation as a “personal brand” or something you invest in does not discriminate in terms of trustworthiness, it discriminates based on investment in reputation-management techniques and tricks. It’s asking to be gamed and managed by intermediaries (reputation.com etc).
Hello Tom,
Wow what an article. It has been a read of my trip (I’m off to OuiShare summit in Brussels) Disclaimer – I am one of the founders of Trustribe, and we build reputation platform embeddable into sharing services.
I must say your post got me thinking really hard, even to a point of brief thought – that reputation might realy not be able to solve trustworthiness problem for sharing services. But after some more thougt – I regained my confidence 🙂 and I am still a firm believer that with certain amount of care, pivoting, and analysis – it is possible to devise a system virtually ungamable by opportunists.
@Trustribe – it’s been long sice we noticed a problem of trustworthiness with current systems (somehow all airbnb hosts are 5 star ones 🙂 ). This is the reason we started building Trustribe.
I know this might ssound a bit cocky, but I guess there is no chioce. Currently accepted model(5 stars to solve it all) simply doesnt hold, and we care about sharing economy – frst of all because we want to live in a world ran on principles of collaboration & sharing. That’s why we see a need to do sonething about it, because with imoroper reputation system shatring world could be destroyed for quite a long time.
So what will we do differently?
First – it will have a built in scarcity – you as rater will not be able to give all the points to all the criterias/categories.
Secondly it will be based on positive criterias – you either have them – in that case it’s a question of ‘which ones, and how many?’ Or whether you don’t have any posotive criterias?
(as you mentioned somewhere – it’s a bit sophisticated form of saying ‘Thank you’)
These are just a few differences. We will have to launch to see if this performs any better. And we are launching at the end of this year with couple lined up pilot companies.
As I was saying in the beginning – I stay a firm believer that a proper reputation system can solve trustworthiness problem, we just need to figure the proper system – a fairly simple task, don’t you think? 🙂