There are two interesting articles in the news, today. They are about two very different companies but, essentially, the same issue: the performative power of the score. Or, in others words, about how much a simple number can influence our life.
The first article is about passenger transportation company, Uber. It was revealed that Uber drivers rate their passengers, and that this information is made available to other drivers, to help them decide whether or not to take up a fare. This means that if you have a low rating, you may find it difficult to book one of Uber’s cabs. The concerns are that the ratings are subjective and may result from factors beyond the passenger’s control. For instance:
“You hear stories from people who missed a pickup because of buggy notifications, for example, and those people all of a sudden just can’t catch a cab. Any kind of technical error can skew the ratings, but because they’re invisible they’re also treated as infallible.” (Source: Guardian)
The second article is about dating website, OKCupid. This company conducted an experiment whereby it manipulated the ratings that signal the extent to which your profile matches that of other users of the website – a higher score signals a better match between 2 persons (and, presumably, a higher chance of developing a successful relationship). During the experiment, users still had access to unaldetred profile information. The experiment revaled that users’ behaviour (i.e., whether they contacted the potential match or not) was highly influenced by the compatibility score provided by the company (as opposed to the qualitative information provided by the users in their profiles). This means that people were making important decisions regarding a potential life partner based on a simple score:
“In one experiment, the site took pairs of “bad” matches between two people – about 30% – and told them they were “exceptionally good” for each other, or 90% matches. (…) Further experiments suggested that “when we tell people they are a good match, they act as if they are. Even when they should be wrong for each other”.” (Source: BBC News)
Scores are not new. For decades, credit decisions have been made based on whether someone’s credit score falls below or above a particular threshold. Over time, the credit screening process has become largely automated as machines are deemed to be more accurate than staff. You see, humans make mistakes, are inconsistent, and are expensive to train and employ. And machines don’t.
However, evidence from default rates in the subprime loans market (yep, the one that triggered the credit crunch we are all recovering from) indicates that customer screening that relies solely, or predominantly, on quantitative data tends to systematically under-predict the default rate. In contrast, screening that incorporates soft data leads to better lending decisions. In other words, decisions that incorporate soft data outperform those that rely only on a quantitative score. See this paper for further details.
With digital technology making it so easy to collect, store, analyse and access data, quantitative scoring is spreading to more and more areas of our lives. As with credit scoring, numbers are easy to understand. They are clear cut. But they provide an incomplete picture of the person behind that score. And, yet, we are making important commercial and personal decisions, like whether to take on a client or contact a potential partner, based on those fallible scores.
I am really interested in this topic and I am trying to set up a project looking at customer profiling in the age of cheap, abundant data. If you want to join or, even better, if you want to fund this project, get in contact 🙂
In the meantime, let me know: Can you remember a situation when you relied solely (or mostly) on a quantitative score to make a decision?