You could probably name half a dozen or so instances where you come across algorithmic decision making in your interactions with commercial organisations. There’s the news displayed on your phone, the adverts shown while you are browsing, the content recommendations in your streaming service, or the product suggestions on your online shopping platform, whether your call gets fast tracked, the prices shown when you are trying to book a flight… and so on. [On a side note, Mike Molesworth’s story “A Heteronomous Consumer Romance”, offers a very interesting take on the ubiquity of algorithmic decision in daily life. You can read the story here, or listen to it, here. I also participated in a discussion about the story, here.]
What about interactions with public sector organisations? Say, the tax office? The police? Local government?
If you are struggling to name more than a couple, you are not alone. I am struggling, too. And when consulting firm “Thinks” set out to investigate “What kind of transparency do citizens expect when the public sector use algorithms?”, they realised that, actually, “There is almost no awareness of the use of algorithms in the public sector”. So, we are not alone.
Yet, as Public Law Project reports, in its register of automated decision-making in government, called the Tracking Automated Government (TAG) Register, there are (at the time of writing this blog post), more than 50 such uses. For instance, algorithms are used to decide where asylum interviews should take place, detection of fraud in welfare claims, or prioritising social housing waiting lists.
Does this lack of awareness about algorithmic decision making matter? And, if so, how?
In the paper “To be or not to be algorithm aware: a question of a new digital divide?”, published in volume 24, issue 12 of Information, Communication & Society, Anne-Britt Gran, Peter Booth and Taina Bucher, argue that the lack of awareness of algorithms “corresponds to a new reinforced digital divide”.
They say that, if people are not aware that algorithms are being used in a certain environment, they can’t deploy the skills to navigate that environment. In this sense, “algorithm awareness is better understood as a meta-skill, a knowledge or understanding that may improve other digital skills and benefits in general (…) algorithmic awareness and literacy, as a meta-skill, are necessary conditions for an enlightened and rewarding online life.”
Gran and colleagues go on to argue that:
“Not only does a lack of algorithm awareness pose a threat to democratic participation in terms of access to information, but users are performatively involved in shaping their own conditions of information access. (…) New divides are created based on the uneven distribution of data and knowledge, between those who have the means to question the processes of datafication and those who lack the necessary resources”.
That is, when it comes to algorithmic decision making, it is not the case that “what you don’t know can’t hurt you”. Part of it will be institutions being transparent about their use of algorithms. But the other part is also about us, as consumers and citizens, becoming curious about how those institutions are making decisions, right?
Ana Canhoto 