Biases in algorithms – the case of Hello Barbie

Sometime ago, I saw a presentation by Val Steeves, Professor of Criminology at the University of Ottawa (Canada), about her research on smart toys. The talk focused on Hello Barbie, a Barbie-branded doll which is advertised as “the first fashion doll that can have a two-way conversation with girls”, and featuring “speech recognition and progressive learning features which provide girls with an engaging and unique Barbie experience” such as playing games, telling jokes, storytelling and tailored conversations based on previous play history.

 

This is the promotional video:

 

In her talk, Steeves unpacked the terms and conditions (T&Cs) of the toy which, as usual, are long, full of technical language, gives the company all the rights, and offers no bargaining power to the owners of the doll. For instance, the T&Cs state that it is the parents’ responsibility to listen to the conversations between the child and the Hello Barbie doll, and to ensure that the child does not reveal any private information. Moreover, the parents can’t prevent Mattel (which own the Barbie brand) or Toy Talk (which processes the recordings of the conversations between the child and the doll) to third parties).

 

Steeves also dissected the more than 8,000 lines of dialogue – the range of questions and responses – that the doll can use when interacting with the child. The analysis of these questions and answers provides really interesting insight into how the doll works simultaneously as a promotional and a market research tool. For instance, the conversations often lead up to references to fashion, make-up, friends and animals, for which there are plenty of Barbie-branded merchandising options available for purchase:

You know, I really appreciate my friends who have a completely unique sense of style… like you!

You know what I want to talk about? FASHION!

 

Moreover, the doll asks various questions about the child’s environment and habits. For instance:

Absolutely! The end of school means the start of summer adventures! What was the last adventure you had?

And where do you usually go on Thanksgiving?

Have you ever gone paddle boarding before?

Well, what was one cool thing that you remember?

What do you want to be when you grow up?

What’s the coolest thing you’ve ever done on vacation?

Your mom? That’s fun! What kind of movies does she like?

 

The analysis of the dialogue menu also reveals the assumptions that Mattel made about the profile of the children playing with the doll. In particular, the biases and implicit stereotyping about the users of the doll. For instance, when it comes to festivities, there are 25 references to Christmas, 25 references to Kwanzaa and 21 references to Diwali, but no reference to Eid or Hanukkah.

 

The qualitative analysis of the dialogue is even more revelatory about the implicit biases in the dialogue’s algorithm. Namely, the 25 exchanges about Kwanzaa and the 21 references to Diwali are largely explanatory, and assume that the child has neither an Hindu or an African background. For instance:

Diwali is a five-day Indian festival that happens at the same time as the Hindu New Year… so many reasons to celebrate, right?

Good question! Hinduism is one of the major religions in India, and one of the world’s oldest religions! Can you guess how old?

During Kwanzaa there are SEVEN principles to reflect on… one for each day of the holiday! Want to hear what they are?

 

In contrast, there are no such ‘educational’ descriptions in the exchanges about Christmas. Instead, the questions and answers reveal an assumption that the child knows about and is celebrating Christmas:

My favorite part of Christmas day is when my sisters and I get our stockings in the morning. Do you get a stocking too?

Well, what are you wishing for this Christmas?

Yes, making everything all Christmassy! And which part of Christmas do you look forward to the most?

You make your own Christmas decorations?! Awesome! Which ones have you made?

 

The same assumption of familiarity and adherence to particular forms of celebration are evident in the questions and answers about two other North-American festivities, Thanksgiving and Halloween. For instance:

Happy Thanksgiving! One of my all time favorite holidays. So…what’s something that you’re really thankful for this year?

There you are! I was just thinking…Halloween isn’t too far away. Have you thought about what costume you’re gonna wear?

 

In summary, Mattel assumes that the child playing with this doll is a girl; whose family celebrates Christmas, Thanksgiving and Halloween; who might be interested in learning about Diwali and Kwanzaa, but not other festivities; and whose parents are attentively listening in when the child plays with the doll, in order to prevent any accidental disclosure of sensitive information. Realistic?

 

I found the recording of one of Steeves’s talk on this topic. It is not the one that I saw, but is very similar, and equally interesting. You may want to check it out.