The value of social media data is not in data itself, but the interpreter and the use

Tim Kourdi’s comment about the value of information, in my latest blog post, reminded me of an interview with Twitter co-founder, Jack Dorsey. Jack challenged criticism that the physical constraints of the platform (specifically, the 140 characters limit for a message) would lead to shallow, value-less content being shared. He did acknowledge that the limit would lead to a very particular type of message being shared: namely, users leaving instantaneous ‘marks’ rather than saying ‘the right thing’. Yet, he felt that those short, instant marks had value, too. He said:

‘It is all up to who receives the message… we can’t judge the value as it goes out. We have to leave it up to the individual to really bring value to that message, to that tweet’.

The full interview is available here, and the excerpt that I am referring to here:

Jack’s comments resonated with me for two reasons.

First, the recognition that the characteristics of the platform shape the conversations that you have – not just the length of the messages, but also (or, even, more importantly) the topic of the messages. This is extremely relevant for those mining social media data, for instance for sentiment analysis.

Second, the value of data resides not in data itself, but in what the interpreter does with it. Value in use, rather than value in availability.

I am doing some work in terms of valuing social media data, and I am very interested in hearing testimonials / examples of gaining (market / consumer / competitive) insight via social media data. Can you help?

Useful illustration of the difference between data and information (and knowledge)

Data and Information. These terms are often used interchangeably, though they actually mean very different things. I recently came across this example*, provided by Sir Nigel Shadbolt, Professor of Artificial Intelligence at Southampton University. He says:

When I give you a number, like 37, you don’t know whether that is somebody’s age or a particular kind of temperature or some kind of stock price.

Until I put it in context.

Then, that data becomes information.

And, if I can do something with that information (I can use it to give you some kind of antibiotic because you got a fever, or sell a share because it is worth selling), that’s knowledge.


So, what we usually refer to as open data, actually is information because it exists in a context. For instance, we don’t just get a database with timings or percentage numbers. We also know that those numbers refer to train timetables or infection rates in hospital. And it is that contextual element which turns data into information and that, in turn, also gives (open) data its value.

* This example emerged during an episode of the BBC radio 4 programme Life Scientific, which, at the time of publishing this post, is available here.

Customers that suffered to get your product value it more than those that didn’t

I am (finally) reading the book Mistakes were made (but not by me) by Carol Travis and Elliot Aronson (affiliate link), which explores why people find it hard to accept responsibility for mistakes.

There is an interesting section in the book, where authors report on findings from psychological experiments that show that:

‘(I)f people go through a great deal of pain, discomfort, effort, or embarrassment to get something, they will be happier with that ‘something’ than if it came to them easily. (…) The cognition that I am a sensible, competent person is dissonant with the cognition that I went through a painful procedure to achieve something – say, joining a group that turned out to be boring and worthless. Therefore, I would distort my perceptions of the group in a positive direction, trying to find good things about them and ignoring the downside.’ (page 15)

IMG_6018Examples of this effect that sprang to my mind, as I was reading this section, include queuing for hours to buy a new product, enter a concert, or be admitted to a restaurant or bar. Or, for instance, going through a series of difficult tests to be admitted into a programme, organisation or group.

This effect also explains why we tend to value very highly things that we put a lot of effort in creating – for instance, assembling IKEA furniture, folding origami, or building Lego. We like those items (the IKEA table, the origami sculpture or the Lego construction) more when we built them than if somebody else did it.

The bottom line: customers that suffered to get your product value it more than those that didn’t.

But before you go mad with this idea, and decide to introduce challenging tests for your customers or move to a self-assembly model, take heed of this section in the same book:

‘(t)hese findings do not mean that people enjoy painful experiences, such as filling out income-tax forms, or that people enjoy things that are associated with pain. What they do show is that if a person voluntarily goes through a difficult or painful experience in order to attain some goal or object, that goal or object becomes more attractive.’ (page 17)

Ladies and gentlemen: meet the user

Recently, I attended a workshop at the HCCM where Russell Davis, Director of Strategy at Government Digital Service, talked us through the process of transforming the government’s online presence. He said that, very early on in the process, he affixed this picture on the wall, to remind everyone of who the websites’ users were:


It is a bit hard to see, but this picture shows random people standing on a street corner. With this simple picture, the team were constantly reminded that they were developing a product to be used by the average person in the street. Not a solicitor or someone in parliament. The result was a series of streamlined webpages and even products, with information organised by ‘problem’, and using plain language. For instance, this one:

You can read about it here.

A simple but effective way of reminding ourselves of who the user is, what they value and how they use our product or service don’t you think? In other words, of why we do what do.

Excuse me for a minute; I need to stick some pictures on my wall.

March 2015 round-up

Spring is here (at least officially), I had one journal and three conference papers accepted, and our lovely friends SP have visited us. So, all in all, the month is ending on a high note. And if it is true that 50% of my sabbatical is now gone :-(, it is also the case that I still have 50% to look forward to :-). The glass is half full!

Looking through the photographs from my 5pm project, I realise that I seem to be indoors a lot, at that time of the day, often supervising homework. I wonder if this pattern will change now that the weather is (meant to be) getting warmer.


As for this month’s highlights, they are summarised below, as usual.


I spent some time thinking about algorithmic decision-making based on social media activity. Algorithmic decision-making refers to the singling out of a customer or user for an offer, or extra monitoring, or even a penalty, because they have been picked up by an automated system that monitors patterns of behaviour (for instance, transactions). It is the typical ‘because the computer said so’ scenario. Algorithmic decision-making (namely, how algorithms are developed and applied) was the topic of my PhD at the London School of Economics, which looked at customer profiling and screening in financial services.

I also spent a lot of time looking at possible sources of funding for research projects, which is a very, very time consuming (and frustrating) activity.


In the first week of the month, I worked on a paper on the use of Social Media in the B2B sector. The deadline was March 8th but, even though I tried really hard to ‘make it happen’, I was unable to finish it on time. I was very disappointed, but it does happen. And, of course, there will be more opportunities for this paper.

On the other hand, the paper that I worked on with Yuvraj Padmanabhan, on the IMG_6962use of Twitter data for sentiment analysis, has been accepted for publication. Yay.

Also, the book The Private Security State? was published, and I had three papers accepted for a conference. So, not a bad month at all.


No teaching this month. Instead, I have been thinking and reading a lot about teaching, from how to make a class more dynamic, to helping students monitor their understanding of a subject (see the header ‘Learning’).


I came across the SOLO framework, which distinguishes different types (and levels) of understanding of a topic, and then I read this book (affiliate link) outlining how to apply the framework to structure teaching and assessment. The book suggests that teachers should create checklists for each topic that they teach, with examples of the application of the topic at the various levels of understanding, from defining the concept, to using it to explain an observation and, at the highest level, to predict what will happen in a particular scenario.

I liked this framework because it makes it explicit that there are different levels of sophistication in the understanding and use of any given topic. I think that incorporating this framework in my course materials would be very difficult to achieve in the time that I have to prepare a module, but wouldbe really valuable for my students, particularly those who are not familiar with the UK educational system and/or who do not have a background in social sciences.

I also made some progress with Python, and I am now learning about strings. My goal is to build a crawler, to collect online data for my research.

And last but not least, the eclipse gave us all (at home) the perfect excuse to learn about the solar system.


What were March’s highlights for you?

UK attitude towards monitoring of internet and mobile communications is surprisingly tolerant

I wonder if you can help me make sense of this.

I came across a survey by YouGov for Amnesty International, published earlier this month, about the attitudes of people from 13 countries* towards government surveillance. According to this data, UK residents are more likely than the (study’s) average to support mass surveillance by government of their Internet and mobile phone communications, including by the US government. The only country consistently more tolerant than the UK is the Philippines.


In this survey, UK respondents are also the most likely to agree that technology companies should provide the government access to secure internet communications such as emails, messages, or social media activity.


I am really struggling to make sense of this ‘permissive attitude’, and wonder if this extends towards collection for commercial purposes. What are your thoughts? And are you aware of other international comparisons about attitudes towards generalised data collection?

* The countries are: Australia, Brazil, Canada, France, Germany, Netherlands, New Zealand, Philippines, South Africa, Spain, Sweden, United States and the UK.