The Twitter ‘Firehose’: Going beyond the tip-of-the-iceberg.

I’ll be posting up a bit more about social media analytics over the coming weeks and months, but today’s news – ‘Hedge fund uses Twitter sentiment analysis to guide decisions’ – provides a useful opportunity to provide a timely taster.

Twitter: We only see the tip of the iceberg
For example, can we use this kind of approach to improve campaigning, media-monitoring, governance or public health? Because if we find a way of really seeing what is being said on social media platforms, we can find out all kinds of timely information that we wouldn’t always know about. Information that – until recently – even money couldn’t buy.

This changes the very nature of decision-making – and by implication, of government and governance. Governments can even foresee revolutions if they know where to look for them.

I spend a lot of time training clients in the use of social media channels and how they can be used to communicate – in the fullest sense of the word. One of the earliest points that I make to users who have a reasonable amount of experience with Facebook or Twitter, for example, is that these media are not as friendly as they may look.

Sure – most of us have a mostly-friendly experience of them because we view them selectively. By definition, we only see the contribution to Facebook by our friends. And there is at least some level of affinity with the people we follow on Twitter. We could make a decision to only actively follow White Supremacists if I wanted to see the other side, but even Glenn Beck has found that to be a career-limiting activity.

Social media: Not always a friendly mirror (click pic for credit)

So. We only see a fairly friendly social media, in the same way that we usually (!) see a friendly face in the mirror.

Even then, we still only see the tip of the iceberg: Most users only have a limited view of Twitter. Twitter imposes limits on what we can search and we only see a fraction of the results that, in an ideal world, we could see.

Unless you have access to the Twitter ‘firehose’.

And unless you have a huge amount of processing power to examine those results.

And unless you have the knowledge and tools to know what you’re looking for, how to find it, and how to extract useful knowledge, or ‘alerts’ from the data.

Here’s why: The most recent statistics that I’ve seen tell me that there are 230,000.000 tweets sent every day. That’s about 160,000 per minute or 2,662 per second.

See? This is why it’s called a ‘Firehose’

So using Twitter to detect Flu outbreaks, for instance, isn’t as far-fetched an idea as it seems. Using it as a customer service tool is a no-brainer.

I’ve been given access to that ‘Firehose’ along with the kind of analytical tools needed to make sense of the information that it lets me see. I’d be interested to talk to people in both government and campaigning roles on how this can be used more effectively.


Leave a Reply

Your email address will not be published. Required fields are marked *

Switch to our mobile site