The Moran Scale of Twitterstorms

I was asked to ‘create’ a Twitterstorm as part of an art project, and I sort of did. While this wonderful Buzzfeed post describes the stages that one goes through, in order to measure the size of a storm and hence the success of my operation we needed a way to describe the extent of a particular one. With Jon Hickman (Degree Leader, Web and New Media at Birmingham City University) I worked up this scale.

It’s an attempt to give a quantitative scale to something that cannot be measured directly in numbers—this is about extent and influence and simple measures are never going to cut it, although as the number of Morans increases so does the number of Tweets and their anger. It’s based roughly on the idea of the news cycle and how the subject of the storm operates within it. We chose the name ‘The Moran Scale’ after Caitlin Moran, whose ability to kick off the storms—and get them featured in the old school media—is unrivalled. As it’s about intensity of storm, a parallel to the Beaufort Scale is entirely intentional.

The Moran Scale of Twitterstorms by Jon Hickman and Jon Bounds

Continue reading The Moran Scale of Twitterstorms

The Eye of the Twitterstorm – xHuming Joseph Priestley



I’ll write up the xHumed talk asap, Joseph Priestley sending this Tweet:


Resulted in this reaction (in statistics) up till about 3 hours later:



Are you in a happy place?

On Friday I got round to doing something I’d been thinking of for a long while. I added location detection to my conversational psychogeography tool. Like the Is Brum Happy? system it takes the latest tweets around a location and rates emotionally sensitive words against a database to give scores for the happiness or emotional wellbeing of the place. If you’re using a HTML 5 browser (you probably are) you can let it reveal your location to the script (it’s not saved anywhere) and it will tell you if where you are (and a mile radius around) is happy right now.

Give it a go.

How Christmassy are you feeling?

The Christmas-o-meter bounder

Just a little fun seasonal project I’ve made with the layout and design help of Gavin Wray.

It works very much like my other sentiment analysis tools, but with a sprinkling of Santa’s magic. Santa’s magic in this instance being that any tweets with the words ‘Christmas’ or ‘Xmas’ in them are weighted doubly—that is the scores are counted twice for the purpose of producing the mean score.

So, try the Christmas-o-meter and see how Christmassy you’re feeling.

Sentiment Analysis of a Football Match

(click through for big)

Last night I turned my sentiment analysis tool on two hashtags: #bcfc and #avfc, the most widely used tags to refer to Birmingham City and Aston Villa during their League Cup quarter final game. It was a chance to see if visualising to ‘competing’ tags around the same event would be a useful exercise.

Caveats that would apply to this:

  • Some people use the tags instead of team names, meaning that they might be used by people supporting the other team (or no team at all)—most fans, though seem to tag with just the hashtag representing their team.
  • Some tweeters use both—these tweets could be removed technically, but make no difference to the comparative scores.
  • If there’s a subject that uses more slang or metaphor than football, it’s not often discussed on Twitter.

There was a generally a downward trend throughout the match, tension? Bad football? It could have been both. The first two goals seemed to have a much bigger impact than the third—this I don’t quite understand, but it seems to be more about the tweets themselves than the tool.

I could see how a special subject-set of emotion words could be created for football, which could cope with more nuanced or unusual words. It’s something to consider.

The sentiment scores in a Google spreadsheet, csv files: #avfc tweets (657 of which were during the game), #bcfc tweets (370 during).

The obligatory Wordle:

Sentiment Analysis and Twitter ‘wormals’

I’ve tried two experiments with the “is Birmingham happy” algorithm in the last few days, as they’re not based on place it makes more sense to use the popular term ‘sentiment analysis’ to refer to what it’s doing in this instance. As they were both reasonably short uses it was posible to update the reading often (and use a smaller number of tweets as the sample, giving more variation in the average scores) and give the sentiment graphs a live ‘wormal’ feeling, watching the ratings change over time.

First was on the Personal Democracy Forum EU conference in Barcelona, for the length of the two-day conference I monitored the hashtag #pdfeu every five minutes:

(click image for larger view)

The highest rating was 64.4% (at 12:45pm on Tuesday), the lowest 49.6% (Monday at 12:14pm during a short power failure). What was interesting to me was that the “arousal” rating seemed to work well as it stayed pretty steady during the power failure  (or even leaped up a little) even as the happiness of the hashtag users  dived. Post-lunch conference lulls and periods of excitement (the big spikes in day two, at least, corresponded with much applause) were mapped quite accurately.

The overall average was 57.29%. If you would like to explore or graph the data yourself, you can see in all in a Google Spreadsheet here.

Secondly I tried a much shorter and more mainstream application, David Cameron’s speech to the Conservative Party Conference:

cpchappyThe emotion tracking tool graphed here ran every 10 seconds during David Cameron’s speech to the CPC and analysed the last 100 tweets with the hashtag #cpc10 and the word “tories”. I chose two versions as I wasn’t sure that non-Conservative supporters would use the ‘official’ hashtag, I theorised that they would be likely to use the word ‘tories’. As it turned out I think that while there was a more even spread of pro and anti political types using the hashtag than I expected, but the ‘tories’ Tweeters were definitely more hostile. (See the data.) There was greater movement across the graph than on any other test I’ve run.

Conclusions? None so far, other than that I think this might be a very useful tool, and that more interesting data is created the more Tweets you have and the more you can afford (server-wise) to poll for results. I’m itching to try it on another big live event with conflicting opinions, that might mean training it on a reality TV event. Roll on the X-Factor.

Is Birmingham Happy?

I’ve been running a, very rough, scrape of the Birmingham (UK) based interweb for ’emotional wellbeing’ since April of 2008. Simply put a script running twice a day read in Tweets, news headlines and (originally) blog posts and compared the words within them to a table I’d drawn up of ’emotion’ words and fairly arbitrary scores.

It was surprisingly interesting to watch: despite its roughness, the internal consistency let patterns emerge. It broadly followed weather and sports results, with some peaks and dips you could map to specific happenings, or news stories.

graph of emotion scores

It lead to a spin off focussing on Tweets from MPs, which I think influenced some of the developments that Tweetminster produced in the next year or so.

It was the patterns that lead me to keep putting off improving the algorithm, but recent Twitter API developments meant I had to do some work anyway and that (together with another project, of which more soon) gave me the impetus to give the project an overhaul. And here’s how it works now…

Twitter’s geolocation services are now much improved, so I can specify a point (the centre of Victoria Square in Birmingham) and a radius (10 miles) and get a reasonably accurate dump of Tweet data back—the algorithm calls for the most recent 1000.

Twitter is now the sole focus of data, in keeping with the ‘conversational pychogeography‘ aims of the project (in essence, words used without too much pre-meditation are more interesting than those written purely for publication). It also provides much more and more reactive data.

The words contained within these tweets are then compared to data from the University of Florida (The Affective Norms for English Words – PDF link). Within that data set each word covered (there are around a thousand in the set I’ve using) is given a score for Valence (sad to happy on a scale 0-10), Arousal (asleep to awake on a scale of 0-10) and Dominance (feeling lack of control to feeling in control  on a scale of 0-10). The scores are then collated and a mean calculated. The overall emotional wellbeing score here is calculated as a mean of the three individual means, although the scores are revealed individually on the site.

I’m unsure if combining the results in this way is the best, which is why the site reveals the working — the Twitter feed just goes with one value for ease of understanding and adds a rating adjective too:

if ($brumemotion<100){$rating="fantastic";}
if ($brumemotion<90){$rating="superb";}
if ($brumemotion<80){$rating="good";}
if ($brumemotion<70){$rating="okay";}
if ($brumemotion<60){$rating="average";}
if ($brumemotion<50){$rating="quiet";}
if ($brumemotion<40){$rating="subdued";}
if ($brumemotion<30){$rating="low";}
if ($brumemotion<20){$rating="dreadful";}
if ($brumemotion<10){$rating="awful";}

The Twitter feed produces results twice a day, and these scores are being saved to visualise more graphically, but the website updates every ten seconds (and will self-refresh if you stay on the site) and also displays a word cloud of the currently found ’emotion words’:

is Brum happy right now?

Thoughts on further development

I’ve been experimenting with more local results (here is a version running on just one Birmingham post code — B13) as well as live graphing. I also have a version that will analyse results for a hashtag—something we may use in conjunction with the Civico player to produce ‘wormals’ (graphs of sentiment) during conferences.

But for now, I’m happy to let the new algorithm bed in—wondering about the amount of data and frequency that will be required to see the most detail—and to see what patterns we can spot.

Feedback welcome. Go see for yourself or follow on Twitter.

We’ve won a Webby (almost)

While it’s not made the final shortlist of five, there’s still a bit to be proud of as Twitpanto has been made an Honoree (US spelling) in the 2010 Webby Awards in the Net Art category.

As that shortlist includes big budget projects like BBC Blast, I don’t think we’ve done badly seeing as it was one crash hot web guy (that’s Matt, not me) and 20 odd (very) Twitterers on a cold December afternoon (which you can relive, of course, here).

In speech mode, we couldn’t have done it so well without the support of the Birmingham Hippodrome — or the hundreds of people that joined in during the play. Sincere thanks for going along with it.

Quite interesting timing as the RSC’s Twitter Shakespeare project launched this week with much fanfare, in some ways it’s a logical step on from Twitpanto and it’ll be interesting to see how it’s sheer scale affects the experience (a slight overreach of scale was one of the problems I felt with this second panto).

To read all about both Twitter panto experiences there are a couple of long and detailed posts here.

RSC Friends

RSC Friends

Pleased to see the RSC Friends (a subscription group who are fans of the RSC) starting to use their blog. I did a day of training with them last year, most were very new to the idea of blogging but enthusiasm for their subject is carrying them through. They’re planning to use it a little like an online magazine, but hopefully to share their love of theatre to more people.

WordPress MU for BEN PCT

That hail of acronyms is my way of announcing that the new site for the Birmingham East and North Primary Care Trust that I’ve been consulting on is now live to the public. It’s based on a WordPress MU (multi-user) installation, which will allow the team to very quickly set a new site live for special events and allow individual services or centres to have their own sites, easily administered by them but controlled from one central area.

NHS Birmingham East and North

The design work is by the team at Substrakt, who also developed a plugin to deal with the wide variety of data that can be pulled from the NHS Choices API. (The plugin would be very useful for local blogs as it would automatically allow search and display of local health services — hopefully it can be released publicly). They’ve done a great job of producing an accessible, clear site with in the NHS brand guidelines.

The site is to be run by the team at the PCT, so a lot of the consultancy process has been about making sure they understand as much as possible about the possibilities and the (simple) procedures needed to administer the site — confidence to experiment has been vauable. We decided on WP MU as there are plans for may sites and sub-sites — a MU install can have a site up and running in a matter of minutes, and with a set of themes designed they can all fit the brand straight away. We’ve also set up a system allowing each site to have a completely separate domain name if required — so no-one need know that they’re related to the main PCT site.

The flexibility of WordPress allowed the team to build a mock-up site very quickly, with a very basic theme, and work on the content internally collaboratively — which was then skinned with the completed theme. For the user, the site offers RSS feeds of any of the categories, and many pages will have commenting on — both standard features.

The first of the sub-sites is under development, as well as experiments with the commentariat theme which is a theme built to make consultation on documents easy. I look forward to seeing the developments.