Category Archives: social media work

Conferences & Talks, my projects, social media, social media work

Sentiment Analysis and Twitter ‘wormals’

7 October 2010 Jon Bounds 2 Comments

I’ve tried two experiments with the “is Birmingham happy” algorithm in the last few days, as they’re not based on place it makes more sense to use the popular term ‘sentiment analysis’ to refer to what it’s doing in this instance. As they were both reasonably short uses it was posible to update the reading often (and use a smaller number of tweets as the sample, giving more variation in the average scores) and give the sentiment graphs a live ‘wormal’ feeling, watching the ratings change over time.

First was on the Personal Democracy Forum EU conference in Barcelona, for the length of the two-day conference I monitored the hashtag #pdfeu every five minutes:

(click image for larger view)

The highest rating was 64.4% (at 12:45pm on Tuesday), the lowest 49.6% (Monday at 12:14pm during a short power failure). What was interesting to me was that the “arousal” rating seemed to work well as it stayed pretty steady during the power failure (or even leaped up a little) even as the happiness of the hashtag users dived. Post-lunch conference lulls and periods of excitement (the big spikes in day two, at least, corresponded with much applause) were mapped quite accurately.

The overall average was 57.29%. If you would like to explore or graph the data yourself, you can see in all in a Google Spreadsheet here.

Secondly I tried a much shorter and more mainstream application, David Cameron’s speech to the Conservative Party Conference:

The emotion tracking tool graphed here ran every 10 seconds during David Cameron’s speech to the CPC and analysed the last 100 tweets with the hashtag #cpc10 and the word “tories”. I chose two versions as I wasn’t sure that non-Conservative supporters would use the ‘official’ hashtag, I theorised that they would be likely to use the word ‘tories’. As it turned out I think that while there was a more even spread of pro and anti political types using the hashtag than I expected, but the ‘tories’ Tweeters were definitely more hostile. (See the data.) There was greater movement across the graph than on any other test I’ve run.

Conclusions? None so far, other than that I think this might be a very useful tool, and that more interesting data is created the more Tweets you have and the more you can afford (server-wise) to poll for results. I’m itching to try it on another big live event with conflicting opinions, that might mean training it on a reality TV event. Roll on the X-Factor.

future web, geodata, my projects, portfolio, social media, social media work, twitter, web development work

Is Birmingham Happy?

29 September 2010 Jon Bounds 4 Comments

I’ve been running a, very rough, scrape of the Birmingham (UK) based interweb for ‘emotional wellbeing’ since April of 2008. Simply put a script running twice a day read in Tweets, news headlines and (originally) blog posts and compared the words within them to a table I’d drawn up of ‘emotion’ words and fairly arbitrary scores.

It was surprisingly interesting to watch: despite its roughness, the internal consistency let patterns emerge. It broadly followed weather and sports results, with some peaks and dips you could map to specific happenings, or news stories.

It lead to a spin off focussing on Tweets from MPs, which I think influenced some of the developments that Tweetminster produced in the next year or so.

It was the patterns that lead me to keep putting off improving the algorithm, but recent Twitter API developments meant I had to do some work anyway and that (together with another project, of which more soon) gave me the impetus to give the project an overhaul. And here’s how it works now…

Twitter’s geolocation services are now much improved, so I can specify a point (the centre of Victoria Square in Birmingham) and a radius (10 miles) and get a reasonably accurate dump of Tweet data back—the algorithm calls for the most recent 1000.

Twitter is now the sole focus of data, in keeping with the ‘conversational pychogeography‘ aims of the project (in essence, words used without too much pre-meditation are more interesting than those written purely for publication). It also provides much more and more reactive data.

The words contained within these tweets are then compared to data from the University of Florida (The Affective Norms for English Words – PDF link). Within that data set each word covered (there are around a thousand in the set I’ve using) is given a score for Valence (sad to happy on a scale 0-10), Arousal (asleep to awake on a scale of 0-10) and Dominance (feeling lack of control to feeling in control on a scale of 0-10). The scores are then collated and a mean calculated. The overall emotional wellbeing score here is calculated as a mean of the three individual means, although the scores are revealed individually on the site.

I’m unsure if combining the results in this way is the best, which is why the site reveals the working — the Twitter feed just goes with one value for ease of understanding and adds a rating adjective too:



if ($brumemotion<100){$rating="fantastic";}
if ($brumemotion<90){$rating="superb";}
if ($brumemotion<80){$rating="good";}
if ($brumemotion<70){$rating="okay";}
if ($brumemotion<60){$rating="average";}
if ($brumemotion<50){$rating="quiet";}
if ($brumemotion<40){$rating="subdued";}
if ($brumemotion<30){$rating="low";}
if ($brumemotion<20){$rating="dreadful";}
if ($brumemotion<10){$rating="awful";}

The Twitter feed produces results twice a day, and these scores are being saved to visualise more graphically, but the website updates every ten seconds (and will self-refresh if you stay on the site) and also displays a word cloud of the currently found ‘emotion words':

Thoughts on further development

I’ve been experimenting with more local results (here is a version running on just one Birmingham post code — B13) as well as live graphing. I also have a version that will analyse results for a hashtag—something we may use in conjunction with the Civico player to produce ‘wormals’ (graphs of sentiment) during conferences.

But for now, I’m happy to let the new algorithm bed in—wondering about the amount of data and frequency that will be required to see the most detail—and to see what patterns we can spot.

Feedback welcome. Go see for yourself or follow on Twitter.

my projects, social media, social media work, twitter, writing work

We’ve won a Webby (almost)

14 April 2010 Jon Bounds Leave a comment

While it’s not made the final shortlist of five, there’s still a bit to be proud of as Twitpanto has been made an Honoree (US spelling) in the 2010 Webby Awards in the Net Art category.

As that shortlist includes big budget projects like BBC Blast, I don’t think we’ve done badly seeing as it was one crash hot web guy (that’s Matt, not me) and 20 odd (very) Twitterers on a cold December afternoon (which you can relive, of course, here).

In speech mode, we couldn’t have done it so well without the support of the Birmingham Hippodrome — or the hundreds of people that joined in during the play. Sincere thanks for going along with it.

Quite interesting timing as the RSC’s Twitter Shakespeare project launched this week with much fanfare, in some ways it’s a logical step on from Twitpanto and it’ll be interesting to see how it’s sheer scale affects the experience (a slight overreach of scale was one of the problems I felt with this second panto).

To read all about both Twitter panto experiences there are a couple of long and detailed posts here.

my projects, social media work

WordPress MU for BEN PCT

5 January 2010 Jon Bounds Leave a comment

That hail of acronyms is my way of announcing that the new site for the Birmingham East and North Primary Care Trust that I’ve been consulting on is now live to the public. It’s based on a WordPress MU (multi-user) installation, which will allow the team to very quickly set a new site live for special events and allow individual services or centres to have their own sites, easily administered by them but controlled from one central area.

The design work is by the team at Substrakt, who also developed a plugin to deal with the wide variety of data that can be pulled from the NHS Choices API. (The plugin would be very useful for local blogs as it would automatically allow search and display of local health services — hopefully it can be released publicly). They’ve done a great job of producing an accessible, clear site with in the NHS brand guidelines.

The site is to be run by the team at the PCT, so a lot of the consultancy process has been about making sure they understand as much as possible about the possibilities and the (simple) procedures needed to administer the site — confidence to experiment has been vauable. We decided on WP MU as there are plans for may sites and sub-sites — a MU install can have a site up and running in a matter of minutes, and with a set of themes designed they can all fit the brand straight away. We’ve also set up a system allowing each site to have a completely separate domain name if required — so no-one need know that they’re related to the main PCT site.

The flexibility of WordPress allowed the team to build a mock-up site very quickly, with a very basic theme, and work on the content internally collaboratively — which was then skinned with the completed theme. For the user, the site offers RSS feeds of any of the categories, and many pages will have commenting on — both standard features.

The first of the sub-sites is under development, as well as experiments with the commentariat theme which is a theme built to make consultation on documents easy. I look forward to seeing the developments.