It was surprisingly interesting to watch: despite its roughness, the internal consistency let patterns emerge. It broadly followed weather and sports results, with some peaks and dips you could map to specific happenings, or news stories.

It lead to a spin off focussing on Tweets from MPs, which I think influenced some of the developments that Tweetminster produced in the next year or so.

It was the patterns that lead me to keep putting off improving the algorithm, but recent Twitter API developments meant I had to do some work anyway and that (together with another project, of which more soon) gave me the impetus to give the project an overhaul. And here’s how it works now…

Twitter’s geolocation services are now much improved, so I can specify a point (the centre of Victoria Square in Birmingham) and a radius (10 miles) and get a reasonably accurate dump of Tweet data back—the algorithm calls for the most recent 1000.

Twitter is now the sole focus of data, in keeping with the ‘conversational pychogeography‘ aims of the project (in essence, words used without too much pre-meditation are more interesting than those written purely for publication). It also provides much more and more reactive data.

The words contained within these tweets are then compared to data from the University of Florida (The Affective Norms for English Words - PDF link). Within that data set each word covered (there are around a thousand in the set I’ve using) is given a score for Valence (sad to happy on a scale 0-10), Arousal (asleep to awake on a scale of 0-10) and Dominance (feeling lack of control to feeling in control  on a scale of 0-10). The scores are then collated and a mean calculated. The overall emotional wellbeing score here is calculated as a mean of the three individual means, although the scores are revealed individually on the site.

I’m unsure if combining the results in this way is the best, which is why the site reveals the working — the Twitter feed just goes with one value for ease of understanding and adds a rating adjective too:

The Twitter feed produces results twice a day, and these scores are being saved to visualise more graphically, but the website updates every ten seconds (and will self-refresh if you stay on the site) and also displays a word cloud of the currently found ‘emotion words’:

Thoughts on further development

I’ve been experimenting with more local results (here is a version running on just one Birmingham post code — B13) as well as live graphing. I also have a version that will analyse results for a hashtag—something we may use in conjunction with the Civico player to produce ‘wormals’ (graphs of sentiment) during conferences.

But for now, I’m happy to let the new algorithm bed in—wondering about the amount of data and frequency that will be required to see the most detail—and to see what patterns we can spot.

Feedback welcome. Go see for yourself or follow on Twitter.

What’s Foursquare for? I think people are still working it out. While the benefits for businesses of being able to target customers and record movement are interesting (and would be much improved with widespread adoption), it’s harder to see just why people are using it. That’s not because I’ve got privacy issues (I do in fact experiment with Foursquare), it’s just that it’s a clumsy way to do most location based things.

Much has been said about the ‘game dynamic’, the badges and mayorships, but it’s a pretty simple and pointless game really. No rules to speak of, no way of easily judging performance, no winner–and horrendously easy to cheat (try logging in to the mobile.foursquare.com site on your computer, you can check in anywhere). My jury is still out on it.

What does interest me is the idea of leaving ‘tips’ at locations, not so much in the “great coffee round the corner” or “Order the special sauce!”–with illiterate exclamation marks–way, but real information.

This is just something I’ve been experimenting with with Michael Grimes, there’s hardly any information in it yet, and no real way to get it out. It’s an attempt at a syntax, and an encouragement to people to try to use these new locative tools to add information. There’s more here, but basically the idea is to add ‘blue plaque’-style information as ‘tips’ on Foursquare and the like.

There are plenty of sites or apps that do things such as overlaying Wikipedia information over maps, but this isn’t quite the same thing. The problem with Wikipedia for location specific information is the same that blue plaques have: namely invisible gatekeepers of what’s “notable” or “historic”. Civic Societies with arcane rules (eg. subject must have been dead for 50 years), or Wikipedia editors waiting to pounce on things without reliable, verifiable, sources make it hard for people to record history as it happens. And it happens in tiny, homely, ways that committees can’t record.

I’ve tried, jokingly really, to liberate the blue plaque before, but this online way might actually take off.

As a demonstration, I’ve knocked up this: a #bp map. It pulls tips from Foursquare that are tagged #bp and lays them on a Google Map. You can change the point of refference of the map by entering latitude and longitude as part of the URL:

http://jonbounds.co.uk/4sq/index.php?lat=52.4999&lon=-1.90001

But there aren’t, as yet, many more ‘plaques’ to look at –why not add a few?

And boundaries were one of the things that open data advocates were most pleased that the Ordinance Survey released for use recently. At at Andrew McKenzie’s Mapitude event in Birmingham the attendees worked on a Ward Comparison site that plotted the boundaries of Walsall Council wards automatically on Google Maps:

Good work, and as Michael Grimes says: “surprising as it may sound, this hasn’t really been done before. Prior to 1 April 2010, UK ward boundary data were simply not available for public use; groups of fools like us could not have spent our free time building this tool for the public good.”

But he also has another really good point that anything like this would be improved by “users choosing boundaries based on their own understanding of geography (administrative boundaries for religious groups or sports organisations, for example) as well as the official civic ones”.

I’m quite obsessed with the idea of defining areas in a meaningful way, have done a fair amount of work with organisations that use internal or official descriptions or designations and then expect the public to grasp what it means to them. People don’t know, or usually care, which NHS PCT they live in and people don’t know where ward or constituency boundaries — it just doesn’t fit in with how people’s lives work.

The problem is that even defined administrative boundaries are confusing — do we mean New York city or New York state?  The ‘West Midlands’ one is a classic where different boundaries of different bodies cause confusion; Governmentally the region is Stoke down to Worcester, the  BBC’s West Midlands also has Gloucestershire in it, where as everyone thinks the West Midlands is just Birmingham and the Black Country — the West Midlands county.

I think most mapping online is using either administrative boundaries or postcode level (where people know which bit they’re in, but not too much about where one might end or begin).

I think that most people grasp:

• County
• Town/City (or ‘council level’) &
• Postcode

as methods or locating where they live but will have a hazy idea of where each starts/ends. For other areas all my be even more hazy.

I had an idea ages ago for a sort of scraper of the social web that helped define “conversational” (here I mean natural language) boundaries rather than administrative ones (for example, where I live the definition of  Moseley stretches well into King’s Heath, Balsall Heath and even Billesley — all neighbouring areas —in conciousness). You could sort of do it with mentions of places vs geolocation on Twitter  — although there’s maybe not quite enough data there yet.

Flickr tried something similar with geolocated photos that also had location information added in the tags, it resulted in boundaries that were quite different (I can’t find the link right now, anyone?). I think we need to pay more attention to these ‘real’ boundaries.

It contains over 200 records, placed on the map by contributors. Zoom in, or see a detail:

It’s available to buy on my Zazzle store, with my other map-based artworks.

Hyperlocal is the buzzword that being used to describe those new models — “The term “hyperlocal” is sometimes used to refer to news coverage of community-level events.” (Wikipedia). Community in this sense has no real definition — except that it’s assumed to be smaller than the “local” of the traditional news gatherer.

It’s understood that online news sources (those that are extensions of the off-web mechanisms) are struggling in part because people don’t read every page on their website — only the stories that interest them. The phrase in the US is “print dollars become online dimes” (or somesuch) — the same content (and a potentially wider audience for each piece) doesn’t generate the same revenues. This is because it’s possible to split, target and assess those eyeballs and click-throughs.

Extend that into the “local arena” and there’s less room for the niche — it’s not feasible at all to suggest that advertising can pay to generate  truly niche content. A stark contrast with niche but non-geographic interests, which can find a audience online that outstrips any a conventionally distributed source can provide.

A quick and dirty example: It’s no good saying that an announcement of (say) a new ukulele group in a suburb is of interest to all in that suburb — it isn’t — and as filtering gets better that information will only reach those that care enough and are local enough. At that level even a specialist shop won’t pay enough to gather than information.

It wouldn’t work in a current (read historical) “local” news source — unless as a little bit of human interest filler, and it wasn’t that item that was attracting the advertisers — it was a place in the whole “package”, which soon will no longer exist.

But there are people and companies that still want to do the local news gathering — why? There are a few competing (or complimentary) models emerging — here’s one way I think we can divide them:

• the very local, volunteer run (one or two people) blog (often no ads, or a trickle of Google ad money — but no real desire to make money)
• the local blog that runs (sort of) like a small newspaper (ads are often sold direct to local business in the same way as a newspaper)
• the network — where sub-sites for areas are created (the ads are sold centrally, the object is to keep the overall site running rather than the sub-sites)
• the aggregator — where content is electronically pulled from various social (or news) spaces — (ads sold by the aggregator for the aggregator)

All but the volunteer-led source are facing the exact same problem as the “traditional operators” — a fight between scale of operation and potential income. The local blog “newspaper” has more flexibility and less costs than the traditional operator and can work on much tighter margins, but it still has to balance between area covered and effort expended. What the two “ground-up” models have as an advantage is that they can feel the size of the area for themselves from experience, they are covering an area that makes sense to them. This might not be at a level that can attract enough advertisements (Philip John is encouraged by the take-up on The Lichfield Blog, but it can’t be paying for much above server costs — will it eventually? I have no idea). What the networks and the aggregators seem to be doing is picking a size that they can sell advertisements too and making that the area on which they focus.

I did a very quick, small and unscientific survey on Twitter asking people what sort of area felt “local” to them — with these results. People had very different ideas of “local” — a road with 35 people, to a country with 4,500,000. Even those picking the same definition (eg my suburb) had wide ideas of what size that was. This isn’t surprising — and many people (rightly) said things to the effect that “my local airport is further away than my local shop” and “it depends”.

Without some system of “soviets” — a network of ultra local sites, each feeding upwards and having new input at each level of scale — there’s no way that one news source (or type of news source) can cover all of the news needs of each people. What’s happened in the past is that the “distribution” scale featured it’s own level and a pick’n'mix of that below — people understandably felt that wasn’t serving them well and have started to create outlets at different scale. These so far have worked in tandem with existing outlets — so aren’t really equipped to replace them.

The worrying (for some) is that the “distribution” scale corresponded roughly with that of legislature (although not a good fit always) — so that’s a gap that is less obviously well filled. Pits’n'Pots – for example – does a great job at the level of Stoke’s Council, but what independent outlet is operating at the scale of regional development agency? Is there anyone that can hold AWM to account (although one might argue that few do anyway)?

It’s my contention that different types of sites will plug these gaps — I could see a “what are they up to” site run for most legislative bodies or quangos, or different sites sharing resources to hold bodies to account. Support networks, collaboration will be what’s needed. There are some businesses there (tech support, local ad sales perhaps), but it’s not huge profits — and there aren’t certainly for content creators.

It’s lucky, therefore, that most content creators are doing it out of duty or love.

We’re in a period of transition — we know that no-one source is enough, but the don’t yet have the methods to pick the bits from separate sources. Aggregation tools as they stand aren’t the answer, and it’s difficult for people to be brave enough to “trust the network” as we have to.

It’ll come, it’s coming — but we’re not there yet — exciting isn’t it?

Describe your idea:

A new delicious tagging plugin that also harnesses the power of location services — so that it bookmarks where people find things interesting.

It could use triple tags to simply add a geo-attention “point” to each item bookmarked.

This tagging could then be used by anyone to see where bookmarks were interesting.

What problem does it solve?:

All attempts to collate and distribute local information are stymied by the fact that placing most information “on a map” is complex (council decisions, govt dept decisions affect discreet boundaried areas, news either has a “spot” and fade in influence or something more esoteric). People don’t do it, or the tech isn’t there for them to.

Coming at it from the other angle would give a sort of heatmap of influence which could prove useful for all sorts of projects — particularly those interested in local news and democracy.

Does that sound worthwhile? Please comment on the MySociety page.

Plotting the data on a map (alongside other data) could show all manner of things — but more importantly raise questions that are worth investigation: are regulations enforced more in certain areas, does enforcement contribute to lowering numbers of accidents on those roads, whatever anyone cares enough about.

But, not easily. Yet. The spreadsheets reveal that the location data in there is just shy of being able to be plotted on Google Maps (or similar) without altering:

The locations aren’t detailed enough for plotting by the tools we might use quickly, for use internally in Birmingham City Council they’re fine. As things readable by humans they’re fine. But to quickly pop something on a map there’s no tool I know of that will let you say “all these roads are in Birmingham, UK” — so the mapping software can’t plot.

Of course you can write a script to add “, Birmingham, UK” to each (or do it by hand) but that’s not simple — it becomes the work of coders rather than “the public”, will enough people be interested?

Yes data in public is better than private, yes get any data out there so some people can use it; but to really unleash the power then that data will need to be collected and stored with “free” use in mind. Are organisations ready for that?

Uploaded with plasq‘s Skitch!

While it isn’t doing anything with the data — yet — the power comes from the way a folksonomy of location can evolve even on services (like Twitter) that don’t offer it directly. The deliberate placement and collaborative ethos may open the path to really usable data being collected. Very interesting.

The first attempt to formalise it was in the 50′s by various Lettritsts and while they were concentrating firmly on the urban (and architecture in particular) the idea was something that continued and was further worked on by Guy Debord in Critique of Urban Geography.

I’ve long been of the opinion that the social web, the blog especially was an ideal canvas for this sort of activity — allowing as it does fast and free publication of thoughts and also, increasingly, opportunities to “tag” the locations easily. Recently though I’ve been thinking more and more about just how perfect our internet of feelings and thoughts was for the study of emotion and location. As I had re-iterated to me at the recent Cultural Mappings Symposium — place is the great unifier and connector of all sorts of data, mapping allows juxtaposition of otherwise unrelated items, and that reveals the questions we should be answering.

A problem with psychogeography as defined by Debourd and the Situtationists is that in order to prepare reports on areas the pyschogeographer (as opposed to the wandering flâneur) must submit themselves to the dérive — a kind of deliberately directionless route with attempts to let emotion act as the guide.

“the dérive, or drift, was defined by the situationists as the ‘technique of locomotion without a goal’, in which ‘one or more persons during a certain period drop their usual motives for movement and action, their relations, their work and leisure activities, and let themselves be drawn by the attractions of the terrain and the encounters they find there’. The dérive acted as something of a model for the ‘playful creation’ of all human relationships.” (‘The most radical gesture: The Situationist International in a postmodern age’ by Sadie Plant)

The dérive attempts to mirror the movement of the residents or users of the area — and to disorientate the pyschogeographer to document emotion rather than topography. It is doomed in this attempt as the observer by his/her very nature is not experiencing the emotion of the inhabitant. This is analogous perhaps to Heisenberg’s Uncertainty Principle (there some pairs of properties on a sub-atomic level that cannot both be know — by measuring one you alter the other), as by becoming a supposedly dispassionate observer you cannot experience the emotions as they are commonly felt.

Psychogeography has stumbled in and out of fashion, and become sidelined as a worthwhile pursuit, referenced more often as a thread in fictions than as a pure study — where in fact the relation between emotion and place is one of the most important pieces of information going.

It’s what I’m calling Conversational Psychogeography — the applying of psychogeographical method to the conversations that take place online.

There are two strands to this:

One is that the social web is the ideal place for contemporary pyschogeographers to work; the ease and speed of publishing in may media and the defined ability to place those reports discretely in geographical space — by geotagging. This is self publishing, as has long been a psychogeographical tradition,  but allowing collaboration on a scale never before allowed — and many and different ways of directly connecting information to maps. This can be cultural mappings, deliberately collected memories and emotion, or projects more related to art (such as my own 11 bus project) — the advantage of the social web is the connectivity and the collaboration.

The second is perhaps more technology-driven and hence only now becoming possible — that is to scrape the social web for emotion as it relates to place.

As a very basic exercise in this is the @birminghamuk emotional wellbeing indicator I constructed just over a year ago. It’s a very simple script that attempts to read news stories published online, blog posts tagged “birminghamuk” and tweets that are located within “Birmingham, UK”. It then analyses this against a static, but unscientific set of “emotion words” each with a rating — an emotion ‘score’ for birmingham is then produced hour by hour. It seems to work, mostly following the weather (despite the words rated but containing weather descriptions):

“Last Saturday, a weekend day of almost unremembered sunshine, the rating hit “superb” and by Sunday tea time (perhaps Villa not in Europe, Blues relegated, that dull ache of an impending Monday morning, Antiques Roadshow) it had dropped to an “average”. I’m not claiming anything scientific here, but a proper study could produce an accepted list of emotions and also factor in other measures. I’ve left the program recording the data, and at some point I’ll get round to graphing it. Overlay temperature, sports results, news items, even the price of a lager in The Spotted Dog and we might have what we need to work out what makes our city chirpy.” (About the project in the Birmingham Post)

This is a trivial example of a powerful idea — imagine all blog posts analysed both for emotion and location, imagine all social web content analysed for both. Using the textual processing used in part on the Mapping the Lakes project, with a more detailed concentration on both place and emotion — many web services already offer location extraction (Yahoo Pipes will do it on the fly for a feed) — a psycogeographical report of areas could be built automatically.

Much content is already geotagged, and with the proliferation of geo-sensitive mobile technology it’s a trend that is steadily on the increase. While more difficult, the processing of speech to extract data from services such as AudioBoo (geotagged audio content) is teetering on the brink of possibility.

In this context conversational pyschogeography allows a slicing of the internet to research almost any part of that relation between emotion and location. The sheer glut of data being created means that there are real conclusions to be drawn.

There are techniques to be drawn from the data mashing culture, and also the GIS research being done — as well as more tangentally related disciplines such as natural language processing or speech to text conversion — but in essence the ability to do this is already available.

It’s so exciting.

Uploaded with plasq‘s Skitch!

It’s also difficult to see the use of the site as it stands is. Interesting the data might be, but in its flat form it doesn’t “do” much.

Freeing the data to my compared and overlaid with other types of information (more than the “bus stops and police stations” that can be toggled here) would produce so much more — imagine house prices, cctv density, education standards, regenerations spends and other info combined. What truths would we be able to see?

Nice start, many more possibilites.