After my talk at Oxford Geek Night I was happy to have a couple of suggestions to see if the algorithm could produce better results. One was to remove retweets from the search, which makes sense as we all know from many Twitter bios “a RT does not imply endorsement”—and that was easy to implement as the basic Twitter search api returns retweets ‘old-style’ with “RT” at the head.
The other was more complex, so I’m going to quote Owen who emailed me directly:
“This morning I thought up an analogy. Suppose you have weather readings for the last 100 days. For each day you have temperature (T), humidity (H) and mm of precipitation (P). What you’re doing is multiplying these all together, presumably because you want to get one number out. Unfortunately this number is meaningless. If you wanted to combine these quantities in some way you should really be thinking about what meaning you’re attaching to the number you get out. I’m ignoring here the fact that you multiplied them all together, when in all likelihood adding them would make more sense. I suggest it would be more meaningful to keep track of them separately, and plot three graphs instead of one. Indeed, this is what is done with weather data.
You spoke about wanting to get a measure of how much spread a set of data has. What you want is the variance, or something like it. The average (more properly called the mean) of a set of numbers is obtained by adding them all up and dividing by the total number. This tells you something very useful, but it loses all information about how spread out the information was. The variance captures that. It’s a bit tricky to calculate. I’ll try to explain it here, but you can always google for more details. Suppose you have numbers a1 up to a100. The average is M = (a1 + a2 + … + a100) / 100. The calculate the variance we have to calculate some intermediate numbers. First, you have to calculate the average. Then you have to calculate the average of each number squared: Z = (a1^2 + … + a100^2) / 100. Now the variance is V = Z – M. I know that doesn’t seem to make much sense. There is a way of calculating the variance which makes it clearer why it’s any use, but it’s a bit harder to actually implement.
You might want to square root the variance to get the standard deviation. This is measured on the same scale as the original numbers you had, so it makes a bit more sense to use that instead.”
So, @IsOxfordHappy and the location sensitive page now do both of those. I’ve removed the ‘word scale’ for the time being till I can see roughly what the numbers are. Thanks everyone for your suggestions.
While working on Siôn Simon’s push for there to be a directly elected Mayor for Birmingham I contributed an ideas paper around the possibilities for a more open system of government in the city. It covered open data (in broad strokes rather than technical details), comms and transparency. It was always a plan for the team that ideas and plans be released as soon as they were ready—happy for any other campaigner to use what we thought were good things.
So, when the referendum went against us I talked to a few people about releasing this stuff anyway. I couldn’t do any harm, I figured. It’s taken me a while to get round to it.
These docs were my own work, so don’t take them as being ‘official’, I’m also not planning to do anything with them at this stage. Some of the ideas may already to part of the political plan but he city’s new council leadership, but to be honest I don’t know.
Feel free to use any of this if it’s interesting.
After moving down to Oxford I did an update of my Birmingham Emotions conversational psychogeography project. That’s now quite simple as I have built a ‘happy monitor’ that can centre anywhere. I’m not as happy myself as I was with the results however, whether due to the increasing volume of the Tweets that it analyses or something else the rating doesn’t move around too much. Such was the problem I proposed in a very quick talk at Oxford Geek Night 27. Here are the slides from the presentation, I think the audio was being recorded and will add if I get hold of it.
I’ve already had a number of suggestions about improving the equation or analysis, if they’re code-able by me I shall try. If not I will have to ask for help…
On a side note, the whole idea of conversational psychogeography came to me when I was thinking of putting an emotional wellbeing indicator in the form of a light at the top of Birmingham’s Rotunda (see how it’s still unfinished right at the top. That was back in 2008, but it seems that London has finally installed something a little similar. Drat.
You can get twice daily Oxford updates on Twitter.
From 18th July until September the large glass edition of the Birmingham Music Map is going to be on display at The Public in West Bromwich as part of their Summer Exhibition. It’s free to visit and has a rather nice coffee shop-cum-bar.
Thanks to Jez from the Birmingham Music Archive for his continued support of this.
Fused magazine, which (as well as sister publication Area) I’ve done stuff for in the past, has just released a very special edition. It’s thick, beautiful and perfect bound and it’s the first volume of what they’re calling the second volume of the mag. It’s got some fantastic photography and illustrations and rather wonderfully for me three of the best interviews I’ve ever done. Read what happened when I talked to bass-god Peter Hook, Barney out of noisecore legends Napalm Death and David Shrigley who’s one of may favourite artists.
Forty two piers in to our circumnavigation of the coast of England and Wales we arrived bleary in Saltburn-by-the-sea one morning. Luckily the breathtaking coastline and the warming sun perked us up. Luckily as it was the only time we were captured on video during the trip. It’s for and, we think, going to be part of a documentary about the pier for its 100th anniversary.
Based on current estimates we’re about halfway or just over to finishing writing the book. Then the edits and long search for a publisher begin in earnest. If you’d like to know more then pierreview.co.uk is the place.
Poorly Collected Works 2010-11 is the title of an eBook I pulled together at the end of last year. It sold a few, and jumped into the Amazon charts when it was part of a promotion, but was more of an experiment. In continuing experimentation, now Createspace is working with Amazon UK I made it into a real paperback too. It’s available to buy now at a cheapish price.
It does contain a few treats not in the similar e-book—amongst them a previously unpublished interview with Barney from the grindcore band Napalm Death, a few pieces focussed on the referendum for an elected mayor that was held in Birmingham in the early part of the year, and excitingly I think a small piece of my half of Pier Review—that real book that I’m writing that will come out sooner rather than later.
Createspace also does DVDs/film downloads as well as books—it’s a fairly simple way of creating work and self publishing. You need to fiddle a little with their formatting and do a lot of checking, but it’s easy enough.
On Friday I got round to doing something I’d been thinking of for a long while. I added location detection to my conversational psychogeography tool. Like the Is Brum Happy? system it takes the latest tweets around a location and rates emotionally sensitive words against a database to give scores for the happiness or emotional wellbeing of the place. If you’re using a HTML 5 browser (you probably are) you can let it reveal your location to the script (it’s not saved anywhere) and it will tell you if where you are (and a mile radius around) is happy right now.
Edgeryders is an EU project to try to work out how creatives and freelancers can make a living ‘on the edge’. One of the people working on it is good mate Chris Pinchen (often found online under the moniker Cataspanglish) and he has been doing a series of interviews. Or more chats really. We had a Skype conversation about how I have been making a living for the past five or six years and he’s cut my ramble into a structured blog post and series of videos.
In this one I apparently talk about how ‘thinkers’ can get the space to think.
You may find it interesting, you may not.
When I returned to the Labour fold and joined the party a few months before the 2010 General Election it was party through fear, partly through duty, but mainly because I thought there was a real opportunity that it could be reclaimed from the New Labour years and start representing the people again. People I trusted trusted Gordon Brown, and the alternatives were too awful to contemplate.
That said the tipping point of joining rather than just voting was because I didn’t want to vote Labour: I had no confidence or trust in the sitting MP that a boundary change had forced upon me. By accounts he was a lifer, remote from his constituents and out of touch. Had the coming election nationally been anything but close I’d have marked my cross next to the semi-independent community focused candidate that looked like she might win. I wanted to vote for her, but didn’t want to be responsible for a situation where we got a tory government by one MP, and not Labour in my constituency. I figured that my membership might offset my ballot-box slip somehow, save my coincidence.
As it turned out it wasn’t that close, neither locally or nationally. But, being a member in opposition felt like the right thing to do. I delivered leaflets, did a little phone canvassing and a little door knocking and was rightly pleased when in the next council elections our man overturned a Liberal majority with a significant one of his own. It might have been national tides that turned the ship but there was a real feeling that the local party was working hard on the ground and really cared. At this point I was as taken with the democratic system as I think I’ve ever been—despite a really disappointing result in the Alternative Vote referendum, an opportunity as I saw it for more representative politics.
I’m fond of quoting that the UK always has a more right wing press than the electorate—there are always more papers supporting the tories than Labour even when the left(er) party wins big in elections. I suspect that if you asked most people about policies in simple terms, and presented real alternatives, most people would be more left wing than any government we get too. Certainly any we’ve had since 1979. So while it felt right to be in the most likely opposition, each failure to control or challenge the dominant shock doctrine narrative by Ed Miliband and his team made it difficult to see what you were working for.
Another referendum offered more hope.
I’d met MP Siôn Simon a few times when he was a creative industries minister, he was trying to look at ways to help the local media through the ‘internet transition’ and I was on the fringes of the local blogging scene that was one possible prop for keeping some news going. He was engaging, engaged, and—you could tell—above all looking for the right solutions to the problems he was tasked with. So when, after the election, he talked to me about the problems of reaching people and told me that he really thought there was an opportunity for change I believed him and believed in him.
That change he said, could be driven by an elected mayor for a city like Birmingham. A position that would have to be backed by a mandate, a leader that would have no choice but to campaign on a manifesto upon which they would be judged. Direct accountability to the people, direct scrutiny by an engaged media, high-profile and with an ability to be strategic. I was sold, and it looked like it was going to happen.
Over the last two years I’ve been variously working on things for Siôn and the non-partisan Yes to a Mayor campaign. The amount of free time I had to volunteer on both ironically helped by the “austerity” cuts the tories disproportionally heaped on poorer areas like Birmingham and on the third sector—reducing my freelance work considerably. I’ve not worked as hard as some others, but I’d like to think I’d made a contribution. To what, maybe, is a more difficult one as ‘we’ lost last week (not by as large a margin as was being reported, but lost all the same).
What’s the main disappointment for me is perhaps the insight it gave me into just how hard change is and seemingly will always be under the current system. Turkeys don’t vote for Christmas, which is understandable—but turkeys don’t talk 364 days of the year about the need for a dry-fowl-meat based religious feast.
Vested interests within the existing party machines are the ones with the access and the voices to the media—which disappointingly went for ‘balance’ (presenting voices from both sides) instead of neutrality (where they could exercise independence and question the arguments on both sides). The main arguments that we heard in the press against the chance for change were all easily researchable and refutable by experienced journalists, but they were all too often let through with a “no campaigners say…” and no challenge. It’s too late to go through them now, but I will anyway as it’s now a pavlovian response:
A mayor would not have cost more than the current system of a council leader and a chief executive, roles which the announced candidates promised to combine. In Leicester, the nearest comparable city the mayor took a salary smaller than Birmingham’s current council leader’s allowances.
The powers of a mayor were defined, in the act of parliament that brought the referendum into being. Extra powers could be negotiated, which in theory they could be for a council leader—if one side was a ‘pig in a poke’ the other was too.
The electorate cannot in practice—whatever the theory—get rid of a council leader. Forget the past, look at the situation currently in Birmingham—a clear Labour majority of councillors that is certainly not likely to change in fewer than the four years a mayor would have—whatever they do and however they perform in the eyes of anyone who cares to look.
Not being challenged on these is the equivalent to the current government not being challenged every time they bring up the ‘cutting the deficit’ line. Statisticians have proved that the cuts are making the deficit worse, in real and structural terms—but time and again that’s not mentioned and the cutters are allowed to dominate the narrative. And for some reason the opposition are too scared to challenge—maybe it comes across as too negative, maybe there is something about the media that they know which means this fight is lost and it’s better for them to move the debate on.
And that’s the crux of it: better for them.
It’s so unusual to see anyone in politics really commit and to do something that might not have benefits for their coterie of contacts. Essentially the party system seems to draw people to compromise for power and not to compromise for good. Siôn’s decision to stand down as an MP to campaign for change in Birmingham always struck me a a courageous decision—not for him the bet hedging of certain other people who were maybe, or maybe not, in favour of real change but would quite like the power if it was available.
And he bravely stuck to talking about the truth of the size of the problems: areas of the city are very deprived, with high—especially youth—unemployment, unacceptable infant mortality. Do you need more? Will rearranging the deck chairs change that? I doubt it. Was that not the line that the media wanted to focus on? Maybe not, but in sea of platitudes and apathy it stood out for me and I was proud to have contributed in a small way.
The Yes campaign bravely tried to rise above politics, and I think that may have contributed to the downfall—often seeing it sidelined in events in favour of people the media already knew about (the BBC WM debate for example had formal ‘no’ people on the panel but no ‘yes’). That presented the argument as a political struggle when maybe it should have been one about opportunity. Some honourable councillors and people involved in the political machines saw that opportunity, but I’m afraid my suspicion is that most didn’t as they were clouded by fear about their own positions and interest.
There were honest people on the ‘no’ side too, but those ‘leading’ it dragged it down some dark paths.
The theory of Overton windows has always stuck me as interesting—it states that there’s only a limited section of the political continuum that’s acceptable mainstream debate at any time. Anything outside it seems too radical to be considered. Decades of right-ist spin seems to have dragged the window to a point where the absurd views of the Taxpayers Alliance are given credence and unions of workers are not—even by the leaders of the workers’ party.
And that’s sad. But more than sad, it’s life and death for countless people in Birmingham, the UK and the World.
I’ve had a brief glimpse of the opportunity as well as a ringside seat at the way the status quo protects itself.
It’s time to be honest, to be open, to be radical. It’s time to challenge obfuscation and apathy as well as disinformation and dishonesty. That can be dragging the window back to the left, or it can be just not letting nonsense past.
Each time you hear someone parrot a political line, think. And if what they say is not true, respond like that. Tell people, bore them stupid.
Reposition the narrative to change.