This is part of my blog, which I have long since stopped maintaining. The page has been preserved in case its content is of any interest. Please go back to the homepage to see the current contents of this site.
The name was a pun on the “Westminster Bubble” in which MPs are sometimes unkindly said to live — implying a lack of awareness of the rest of the country — and “Hubble” alluding to the Hubble Space Telescope, which has allowed us to see distant objects in more detail than ever before.
Westminster Hubble was a website that aimed to bring MPs and their constituents closer online by providing a single location to find contact details for an MP, in real life and on social networks. It also provided customised feeds of MPs’ activity from a variety of sources, from YouTube videos to speeches made in the House of Commons. At its core was a RSS-parsing engine powered by SimplePie that pulled in content from all the sources it knew about as quickly as it could, stashing the results in one giant database table. The contents of this would then be served to users as HTML, or as an RSS “meta” feed to users who preferred to get the data that way.
Westminster Hubble’s main “feed” page for an MP, in this case tech-savvy MP Tom Watson.
Amongst my favourite features were the Google Maps / They Work For You mashup that allowed users to find their local MP in an intuitive way, and the “badges” awarded to MPs for particular dedication (or just a lot of tweeting).
Westminster Hubble’s “find your MP” map
We launched just after similar service Tweetminster really took off, and although we never achieved their relevance or their Wired UK features I still feel that we were offering separate complimentary services — Tweetminster curated tweets around particular subjects for use by those in and around Westminster, while we pulled together tweets and other items from particular people inside Westminster and provided them to those on the outside.
In many ways, Tweetminster provided a destination, somewhere people would go to get information, whilst Westminster Hubble was designed to fade into the background and become part of the plumbing of the internet — RSS feeds went in, RSS feeds came out in a more structured form as chosen by the users. In many ways, then, it shouldn’t be surprising that this week I am closing Westminster Hubble due to a lack of use. Without the user appeal of being a “destination”, the users didn’t come — didn’t spread the word.
Westminster Hubble “badges”
In recent months, the web itself seems to have turned a corner from the heady days of the early 2000s; the Web we lost. Twitter’s discontinued API v1 takes with it the availability of RSS feeds for a user — parsing Twitter feeds now requires a “proper” Twitter client that must authenticate and use the JSON API. Facebook pages no longer advertise their RSS feeds; third-party tools must often be relied upon instead.
It seems the days of mashups, of open services that exposed their data in freely-usable machine-readable formats, are fading. Facebook, and to a lesser extent Twitter, are realising that to maximise their profits, they need to keep users on their sites rather than accessing their data from elsewhere. They are becoming walled gardens in the tradition of AOL, a transition that is fundamentally bad for the free and open web that most of us enjoy today.
If I were more of an activist, I would keep Westminster Hubble alive and fix its links to Twitter and Facebook precisely for the reason that this trend needs to be fought — that the British public should have the right to see what MPs post on “walled garden” websites without the members of the public themselves needing to enter that garden. But the fact of the matter is that Westminster Hubble has failed to become a popular service. In the past month there have been exactly six unique visitors, and that includes consumers of the RSS feeds.
It is tempting to leave the service running somewhere in some capacity — its database currently contains nearly a million items posted by MPs over the course of 16 years. (Westminster Hubble has only been running for three years; it retrieves old posts from feeds when it can.) However, there seems little point in maintaining the domain name, the Twitter account and the Facebook page for a service that now sees so few users.
For anyone wanting one last play with the site, on the understanding that many social network integration features no longer work, can do so on the Westminster Hubble temporary server. On request I am also happy to host the complete (~420MB) database dump, in case anyone wants a large data set of MP activity on which to run some analysis.
To everyone else who has used Westminster Hubble over the years, thank you. I hope it proved useful, and I like to hope that maybe even one of you was inspired by it to support an open government, to campaign for it, or to follow in the footsteps of Chris and I and build your own tools to make it happen.
After many MPs have held Hubble’s “badges” over the years, I’d like to award one special, final badge of honour. The Westminster Hubble award for Social Network Mastery could go to nobody else: ladies and gentlemen, Ed Balls.
Ed Balls— Ed Balls (@edballsmp) April 28, 2011
So long, and thanks for all the fish.
My reason for saying “no” every time is that it would tip me over the edge into buying things I can’t afford — and the AR.drone that Nodecopter events use wouldn’t even be the most expensive part of the build.
With the knowledge that I will probably not build this project any time soon, if at all, I figured I might as well put the design out there in case anyone else is up to the challenge (and the expense). So without further ado…
Introducing the ChromeCopter 2000
Despite the most recent inspiration for the design being William Gibson’s 2010 novel “Zero History”, the concept — and hence the name — belongs more to an 80s cyberpunk vision of the future; where the Internet overlaps with the world in the form of virtual reality cyberspace rather than app stores and push notifications.
This, I feel, is the closest we may get to “jacking in” in the next few decades.
Phase 1 — A Few Simple Parts
The initial “proof of concept” phase is technically simple, although unfortunately very expensive.
The AR.drone comes with a great Android and iOS app that allows you to fly the drone intuitively using the phone’s accelerometer and on-screen controls. The simple addition of a modern VR headset such as the Oculus Rift attached to the phone via an mini HDMI to DVI converter will give 90% of that “jacking in” sensation for 1% of the effort that Phase 2 will involve.
Aside from the price and the Oculus Rift’s lack of a release date, the big problem here is that the pilot will have to operate the phone’s on-screen controls — such as altitude — by feel alone because she won’t be able to see her own fingers.
Replacing that on-screen control with motion sensors is the other 99% of the effort.
Phase 2 — The Inevitable Raspberry Pi
The one limitation that stopped me rushing out to buy an AR.drone on release day was that it cuts a few corners to make it (relatively) cheap. Its props aren’t powerful enough to allow it to support a decent payload (read: onboard autonomy controller), and the core control software is closed and unhackable.
What this means is that any interesting capabilities have to be implemented off the drone itself. While this isn’t ideal, there is a decent SDK, and now there is a very popular Node.js ar-drone library on which Nodecopter code is based.
This is PC-based, so the logical place to start would be a PC that fits in your pocket — the Raspberry Pi.
(The AR.drone SDK also supports Android, so this bit would be possible with something like an Android phone and an IOIO board. This would rule out the use of Node.js, though.)
With the Raspberry Pi running Nodecopter software, it becomes pretty easy to integrate a number of sensors and feed the control through to the drone over WiFi. Plus, a portable “phone charger” type battery pack would make the processing module more-or-less pocketable.
For the control of the drone, the feeling of “jacking in” is enhanced by applying control through body motion. The Oculus Rift includes sensors which could allow the pilot to rotate the quadcopter by rotating her head. Cheap solid-state magnetometers like those used in the Raspberry Tank can be embedded in gloves to control the three-dimensional motion of the drone. For example, tilting the right hand forwards and backwards could decrease and increase altitude.
Finally, a safe “hover” mode is required that will allow the pilot to move her hands without controlling and potentially crashing the drone. This could be achieved by metal contacts on the thumb and forefinger, which must be touching in order to actively command the drone.
These sensors can easily be fed into the Raspberry Pi via its GPIO port, with a C program exposing a TCP socket via which the Node.js code can retrieve the data.
When flying a drone, speed is of the essence, and a lot of work will be required to decrease the time between the pilot moving and the drone responding. Delays here can cause difficulty in piloting the drone, not to mention the possibility of motion sickness since the pilot’s field of view consists only of the (possibly delayed) video from the drone.
That’s my crazy cyberpunk invention of the week. As a design, its BOM cost of over £600 puts it out of my price range for now — but if anyone else builds something similar before I get the chance, I’d love to hear about it!
As my previous post on closing my Last.fm account might indicate, I’m currently on something of an account-closing spree.
Closing online accounts seems to be a rarity these days, and more than once I’ve been asked why I bother. My reasons are pretty simple, so I suppose I care about them more than most people.
- The service knows my email address, which they can use to spam me about a product I no longer care about.
- The service knows my password in some form — if I’m lucky it’s suitably salted and hashed, but in many cases it won’t be. And like everyone else on the internet, I re-use passwords on several sites because I need my brain to remember more important things.
- In some cases, the service knows my credit card details, which I really don’t want falling into the wrong hands.
Closing a Last.fm account was pretty painless. Re-enter your password for security, click a couple of confirmation buttons, and you’re done — with the slightly odd proviso that it doesn’t delete forum posts in your name. Presumably this is so that the forum archives are still readable rather than being peppered with one-sided conversations, but it does suggest that after deleting your account, there is no longer any way of removing your forum posts at all.
Not the worst offender, though, by far.
Today I tried to close an account with an old web hosting provider, 34SP. There’s no obvious way to delete your account through their admin panel, so I contacted support and was told:
Unfortunately we cannot ‘delete’ your account. We keep accounts active for our records should any future changes need to be made.
What future changes? I am leaving your service.
This is far from the first time I’ve encountered this — the internet is littered with accounts of mine that still exist only because the service provider does not provide any means to close an account, often as an actual policy rather than just a programming oversight. It’s not exactly the hardest thing to program anyway, so there should be no excuses on that front. Here’s SuccessWhale’s account deletion dialog — enter your password, click a button, and everything you ever did is permanently erased.
I’m not sure when this practice came about, but it’s particularly frustrating to know that your email address and weakly-encrypted password are held by a company and there is nothing at all you can do to stop that being the case. Although the EU’s “Right to be Forgotten” by internet companies is largely unworkable and unlikely to make it anywhere near a book of law, it would be wonderful to see it demand that account deletion is actually possible.
In the mean time, I intend to stop by Terms of Service, Didn’t Read before signing up to any new online services, so I can make sure I only create accounts that I can one day choose to delete.
On TOS;DR, “No right to delete account” is a worryingly common sight.
I started using Last.fm back in 2006, in the final months of my time at University, and have carried on using it up until a few months ago, despite coming to the conclusion that I should stop back in 2011. Although the social media narcissism of “everyone must know what I’m listening to!” is no longer appealing in these days of over-sharing, I kept my Last.fm account around for its free “recommendations” streaming services until deciding earlier this year that a Spotify subscription was a worthwhile investment.
I was reluctant to delete my account, though, as seven years of listening to over 30,000 songs is a lot of data — so much that it feels wrong to click a single button and pretend it never happened.
Luckily, I’m far from the first person to want to turn their years of recorded listening habits into some kind of accessible permanent record. The most famous such service, LastGraph shut down earlier this month — annoyingly on the very day that I intended to use it — but there are many other ways to get interesting data from a Last.fm history.
Last.fm offers their own visualisation tools in their “Playground” site. Many are for subscribers only, but even free users get access to some interesting graphs.
For example, the Gender Plot uses your history to guess your gender and age. As you can see below, Last.fm pegs me as 24 (I’ll take that as a compliment) and it’s pretty indecisive on my gender — a largely manly playlist conflicts with my fondness for Tokio Hotel, apparently only listened to by 18-year-old girls.
Last.fm Graph is a third-party Java app that takes your favourite artists and displays them as a network graph, showing the interlinking between them. The result is interactive and designed to be played with, which unfortunately makes for a pretty poor screenshot.
According to my output, my main genres of metal and EBM don’t intersect anywhere — perhaps they would have if more industrial acts had made the “top 50 artists” cut-off that I used for the data set. My 2006-2007 J-Pop phase is sitting on its own separate from everything else (and deservedly so).
Last.fm Extra Stats
Last.fm Extra Stats (Windows only, .NET 2.0) generates much the same graphs that LastGraph did, more configurably but perhaps a little less pretty. Everyone’s favourite is the “Wave chart” view, showing trends in listening to your most popular bands over time.
Here, the amount of music I listened to — or at least, the number of tracks I scrobbled to Last.fm — dominates the chart causing a very bumpy output, but it’s all there. The sheer volume of Kotoko and Scooter tracks I’ve listened to are now laid bare for the world to see and silently judge me on.
My favourite of the bunch has to be LastHistory (OSX only). It’s not the prettiest visualisation, but what it does do is not just plot your listening over time on a day-by-day basis, but minute-by-minute. The resulting visualisation displays information about your life, while others simply display your music.
In this history I can see my varying sleep patterns as I changed from student to office worker to father. I can see the all-nighters I pulled and what music I chose to accompany me. The days when I listened to music only on my commute, and the rarer interludes where I managed a whole day of listening.
Reminiscence rears its head in strange places, few stranger than a 30,000 point data set began one day with a 20-year-old thinking people on the internet would be interested in his music.
Today I delete my Last.fm account, thankful for the opportunity to look back over seven years of my life summarised in scrobbles. I hope this page proves useful for anyone else in a similar situation, looking to extract pretty graphs — or even memories — from their Last.fm history.
There are a whole host of decisions involved with starting a new software project. What’s my target audience? What language shall I write it in? Which libraries shall I use? And of course, “What shall I call it?”
So picking a name for an open-source project or tool is IMPOSSIBLE it seems...— Mark Embling (@markembling) April 22, 2013
For anyone looking to give their new project a unique name, there’s an annoying process to go through of searching for each idea to see if something already exists by that name. Linux packages need to have unique names, as do SourceForge projects, Ruby Gems and projects on many other distribution systems.
As of 4pm yesterday, there was no simple way of querying all these repositories and package management systems together, to see if your chosen name was already taken by someone else.
So at 8pm I sat down to code. And by 11pm, there was a way to do exactly that.
Meet CICI, or “Can I Call it…?”
CICI is a simple website. You give it a name you would like to use for your project, it checks against a bunch of services, and tells you if your name is unique – i.e., you can call it that – or not.
Currently, CICI looks up information on packages and projects using Github, SourceForge, Ruby Gems, PyPI, Maven, Debian and Fedora, but it’s easy to add more. CICI itself is a simple Ruby script (full of ugly hacks, as is befitting for a program that I knocked together in a few hours), which you can download and contribute to on GitHub. It’s all BSD-licenced.
Of course, you can play with CICI on the web right here:
As we have also discovered, typing random words into the search box to see what it finds is surprisingly addictive… See what odd (or even useful) things you can find on CICI, and good luck with your new projects – whatever name you end up giving them!
This morning I tweeted my annoyance with Facebook’s privacy model, and since that provoked some (albeit minor) reaction, I thought I’d follow it up with a better explanation of what I’m on about.
Have you ever used a third-party app to access Facebook – such as TweetDeck, FriendCaster or my very own SuccessWhale? If you have, did you notice that some of your friends’ posts and notifications just don’t appear in the app, whereas on the Facebook website they are perfectly visible? Have you seen some odd comment threads where certain friends’ comments are missing when you view the thread from an app?
If you have, the problem isn’t with your app. It’s a problem with the settings that the users you can’t see have set – and that problem is that they have their privacy settings set correctly.
This is a pain in the arse for me and many others who access Facebook primarily through third-party apps, because not only do we miss out on important updates from these people, but the only way to ‘fix’ the situation is to ask them to degrade their privacy settings. As a big fan of online privacy, that’s not something I’m willing to do.
So how does this problem come about?
Well, let’s say we have two characters who want to communicate on Facebook. I’d suggest Alice and Bob, but after 20 years learning about cryptography, neither of them are willing to trust their paranoid conspiracy theories to Facebook’s messaging system. No, I started this post with an oddly-placed reference, and I’m going to persevere with it.
We have two characters, Ahab and Ishmael, who are friends aboard the good ship Facebook. Ahab fires up his favourite whale-themed client, SuccessWhale, and links it to his Facebook account. He gets a dialog like this:
Ahab clicks “Allow”, and as he granted the “Access posts in your News Feed” permission, he starts seeing posts from his friends. But not Ishmael.
Why not? Because Ishmael is a privacy-conscious sailor, and has previously found the “Apps” section of Facebook’s “Privacy Settings” dashboard. That section not only contains settings for what personal data apps you use can see, but also settings for the apps that your friends use. By default, it looks like this:
Ishmael saw that section and, quite rightly, thought “So if a friend of mine uses Farmville, this means Zynga can see everything I do on Facebook without asking my permission? Fuck that!”, and promptly unticked all the boxes. His data is now safe from unscrupulous apps used by his friends – but his data is also hidden from ‘nice’ apps too, like SuccessWhale and TweetDeck.
Arguably, this privacy model is also “right” – Facebook can’t control what apps do with the data they can see, so it has no way to distinguish between SuccessWhale (which needs to see friends’ posts in order to be useful) from FarmVille (which has no business looking at friends’ posts at all).
Not only are two sets of privacy settings making apps like mine annoying to use, but the annoyance is doubled by the fact that both of those privacy settings are arguably the right decision on the part of both our Ishmael and Facebook.
Fig. 3. A Successful Whale (artist unknown)
It’s no secret that, since the launch of version 2.0 back in July of 2011, my SuccessWhale social network client has stagnated somewhat. It had reached that point at which it did everything that I needed it to do, and so my enthusiasm for updating it kind of disappeared.
Well, no longer. Twitter discontinued TweetDeck, the only Android client that merged Twitter notifications and Facebook feeds without sucking. At the same time it discontinued TweetDeck’s desktop client, and removed Facebook support from the web-based client.
That really sucks.
And that’s where SuccessWhale comes in.
I’m no longer content with the ways in which I interact with Twitter and Facebook, particularly on mobile devices, so we’re going to fix it.
SuccessWhale began as a “my first PHP application” kind of affair, and right now it still is. The code behind it is an ugly mash of model, view and controller without a decent structure. SuccessWhale version 3 will be rebuilt from the ground up with proper design principles behind it.
Even better, haku is making an Android client called OnoSendai which will feature the combined feed columns that are SuccessWhale’s major feature. We will bring TweetDeck’s feature set back to Android with a lot more besides, offering the users the ability to mix together the feeds in their social network client like never before.
And to prevent our software going the way of TweetDeck – being bought up and eventually scrapped – SuccessWhale and OnoSendai are open source software. A version of SuccessWhale’s API, operating on the main database at successwhale.com, will be open for anyone to use and build clients for. SuccessWhale is released under the BSD 2-clause licence and OnoSendai under the Apache 2.0 licence, meaning that even if we were to be bought out, anyone on the web could simply grab our source code and run their own SuccessWhale.
We’re bringing TweetDeck’s features back to Android and to the web. We’re making SuccessWhale an application to be proud of. We’re free, we’re open, and we’re Twitter-proof.
Today, Google announced that its popular RSS reader, Google Reader, would be retired. This has provoked an outcry from certain corners of the internet, as GReader was a well-loved product. In many ways it was – and still is – the gold standard of RSS readers. It was massively more popular than the alternatives, so much so that many other RSS reader applications added a “sync with Google Reader” option so that the offline application could stay in sync with what the user read on the web in GReader.
A year and a half ago, a similar public outcry met Google’s decision to remove Reader’s sharing feature and instead integrate the “+1” button from its Google+ social network. The sharing feature was almost a little social network in its own right, just between Reader users who were already “friends” on Google Talk. But Google’s flagging “proper” social network needed those “shares” to drive traffic there, and so sharing within Reader was removed.
We should have seen the writing on the wall then, because the reason the service is being withdrawn is just the same – Google+ offers the ability to read RSS feeds, so by removing Reader they can drive more users to Google+. Why? Because Google+ creates advertising revenue, making Google money. Reader does not.
Never mind the fact that many Reader users explicitly preferred not to use Google+. Never mind that the feed reading UI on Google+ is cluttered with social things, and the management interface significantly worse. The decision to close down Google Reader is justifiable on financial grounds alone.
Does this fall afoul of Google’s “Do No Evil” policy? Possibly not – discontinuing a product that doesn’t make a company enough money is hardly evil – but it certainly serves as a reminder that the Google of today is not Larry Page and Sergey Brin getting together to make cool websites that make people’s lives easier. Google is a massive multinational company and is beholden only to regulators and to its shareholders.
If, one day, they decide it is financially advantageous to roll GMail into its Google+ service, they will do it in a heartbeat.
We all trust a lot of data to companies like Google. There is nothing wrong with this per se, but we must remember that they do not offer us services such as GMail and Google Reader for free out of the goodness of their hearts. They offer them to us because the economics of online advertising mean that they make more profit if the service is free than if it costs money. When those services no longer make a profit, they will very quickly cease to be available.
Centralised “cloud” services have their advantages: they are easy, simple, and social. But they have one big disadvantage: the users will always be completely at the mercy of the service provider.
It should have been obvious when TweetDeck was acquired by Twitter back in 2011 that it wasn’t long for this world. Even more so when the only significant update in the intervening period was to remove a feature (handling tweets over 140 characters).
Although Twitter started out by enthusiastically embracing 3rd-party app developers, its quest to find a way to monetise its service has led the company to grab more and more control over how its users interact with the platform. Users who use Twitter’s website and mobile apps can be served ads, or “promoted tweets”, much more easily than those using 3rd-party clients. The transition was an obvious one, but not a pleasant one – many developers turned on Twitter, accusing it of being actively hostile to developers.
I would be hard pushed to disagree. Westminster Hubble relied on Twitter’s RSS feeds to let people follow their MPs more easily – a feature broken by Twitter’s API changes. SuccessWhale survives, but hardly with Twitter’s blessing – if it were to ever have 100,000 users, it would be banned.
TweetDeck’s Merged “Mentions” and “Notifications” Column
And today we lose the TweetDeck app on desktop and mobile platforms.
I am an avid user of TweetDeck for Android (actually, its fork “TweakDeck”, though they are very similar). This is for the simple reason that it is the only Android app that can combine “mentions” from multiple Twitter accounts and “notifications” from a Facebook account in a single column view. Surely this is a feature that plenty of people would like in an app. But check out the competition. This is a list of all the Android apps that are both Twitter and Facebook clients:
- TweetDeck is dying – service outages are forecast before it is killed off completely in May.
- TweakDeck is an old fork of TweakDeck, not under Twitter’s control – but the API changes that kill TweetDeck will take TweakDeck with them.
- Seesmic offers combined Twitter and Facebook feeds in its paid version – I don’t object to paying £1.89 for an app, but Seesmic has been acquired by HootSuite and will be phased out.
- HootSuite itself does support multiple Twitter and Facebook accounts, but its user interface offers no way to merge feeds together or even swipe between columns from different accounts.
- Scope offers a merged mentions/notifications feed, but only supports one Twitter account, has performance issues (on my devices at least) and has odd defaults (all retweets are also posted to Facebook, Tumblr etc unless manually turned off every time).
- UberSocial (formerly Twidroid) supports only one Twitter account, and adds Facebook as an afterthought with no merging of feeds.
- Plume (formerly Touiteur) supports multiple Twitter and Facebook accounts, but only supports Facebook’s posts feed, not notifications.
- StreamLife is intentionally low on functionality, and only shows “home” timelines, not mentions/notifications.
The functionality that Twitter is removing by retiring TweetDeck is simply not found anywhere else in the Android ecosystem. Until some other application steps in to fill the gap, a function that I and many other users love is simply and infuriatingly impossible to achieve on Android.
Just like with Facebook, it is the network effect that keeps me – and countless other developers – using Twitter despite its increasingly developer-hostile control over the ways in which we interact with it.
One day, perhaps the “next big thing” in social networking will be a platform that starts open and stays open.
- The fact that they’ve chosen to lead with a photo of a Taranis UAV, even though in the photo caption they acknowledge that it isn’t autonomous in any way they are concerned about
- The fact that in the Guardian’s article about the campaign, the newspaper has instead opted to lead with a photo of the Terminator
- The fact that the whole thing – Terminator image and all – is distinctly reminiscent of the year 2009.
The Human Rights Watch campaign centres upon the idea that military robots with an autonomous killing capability (i.e. those that can conduct a lethal attack without human oversight) are not only in development but, when ready, will be used naïvely and with no consideration for the law.
Yes, military robots with increasing levels of autonomy are being developed. The Guardian article rightly cites the X-47B, which is capable of autonomous landing on the deck of an aircraft carrier and is soon to undergo autonomous in-air refuelling trials. But at no point do Noel Sharkey, the Guardian or Human Rights Watch offer any examples of systems capable of lethal autonomous behaviours. Nor do they offer any evidence that the developers of these vehicles intend them to be put into service despite the fact that they “could not meet the requirements of international humanitarian law” and “would also undermine non-legal checks on the killing of civilians” (quotes: HRW).
Technology is advancing at a relentless rate, and they – we – are right to be concerned about the possibility that one day a robot may be programmed to take a human life. But the US Military at whom the campaign is targetted has been publishing studies such as “Governing Lethal Behaviour in Autonomous Robots” (part 2) since long before this HRW campaign and before 2009’s Terminator-themed article too.
Campaigns titled “Stop the Killer Robots” do little but persuade the public that the people and companies who develop these technologies are incompetent, or worse, actively opposed to the idea that human rights and legal conventions should govern robotic warfare.
Also, they make the robots sad.
Screw you, humans. I ask for a torpedo launcher for my birthday and *this* is what you come back with? is.gd/ug7fCc— Wandering AUV (@WanderingAUV) February 26, 2013
Please, think of the robots.