Some people would say we’re mad, but last Saturday, on the hottest day of the year so far, a group of people eschewed barbecues and beer in the garden for a whole day spent working with mapping data and other bits of open data to help people understand the benefits of open data at Mapitude, an event aimed at developing understanding and practical collaboration between web developers and mappers.

As well as unconference style talks and discussions, there was also a hack day, where a group of us got together to try and solve practical problems using open data in just one day. The brief we decided on was to map a council ward, add some statistical information to it, and then compare it with a neigbouring ward.

As Dan Slee from Walsall Council was in attendance, he was keen that we map St. Matthew’s ward in Walsall, a deprived ward in the centre of Walsall, and compare it with Aldridge Central and South, a neighbouring, but significantly less deprived area.

Once the brief was written up, I got together with Chris Taggart of OpenlyLocal to identify some data sources. The first thing we noticed was that OpenlyLocal didn’t have much information about the wards and councillors in Walsall - however, after a bit of trawling Walsall’s website and adapting an existing screen scraper, OpenlyLocal had Walsall’s data up, this made it easier to get the councillors for each ward.

The next challenge was getting the ward boundaries. Now, a few months ago, this would have been nigh on impossible without either breaking the law, or physically walking the boundaries with a GPS tracker (which, even then, might have been dodgy). However, thanks the the release of Ordnance Survey data, this is now significantly easier.

However, even with Open Data, this was still not an easy task. The boundary data comes in ESRI shapefile format, which, on it’s own isn’t a particularly friendly format for web developers, as it’s designed for desktop GIS software, so the first task was converting the Shapefiles to something we can easily work with online.

Thankfully, I’d been doing some research a few days prior, and chanced upon this bit of work, which includes a program that converts ESRI Shapefiles to MySQL tables, a format that is much kinder to web developers. Once that was done, we then had to convert the boundaries themselves from Ordnance Survey National Grid References to latitude and longitude, which was easily done by this PHP script.

We now had councillors and shapefiles, and already time was against us. The group who put together the brief had decided on population, families claiming child benefit, average income, and number of unpaid carers. This could be retrieved from the National Statistics Data Exchange, but we ran out of time, so ended up simply copying and pasting the data. However, given enough time, we knew we could’ve done it.

You can see the results of our work here, and some of the comparisons are quite stark. We would’ve liked to add crime data from the Police API, but again, pressures of time meant we didn’t get round to it. However, if you look at the crime stats for St Matthew’s and Aldridge Central and South, you can see there’s quite a difference, with crime in St Matthew’s being much much higher. However, this may be slightly skewed by the fact that St Matthew’s takes in much of Walsall town centre where there are a lot of pubs and bars, so fights etc at chucking out time are, sadly, common.

Where next?

So, was it all worth it? Well, I know Chris Taggart will be adding similar functionality to OpenlyLocal in the future, so in that respect it was useful. It also helped me get a handle on how to work with the OS boundary data, and yesterday, I released the Lichfield portion in a much more usable format on the Lichfield District Council open data section.

The most important aspect though, I think, is serving as an example of what can be done with open data. For too long, us open data folks have been banging on about how great open data is and everyone should do it. However, the people who have the power to really open stuff up aren’t always convinced, they might be wary, or not understand the issues. With examples like this, we can start with a problem, and in less than a day, have a solution. It’s these solutions which open people’s eyes to the power of data, and convince them that this is the future.