History Hack Day – It was, therefore it now isn’t!

Image of King Henry the Eighth, on his way to History Hack Dayhenry8

For the first time as far as anyone can remember,

on a cold and damp London day,
a group of souls with a weekend to spare,
gathered to plot on History Hack Day.

As far as we know, no governments were overthrown this past weekend, and the Queen is still on her throne. But a lot of fun was had by throwing together a group of history buffs with a group of web geeks, to see if they could come up with any new and innovative insights, visualizations, and applications from the wealth of history data out there in the world.

History Hack Day is the latest in a series of events that highlight a growing appetite for subject-matter hack days. It follows the format of the successful Science Hack Days, which took place in London and San Francisco in 2010. I expect to see a lot more of these popping up, as it's hard to think of a subject that wouldn't be suitable for a Hack Day. Sports Hack Day? Yes!; Finance Hack Day? Why not?!; Celebrity Hack Day? Well, maybe.

Big thanks to Matt Patterson for organizing the event. Matt explained that the idea came to him one evening, when he obviously had too much time on his hands. He was wondering why the block of flats where he lives in London was there, amongst the other types of housing in the area. After a bit of research, he discovered that the original houses in the plot had been bombed in the war, and replaced with the block of flats. This is fairly common in London, which took a bit of a beating during WW2. Then, Matt started to think about how the place where he grew up has changed over the years, from an industrial heartland to residential properties. How would anyone to know what used to be there? The old maps have all been updated and the new ones don't show what used to be there. There's no version control repo for maps that we can easily access and see how they change over time.

In recent years, we've seen the emergence of digital maps and Google Street View, which could easily be used to add a time dimension to maps and photos of areas of the world. However, this is much harder to do with pre-digital data. Some parts of the past are lost forever. How cool would it be to be able to play Google Street View back in time? Very!

This was a great introduction to History Hack Day; it set the scene well for the rest of the event. There's a wealth of digital data out there, which is growing exponentially. As the years tick by, web applications like Twitter and Flickr are recording history in a way that will make it much easier to use and visualize for coming generations.

The Speakers

To get the event off to a good start, a number of speakers inspired the hackers with ideas and approaches to using historical data. There were folks with data to hack on, and others with technologies that could be used to make hacking easier. Max Gadney kicked things off with a discussion about his experience of creating visualizations of data. He discussed the importance of providing an epic or eye-catching "way in," and a useful metaphor to help viewers quickly understand what the data refers to and means. He also discussed the difficulty of visualizing timelines effectively.

Tom Pollard from the British Library

Tom talked about the British Library dataset program. He discussed how all good research is built on good data, but that data used in the production of an article is often discarded or neglected after an article is published. This makes it difficult for anyone at a later date to rigorously interrogate and verify an article's claims. To fix this problem, they have come up with a program called DataCite, which supports researchers by enabling them to cite data with confidence. This essentially means linking articles to the underlying data using a unique reference called a Digital Object Identifier (DOI). He also briefly discussed another project by the British Library to archive British websites, and a website that lets you search through the British Library Archives. Enjoy!

Dan Pett from the Portable Antiquities Scheme

The Portable Antiquities Scheme records where the public find valuable antiquities (think of guys wandering round in fields with metal detectors). You can search their database of hundreds of thousands of artifacts to see what's been found where. They also have an API, and make heavy use of YQL and GeoPlanet. Despite some high profile examples of people getting rich finding hoards of gold in British fields, Dan noted that the average find is about £50--about the same cost as the average metal detector!

Mano Marks on Google Fusion Tables

Mano talked about Google's Fusion Tables, a low-tech way of visualizing tabular data on maps and on timelines, which can then be easily embedded on web pages. The Guardian used Fusion Tables to visualize the Wikileaks data on Afghanistan War Deaths.

Francisco Jordano on YQL

Francisco talked about using the power of YQL to easily retrieve and combine data from multiple sources with its simple SQL-like syntax. He talked about adding your own tables, and how this can be made even simpler using Zach Grave's TableSaw, a tool to help you create the basic scaffolding for your own open table. He also discussed how YQL provides the ability to execute JavaScript code into your tables. This means that you have a programming language to alter results, execute more queries, even run JavaScript in the cloud. Francisco walked us through an example, using a Eurovision dataset. All of his materials and references can be found here.

Matthew Shirak from LastFM

Matthew described himself as a storyteller. He told us about the power of storytelling using as many references to Dr. Who as he could fit in. His key message: in this hack day, we should aim to make compelling stories out of data. "After all, history is about stories, and stories are about adventure, and we all need a bit of adventure." He also cautioned us not to be afraid to ask stupid questions, like "Have Old Bailey trials ever included monsters?" or "What plays did people get kicked out of during the blitz"? Often, finding humorous angles within stories and data helps provide a way in for people to interact with the data.

history hack winner shakes hands with Matt Pattersonhhd11_handshake

History Hack Day Presentations

The hack presentations were done fairly slickly as the Guardian meeting room was set up with two sets of screens. So, the next contestant could prepare their presentation when the previous one was doing their pitch.

Best in Show: A History of the world in 100 objects

The winner of the event was a mobile hack by Cristiano Betta, which was based on a recent BBC Series called A History of the World in 100 Objects. All of the objects were from the British Museum, and the Podcast audio files for each episode are still available online. The hack can be useful wherever you are, but it will be most useful if you happen to be in the British Museum. The hack shows you where in the museum that each object is located, and then allows you to listen to the podcast of the program when you've found the object. This hack scores high on the value scale. We can see it being used by visitors to the museum, and it is very well polished. The code for the hack is on Github.

Best Visualization: A History of the World in 100 seconds

This sounds very similar to the Best in Show hack, but it really isn't at all. It's a video hack, that shows the emergence of civilization (according to Wikipedia). The hack was created by Gareth Lloyd and Tom Martin, and here's Gareth's summary of what it is:

"Many Wikipedia articles have coordinates. Many have references to historic events. Me and Tom Martin cross referenced the two to create a dynamic visualization of Wikipedia's view of world history. Watch as empires fall, wars break out and continents are discovered.

This won "Best Visualization" at the Guardian's @historyhackday. To make it, we parsed an xml dump of all Wikipedia articles (30Gb) and pulled out 424,000 articles with coordinates and 35,000 references to events. Cross referencing these produced 15,500 events with locations. Then we mapped them over time."

As if that wasn't enough, they then went on to run their algorithm across the French version of Wikipedia to see if that told a different story of how civilization emerged, and it did. On the French video, there was more activity in Africa and South America, particularly in areas that used to be French colonies. This shows biases in the data sources, which in itself was very interesting and revealing. Other interesting biases were that 60% of the events happened after 1650 and 80% of the events happened above the equator. All in all, this was a great visualization hack, topped of with a great sound track!

Next Best Visualization: GeStation

This hack by Simon Harriyott shows the emergence of the British Railway system on a map. The points that appear represent railway stations in the order in which they were opened. Another great hack which does a good job of visualizing the geo-spacial timeline Captain!

Spent the weekend in a small interview room with @adactio, @briansuda and @zzgavin #hhd11

Most Unexpected Hack: Did you make it off the Titanic?

The most unexpected hacks are always my favorite category. This one by Brian Suda uses a Telephony service called Tropo, which allows you to add voice, SMS, Twitter, IM to applications. It's well worth checking out. Brian used a data set consisting of the details of those who were aboard the Titanic, including their age, sex and the class of their ticket. When you call the number, the system asks you for your sex and age. It then tries to match your details with a real person who was on the titanic, and tells you whether that person made it to safety or sadly passed away as the ship sank. Whilst this hack clearly embraces a sensitive subject, I think that it helps you to grasp and relate to the reality of the tragedy that impacted so many lives. You can experience the hack yourself by calling one of these phone numbers: +1 (804) 316-9215 (US), +44 2035142721 (UK) or your could also listen to it on Audioboo.

Next Most Unexpected Hack: Magical Mystery Ley Line Locator

Ley liness are mysterious lines of force between ancient monuments (obviously), and this hack by Tom Scott can tell you which, if any, Ley lines run through any UK post code, as well as any interesting historical places that lie on those Ley lines. There are a total of 1500 Ancient Monuments in the dataset.

Best Game: Wikipedia Top Trumps

This hack by Mike Stenhouse allowed you to play top trumps with any figure on Wikipedia. You can play any person in Wikipedia against any other, and they area rated on a number of stats that Mike aggregated from a whole host of places. For example, he has a measure of historical importance which is calculated by how many characters are names after the person in IMDb, and a measure of academic importance by the number of times a character is mentioned in academic articles. The game allows you to challenge friends on Twitter, and all you have to do to play is paste in the url of a person on Wikipedia. Clearly Mike went to a lot of trouble to put this together. I only wish I had the URL to show you. You can listen to Mike talk about his hack over at amplified10.com.

Next Best Game: Plaqathon

Commemorative plaques are little round blue plates that you often see on the sides of buildings over London. They mark the fact that something interesting happened here, such as [you name it] famous person in history lived here. Plaqathon is a mobile game that is built on Facebook Places and takes it data from the Open Plaques web site. The idea is that you compete against your friends to checkin to as many of the Plaqued locations as possible. This could be a fun way to learn about the historical places in London.

Best Public History: Wiki Loves Geo

Wiki Loves Geo is a mobile application created by Tom Morris that aims to help Wikipedia solve the problem that there is a distinct lack of photos on certain sections and pages on the site. Wikipedia authors often request photos for those sections. When you check into Foursquare, this application will alert you when you are within photo-shooting distance of a place where Wikipedia would like a photo. You can then take and submit a photo to the site. You can listen to Tom Morris talk about his hack here. A hack that good citizens like us can use to make a difference in the world! Follow @wikilovesgeo on Twitter to learn more.

Further Sources of Information

There were many other great hacks, but I think I'm lucky to have kept your attention this long. So, if you want to learn more about History Hack Day, then you can find it yourself at the following locations: