A simple hack to geoplace images

I really enjoy reading Shorpy, a website that features images scanned from the American National Archive. It's an interesting, candid insight into American history and offers some great amateur photographs of scenes that wouldn't normally be featured in the history books.

Shorpy website

Sadly, however, my knowledge of American geography is terrible, so being able to put a place to a photograph geographically by description alone wasn't easy.

So, using YQL, a little Javascript, and some HTML, I made a site that would solve this problem for me.

The site uses YQL's ability to both read RSS feeds and — using a description of each photograph — discern any places mentioned. Since Shorpy uses a fairly standard format for describing each photograph it posts, I knew I had a good chance of coming up with a decent location for each.

First I needed to retrieve the description for each item in the RSS feed. The YQL query for this is simple: "SELECT description from rss where url='http://feeds.feedburner.com/shorpy?q=rss.xml''. (You could easily replace the 'SELECT description FROM ...' with 'SELECT * FROM ...' if you wanted to get more information about the RSS feed, but for the moment we're only concerned with the description for each item.)

I used the handy JQuery YQL plugin to make this call:

	function getRSSDescription() {
"SELECT description from rss where url=#{rss}",
rss: "http://feeds.feedburner.com/shorpy?q=rss.xml"
function (data) {
// Code to manipulate our results

How simple was that?! I created a YQL statement and passed in an argument (our RSS feed location), and now I have a JSON object (the 'data' variable in the code) that consists of a results array of a description for each item in our RSS feed. I didn't have to bother with explicitly calling an API, handling URL encoding/decoding, or parsing XML.

So what do we do with this description? Well, for each result in our data object I wanted to see whether it was possible to geoplace a location for the photograph. Since we're handling a JSON object with a known, specific structure, iterating over the results is very easy.

	var resultsArray = data.query.results.item;
var arLen = resultsArray.length;
for (var i = 0, len = arLen; i < len; ++i ) {
var locationString = resultsArray[i].description;
// Code not shown here parses the year from the locationString
// using a simple regular expression, and we pass the original
// HTML from the RSS feed so as to embed it in our map.
getMapLocation(locationString, WINDOW_HTML, year);


After slightly pruning the resulting description to remove unnecessary embedded HTML, I made a second YQL query, this time to the YQL Placemaker table. This time I passed in the location description (returned from the query above) as an argument.

	function getMapLocation(locationString, WINDOW_HTML, year) {
"SELECT * FROM geo.placemaker(1) WHERE documentContent = '#{location}' AND documentType='text/html';",
location: locationString
function (data) {
var latitude = JSON.stringify(data.query.results.matches.match.place.centroid.latitude);
var longitude = JSON.stringify(data.query.results.matches.match.place.centroid.longitude);
var JSONString = JSON.stringify(data.query.results.matches.match.place.centroid.latitude);
// The function addPoint adds the resulting lat/lon to the Google Maps object, alongside
// the original HTML from the RSS feed.
addPoint(data.query.results.matches.match, unescape(WINDOW_HTML), year);

This time our YQL query has returned a returned a single result — we expressly asked for this by selecting from 'geo.placemaker(1)'. Again, as the result is a JSON object, it is easy to reference the necessary attributes using JavaScript (in this case, the latitude and longitude).

So now I have a geographical location for each photograph in Shorpy's RSS feed, and all I've done is make two YQL queries!


At this point it's possible to whatever you want with the data. I passed this to the Google Maps JavaScript library to plot on (see the addPoint() method in the source itself to see how this is done). But you could also write it out, store it, or do whatever you wanted with it.

While there are doubtless ways that JavaScript could be made more efficient — I am by no means a Front-end Developer — I find it amazing how simple YQL and Javascript make it to mix interesting datasets (like RSS) with dynamic services (like Placemaker and all) without any heavy lifting on the server-side. I highly recommend just reading through the datasources available as YQL tables to get an idea of what amazing things YQL is capable of.