Kelkoo goes microformatic

Microformats are a big deal. Regular readers will have seen a lot about them on these pages before, and in the past eighteen months every other web development journal worth its salt will likely have mentioned how to mark up contacts and calendar events using the hcard and hcalendar patterns.

However, microformats — the popular, reusable HTML patterns produced at — step out beyond those initial use-cases. Formats for reviews (hreview), blogs, news and syndication (hatom) and curricula vitæ (hresume) are also spreading.

Last month our teams at Kelkoo and Yahoo! Europe pushed out version 10 of Kelkoo. It brings with it the single largest deployment of the hListing draft so far.

hListing is a new format to mark up classifieds, services and product listings. It is quite generic, and can be used to mark up physical products, flat shares, services, events, requests for help and all sorts; consider the breadth of listings on a site like Gumtree or Craigslist. For Kelkoo and retail sites, it provides an excellent foundation for describing product information: Item names, descriptions and prices are all marked up.

Of course, the challenge that faces any new format — be that HTML pattern or W3C specification — is adoption. The need for tools for consumption and the motivation to publish tend to be at odds with one another. Whilst hListing is still a draft, and there’s work to do to complete the specification, it’s been stable for over a year so we’ve worked to see whether the core of the format really works in the real world. You may consider the following as a resounding ‘yes’:

We’ve just published 26,456,448 hListings on Kelkoo. Allow me a little indulgent emphasis, that’s twenty-six and a half million hListings in the wild, right now.

Alongside that gigantic number, we’ve also published 6500 hcards for the details of our merchants and stores; the companies listing the products on Kelkoo. There’s also some XOXO on the sitemap and what I can best estimate as ‘lots and lots’ of hreview, too.

This bumper injection of structured data into Kelkoo’s pages makes it ripe for re-use, be that browser extensions to draw out product information on our pages, indexing services aggregating product listings together or mashing up the data for reuse in widgets. To prove the concept, this piece concludes by building a ‘related products’ widget to accompany tagged blog entries.

Aside from taking the opportunity to lay praise on the developers involved in this release, hListing is a very, very interesting microformat to have in the wild, and we really hope will gain more traction.

### Consume!

Microformats are getting easier and easier to consume in a RESTful manner, thanks to all manner of tools from the likes of Brian Suda ’s X2V scripts, Dmitry Baranovskiy ’s Optimus parser and validator, and Drew McLellan ’s hKit.

In this case, a webservice is running hKit with a profile written for the current hlisting deployment. It parses a URL and returns a JSON structure for the page.

On Kelkoo, all of our search result pages, category pages and store pages are marked up with hListing, so any product listing can be converted into JSON. So for example, this URL finds products with keywords ‘Canon’ and ‘400D’:

If working specifically with Kelkoo’s results, note the query terms are always separated by OR in the query, never AND. By default you’ll receive 20 results sorted by popularity.

Next, we construct a URL to use the hKit webservice and extract a JSON object from the search page:

The URL breaks down into a call to parse hlisting into a JSON structure of the information on the Kelkoo results page, wrapped in a callback named getProducts.

### Example: A contextual products widget

To show this in action, here’s a widget to pull in related products based on the tags of a blog entry. (Tags are perhaps the most adopted microformat of all, using rel-tag.)

#### Start with a blog entry

Here’s the HTML for a typical blog entry (marked up with hatom), with some tags:

<div class="hentry">
  <h1 class="entry-title">Shiny new Mac!</h1>
  <div class="updated">
    Posted yesterday
    (<span class="value">2008-03-10</span>)
  <p class="entry-summary">250 words of passionate gadget talk,
    now slightly faster.</p>
  <div class="entry-content">[…]</div>
    <li><a rel="tag" href="/tags/apple">Apple</a></li>
    <li><a rel="tag" href="/tags/iMac">iMac</a></li>

#### Read rel-tag

When the widget is initialised, it reads and parses the tags from the page. Anchor elements which are tags are identifiable with where the rel attribute has a value ‘tag’.

getTags : function() {
// Get tags (as strings), from each rel-tag microformat on the page
var tags = [];
function(anchor) {
var rel = anchor.getAttribute('rel');
if(rel !== undefined && rel !== null) {
return (rel.indexOf("tag") > -1);
else {
return false;
function(a) {
// The rel-tag specification states that the tag itself is not
// the inner text on the anchor, but is instead the last fragment
// of the URL.
// See
var segments = a.href.split('/');
var tag = segments[segments.length-1];
return tags;

This makes full use of the very powerful getElementsBy function in YAHOO.util.Dom. It takes a function to filter elements in the document, and a function to process each element when found in the document.

As well as producing a more concise piece of code, it’s efficient and saves looping over the tags twice; once to find them and once more to process them.

Generate a search URL, transform into JSON

The tags come back as an array, and since Kelkoo’s search URLs are quite simple, the tags are just appended to search URL as keywords. The widget makes the search URL configurable.

queryListings : function(tags) {
var self = YAHOO.EU.BenWard.HListingProductWidget;
var products;

// If there are tags, search for products
if(tags.length > 0) {

// The query is cross-site, so insert a new script element with
// the webservice URL, and then pass it a callback function

var qscript = document.createElement("script");
qscript.type = "text/javascript";
qscript.charset = "utf-8";

var query = tags.join("%20"); // create an encoded search string
self.datasource = self.provider + query;

// self.parser is a string, pre-set as:
// "
// profile=hlisting
// &output=json
// &callback=YAHOO.EU.BenWard.HListingProductWidget.callbackListings
// &url=",

qscript.src = self.parser + self.datasource;

Passing in the <var>callback</var> parameter, when the JSON is returned it will immediately execute the function YAHOO.EU.BenWard.HListingProductWidget.callbackListings.

#### Render

When adding content to a page in a way that’s reliant on both client-side JavaScript and a dynamic data request, it’s essential to progressively enhance static content with the dynamic results.

Not only does this provide content to script-less user agents, but it also means some useful content can remain if there are no matching search results for a particular blog entry. For this widget, the placeholder content can be set to anything at all; a technique well suited to the varied contexts of blogs.

Using a mock YUI blog page, the static base looks like this:

<div id="related-products">
<h3>Buy into this post?</h3>
<div id="related-listings-results">
Search <a href="">Kelkoo</a>
to compare prices of products of all shapes and sizes,
perhaps even related to what you've just read…
<p><strong>Powered by <a href="">Kelkoo</a>,
<a href="">microformats</a>
and <em>Colville brand coffee</em>.</strong></p>

The widget container (related-products) is styled by the site itself and includes some framing content (the title and ‘powered by’ statement). The related-listings-results element contains some generic, static text which makes sense alone, but will be replaced by more relevant search results where available.

The widget is initiated when the page loads:


The widget init function takes the ID of the element to replace with results, and the search provider URL for Kelkoo. Changing the listings to a different provider can be as simple as changing this one parameter.

Pulling it all together, once initialised the widget extracts tags and builds a search query from them. From the post-query callback function, any results are then rendered to the page.

// Initialisation takes an element ID name, provider URL and
// optional preferences structure.
init : function(el, provider, options) {
var self = YAHOO.EU.BenWard.HListingProductWidget; // shorthand

// Get the widget container and confirm its existence
self.widget = document.getElementById(el);
if(!self.widget) {
return false;
// Here the specified container doesn't exist, so fail

// Similarly, if no search provider URL is provided, the script fails
self.provider = provider;
if(self.provider == undefined || self.provider == '') {
return false;

// The widget is somewhat configurable, for the number of results to
// display, for example. Using an object for all options makes this
// extensible in a tidy manner.
if(self.options === undefined) {
self.options = {};
self.options.resultsLength = self.options.resultsLength || "5";

// With the widget configured and primed, run the first steps to load the
// search result data
var tags = self.getTags();

The queryListings function instructs the webservice to callback to callbackListings. That handles the final parts of processing the results and rendering back to the page.

callbackListings : function(listings) {
var self = YAHOO.EU.BenWard.HListingProductWidget;

self.listings = listings;
renderWidget : function() {
var self = YAHOO.EU.BenWard.HListingProductWidget;

if(self.listings !== undefined && self.listings.length > 0) {

// With it verified that there are results to display,
// replace the entire static content of the ‘widget’ element

self.widget.innerHTML = "";

// Results are going to be displayed in a list, ordered by
// popularity (the same order as Kelkoo sends back by default)
var list = document.createElement("ol");

var i=0;
(listing = self.listings[i])
&& (i < self.options.resultsLength);
) {
// For each result (up to the {resultsLength} limit set in the
// widget options), create an LI, fill it with content and then
// append it to the list.

var li = document.createElement("li");

var item = document.createElement("a");
item.className = "fn";

if(listing.item.url !== undefined) {
item.href = listing.item.url;
else {
// Microformats are flexible to publishers, so handle
// situations where a product might not have a URL of its
// own, and instead link back to the datasource that the
// results were parsed from.
item.href = self.datasource;

var lister = document.createElement("span");
lister.className = "lister";

var price = document.createElement("span");
price.className = "price";




// Finally, append the list to the now empty widget container element.


These are the core components to parse a simple microformat from a page, use it to query a more complex web-service and progressively enhance a page with the transformed content.

Once upon a time, HTML pages had to be scraped to reuse content. Microformats change that. We go from a world of scraping to a world of real, more reliable parsing. Whether parsed from the DOM in JavaScript, or transformed remotely, microformatted information is an incredible enabler for content reuse across the internet.

Kelkoo is just one provider, and we’re looking forward to others following suit with support for hListing. Reviews are prevalent all over the internet, contact information even more so. After years of waiting for technology to move the web forward, it’s happening. There’s information our there now to pull of functionality we never had before. As web developers, there’s little to do but slip in microformatted mark-up wherever we can, and start having fun in consuming it.

Download full code listing here