It's been three weeks since we began the limited preview of Yahoo! Search's new open developer platform, SearchMonkey. Today, we're officially opening up the doors to all developers -- professionals and hobbyists -- to begin building applications that enhance the usefulness and relevance of search results.

There are three components to this open ecosystem:

• Site owners share structured data with Yahoo!, using semantic markup (microformats, RDF), standardized XML feeds, APIs (OpenSearch or other web services), and page extraction.


• Third party developers build SearchMonkey applications.


• Consumers customize their search experience.

So, what's in it for developers?

With SearchMonkey, developers have a hand in shaping the next generation of search by building customized search results and mash-ups that users can add to their Yahoo! Search experience. By leveraging structured data from sites like CitySearch, StumbleUpon, eBay, or Epicurious.com, developers can add navigational links, reviews, contact information, and even locations to provide enhanced search listings.

Developers can build two types of applications using SearchMonkey: Enhanced Results and Infobars. Enhanced Results replace the current standard results with a richer display. All the links in the Enhanced Results must point to the site to which the result refers. Infobars are appended below search results and can include metadata about the result, related links or content, or links for user actions (such as adding a movie to a Netflix queue).

The process for building SearchMonkey applications is very straightforward:

1) Application Type -- Decide what type of app you want to build (Enhanced Result or Infobar) and enter basic info such as application name, description and icon.
2) Trigger URLs -- Decide the URL patterns that will trigger your app. For example, for the Enhanced Result above, the pattern would be "acmemovies.com/*"
3) Data Services -- Data Services are the structured data on which SearchMonkey apps are based. They can be created using data available in the Yahoo! Search index (via data feeds or page markup such as microformats or RDF) or by using APIs or page extraction.
4) Appearance -- Use PHP to configure how structured data should appear in the application.

Announcing the SearchMonkey Developer Challenge

To foster innovation and creativity on the SearchMonkey platform, we're hosting a good old-fashioned competition. The SearchMonkey Developer Challenge will recognize innovative applications within four categories: Best Enhanced Result, Best Infobar, Most Innovative Use of Structured Data, Best Data Service, and Grand Prize (best over all categories). You have until June 14th to submit your applications for a chance to win up to $10,000.

And don't forget to come kick things off with us this evening at the SearchMonkey Developer Launch Party. Catch live demos, meet the product team and enjoy free food, beer and, of course, schwag at Yahoo!'s headquarters in Sunnyvale, CA.

Whether you can join us for the party or not, keep in touch -- visit our suggestion forum or leave us a comment below. We want to know how the tool is working out for you.

We look forward to evolving web search with you.

Amit Kumar
Director, Product Management
Yahoo! Search

Microformats are community-driven standards, put together and maintained by volunteers outside of a formal organization like IETF. Typically, they cover well-worn use cases, a concept called "paving the cowpaths." A number of microformat specifications in various stages of development are available at microformats headquarters.

Initially, the Yahoo! Search indexer supports the following microformats:

* hCard for personal or organization contact info
* hCalendar for event descriptions and timelines
* hAtom for syndicated content as might appear in an RSS feed
* hReview to record review ratings such as "8.5 out of 10"
* XFN to track relationships on the social graph in a lightweight fashion

The Web has a huge number of helpful articles and tutorials on using microformats. If the structured data your site exposes falls into any of categories above, then microformats are probably a good choice for you.

Here's a simple example. If your personal site already has markup like this, pointing to one of your other sites:
<a href="http://myothersite.com/blog">My site</a>

Add XFN with a single attribute, like this:
<a rel="me" href="http://myothersite.com/blog">My site</a>

The value of rel="me" indicates that the other site is also representative of you.

A more involved example requires changes across more than one element. Let's say a page mentions a review of an iPod, like this:
<div>Overall, I give the iPod a rating of 8 (out of 10)</div>

To add hReview markup to this, a few additional wrapper elements are needed, like this:

<div class="hreview">
   Overall, I give the
   <span class="item">
      <span class="fn">iPod</span>
   </span>
   a rating of
   <span class="rating">
      8
   </span>
   (out of
   <span class="best">
      10
   </span>
   )
</div>

This is simplified markup. Consult the microformats.org site for specific details. In general, these changes indicate that the overall structure is a review ("hReview"), that the item being reviewed is an iPod (additional details such as a URL are helpful), and the rating is 8 out of a possible 10.

What if your structured markup needs go beyond the list of supported microformats? Please provide us with feedback, as we are continuously evaluating and adding support for additional microformats. On the other hand, you might want to consider using more expressive RDF markup. If you want to expose structured data but aren't ready (or able) to make site changes yet, then you might consider writing a custom data service. Stay tuned, we'll cover these topics in future blog posts.

By the way, we'll be talking about microformats among other things later today (Thursday, May 15) at the SearchMonkey Launch Party at Yahoo!'s Sunnyvale headquarters. Here are the details – join us for a bit of SearchMonkey talk and a plenty of beer, food, and schwag.

Micah Dubinko
SearchMonkey Team Alumnus

Like the London Tube Map, recording and retrieval of locations and their relationships doesn’t always have to do with their Lat/Lon. There is a more elegant way to abstract the relationships of location, and unambiguously describe places in a permanent, language-neutral manner. Since Where on Earth joined Yahoo! in 2005, Yahoo! has geo-enabled the network, including geo-tagging advertising, Flickr photos, Yahoo! Maps, and so many of the location-based services that Yahoo! has offered.

Yahoo! is now offering a developer preview of this Yahoo! Internet Location Platform.

Here’s an example of the new platform in action. Check out this photo on Flickr. This photo was geo-tagged by the user, and since it was placed on a map, we were able to give it a set of these location tags. If you browse in to the tagged metadata using the Flickr API, you’ll notice a set of geo-tags called WOEID (Where On Earth IDs) which are permanent, unambiguous, language neutral tags that represent that location.

Since we have content with a tag, we can find out some interesting things, tag 727232 for example, is Amsterdam, and we can use the new APIs that we’re releasing to find out all sorts of relevant relationship information, including:

• The parent, the administrative region of Amsterdam
• Neighbors, such as Landsmeer, Zaandam, and Watergang.


• Belong Tos, such as North Holland, Western Europe, and the Europe/Amsterdam Time Zone


• and more!



(Note: The links above are in XML, and may not be viewable in all browsers.)

This service allows you to discover location relationships from free-text place names, tag your content with location IDs for easy indexing, disambiguate numerous objects tagged with the same location, and so on.

Go ahead and check out the documentation on the Yahoo! Developer Network at http://developer.yahoo.com/geo/.

Congrats to the Yahoo! Geo team on this preview.

Michael Lawless
Sr. Product Manager, Yahoo! Maps

P.S. For more on subject, check out the post by Dan Catt on geobloggers.com.

The event was Ignite at O’Reilly’s Web 2.0 Expo last month. This was the most recent in a series of slideshow “slams” that Brady Forrest has been putting on all around the country, and the second one held at Jamie Zawinski’s DNA Lounge in San Francisco (the first was last fall).

The format is inspired by pecha kucha. Each speaker is limited to 20 slides and five minutes. The slides advance automatically each 15 seconds. Each speaker is encouraged to tell a roomful of geeks how to do something.

The tight strictures of the format force the speaker to get to the point right away and pack as much information as possible into the allotted time. I’ll be honest: it was a bit intimidating.

The format itself turned out to be exhilarating, bringing out great performances from a number of the speakers, notably Kathy Sierra (how to make your users kick ass), Annalee Newitz (why giant monsters rule), and Lane Becker (the gruesome physical consequences of running a marathon).

I discussed some of the social design patterns (and antipatterns) that I’ve been gathering as curator of the Yahoo! Design Pattern Library. You can hear the whole talk, including facetious arena-rock shout-out at the beginning, synched up with the slides (which are harder to read in the video), by watching this slidecast:

| View | Upload your own

Christian Crumlish
Pattern Detective

The problem for developers of networks is that you don't want people to leave your application to see a photo or a video, which means that you need a way to only get the media and embed that. Normally this happens using the media provider's API or - if they don't offer one - screen scraping (loading the whole document and programatically finding and extracting the media).

oEmbed is a great idea to work around that issue:

oEmbed is a format for allowing an embedded representation of a URL on third party sites. The simple API allows a website to display embedded content (such as photos or videos) when a user posts a link to that resource, without having to parse the resource directly.

Flickr, Viddler, Qik, Pownce and Revision3 are the first services to support oEmbed which means you can easily access photos and videos with a simple URL.

Say for example find a nice photo on flickr:

Original URL:

oEmbed URL:

Result:

<oembed>
  <version>1.0</version>
  <type>photo</type>
  <title>? too much myspace error</title>
  <author_name>codepo8</author_name>
  <author_url>http://www.flickr.com/photos/codepo8/</author_url>
  <cache_age>3600</cache_age>
  <provider_name>Flickr</provider_name>
  <provider_url>http://www.flickr.com/</provider_url>
  <width>500</width>
  <height>375</height>
  <url>
    http://farm4.static.flickr.com/3128/2475016321_982666ec95.jpg
  </url>
</oembed>

You can define the output format and the maximum width and height with URL parameters:

oEmbed URL:

Result:

{
  version: '1.0',
  type: 'photo',
  title: '? too much myspace error',
  author_name: 'codepo8',
  author_url: 'http://www.flickr.com/photos/codepo8/',
  cache_age: '3600',
  provider_name: 'Flickr',
  provider_url: 'http://www.flickr.com/',
  width: '100',
  height: '75',
  url: 'http://farm4.static.flickr.com/3128/2475016321_982666ec95_t.jpg'
}

Supported formats for responses so far are photo, video, link and rich.

Christian Heilmann
Yahoo Developer Network

We wanted to remind you that we're kicking off this launch in true SearchMonkey style with a Developer Launch Party next Thursday, May 15. Come get the inside scoop on SearchMonkey, meet with our product managers and engineers over tasty (read: free!) food and beer, see live demos, and take a closer look at the Developer Tool.

When: May 15, 2008, 5:30 -- 8:30 p.m.

Where: Yahoo! Headquarters @ URL's Cafe

701 First Ave.

Sunnyvale, CA 94089

RSVP: Email your full name and company name directly
to searchmonkeyevent@yahoo-inc.com. Space is limited.

For more information on the agenda and logistics, check out the event page.

Events include the by now famous @media conference, the fourth London BarCamp, a Microformats vEvent and Web Roots. To learn about all the events, check the schedule on the web week site.

To kick things off in the right fashion, we will sponsor the London Web Week Opening Party on Monday, May 26, located at the Ruby Blue Bar in London's West End.

If you are in the area, or you are coming to @media anyways, make sure to check out the sign-up page on event wax to secure a ticket, as the space is limited to 200 people.

Hope to see you there and have a blast before making this week even more special than it already is.

• Sign up for a party ticket
• Learn more about the event at upcoming

Christian Heilmann
Yahoo Developer Network

I took this chance to vent some of my ideas on the subject and counteract some of the criticisms I heard about the need for enhancing web solutions progressively.

I've come up with the following "Seven rules of progressive enhancement":

1. Separate as much as possible
2. Build on things that work
3. Generate dependent markup
4. Test for everything before you apply it
5. Explore the environment
6. Load on demand
7. Modularize code

I've explained them in further detail in the slides available on slideshare:

Furthermore I put some more information in an article on the subject: Pragmatic Progressive Enhancement.

The article is licensed with creative commons, so you are very much invited to use and remix it to your needs.

Chris Heilmann
Yahoo Developer Network

It's been a long time coming, but last month I finally pushed out a new design for my personal website, Click on Tyler. I rebuilt it from the ground up using two key tools from the Yahoo! Developer Network:

• YUI Grids and
• YSlow

The new design is really a refresh of the previous look with a focus on readability and speed. I want to take a few minutes and touch on what I learned during this go-round so (hopefully) others might benefit.

Color Scheme

Although I really liked the darker color scheme from before, it was too hard to read. There simply wasn't enough contrast between the body text and the black background. I tried my best to make it work. I searched around for various articles about text legibility on dark backgrounds. I increased the letter spacing, the leading, narrowed the body columns, and everything else I learned in the intro graphic design class I took in college. The results were better, but my gut agreed with all the articles I read online which basically said "don't do it." So I compromised and switched to a white body background, while leaving the header mostly untouched. I find the new look much more readable -- hopefully this will encourage me to begin writing longer posts.

CSS and Semantic Structure

The old site was built piecemeal over a couple months and, quite frankly, turned into a mess font-wise. I had inconsistent headers, font-weights, and anchor styles depending on which section you were viewing. With the new design, I sat down (as I should have before) and decided explicitly on which font family, size, and color to use for each header. I spec'ed out the font sizes using YUI's percent-based scheme which helps ensure a consistent look when users adjust the size. (Go ahead, scale the font size up and down.) An added bonus was that it forced me to think more about the semantic structure of my markup. (If you have Firefox's Web Developer toolbar installed, try viewing the site with stylesheets turned off.) If there's one thing I learned working for Sitening, it's that semantic structure plays a huge part in your SERPs.

Optimizing With YSlow

At OSCON last summer, I attended one of the first talks Steve Souders gave on YSlow -- a Firefox plugin that measures website performance. That, plus working for Yahoo!, has kept the techniques suggested by YSlow in the back of my head with every site I build. But this redesign was the first time I committed to scoring as high as I could.

As usual, I coded everything by hand, paying attention to all the typical SEO rules that I learned at Sitening. Once the initial design was complete and I had a working home page, I ran YSlow.

Ouch. A failing 56 out of 100. What did YSlow suggest I improve? And how did I fix it?

• Make fewer HTTP requests - My site was including too many files. Three CSS stylesheets, four JavaScript files, plus any images on the page. I can't cut down on the amount of images (without resorting to using sprites - which are usually more trouble than they're worth), so I concatenated my CSS and JS into single files. That removed five requests and brought me up to an "A" ranking for that category. (I'm further toying with adding the YUI Compressor into the mix.)
• Use a content delivery network - At Yahoo! we put all static files on Akamai. Other large websites like Facebook, Google, and MySpace push to their own CDNs, too. But what's a single developer to do? Use Amazon S3 of course! I put together a quick PHP script which syncs all of my static content (images, css, js) and stores them on S3. Throughout the site, I prepend each link with a PHP variable that lets me switch the CDN on or off depending on if I'm running locally or on my production server. (And, in the event S3 ever goes down or away, I can quick switch back to serving files off my own domain.)
• Add expiry headers - Expiry headers tell the browser to cache static content and not attempt to reload it on each page view. I didn't want to put a far future header on my PHP files (since they change often), but I did add them to all of the content stored on S3. This is fine for images that should never change, but for my JavaScript and CSS files it means I need to change their filename whenever I push out a new update so the browser knows to re-download the content. It's extra work on my part, but it pays off later on.


• Gzip files - This fix comes in two parts. First, I modified Apache to serve gzipped content if the browser supports it (most do) — not only does this cut down on transfer time, but it also decreases the amount of bandwidth I'm serving. But what about content coming from S3? Amazon doesn't support gzipping content natively. Instead, in addition to the static files stored there, I also uploaded their gzipped counterparts. Then, using PHP, I change the HTML links to reference the gzip versions if I detect the user's browser can handle it.


• Configure ETags - ETags are a hash provided by the webserver that the browser can use to determine if a file has been modified before downloading it. Amazon S3 automatically generates ETags for every file -- it's just a free benefit of using S3 as my CDN.

So, all of the changes above took about three hours to implement. Most of that time was spent writing my S3 deploy script and figuring out how to make Amazon serve gzipped content. Was it worth it? See for yourself.

Wow. Three short hours of work and I jumped to a near perfect 96 out of 100. The only remaining penalty is from not serving an expires header on my Mint JavaScript.

Do these optimization techniques make a difference? I think so. Visually, I can tell there's a huge decrease in page rendering time on both Firefox and Safari. (IE accounts for 6% of my traffic, so I don't bother testing there any longer.) More amazing, perhaps, is the site's performance on iPhone. The page doesn't just load -- it appears.

I've made a bunch of vague references to the S3 deploy script I'm using and how to setup gzip on Amazon. In the interest of space, I've left out the specifics. If you're interested, contact me with any questions and I'll be happy to help.

Tyler Hall
Front-end Engineer

The example was the ease of being able to annotate and caption photos in Flickr enticing a lot more users to enter sensible captions and text which in return makes it easier to find the photos and describe them to non-sighted users.

Now we found out via Amy Cohen Efron's blog "Deaf World as Eye see it" that deaf users have discovered Yahoo Live as an opportunity to chat online.

This is wonderful news, and proof that products that are on the bleeding edge and use technologies with a reputation for inaccessibility can actually benefit users overcome boundaries and make them communicate using "web 2.0" products despite their disability.

It is also a great example that accessibility is about all kind of disabilities, and that technology can be the solution rather than a hindrance.

More reading:

• LaRonda Zupp on the addictiveness of Yahoo! Live for the deaf community
• Scott Gatz on decisions being made to make Yahoo! Live inclusive for all users

As a follow-up, Neal Sample (YOS Chief Architect) gave a 40 minute deeper dive presentation last Friday. In this video he begins with a recap of the information presented in Ari's keynote and then spends time providing more in-depth explanations and examples of the various YOS platform components: update notifications, social connections, the activator, and more.

Expect to see more and more in-depth videos with various YOS engineers in the coming weeks and months.

Jeremy Zawodny
Yahoo! Developer Network

Around 50 developers met yesterday in London, England to hear all about Yahoo's new location based service Fire Eagle. In two short presentations and a long Q&A session Tom Coates and Seth Fitzsimmons from the Fire Eagle team helped developers get their head around the concept and technicalities of Fire Eagle. They were aided by Steve Marshall, developer of the Django Fire Eagle Authentication wrapper and co-developer of Fireball.

The NetSquared Mashup Challenge: Hacking for Social Change

For the first time, NetSquared is running a really cool program called the Mashup Challenge in which they are matching up ideas from non-profits and NGOs involved in all sorts of social change to people like you (i.e. developers, product managers, and designers) who have the skills to implement them. I have been helping NetSquared promote the Mashup Challenge (see their recent YDN Theater video) because I think it's a very practical roll-up-your-sleeves way of getting people to work together across many boundaries (company, international, etc) to produce something exciting and useful that benefits the world at large. It's very much in the spirit of our own Yahoo! Hack Day.

The Projects

The community around NetSquared has already submitted dozens of ideas for theMashup Challenge and the NetSquared community voted for the top 20, which are listed here (there are 21 since there was a tie!). You can see ALL of the ideas here.

How to Participate

I'm working with Billy Bicket, NetSquared's organizer, to recruit Project Leads interested in participating in a 4-week sprint leading up to NetSquared's conference, happening on May 27-28 at Cisco in San Jose, CA.

By contributing to NetSquared in the capacity of Project Lead, participants will be working with a leading social innovator working on a range of challenges including:

1. refining some of their ideas about how the data they are using could be rendered in mashups; and

2. in some cases, building mashups using a variety of APIs, products, or coming up with creative technical solutions to particular problems they are facing.

Project Leads will receive access to all NetSquared Hack Days. The next invite-only event is happening on May 4 at Moscone Center in San Francisco and is hosted by Sun and NetSquared. Project leads will also receive two free tickets to this year's NetSquared Conference.

Here is a quick run-down of the role and expectations for Project Leads and here are the simple next steps if you would like to participate:

1. Scroll through the list of 21 Featured Projects

2. Send Billy Bicket (billybicket -at- yahoo.com) an email with your favorite three (3) featured projects, and confirm your interest in participating by Wednesday, April 30 at noon.

4. Once you send Billy an email, he'll send easy next steps (including link to conference registration) and details about engaging with featured projects for expedient approach and execution.

Bringing It Together: The NetSquared Conference

The Mashup Challenge will culminate in the annual NetSquared Conference on May 27-28 in San Jose. At the conference, project teams will have an opportunity to display and discuss their mashups and attendees will vote to select the top three (this is only several weeks away and some of the ideas are complex, so the projects don't have to be 100% done, just underway). All 20 projects at the conference will receive a share of $100,000 in prize money. The share will be determined by voting at the conference.

It's a good cause, a great opportunity to meet new people, and an even better opportunity to use your various web superpowers. Join a project today!

Chad Dickerson

Today we’re introducing the Yahoo! Open Strategy (or Y!OS for short), the details behind Yahoo!’s big bet to "deliver open, industry-leading platforms that attract the most publishers and developers."

Y!OS platforms will harness Yahoo!'s unique strengths – our rich and relevant user experiences (we’re #1 in 7 verticals), our massive audience (half a billion users/month), and our deep data repositories (content, content, content) – and open them to the innovations of the developer community. Our aim: to fundamentally transform how people experience Yahoo!.

With Y!OS, we’re moving from a model in which each Yahoo! property develops much of its own technology to one where we share common data and frameworks that can be easily surfaced across multiple Yahoo! properties and off the Yahoo.com network.

It’s a major rewiring of Yahoo!.

And the good news for developers is that Y!OS will allow you to access to these assets, build applications around them, and then get distribution on Yahoo!’s monumentally popular properties (and/or use them in your own websites).

Specifically, Y!OS includes:

A Single Social Platform for Yahoo!

The Social Platform will enable a single social "connective tissue" across all Yahoo! experiences letting users view Yahoo! through a relevant social lens. The Social Platform isn’t yet another social networking vertical from Yahoo!. It's a common social infrastructure that will span all of Yahoo! and the web beyond. We’ll start by collapsing the Y! 360 and Mash connection lists into one social repository (and add a new user interface to it), then we'll take a proactive approach to tapping into the TEN BILLION aggregate relationships in Mail, Messenger, Address Book, and other social areas of Yahoo! to recommend connections to our users. Plus, we’ll begin surfacing users’ profile and connection information throughout Yahoo! as the whole Yahoo! network becomes more social, which will motivate users to activate connections. Developers will gain access to this Social Platform via RESTful APIs, giving you ways to query for a user's profile data and connections data (our Social API), ways to update a user’s presence across the network (Presence API), and an activity stream with an API for reading/writing a user’s activity (we call this “Updates”).

Lotsa Standardized Web Services

You want data? Have we got data for you! And we’ll be delivering a lot more of it in the coming months. We’re working to get it to you in ways that won’t require "RTFM" every time we release new APIs. We’ve created a RESTful API standard to which most new Yahoo! API releases will adhere, including all of the Social Platform APIs mentioned above. And some more news: we’ll be supporting OAuth – look for the OAuth Consumer Key to replace the YDN App ID for new API signups here on the developer website very soon.

The Yahoo! Application Platform (YAP)

So now you have an out-of-the-box social graph, and wicked cool data to mashup. How about some traffic? How about some traffic from Yahoo! Mail, the Yahoo! Front Page, Yahoo! Search, My Yahoo! and the revamped Y! Profile? The Yahoo! Application Platform will allow developers to build applications that plug directly into these Yahoo! pages and properties, starting today with an exciting developer's preview release into Yahoo! Search called Search Monkey.

As Yahoo! is a member of the OpenSocial Foundation, YAP supports OpenSocial. Apps you’ve written for other OpenSocial-compliant application platforms should function on Yahoo! with minimal tweaking (and vice versa) -- or you can write apps that go directly to the Social Platform REST APIs; it’s up to you. And YAP offers you the choice of hosting your app with Yahoo! or using your own server to proxy your app back to us.

Today’s announcement is just a hint of things to come. Get started today with Search Monkey and stay tuned for a cascade of more announcements as pieces continue to come online.

We couldn’t be more excited to work with you to change Yahoo! and change the web.

Say it with me now: “Developers! Developers! Developers!” ;)

Cody Simms
Y!OS Product Management