The Yahoo! Firehose: Get access to our public Updates activity stream

The Yahoo! Developer Network is excited to announce the initial release of the Yahoo! Updates Firehose service. The Updates Firehose is a web service for accessing and searching the full, real-time index of Yahoo! Updates.

Yahoo! Updates aggregates social updates from Yahoo! and across the Web: It includes a real-time feed of every public action taken on our network (such as the 45,000+ comments on the News story about the Healthcare reform bill recently passed in Congress) and elsewhere around the Web that users have authorized Yahoo! to make available. From these, the Firehose feed aggregates activities, including:

  • 750,000+ ratings a day (including those from Yahoo! News, Buzz, Movies, Travel, TV and more),
  • 8,000+ reviews a day (including Yahoo! Local, Shopping, Cricket, Travel and more),
  • 150,000+ comments/day (including Yahoo! News, Buzz, Sports, OMG stories and more),
  • status updates,
  • Flickr uploads,
  • Delicious bookmarking,
  • Open App activity,
  • YouTube favoriting,
  • Last.fm listening,
  • and many, many others.

How it works

The Yahoo! Firehose is accessed via Yahoo! Query Language (YQL), a simple SQL-like query language that is increasingly becoming our primary vehicle for accessing Yahoo! data and services. It allows you to call, update, and combine data from across Yahoo! (and any other data source on the Web made available through open tables) for use on your sites and applications — all while leveraging the Yahoo! cloud.

Firehose Console

Using YQL, you can now query the Updates Firehose for:

By accessing the Updates Firehose’s rich data stream through YQL, it is significantly easier to mash its output with other sources, enhance its display, or even translate it. YQL empowers developers to take the best of Yahoo! data and technology, combine them with other best-of-breed services, and produce rich, engaging experiences wherever users travel around the Web.

The new Firehose feed is part of the family of Yahoo!'s Social APIs and solutions, including:

  • Social Directory API: access Yahoo! user profile data and relationships
  • Updates API: collect and distribute user Internet activity streams
  • Status API: read and set status information for a user
  • Contacts API: tap into information about users’ contacts from one of the Web’s richest and longest-serving Address Books
  • Yahoo! OpenID and OAuth: A simple solution to let users sign in with their Yahoo! ID and get read/write access to their Yahoo! data on your Web page or in your application
  • Applications platform: Deploy your content and experiences with a social lens across the Yahoo! Homepage, My Yahoo!, Yahoo! Toolbar, and more

And off you go

The real-time web could be called the modern equivalent of Adam Smith's invisible hand, organically guiding the web marketplace based on the seemingly chaotic click choices of millions of users. We believe the Updates Firehose will provide the tools necessary for you to make structure and sense out of such an activity stream, and we're excited to see what products you build using the Yahoo! Firehose. (And, when you do, let us know!)

Editor's Note: Erik Eldrige's post, "Two-legged OAuth client/server example," provides instructions on using OAuth to sign YQL requests.