The Yahoo! Developer Network is excited to announce the initial release of the Yahoo! Updates Firehose service. The Updates Firehose is a web service for accessing and searching the full, real-time index of Yahoo! Updates.
Yahoo! Updates aggregates social updates from Yahoo! and across the Web: It includes a real-time feed of every public action taken on our network (such as the 45,000+ comments on the News story about the Healthcare reform bill recently passed in Congress) and elsewhere around the Web that users have authorized Yahoo! to make available. From these, the Firehose feed aggregates activities, including:
- 750,000+ ratings a day (including those from Yahoo! News, Buzz, Movies, Travel, TV and more),
- 8,000+ reviews a day (including Yahoo! Local, Shopping, Cricket, Travel and more),
- 150,000+ comments/day (including Yahoo! News, Buzz, Sports, OMG stories and more),
- status updates,
- Flickr uploads,
- Delicious bookmarking,
- Open App activity,
- YouTube favoriting,
- Last.fm listening,
- and many, many others.
How it works
The Yahoo! Firehose is accessed via Yahoo! Query Language (YQL), a simple SQL-like query language that is increasingly becoming our primary vehicle for accessing Yahoo! data and services. It allows you to call, update, and combine data from across Yahoo! (and any other data source on the Web made available through open tables) for use on your sites and applications — all while leveraging the Yahoo! cloud.
Using YQL, you can now query the Updates Firehose for:
- common keywords used across all Updates (e.g., cryptozoology)
- all Updates from a specific application (e.g. Mafia Wars)
- all Updates of a specific language (e.g., Spanish and Portuguese)
- Updates associated with a location (e.g., New York City)
- Updates associated with a specific URL (e.g., a Huffington Post article on net neutrality)
- Updates of a specific type (e.g., status updates, ratings, comments, and reviews)
By accessing the Updates Firehoses rich data stream through YQL, it is significantly easier to mash its output with other sources, enhance its display, or even translate it. YQL empowers developers to take the best of Yahoo! data and technology, combine them with other best-of-breed services, and produce rich, engaging experiences wherever users travel around the Web.
The new Firehose feed is part of the family of Yahoo!'s Social APIs and solutions, including:
- Social Directory API: access Yahoo! user profile data and relationships
- Updates API: collect and distribute user Internet activity streams
- Status API: read and set status information for a user
- Contacts API: tap into information about users contacts from one of the Webs richest and longest-serving Address Books
- Yahoo! OpenID and OAuth: A simple solution to let users sign in with their Yahoo! ID and get read/write access to their Yahoo! data on your Web page or in your application
- Applications platform: Deploy your content and experiences with a social lens across the Yahoo! Homepage, My Yahoo!, Yahoo! Toolbar, and more
And off you go
The real-time web could be called the modern equivalent of Adam Smith's invisible hand, organically guiding the web marketplace based on the seemingly chaotic click choices of millions of users. We believe the Updates Firehose will provide the tools necessary for you to make structure and sense out of such an activity stream, and we're excited to see what products you build using the Yahoo! Firehose. (And, when you do, let us know!)
Editor's Note: Erik Eldrige's post, "Two-legged OAuth client/server example," provides instructions on using OAuth to sign YQL requests.