How YQL powers Cocktails, the technology behind Livestand

YQL is a query language platform. While it has been popular with developers outside of Yahoo! for some time now, it should come as no surprise that YQL also serves a vital role as a key part of the Yahoo! ecosystem, processing a substantial volume of queries reaching well over hundred billion per month.

Incidentally, YQL is also one of the key technologies behind Livestand, along with the Cocktails development platform. While Cocktails provides a powerful cross device application runtime, YQL steps in with a scalable, data execution environment for hosting data-tier logic that is easily accessed via familiar query language and YQL Tables. YQL is a key data pipeline between the application and discrete specialized content services within Yahoo!. YQL is a front-end serving platform for various content pieces such as sports articles, weather, stock quotes, advertising and user preferences to name a few. YQL is used to deliver information to the application as well as send data from the application to target web services responsible for particular content type. In this context, YQL can be thought of as content multiplexer with a query interface.


With YQL's flexible table encapsulation model, expressed through environment files, applications can bundle multiple table implementations in a decoupled manner from the user interface and the client application itself. This allows the application user interface to rely on established content data class, rather than on multiple content endpoints, each with its own semantics. This is a critical piece of abstraction, especially considering the need to manage thin client applications, which often run native code, relying on a software update cycle. The problem is particularly pronounced when this update is controlled solely at user discretion. With YQL, and in this distributed model, the binding with a content provider is encapsulated within YQL table. This enables Livestand to switch out with ease a content component, and more importantly, also enables flexible content enrichment.

YQL helps solve a key content programming problem at scale, it also empowers application developers with simple self-describing Tables, which make data selection and extension easy. Tables written once are reused in multiple application contexts lowering development time. This approach also enhances support for the MVC model, with data layer validation performed independent of UI itself across multiple uses and content providers.

While YQL delivers value to distributed applications and in particular Livestand, it does so by tackling a fundamental problem of isolation of data. YQL ties web services with its own, isolated access characteristics that require applications to consider each one independently and code to its unique access schema. YQL offers a single, seamless interface to access all those disparate data services through one language construct. The service enables developers to query, filter, and combine data across various web data sources. The exposed syntax is both familiar to developers and expressive enough for getting the right data.

With YQL you can access data in multiple formats such as HTML, XML, JSON, RSS, Atom and micro-formats. Additional access formats are available with community contributed YQL format extensions, such as SPARQL and even SOAP. While YQL offers a wealth of SQL and data shaping functions as well as ready-to-use Tables, it also provides flexible IO features such as YQL Streaming, and callback based asynchronous processing to name the most recent.

Since its initial launch in October 2008, YQL has been very well received. With addition of Open Data Tables, developers anywhere could contribute YQL Tables to describe any data source on the Web, making YQL easily extensible to any application needs. Today, YQL offers a total of nearly two hundred built-in tables, and well over a thousand community contributed tables, and more are being added daily. You can easily build your own data table by following instructions on creating Open Data Tables, you may also check out source code of already committed tables on github.

Over the past two years the YQL developer community has grown from serving over a million table queries per month to serving over twenty billion table requests per month for various applications worldwide.

YQL taglines and articles from community:

    "I Tried YQL Execute and All I Got Was an Authenticated Javascript API Processing Layer in the Cloud”
    “The Yahoo Query Language is a great tool that’s guaranteed to speed up your web development time”
    “YQL becomes increasingly valuable to make the use of these APIs easier”
    “I’m in love with YQL”

From Twitter:

    “YQL just saved me. I just made a huge social stats app in 2 days. Easy.” by MikeNGarrett

    “YQL + Raphael == coolness “ by dalmaer Dion Almaer

    “Creating your own YQL Webservice with No Server!” « Triptych by triptych Andrew Wooldridge

Starting with YQL is trivial; simply navigate to YQL Console to run the YQL statement. You can follow YQL on Twitter and check out the blog for latest on product updates and new features.