BOSS Mashup Framework

The BOSS Mashup Framework is an experimental Python library that provides developers with SQL-like constructs for mashing up the BOSS API with third-party data sources.

The library exposes data constructors that will "best effort" unify response formats regardless of their XML, JSON, RSS/RDF structures. In this paradigm, the developer does not need to specify the parsing and conversion logic per data source, resulting in concise, declarative and simple-to-interpret code.

Features

Current features of the Mashup Framework include:

• Support for SQL-like functions such as select, group (reduce), sort, union, inner join and user defined functions (map)
• Text normalization and duplicate removal
• Auto-transformation of resource-oriented API results into tables without specifying any parsing logic (unifies XML and JSON responses based on inferred data format)
• All-in-memory storage and retrieval operations
• Ability to join lists of tables via an arbitrary predicate function
• Mashup output available in XML or JSON
• Includes UI templates to allow developers to easily render mashed up search results on a customizable search results page template
• Documentation included in the download as a README

Code Example

In less than 15 lines of code, one can search for "iphone" in Yahoo! News and overlay over each result a related result from the del.icio.us popular feed, and display the combined results in a provided search results page template.

INPUT:

from templates import publisher
from util import text, console
from yos.boss.ysearch import search
from yos.yql import db, udfs

dl = db.select(name="dl", udf=udfs.unnest_value, url="http://feeds.delicious.com/rss/popular/iphone")
yn = db.create(name="yn", data=search("iphone", vertical="news", count=50))

def overlap_predicate(r1, r2):
  return text.overlap(r1["title"], r2["title"]) > 1

tb = db.join(overlap_predicate, [dl, yn])
tb = db.group(by=["yn\$title"], key=None, reducer=lambda x,y: None, as=None, table=tb, norm=text.norm)

serp = publisher.Serp(template_dir="templates/california", title="boss 'iphone'", endpoint="http://yahoo/search")

for row in tb.rows:
  serp.add(url=row["dl\$link"], title=row["yn\$title"], abstract=row["yn\$abstract"], dispurl=row["yn\$sourceurl"], source=row["dl\$creator"])

serp.dump("iphone.html")

OUTPUT:

Licensing

The BOSS mashup framework is a non-supported code library, and is licensed under the terms set forth in the Yahoo! Search BOSS Terms of Use.

Feedback

We hope the mashup framework is a useful tool for developers using the BOSS API. Please let us know if you have any feedback or suggestions by sending us an email.