Thanks to the over 200 developers who came to Yahoo! recently for our monthly Hadoop User Group meeting. The energy in the packed room was phenomenal, and conversations continued long after the formal sessions.
Hundreds of Hadoop Fans Flock to Yahoo! for the Hadoop User Group
The event started with Nitin Motgi from Yahoo! describing the challenge of content optimization at scale and how Yahoo! is leveraging the power of Hadoop Stack to conquer this challenge. Hadoop, along with Hive and HBase, is the technology that enables mass content personalization for Yahoo! users. Nitin talked about the high-level architecture and modeling challenges. He concluded with a summary on the learnings so far.
Anil Madan from eBay discussed the adoption of the Hadoop Stack at eBay and their future adoption plans. He also touched upon how eBay is leveraging the Hadoop clusters to enhance search relevance and extend catalog coverage for eBay. He described in detail eBay's data-sourcing patters. Anil concluded with the opportunities for the road ahead.
Next was Doug Cutting of Cloudera, who introduced Avro, a generic serialization system. He talked about the salient features of the system, contrasted the existing serialization system, and highlighted how Avro can help solve the backward-compatibility issues with Hadoop. He walked us through samples on Avro schemas and Avro Map Reduce APIs. Do view the presentation and video for a quick overview and summary of Avro.
We at Yahoo! embrace Hadoop, and we understand and relate to the hurdles eBay is facing, and stumble upon the issues that Avro promises to solve. As always, we are looking for exciting technologies and experiences you want to share. Please contact me via the Hadoop Bay Area User Group Meetup page.