At Yahoo!, Hadoop plays a central role in providing personalized experiences for our users and creating value for our advertisers. To serve Yahoo!’s emerging business needs, the Cloud Engineering Group is working on a next generation platform that enables the convergence of big-data and low-latency processing.
Yahoo! is enhancing its web properties and mobile applications to provide its users personalized experience based on interest profiles. To compute user interest, we process billions of events from our over 700 million users, and analyze 2.2 billion content every day. Since users' change interest over time, we need to update user profiles to reflect their current interests. Figure 1 illustrates a conceptual architecture that describes how low-latency processing and batch processing are leveraged to update users' interest profile for personalization.
Enabling low-latencyRead More »from Storm and Hadoop: Convergence of Big-Data and Low-Latency Processing