Blog Posts by Yahoo! Developer Network

  • Multi-Core HTTP Server with NodeJS

    NodeJS API viewNodeJS has been garnering a lot of attention late. Briefly, NodeJS is a server-side JavaScript runtime implemented using Google's highly performant V8 engine. It provides an (almost) completely non-blocking I/O stack which, when combined with JavaScript closures and anonymous functions, makes it an excellent platform for implementing high throughput web services. As an example, a simple "Hello, world!" application written for NodeJS performs comparably to an Nginx module written to do the same.

    In case you missed it, NodeJS author Ryan Dahl (@ryah) gave an excellent talk at a Bayjax event hosted by Yahoo! a few weeks ago.

    Here at Yahoo!, the Mail team is investigating the use of NodeJS for some of our upcoming platform development work. We thought it would be fun to share a bit of what we've been working on, which includes

    • Performance characterization of NodeJS vs. other web stacks in various workloads: straight HTTP proxy at different latencies, scatter/gather HTTP proxy, etc.
    Read More »from Multi-Core HTTP Server with NodeJS
  • Hadoop2010: Hadoop Security in Detail

    allowFullScreen="true" src="https://s.yimg.com/m/up/ypp/default/player.swf" flashvars="vid=20889997&autoPlay=1">

    iPod: Download high-resolution version

    Interested in knowing what the Hadoop Security release comprises and how to leverage it to solve business challenges? The talk will primarily cover the user/developer impact and a peek at the implementation. Owen O'Malley is a software architect on Hadoop working for Yahoo's Grid team, which is part of the Cloud Computing & Data Infrastructure group. He has been contributing patches to Hadoop since before it was separated from Nutch, and is the chair of the Hadoop Project Management Committee.

    Baycat logo Media Production by BAYCAT, a non-profit community media producer that educates and employs underserved youth and adults in the digital media arts.

    Read More »from Hadoop2010: Hadoop Security in Detail
  • Hadoop2010: Hadoop Grows Up

    allowFullScreen="true" src="https://s.yimg.com/m/up/ypp/default/player.swf" flashvars="vid=20793363&autoPlay=1">

    iPod: Download high-resolution version

    Hadoop project founder Doug Cutting gives us his unique insight and perspective into how the project has evolved over the last 4 years, as new sub-projects have grown up and out. He then give us a glance at how he sees things changing going forward as the data management industry more broadly embraces Hadoop for information and analysis. Doug is the creator of numerous successful open source projects, including Lucene, Nutch, and Hadoop. Doug joined Cloudera in 2009 from Yahoo!, where he was a key member of the team that built and deployed a production Hadoop storage and analysis cluster for mission-critical business analytics.

    Baycat logo Media Production by BAYCAT, a non-profit community media producer that educates and employs underserved youth and adults in the digital media arts.

    Read More »from Hadoop2010: Hadoop Grows Up
  • Hadoop2010: Amazon Elastic MapReduce

    allowFullScreen="true" src="https://s.yimg.com/m/up/ypp/default/player.swf" flashvars="vid=20793309&autoPlay=1">

    iPod: Download high-resolution version

    As the demand for cloud-based analysis of large data sets explodes, customers of Amazon Web Services have wanted to leverage Hadoop in the cloud. Last year AWS introduced enterprise-ready Amazon Elastic MapReduce, which manages the complex and time-consuming set-up, management and tuning of both Hadoop clusters and the compute capacity upon which they sit. A user can instantly spin up large Hadoop job flows and begin processing within minutes. Peter Sirota is the General Manager of Amazon Elastic MapReduce.

    Baycat logo Media Production by BAYCAT, a non-profit community media producer that educates and employs underserved youth and adults in the digital media arts.

    Read More »from Hadoop2010: Amazon Elastic MapReduce
  • Hadoop2010: Scaling Hadoop

    allowFullScreen="true" src="https://s.yimg.com/m/up/ypp/default/player.swf" flashvars="vid=20793317&autoPlay=1">

    iPod: Download high-resolution version

    At Yahoo!, we’re engineering for global scale, and constantly expanding the frontiers of Hadoop. Hear how we’re scaling Hadoop to handle larger clusters, growing application complexity, and evolving business needs. Eric Baldeschwieler is Vice President of Hadoop at Yahoo!. Eric leads the development team responsible for designing and deploying Hadoop across Yahoo!’s global infrastructure, the world’s largest implementation of the technology.

  • Hadoop2010: Internet-Scale Cloud Computing

    allowFullScreen="true" src="https://s.yimg.com/m/up/ypp/default/player.swf" flashvars="vid=20793307&autoPlay=1">

    iPod: Download high-resolution version

    Looking to the future, Yahoo!’s vision for the cloud is closely intertwined with open-source computing. As implementer of the world’s largest Hadoop deployment, Yahoo! serves as the canary in the coal mine signaling emerging trends and challenges that the industry will face as Hadoop adoption continues on its sharp increase. Shelton Shugar is Senior Vice President of Cloud Computing at Yahoo!.

    Baycat logo Media Production by BAYCAT, a non-profit community media producer that educates and employs underserved youth and adults in the digital media arts.

    Read More »from Hadoop2010: Internet-Scale Cloud Computing
  • Hadoop2010: Large-Scale Analytics at Facebook

    allowFullScreen='true' src='https://s.yimg.com/m/up/ypp/default/player.swf' flashvars='vid=20793313&autoPlay=0'>

    iPod: Download high-resolution version

    Facebook has one of the largest Hadoop clusters in the world. Mike Schroepfer, Facebook's VP of Engineering, talks about how the social networking service uses Hadoop and Hive to analyze data at scale. The presentation covers how the company deploys the technology and what products utilize the power of Hadoop and Hive. Mike Schroepfer is the Vice President of Engineering at Facebook. Mike is responsible for harnessing the engineering organization's culture of speed, creativity and exploration to build products, services and infrastructure that support the company's users, developers and partners around the world.

    Baycat logo Media Production by BAYCAT, a non-profit community media producer that educates and employs underserved youth and adults in the digital media arts.

    Read More »from Hadoop2010: Large-Scale Analytics at Facebook
  • Hadoop2010: Big Data & the Power of Hadoop

    allowFullScreen='true' src='https://s.yimg.com/m/up/ypp/default/player.swf' flashvars='vid=20793320&autoPlay=0'>

    iPod: Download high-resolution version

    Behind every click across Yahoo!, there is a global cloud computing infrastructure, heavily reliant on Hadoop. We see the magic in crunching unimaginable volumes of data to create increasingly relevant Internet experiences for people around the world — and Hadoop is the technology behind that magic. Blake Irving is Executive Vice President and Chief Products Officer at Yahoo!. As leader of Yahoo!'s Products organization, Irving is focused on building unique and highly personal experiences for Yahoo!'s consumers, delivering on Yahoo!'s promise of Science, Art, and Scale to its advertisers, and continuing to deliver more and faster innovations to the market.

    Baycat logo Media Production by BAYCAT, a non-profit community media producer that educates and employs underserved youth and adults in the digital media arts.

    Read More »from Hadoop2010: Big Data & the Power of Hadoop
  • GIDS 2010 report: Y! thought leadership

    Yahoo! reached out to a 2,000-strong audience at the recent Great Indian Developer Summit held in Bangalore, April 20-23. We connected with the developer community through showcasing our innovations and evangelizing Yahoo! offerings at our presentations and our booth.

    Yahoo!'s senior director of engineering in Bangalore, Nishad Kamat, spoke on "Longevity of Scalable Systems." He drew a large audience of nearly 100 folks. The topic offered an overview of how an engineer needs to be prepared to build long-term scalable systems in fast-moving markets.

    Nishad Kamat at GIDS

    Interaction designer Anshuman Kumar, who showcased Yodel, says “GIDS was a great forum to portray Yahoo!’s surge in innovation and the loads of work it is doing for the open-source community in the form of YUI libraries. The audience was very receptive and enthusiastic to know what Yahoo! had to offer.

    "I covered the aspect of prototyping," Kumar explains, "and how it influences the product development cycle. The developers in the audience

    Read More »from GIDS 2010 report: Y! thought leadership
  • BarCamp Charlotte – The Southern hospitality startup

    Last weekend, I headed out to Charlotte, NC, to attend their third annual BarCamp, held at the Area15 startup incubator space. Somewhere in the neighborhood of 150 developers, designers, entrepreneurs, and fellow travelers showed up to talk about a huge range of topics, including:

    The first talk I was able to attend was Jeremy Olson's discussion about promoting iPhone apps. Olson was speaking about his experience developing and marketing Grades, a tool for helping students (and their parents) keep track of their grades every semester. Grades has had plenty of buzz in the iPhone development community and has even been featured in the App Store on a few occasions. Olson walked us through how he was able to garner so much attention. Key takeaway: Don't be shy.

    In the second session, I gave a talk about developing mobile web applications. I made a couple of brief

    Read More »from BarCamp Charlotte – The Southern hospitality startup

Pagination

(91 Stories)