In my previous post Do you have what it takes to join Yahoo!'s US Hadoop Team? I talked about a number of technical positions we have open in the Yahoo! Hadoop team. The post was a big success. We received a number of excellent resumes, and ended up hiring a few great engineers.
But today I want to talk about a particular challlenge we have - testing Apache Hadoop. Testing distributed systems is a hard job. In addition to testing functionality, one has to deal with scalability, reliability, security, non-determinism, and a number of other tough challenges. The larger the distributed system, the harder it is to test. Yahoo! currently runs Hadoop on clusters of up to 4000 servers, and this is not the limit. These are some of the largest distributed systems in the world.
Also, the way Yahoo! uses Hadoop is changing. Previously, most Hadoop users at Yahoo! were researchers. Reseraches are usually hungry for scalability and features, but they are fairly tolerant of failures. Few scientists even know what "SLA" means, and they are not in the habit of counting the number of nines in your uptime. Today, more and more of Yahoo! production applications have moved to Hadoop. These mission-critical applications control every aspect of Yahoo!'s operation, from personalizing user experience to optimizing ad placement. They run 24/7, processing many terabytes of data per day. They must not fail.
So we are looking for software engineers who want to help us make sure Hadoop works for Yahoo! and the numerous Hadoop users outside Yahoo!. We are not looking for regular testers. We are looking for outstanding software engineers and architects who are interested in quality and testing. We are looking for people who understand what testing distributed systems is all about, and who know Java and SQL cold. We are looking for people who understand performance, scalability and reliability testing. People who understand how to automate complex use cases and failure scenarios. People who can lead teams and mentor junior staff.
So do you have what it takes to test Hadoop? Please send your resume to firstname.lastname@example.org
Director of Engineering