In many projects, it's painful to ensure that all unit tests are run on every patch before committing. Add to this some other basic checks:
- no new javac compiler warnings
- no new Findbugs warnings
- zero javadoc warnings
- zero @author attributions
and committing patches becomes very painful and time consuming indeed.
Something that painful, of course, is begging to be automated -- and that's exactly what we did with our patching process for the Hadoop project.
Every patch for Hadoop must be attached to a Jira issue. When a Jira issue is moved into the Patch Available state by a developer, the Hadoop continuous integration server automatically picks up the issue's latest patch, applies it to a fresh checkout of trunk, builds the software, runs all the unit tests, and verifies all the other items listed above. Once the build and testing is complete, a comment like this is automatically added to the Jira issue:
-1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12368715/1185_20071030b.patch against trunk revision r590273.@author +1. The patch does not contain any @author tags. javadoc +1. The javadoc tool did not generate any warning messages. javac +1. The applied patch does not generate any new compiler warnings. findbugs +1. The patch does not introduce any new Findbugs warnings. core tests -1. The patch failed core unit tests. contrib tests -1. The patch failed contrib unit tests.Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1033/testReport/ Findbugs warnings: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1033/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1033/artifact/trunk/build/test/checkstyle-errors.html Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1033/consoleThis message is automatically generated.
This automated system diminishes a Hadoop committer's burden to simply ensuring these two comments exist on the Jira issue:
- a "+1" comment from the automated patch build, and
- a "+1" comment from a code reviewer.
No more committing pain!
Grid Computing QA Lead
Want a fun job testing open source? Passionate about software quality? I'm hiring! Talk to me at: ndaley at yahoo-inc dot com. Testing and coding experience required.