Large scale Ruby development with TDD

July 23, 2009 by kowsik

We use Ruby, a lot. Everything from one-off scripts to modeling protocols to building mutations to packaging these things on to the product and then some. We push the language features quite a bit with native C extensions, eigen classes, DSL’s, method yanking, method redefinition, open classes, etc. We have parsers, code generators (that output C, Java, C++ or Ruby in some cases), document generators (that output wiki pages) all written in Ruby. The online help in the product is validated during build time and then aggregated into a set of HTML pages. Our 3rd generation protocol engine which powers all of our protocols and Studio has 2,323 Ruby files in trunk with a total of 215,621 lines of code!

The challenge with such a large scale project using a language that does run-time duck-typing is that if you haven’t tested it, it prolly doesn’t work. The only reason we can innovate at such a rapid pace is because we are a big TDD (Test Driven Development) shop. This mean that we have very large number of unit tests which we continue to develop, maintain and test. This allows us to measure and understand the risk of a major code surgery before we push it out to customers. As someone once mentioned to me, “we don’t want to be testing test-gear“. And we respect and honor that sentiment.

For Ruby unit tests we use runit which is a simple and powerful unit testing framework. We even have unit tests that fail the build if it detects that some newly added class doesn’t have a corresponding unit test. :-) When we forget to write unit tests while in a hurry, the email from the failed build reminds us ever so gently. Unit testing is so important that we pay attention to it.

So speaking of unit tests, here are the results from a recent run:

109398 test methods, 383461 assertions

A serial run of all unit tests in integration mode (where we test more things than in unit testing mode) takes about an hour. Obviously this is a little too long for the write-test-debug cycle. So Aaron wrote a worker-bee farm built out of 4 monster boxes each with 2x Xeon 3.16GHz quad-core processors. Built using DRb and Rinda::Ring, we just pick the free DRb servers and parallelize the unit test execution. When all workers are available the unit testing times drops to about 7 minutes, which is much more reasonable for interactive use (just enough to stretch out before the next coding marathon).

All these stats are just for the core protocol engine. For UI and REST/WSDL API testing we also use Selenium and soap4r. We are proud of our test coverage and more importantly the automated testing we do. All in all, it just means cooler features and faster time to market and lots more innovation with a high regard for quality. With marquee customers using our product, it’s imperative that we keep a very close attention to the overall quality of what we deliver; and we take this seriously.

What’s the largest Ruby code-base that you are maintaining? We would love to hear about it.

Categories: Ruby, Tools
882 Comments
Bookmark and Share