A recent post by William Pietri on the XP list caught my eye:
Alas, I don’t have the data Brad is looking for on requirements gathering, but I did just happen to come up with some data on that eternal novice XP question: Doesn’t writing all of that test code slow you down?
I just ran the code from a previous project through LOCC:
http://csdl.ics.hawaii.edu/Tools/LOCC/LOCC.html
This is from a pure XP project, a Java-based web application. Total lines of code:
Prod: 34961 LOC
Test: 35176 LOCI haven’t checked exactly, but this was about four developers for 8 or 9 months. Looking at numbers from Steve McConnell’s Rapid Development, and assuming that web applications are more or less like shrink-wrap apps, he has three sets of schedule numbers for 35 kLOC:
shortest possible: 7 months 44 developer-months
efficient: 10 months 39 developer-months
nominal: 12 months 71 developer-monthsSo counting only production code, we did well, slightly beating the efficient schedule. But what if we count the test code? At 70 kLOC, the numbers look like this:
shortest possible: 9 months 120 developer-months
efficient: 13 months 100 developer-months
nominal: 16 months 175 developer-monthsSo assuming McConnell’s numbers are in the ballpark, there are two possibilities. Either we were two or three times as good as the best teams in the source data, or test code is essentially free. Personally, I think it’s the latter.
As much as I like the idea of TDD, this is a pretty bogus comparison. There’s a big difference between 35k lines of unit tests, which by definition are only coupled to a small part of the functional program, and 35k lines which actually extend the program’s functionality over the existing 35k lines of functional code. McConnell is clearly talking about the second case, which is bound to involve many, many more complex interactions than the “35k lines of unit tests” case.
My personal experience has convinced me that writing unit tests actually makes the project go faster. This is because unit tests enable refactoring, which promotes clean, appropriate design and good design lets the project develop quickly. I’ve written more here:
http://www.pycs.net/users/0000323/weblog/2005/08/21.html#P73
(To Anonymous):
I’m not sure if I would call it bogus. After all, he met the timeline based on his code without tests based on the Rapid Development book. But in reality he had written twice the amount of code.
Also, because he has those tests, he can now make changes, refactor, and add features with a lot more confidence then if he didn’t have them. Or, at least that has been my experience. I’d love to hear if you have had a different experience.
I agree that the anonymous commenter has a point. The larger a code base is, the more work something takes, and 70 kloc of code+tests may not be directly comparable to 70 kloc of pure production code.
On the other hand, I think my point still stands: people generally think that TDD and pair programming will slow you down, but even counting only the production code, we were at or better than the top end of the metrics.
And I should mention that this went along with fantastic bug statistics. I think this system had a total of two bugs in production since launch. So code debt is probably, as you’d expect, much lower for a well-tested system.