Most of the time, you just need to get the guts to make the hard decision than to fear the troubles and tolerate. Everything else will just follow. Though some will be tough, you are going to make it, because it is an equally hard decision to give up in the middle once you start, especially when the action you are taking now is the result of a prior taking-guts-to-make decision.
It seems to me that most of the people criticizing the metaphor or Jeff are only looking at the idea from a very narrow perspective.
To pick on just one, the idea that most refactoring would be a waste of time based on 10% being the average amount of technical debt on a project seems ridiculous to me. Even if 10% were the average, it is representative of such a small slice of the projects going on out there that it proves nothing about whether or not refactoring in general is worth while.
In the real world on projects of all sizes, technical debt can be accrued for so many reasons outside of developer control, from customer pressure for design changes to hardware and environment changes that the technical debt can fluctuate dramatically within a single day. Furthermore, it’s impossible to predict all of these changes and account for them in the architecture from day one or you would be wasting an incredible amount of time not having a working site/application that is bringing in no money.
The biggest take away from the article should not be whether you think you are smart enough or more likely lucky enough that you haven’t run into the issue on your projects, but to be aware that it does happen on a large number of real world projects. I’m not saying that getting into too much technical debt is an everyday occurrence or that it always happens fast or that it is never the developers’ fault. Whatever the reason, speed of accrual or where the blame lies, it does happen, you should be aware that it happens, and you should be critical enough to recognize when paying down the debt makes sense.
Great article btw =)
I have a severe back problem that, for almost a year, has left me unemployed except for a little paid work programming. Programming, which I had previously dabbled with as a hobby, has become my new career out of necessity, but mainly love for the craft.
Since I can’t go to a University, I am learning all I can from the internet. I do not have the advantage of the give and take discussions with Professors and fellow students.
I wrote all of tht to say this: You and Steve, among others, provide me with invaluable insights into the world of software development. And the comments are also very helpful.
Your battles with the Coding Horrors of programming encourage me since I face the same battles, although on a much smaller scale.
Criticism can be helpful but sometimes it goes too far. To your unreasonably harsh critics, I recommend they try taking on all the burden of responsibilities and work that you do and write quality articles about programming 3 or 4 times a week. Let us see how well they do it! When they do all of that and get as many hits on their blog as you do, then they may see things a little differently.
Keep up the good work, Jeff.
Re - changing number of arguments to a function…
(1) The eclipse refactoring for Java change method signature will rewrite all the callers to the function for you, supplying a default value, so that’s not a lot a work.
(2) if you have more than (approximately) 10 tests/test-setups calling a particular method, you have too much duplication in your tests and/or the function being tested is too large.
Use mock objects, stubs, fakes, etc., to decouple classes so that a single test only tests one interesting bit of behavior of one method of one class, not testing the whole pyramid of behaviors invoked by a high-level method. You still want some integration and acceptance tests, but not everything has to be tested that way.
With test-driven development, refactoring, and paying attention to code smells [like Large Class, Large Method, Duplicate Logic] and test smells [like Expensive Setup, Duplicate Logic, Large Test], you don’t have the problem of one change making lots of tests fail.
Really.
There is also the technical tax that accumulates when your projects rely on major components and underlying platforms you have no control over.
Vendors usually try to maintain backward compatibility, at least over a short sequence of versions, but changes can creep up. At some point if you skip major releases you can find yourself in unsupported territory and face a major rework that doesn’t add any value in itself.
Yet paying this tax as you go can be a drag on resources too. Of course the alternative means letting the taxes accumulate and compound.
This is less of an issue for a product with a 5 year lifespan than one that must live 10 years or more though.
I tend to agree with this metaphore, but it raises a big issue:
The developper have a company check book out of control…
Make this metaphore widely adopted would create a strong movement
coming from finance to control the use of this hidden checkbook. Lot of headache in perspective …
In the current time, knowing that some group of people can create debt out of control taking advantage that what they do is too technical to be monitored will trigger very painfull memories that are very fresh.
In addition, finding some else to blame, can be very appealing for those sitting on the grill today 
any idea how to explain to the company that what they expect costs much more than they know ? And how to explain we need them to still fund it because we need money to live
or even worst, telling the truth:
We have no idea how much it will cost utlimatly, but for the fragmented data we have the purposal you have in hand is the most appealing that we can write.
Of course we can intend to collect enough data to have a more precise idea of the real cost without technical debt, but that study alone will likely cost more than the whole project that you have in mind…
so let’s be reasonable, let’s admit that organizations are too complex and irrational to know what we do. Going agile is not so bad in that perspective, we throw away money in small chunk and add more if we get a return that makes us happy…
Paul, have you even seen the statistics for the site? 500k page views per day. Millions of unique visitors per month. Yeah, sure, everybody is laughing at the lack of unit tests.
Sorry, have I wondered into YouTube by mistake? I’m looking at some of the comments here, and I can barely tell the difference. I guess you’ve reached some critical readership mass that tipped you over some kind of wanker-attraction threshold… 'tis the price of useful feedback, I guess, but it’s most unwelcome.
See this story from the early mac days:
http://www.folklore.org/StoryView.py?project=Macintoshstory=Mea_Culpa.txt
They used the upper 8 bits of 32, because in the early days only 24 were used. That was a big burden, when the 68020 got reality …
No more metaphors! PLEASE
A couple of interesting posts linked to here, but I feel it would have been more valuable if you could have added some of your own experience to them.
For example, with reference to Steve’s article, what sort of debt did StackOverflow aquire? Type I.A.1 or I.A.2? This would be interesting to know as some of your recent posts might suggest you were going for the bad sort of debt - lack of comments, generic variable names, not following coding conventions. The linked article suggests this is never a good strategy, even in the short term.
By the way, I wish you had credited Martin Fowler for the first quote.
Hey,
I love you Jeff, and all, but this continued aversion to testing makes you sound like an idiot.
Testing massively reduces the fear associated with refactoring. Yes, of course you break tests while making changes, and yes you have to fix them. But that is part of the process of finding out and compensating for the effects your changes have had across the system. The alternative - to cover your eyes and hope for the best - is hopeless. You have a responsibility not to lead people so far astray like this.
Big hugs though, I really do love you and your stuff,
Jonathan
Just now I read an article about how FriendFeed use a weird schema to make their MySQL more responsive. And it looks like you have database scaling issues as well. Perhaps it will give you ideas on how to handle it :
http://bret.appspot.com/entry/how-friendfeed-uses-mysql
Even though personally I think they should not have chosen to use MySQL in the first place, something more robust and scalable like PostgreSQL would probably not lead them to the problems they are facing. But afaik you have the M$ DB, so it might not be relevant for you at all.
To all the folks who say stuff like in the real world, Jeff is 100% spot on and anyone who says differently is dreaming… This is either an ad hominem attack or an inept appeal to authority. Jeff is making some pretty vague claims here about both the scope and necessity of an impending refactoring, and deserves to be questioned. Personally I think it sounds like he is promoting having a permanent version 1.0 by taking time off to refactor major parts of his software. I have never seen that work well.
I’d also argue that accumulated technical debt becomes a major disincentive to work on a project
This is more important than it is usually given credit for. Good programmers dislike dirty code. They want to get rid of it, they don’t want to be around it and they certainly don’t want to be associated with it.
The point is that a technical debt is not just an undocumented hole in the budget, it’s an undocumented HR problem. It lowers moral and raises staff turnover.
If your project has a high level of technical debt it makes everyone who works on it look inexplicably inefficient and weak. People procrastinate rather than touch such painful code, they try to avoid the project, their more tempted to make quick and dirty fixes just to get away from it faster and in the worst case they’ll leave the company to leave the mess behind. Likewise attracting and retaining quality new staff is going to be hampered by having a high technical debt.
The point is to look at technical debt as both a financial burden and an HR burden. Programmers working in a clean, powerful, elegant development environment are more motivated, more interested in producing more code of a similar standard and less interested in leaving the company.
Interesting, seems like you would benefit from SOLID in your designs … 
Many don’t understand that sometimes, code needs to be rewritten partially or almost completely to be able to go further. It’s a shame, but that’s life, but don’t blame me, blame my boss for his incompetence!
The idea of technical debt is valid and the other assertions of, Michael on February 28, making business people drool is also important. Enabling software development to be taken seriously by business is important because the majority see it as a cost. Whilst I hope to not indulge in jargon bingo, development needs to create its own that business people can understand and cost from both a productive and efficiency led perspective.
Living in the dark and hoping to achieve validation by not examining your own profession is tantamount to crazy. Technical debt is one in a series of phrases and measurements software development needs hold itself accountable to because business cannot understand its value.
As far as the sniping that has occurred I believe that Jeff has raised an area of debate that can be used to initiate a debate to bring together IT and Business to a common goal; good maintainable software that is inexpensive to develop and easier to maintain. This is balanced with an understanding of the process of development and the degeneration of the product itself.
I’m with Jeff and Joel on unit testing. I would rather have functional tests to make sure all the end user functionality works.
Have façade interfaces that expose all the end user functionality to the UI (ie. the UI is very thin wrapper around the façade). Then write test against those interfaces.
Great post. I work for a Fortune 100 company. We face technical debt in two ways. The first, we have a large legacy base (all that COBOL stuff the business really runs on). Over 30 years additions, changes, and hardcoding have been added to solve problems but rarely has anyone refactored. A lot of this is because its hard to refactor COBOL into anything but long complex COBOL programs with confusing global variables. Today we have less than 5 or 6 people who would even be qualified to touch the code, everyone else is scared. We’ve ignored it so long the people best qualified to understand it have retired (or died) and the people left are too busy keeping it working. We are ultimately facing bankruptcy on the code and start over with a new system.
The second is on new projects that are run by non-technical MS Project leaders. They want to stick to the project plan and meet their timeline. This is admirable and I agree they are necessary to make sure we actually deliver on time. But combine this with a large outsourced Indian development work for the focus is on make it work now and ignore the future. Once a feature is written which does basically what it should do and with reasonable performance its marked done and not touched again.
I deal with both legacy and new code. My mantra is don’t be afraid to rewrite code. Its scary to do so, but the benefits are huge. Even if you don’t do it any better, you at least didn’t blindly leave it as a mess and will have a better understanding. I always try to have at least one developer on the team that does code maintenance, they understand the pain later by not being willing to rewrite code.