Paying Down Your Technical Debt

@ John Gallagher

… because someone should call Atwood out for what he is – a shill, selling himself, effectively regurgitating more intelligent individual’s work (eg, in this case, Ward Cunningham Martin Fowler).

He’s polluting the technical space with poorly thought out – if not just plain wrong – ideas, and nearly anything good he publishes, he got somewhere else.

Or to misappropriate an unrelated (!) quote from Charlie Stross, originally in regards to Kerzweil – I believe this applies equally well here.

I rate him as being a publicity hound rather than a pioneer, and I tend to assume that people who think he’s got something useful to say aren’t familiar with the original sources he strip-mined.

Wow. What a truly awesome take on things. This is the best way I’ve heard it explained. This post not only notes the benefits of well defined code for a large multi-coder project, but justifies the use of quick, dirty code for small projects or one-time-run programs. You reap what you sow on a large project, but on a small project you usually don’t make it to the reaping part more than once.

Great article.

This is an interesting topic! I think everyone working for some time as a developer has had to (or ought to) pay off technical debt many times.

@Jeff,
Since Stack Overflow is a relatively open site and many of your readers are somewhat familiar with it, this seems like a great opportunity for a series of articles about this in more detail

How did Stack Overflow handle this particular down payment.

What symptoms did you notice to make you want to do it, what did you conclude it was caused by, and how did you decide to address it?
What was the design like before and after, how did you practically carry it out to avoid downtime, and what was the result?

What you guys have done with Stack Overflow is inspiring and impressive, I’m sure you have material for a whole book by now. Dreaming in code 2 - Stack overflow I’ll pre-order! :wink:

Excellent post!

On my last major project (18 months, 8 developers) I tried to convince various influential people about the importance, relevance and inevitability of technical debt but kept being told: refactor as you go.

In practice, refactoring as you go is NOT sufficient to change major early decisions. And, even if it was, by constantly refactoring, you’re never able to benefit from accruing SOME technical debt. It is a perfect metaphor as far as I can tell.

In the end the dev team forced through major changes to the database and core entity model. This broke just about everything so we had to spend time making the system work again instead of implementing new features. Subsequent releases however almost certainly arrived quicker and more feature rich as a result of vastly improved productivity brought about by these major changes. We discussed the situation at great length and nobody could see how such changes could have been achieved in an incremental, refactor as you go, way. The impact on developer morale should not be underestimated either - the entire team was invigorated by this event.

With a large database, even small structural changes can be difficult to implement (think about complex data migration scripts). But technical debt accrues in the database design just as much as in the code so can’t be ignored.

We did not do much unit testing and while I agree they can help with the refactoring as you go approach, they would not have helped us during this major change. Well over 80% of any unit tests would have needed to be substantially rewritten. In fact, looking at the 18 month project as a whole, I believe having a comprehensive unit test suite would have substantially reduced overall productivity without any significant (and entirely unnecessary) increase in quality.

I wonder if any controlled tests have been conducted to compare two equivalently skilled teams making identical changes to a codebase where one uses unit tests and the other doesn’t. It would be interesting to see how the productivity/quality metrics compare between these teams as they are asked to make different kinds of changes and where different amounts of technical debt pre-exist.

Very good post, I’ve always known about technical debt I just never knew a word for it so thnx for the word.

Is this technical debt caused by a rush to get features out? I think SO is very good which is the irony that maybe you must create this debt to produce something good out of the code? I would hope not but it seems that way sometimes.

Keep all debts manageable.

This is why I’m proposing a new methodology called Revenue Driven Development. What you do is you figure out the simple possible functionality you can charge money for, you write a test to prove out that functionality, you write the code to perform that function, you release (either internally or externally), you move on to the next simplest possible piece of functionality you can charge money for.

There are supporting practices involved… but they all work just like XP. The difference: under Revenue Driven Development they all have monikers that make marketing and financing people drool.

This way you figure out how to make more than the minimum monthly payment on your technical debt before you max out your credibility card. Stopping all new development on your project to do something other than fix actual reported bugs is like taking a second mortgage on your house to pay your credit card bill off. You still have the debt, perhaps on better terms, but if you don’t learn to prevent the debt from building up in the first place… you’re going to be in even more trouble later.

As a counterpoint to those who have espoused the ideal of do it right the first time, I still assert that Technical Debt accrues. Languages improve; hardware improves; operating systems improve. Space-Time tradeoffs change. What may have been the provably best solution in 1992 is likely to be sub-optimal given current theory and technology.

For a simple case, suppose a sort algorithm were discovered that ran in O(n loglog n) time. What would it take for you to rewrite everything to take advantage of this theoretical improvement? Would it be worth it?

Got technical debt? Can’t pay your software-quality mortgage?

Call Washington NOW for YOUR bailout!

Billions of LOC available!

1-800-CONGRESS

ah yes, but Fowler and Cunningham want you to realise the technical debt when it’s staring you in the face, before you’ve made the decision to suck. e.g. quick but messy versus clean but longer implementation.

The idea is that you’re not doing anyone a favor by cheating with the quick and messy, and the analogy is using your credit card versus paying cash. Use the card, you pay interest in addition to the effort, and the net outcome is paying more. You think you’re being clever, but you’re not. Pay with cash and you don’t have to pay again. You’re done.

There’s also a peculiar notion expressed by some of you that seems to feel that refactoring is only to do with small code changes. Refactoring can be anything. It just means spending the time to make changes that are state-preserving. You change focus from adding features to cleaning code. Refactoring your architecture is still refactoring.

Great post Jeff. +1 :slight_smile:

Given the nature of some of the comments here, one could be forgiven for thinking that some of the commenters were forced at gunpoint to read and recite this post whilst their eyelids were stapled open, until it was burned into their psyches.

This experience being in stark contrast to their otherwise utopian existences, where stakeholders and investors do give a shit about unit testing and quality metrics, and never threatened to offshore a project because they just want it to market, like yesterday.

For a great many of us, getting to market fast, cheap and good enough is the driving factor, and what puts food on the table. We will make some bad decisions along the way, if the project is lucky enough to still be running in 12 months time, they may be fixed, and fixing them will quite possibly break some other shit, there was no budget for unit tests, because the investors couldn’t see them as a valid deliverable.

Ideal? Hell No. Would we prefer that to be different? Hell Yeah.

But for a lot of developers that is the grim commercial reality of life, if you want unit tests, you can write em on your weekends, but on weekdays, investors want to see dem blinkenlights right now, and they want their new, pretty, and probably imperfect blinkenlights on the interwhatsit before their competitors.

It becomes even more seat of the pants when you put your own assetts on the line for a startup, as I dare say Jeff did to some extent. In that case, willing to take technical debt in return for cashflow, pagerank and less time to market anyday.

If it floats, you may have a chance to fix it, if it sinks, move along. Unless you make an absolute dog of a product, market factors and bling on your UI will be much more a determinant of your initial success than software quality. If it does make money, and no one really notices the imperfections, they may never get fixed, because it’s probably not a weapon aiming system.

For those who don’t exist in this sometimes dystopian place for lesser mortals, that seemingly offends you so, why are you here?

You’re just increasing traffic and ranking for this blog… it’s an imperfect blog, but maybe read more than yours.

I feel as you do, Jeff, but I have no problem believing that Joel balked at this.

On the flip side of what you feel is this article by Joel: http://www.joelonsoftware.com/articles/fog0000000069.html – that’s the one titled Things You Should Never Do, Part I about how stupid Netscape was to rewrite their browser.

Nothing is cut and dried, but how scary is it to tear down code that you know works and replace it with all new code? Do you think you might get into trouble at some point in your refactoring?

Technical debt accumulates on every project, because we have not yet managed to invent the mind reading, prescient programmer.

Don’t you mean architect/designer? At least in the case of Stack Overflow I fear you are correct!

@Simon - In reality though if the performance gain from the refactoring allowed the website to run efficiently on the current hardware for longer without requiring upgrades as the website grew or if it prevented hitting a wall in later development where more major changes would need to be made it is enitely worth it.

I think it is best to make these changes early, when there identified rather than waiting for them to become a major performance issue.

Atwood == YAWN.

Talk about flogging. You post, and you post, and you post some more, and it quickly becomes apparent that 1) You talk too much (especially about yourself), and 2) You’re really not very good at this whole ‘computer science’ thing.

You -are- pretty good at the self-promotion, though.

I think it’s excellent that you’re recognizing the technical debt and taking time to pay it down. However, the level beyond where you are at is where you continuously pay down technical debt. Once you reach that level, having to separately schedule it is a sign you’ve let too much pile up.

Also, unit tests are a big help with managing technical debt. Fear is one of the biggest factors keeping the teams I work with from changing things they know they should change. If you are afraid of breaking something, that means you don’t trust your tests to catch you.

@kbbartum If Atwood bores you, and you don’t have any constructive criticism of his writing, why are you wasting your time commenting? Just don’t read his blog any more.

@Atwood Fantastic article. This is something that’s relevant to most developers I would expect, but is particularly relevant to me right now as I’m making the first prototype for my first proper software project. Yikes, it’s scary to be making a mess. But I’m making a mess knowing that this will not be the actual project. And when I do get going with the actual codebase, I’ll know that technical debt is something to treat as I would interest. It’s a great analogy.

One more thing - you included a link to a quote from Flash Gordon on YouTube, therefore you rule. I can only aspire to such greatness.

This post pretty much boils down to we have to refactor/rewrite some stuff on our software project - Big deal, so do I; and probably so does the poster above me, and the one above him.

Details Jeff, is what makes a technical article… technical. Otherwise this could have been written by a 2nd year business major.

Respect is due for having built such a useful site as Stack Overflow. (And for having stuck the boot into the much hated Experts Exchange.)

As other commenters have suggested on this post and others, Joel and your position on automatic tests seems a little extreme though. Sure, tests do have a cost. But as a code base matures, interfaces (in the loose sense) to larger units should tend to stabilise. (Unless there is no architecture at all - ie. big ball of mud.) So the ongoing cost of testing at least some of the code should go down, and change the cost-benefit equation. Even if you don’t buy into the 100% test-infected stuff, I think you are probably missing out by rejecting automatic testing so absolutely.

Way to rip off Martin Fowlers last blog post. At least you added some pretty pictures.