I both subscribed martinfowler’s and your blog. and most of us did so.
I enjoyed this post (too) 
Thanks
I personally find metaphors very useful. Not only do they explain a concept quickly (by reusing past experience) but they can also help you pass the things you learn in life across all disciplines.
Just as the lessons you learn in everyday life can be applied to developing software, the lessons you learn in developing software can be applied to everyday life.
This time I will keep my filling knife clean…
http://www.mattburns.co.uk/blog/2009/03/02/keep-your-filling-knife-clean/
Jeff -
What exactly do you mean by refactoring? That is a buzzword I see on a lot of PP charts. If you mean rewriting code to embrace new architectures, etc. (multi-core, openmp, etc), please say so. If you mean re-designing projects so they will score less on the stinkometer, please say that too.
(I’m chuckling quietly to myself, but I’m wondering:)how do I resolve the focus on refactoring with your previous suggestion about avoiding re-inventing the wheel?
I’ve not read more than just the first book, but my favourite Dune litany rip-off has always been the one based on the Mentats:
It is by caffeine alone I set my mind in motion.
It is by the Beans of Java that thoughts acquire speed,
the hands acquire shaking, the shaking becomes a warning.
It is by caffeine alone I set my mind in motion.
Technical debt accumulates on every project, because we have not yet managed to invent the mind reading, prescient programmer.
This is no longer strictly true, because after my spice-binge last weekend, I am the…
Like the real-estate meltdown I forsee a software meltdown.
There will come a time when this stuff becomes so complex that no single or probably dozen human minds will be able to comprehend it.
That will probably be the time when the machines decide to refactor us…
What does this have to do with technical debt? There is a hidden and very real cost in complexity. I’m not at all sure that I am happier today than when I was a boy.
Sure, we did not have video games, computers, and all the gadgets and gizmos of today. However; I do believe that I was happier then, the world was less complex and less daunting.
I see kids today going from zombies training for a future war in front of an XBox to totally overloaded in certain school situations to sometimes outright disaster with things like facebook.
I’ve worked in this industry for the past 30 years and most of the time it’s been relatively kind to me.
So, what can I do without?
Cell phones: don’t need it, don’t want it.
Home Computer: This probably only applies to me; I work with one all day long, why do I want to see it in my house?
Internet: This is blasphemy I know but; I got by just fine before it became all the rage, I make money on it now… Has it really changed my life for the better? Yes because I work on it but, if I was not in this industry would I really need it? Probably not.
What do I not want to do without:
Medical Machinery breakthroughs such as MRI and CT scans, dialysis machines, pace makers, heart monitors, etc. Life saving applications are among the greatest applications for technology.
Electronic Ignition: The best thing that ever happend to the internal combustion engine.
Automated alarm systems: Unfortunately, we still have crooks in the world.
And the ONE THING I HATE THE MOST:
Amazing…I just finished reading Dune for the first time last week. Nice tie-in
Ok, where is the print view of this blog post? My boss urgently needs to see this before he comes in and asks for another new feature when I’m busy fixing spaghetti.
Ahem!
I will use stack overflow. I will read joelonsoftware. I will read codinghorror. If I find something nice, I will take it and use it. But If I don’t agree, I’ll make sure that the whole wide world knows about my strong opinion about what I think about Jeff’s shortcomings.
I, ladies and gentlemen, am Master Troll! I rule the blogosphere, I am king, Lord and Master of the Dark Side. I will never allow anything to go unnoticed if I don’t like it. I have the God given right to insult, attack and defame anyone in the blogosphere. I am The Law. I will bite and chew off every little nit in this article. I will spread doubt, fear and poison in the minds of all the readers. I am the One they should listen to, or face my words of wrath. There shall be no fame or respect without my permission,
I am the Lord of Trolls!
Evgeny,
FriendFeed’s problems are just weird.
http://bret.appspot.com/entry/how-friendfeed-uses-mysql says, In particular, making schema changes or adding indexes to a database with more than 10 - 20 million rows completely locks the database for hours at a time.
My jaw just dropped. 10-20 million rows is SMALL. Any database smaller than 50 gigs is small these days.
They want advice on how to deal with performance issues, then they should probably contact a consulting company who understands high-performance MySQL. For instance, Proven Scaling LLC. http://provenscaling.com/
Alex,
@Great post. I work for a Fortune 100 company. We face technical debt in two ways. The first, we have a large legacy base (all that COBOL stuff the business really runs on). Over 30 years additions, changes, and hardcoding have been added to solve problems but rarely has anyone refactored. A lot of this is because its hard to refactor COBOL into anything but long complex COBOL programs with confusing global variables.
Sorry but this isn’t true. COBOL refactoring tools do exist. Netron Fusion is an example. It won’t solve all your problems and may not even realistically be worth it for you, but there is a book that talks about properly factored COBOL code (if such a thing exists!): Framing Software Reuse by Paul G. Bassett.
There’s a fine line between not incurring technical debt (i.e. coding until the job is done properly), and gold plating.
Love the Dune quote!
Post the DB schema, and write up the changes.
Breaking tests doesn’t have to mean your code has a bug.
It could just be for example that function A now takes 3 parameters instead of 2. All those units tests will then need to be modified to take that into account.
Let me start by saying I know nothing about your project. However, let’s take the technical debt analogy and run with it.
How much are the interest repayments on that technical debt? 25%, 15%%, 10%, 5%, 2%, 1%?
However, for most projects I think that 10% is typical for most projects. Let’s say you do two weeks (10 working days) of refactoring to get the technical debt interest rate down to 1%.
Before the refactoring you were working at 90% effectiveness and after the refactoring you’re operating 99% effectiveness. A straightforward calculation shows that it would take 124 working days (or 29 weeks) to be ahead where you would have been if you hadn’t have refactored at all.
This is why your business partner wasn’t convinced. If you’d spent that two weeks adding or improving pre-existing features, it’s obviously going to add a lot more value to the project than the time spent refactoring.
I find that the technical debt analogy actually proves the opposite of what the proponents of term what to demonstrate: that refactoring is typically a really bad use of project time!
Unit testing, in this context, is irrelevant; you WILL make imperfect architectural decisions early on.
Very true Jeff. Having unit tests does nothing to eliminate technical debt. In fact unit tests themselves, being more code, can have their own technical debt too.
Now you could argue that having a lot of unit tests makes it easier to make sweeping architectural changes.
Yes!!! And this is the key argument for writing tests. It’s not just a side benefit of having them.
For any kind of refactoring - including eliminating technical debt - having unit tests, and as importantly, having functional or full stack tests can be a huge help. More than enough to make it worth having taken the time to write the tests in the first place.
I call this the software quality mortgage:
http://blog.asmartbear.com/blog/software-quality-mortgage.html
See that link for why IT IS GOOD to have accumulated technical debt, and techniques for convincing non-technical management to allocate time to clean it up later.