The Big Ball of Mud and Other Architectural Disasters

Okay maybe that was mean. But perhaps you have some advice about strategies to avoid these patterns from emerging? Certainly it’s not as easy as “now that you know how to do it, don’t!”

These are all very natural ways for software projects to evolve (“natural” meaning “following the path of least resistance”). After all, every programmer worth his pocket protector should be able to tell that you don’t sell architecture, you sell a product. Clients (in general) don’t give a crap if you spent time perfecting your design, they just want something that works. And in our capitalist system where the money goes to the lowest bidder, there is a powerful incentive to nix the designing and the refactoring and just spin out code until you’ve got something you can hand off.

Now you and I know that time invested in design and test driven development and prototyping and rewriting and so on can save time and money in the long run (“a stitch in time saves nine”), but to an accountant an expenditure NOW isn’t the same as a possible expenditure, oh, somewhere in the future. So how do you strike the balance between “design it perfect” and “git 'er dun”? What methods do you use to keep a project from going down a dark road, or save one that has started that way?

Yeah I guess that’s a tall order for a mere blog post, but that’s what I’d like to read.

Hillbilly,

That’s some hilarious sh@@! I’m sure you know that Indians and others added straw to the clay to make stronger bricks. You are to be commended.

Few people know that it is harder to do what you are doing than to design something from scratch, which is usually abandoned anyway.

You are are true hero. You keep a pile of crap bringing in money and employing people. Bravo!

Um, guys, are you sure you know how Big Balls of Mud are formed? Let me tell you a little story…

DAY 1
Boss: "Okay, we’ve got this new contract. It’s a generic e-commerce website, except it has to have X, Y and Z. Oh, and T. And - lemme see - U, V, W… and a few other minor features. About 100 of them."
Team: "That will be long and expensive."
Boss: "I realize that. Give me an estimate, boys."
Team: "We’ll need to do some speccing and planning first."
Boss: "There’s no time. We’ve got to have a demoable product in two weeks and a production version in two months."
Team: "No way we can do it that fast. Wait, didn’t you just say you want us to estimate…?"
Boss: "Actually, I’ve already signed the contract."
Team: “…”

DAY 2
Team: (scrambles to put the finishing touches on the currently underway project)
Project manager: (scratch that, there’s no project manager)
Senior programmer (let’s call him Steve): "Come on, guys, put that away. We’d better get started with the new project."
Team: "Sure, let’s sketch a quick plan."
Steve: "No time for that. Bob, you mock up a user interface."
Bob: "But I don’t even know what will go in there!"
Steve: "Never mind, we need it quick. Marc, you get started on the database layer. You know, the usual e-commerce stuff: authentication, products, shopping cart…"
Marc: "What about those extra features the boss mentioned?"
Steve: “We’ll get to them later.”

I think you know (or else you can guess) the rest. There’s no mistery. Greed and lack of patience, nothing else. And it happens over and over, while a lot of smart people are looking for technical causes. Can we wake up now?

You are absolutely right about the temptation to go on using a prototype and enhancing it to be good enough so we can add it to the production code. So we usually write the code on a VPC which is nothing but spaghetti… later we refactor the “POC” to be more presentable and then get it reviewed… once reviewed we throw away the VPC and start all over…

the VPC works as a good way to ascertain discipline…

Very understandable post. I was at PLoP in '97 (was a student at UIUC), and I was immediately struck by the notion of the Big Blob that Foote and Yoder presented.

In your pictorial summary, it’s plain to see that analogizing the software development world to the urban planning world is helpful to the extent that we can better understand why we build software shantytowns, and engage in software sprawl. These ‘patterns’ have worked at small scale or when under severe time pressure.

The fact that these agglomerations continue to exist in the real world gives no reason to hope that their eradication from the world of software development is at hand–but hope springs eternal because we believe that the power to transform the virtual is far more within reach than that to transform the physical.

(Jeff: would you mind fixing your “Read older entries” link at the bottom of your main page so that it goes to an earlier page of posts, rather than to a single post? Thanks.)

My project sucks.
So now what?

Its depressing to have enough experience to know what you are doing is wrong and yet not know how to fix it.

We don’t exactly have defined career paths in the software industry. Its not like becoming a doctor. I would have gladly done a residency, taken a bar exam… whatever.

Reading this and other blogs as well as a multitude of Patterns and Practice book does me no good. As someone who has only been programming for 3 years, its depressing.

I wish i were ignorant of good design. Maybe then I wouldn’t hate this career.

Most of my projects end up looking like the Bronx circa 1982, or some Cuban dictatorship where everyone drives around in “Classic ASP” cars. Yuck.

But don’t worry folks…

VS.NET 2008 + Vista promise a “Gatacca” type of future. However I suspect it is more going to be like “Logans Run” where on the the inside it is based on fantastic and perfect ideologies (which not everyone understands or agrees on) - but when you escape that, its the same old horrible mess on the outside… based on the 1980’s x86 architecture.

It’s so unfortunate that most of the time we aren’t choosing to make the shantytowns or the throwaway code… we’re simply the building crew. With the exception of brand new projects, there’s no way to see what you’re walking into, and when you’re there, you don’t always have the authority to change large sections of code.

I think a lot of the reason why it’s easy to get jaded in this field is just because management understands only immediate needs and not the necessity of clearly written, efficient code and the time it takes to write it.

Matt - “I’d seriously like to find out from other software developers if anyone has been involved in a project that you were very proud of the code.” “So, has anyone worked on a project that was planned (or refactored) so well that maintenance is a joy (relatively speaking), new features can be added logically without screwing up the main plan, etc?”

Yes, but I was working for a small company that sold the software I was writing when I did it. I completely re-designed/developed a big ball of mud VB application in java swing, more or less by myself over the course of 8 months. My boss was not a technical person, and my co-worker/manager trusted me to do the right thing. If I wanted to re-factor something I didn’t feel I did a very good job on I was free to do so at my own discretion. If I wanted to spend a few days to make structural changes that I knew would save me time down the road, I could just do it. If I thought of ways to make the user interface better, I could just do it (of course I would get a thumbs up or down sometimes).

I then maintained it for another year and a half. Again I had the same level of complete control over the application. Some things were not perfect, but I am still proud of about 90% of it. Everyone who did occasionally work on the project always had positive things to say about the code base… so I know I am not just blowing sunshine up my rear. :stuck_out_tongue:

#8 - Accidental Architecture

“Every interesting software-intensive system has an architecture. While some of these architectures are intentional, most appear to be accidental” Grady Booch

http://www.computer.org/portal/cms_docs_software/software/homepage/2006/009-011.pdf

I would say the Accidental Architecture could encompass all of the categories above or maybe we are just stuck in the Liquid Goo phase…

My view is that we still don’t know how to even design software structures yet. Heck, we are still arguing how to even draw them. At least architects have symbols that most everyone agrees upon, just like electricians have circuit diagrams and symbols. What do we have? UML stickman? It’s embarrassing.

This is in addition to the negotiating technical debt as described by others. Tim Dudra, I understand your plight, but you are still reading a software blog – you haven’t given up completely have you :wink:

copy + paste = blog!

I added photos, too! But seriously, I wanted to highlight this particular article by providing a Reader’s Digest condensed version of it. It’s great, but it’s giant – not everyone has time to read all of it. Hopefully they can get a good summary of it here and dig in to the sections they’re interested in.

I am a bit confused about the “keeping it working” section.Jeff,are you saying that software versioning is bad practice ? Versioning is maintenance of existing software to improve the software structure,fix bugs,add new functionality etc to produce more efficient software right? If so why is that a bad practice.I may have misunderstood what you were putting across,if so enlighten me.

“Embrace the beast and learn to work around it. It may be a shanty town, but it’s still home to someone.”

A shantytown is a palace for someone who lives on the streets. Bad programmers love shanty towns, good programmers go there to die.

Cheap, in this context, means how much effort went into making the tools and how much they help you do your job. OSS is free, but not cheap in terms of the human capital invested in it.

They mean blunt saws and drills that have been repaired a million times that have to be borrowed from someone. I think that the article was written before OSS became so ubiquitous and muddied the metaphor.

That’s not Fulton County Stadium.

http://cache.eb.com/eb/image?id=66862rendTypeId=4

VS.NET 2008 + Vista promise a “Gatacca” type of future. However I suspect it is more going to be like “Logans Run” where on the the inside it is based on fantastic and perfect ideologies (which not everyone understands or agrees on) - but when you escape that, its the same old horrible mess on the outside… based on the 1980’s x86 architecture.”

More correct than you know, including the part where the ideal society is not ideal at all, nothing truly new is ever done, and you are killed when you get too old …?

I agree with Andrew a little bit. These patterns can be interpreted as natural solutions to many programming problems, that arise from the need of solving complex problems.

TDD creates some shanty towns, since it builds things bottom-up

XP prototypes are some sort of Throwaway Code

Shearing Layers sounds like “structural pieces of the architecture last longer”, but I also don’t agree with that. Sometimes you’re forced to change major pieces of the architecture due to several reasons

I think the post is still valid, but I appreciate Andrew’s comment because we must always read everything critically.

cheers

Been thinking about different ways to approach architecture and came up with an idea i’d like to throw out there.

How about taking a cue from ants where each individual or team follows a common strict ruleset on how to approach the problem/solution scenario.

Consider it like the piecemeal growth model but with a strict set of rules.

These rules will obviously need to be worked out first. Maybe even a community effort to provide standards etc.

What does everyone think about this? does this already exist?

Regarding #5, if you want some more interesting thoughts from Stewart Brand, he did a seminar entitled “Cities and Time”. The mp3 of the seminar seems to be unavailable, but it’s on google video at :

http://video.google.com/videoplay?docid=3409261905127173946

The slides he uses are online at :

http://media.longnow.org/salt-slides/Brand.html

“So you agree that the use of cheap tools is not a mistake?”

Cheap != Inexpensive. Just because a tool is inexpensive does not mean it is cheap. If GCC was useless then why would every major operating system besides windows use it as it’s default compiler? When Jeff is talking about “Cheap” tools, he is talking about tools that are more or less worthless. QBasic anybody?

Also, just because he cuts and pastes, does not mean he agrees. Oh yes, most of the time he does agree, but many times he pastes something just because it’s interesting to think about and make your own decisions about.