In Microsoft Project and the Gantt Waterfall, many commenters wondered why software projects can't be treated like any other construction or engineering project:
Thanks for posting this.
I liken software development to driving to a strange town, in a vehicle your only marginally familiar with, without a map, and not knowing the driving conditions.
Yes, you can get there. You can define steps. You can state your goals. But you’re not likely to know every step you’ll take. When you get there, you have success or failure (depending on your definition of “success”).
Not a perfect analogy, but it fits every software project I’ve ever worked on.
It’s interesting that you talk about software being invention rather than construction. I’ve been told that the german word for software developer (which is software entwickler) actually translates to software inventor.
As someone who works in software in the construction industry, I think you’re making a lot of assumptions and generalizations. Construction has a lot of the same lifecycle issues that software has - including bad or incomplete design, environmental issues, and insane/stupid clients.
Construction also has a lot of issues that software development does not have. One being weather. Construction sites get rained out, or a storm does some major damage, or it’s too hot to safely work. And with construction starting to get more tech-savvy, they’ve got a lot of the same issues we have in software development. Bad internet connection, the server is down, the server is dead, poor source/document control
Finally, construction jobs are often completed late and over budget. This is due to some of the things I’ve already mentioned - but it also includes: materials shortages, equipment shortages, overbearing owner/client/government requirements for minority/female/apprentice/local workers.
But the thing that construction has over software is the abundance of skilled laborers. In construction you can have a night shift come in to help get the job done on time. In construction you can get specialized workers fairly easily. These things are either impossible or very hard to do in software. In construction you can add another electrician to the job at 8am and have him be fully integrated into the project in a number of hours. In software it can take days for a developer to get completely up to speed with the project.
But overall, construction carries much higher risks than software. That’s why most firms have Risk Management and Legal departments.
From my friends who have had contractors do work for them, almost every project has been overrun either in time or costs.
So using kitchen remodeling as an example only bolsters your original point.
Wow, to use remodelers as an example of an industry who makes accurate estimates is mind-boggling. Ask anyone who’s been through the process…it takes longer–A LOT longer–and costs more than the contractor ever estimated. Sounds a lot like software engineering to me!
Then throw in all the other software engineering-unique issues and there you go. We might be able to figure out accurately and repeatably how long it takes a bag of concrete to dry once mixed, but how long to develop a user interface where how much concrete is needed to even start is a question neither the business nor the technology sides can anticipate 100% up front.
Oh, and hey, can anyone spell Big Dig? Why does software engineering alone get such a bad wrap in this area?!
Have to disagree with much of this.
I have remodeled several kitchens and they are never the same.
Part of what developers do is always new, but perhaps not as much as we think. Of course many times the shifting sands of dev tools affects us too, and so you are not god. You have an OS, a framework, a fancy UI, etc., that you must obey.
A kitchen and a software project are not the same of course. But just as you will be penalized for not thinking thru a remodel, you will pay the price for half-baked specs and users who are allowed to change their minds at will with no penalty – and you especially will pay the price if you assume that software projects cannot be managed well, waterfall or otherwise.
I think part of the problem is that generally people aren’t willing to pay for software engineering like they pay for other kinds of engineering. It’s a lot faster and cheaper to do a shoddy job, and since nobody’s life is on the line, people are willing to pay substantially less for software that only sort-of works.
As a result, if you’re building software on top of other, existing software, and that existing software only sort-of works, it’s difficult to predict how the resulting beast will react when it’s subjected to your particular set of circumstances.
Software development will become predictable when the “standard parts” we use become reliable.
Brooks: you’re still trying to compare construction - where generally unskilled or low-skilled labourers take an actual design and build it. They’re not changing the properties of something as they build it. They know how many (identical) bricks they can lay in a certain amount of time, but the weather determines how long it takes for your concrete to harden, so the project isn’t going to be entirely predictable. They also have to allow for mistakes in the design; if this occurs, and part of the structure turns out not to be strong enough, they have to go back to design.
We have unskilled labourers to construct our software. They’re called compilers, linkers, make tools, continous integration tools. We’ve so automated our construction activity that it doesn’t look like construction anymore, so our detailed design activity is confused for construction. If we make a serious enough mistake in our design, the labourers go on strike (produce error messages). But when we make a mistake that the labourers don’t catch, it’s not as readily apparent as when part of the house falls down when you’re trying to build it.
Architects are also not generally trying to construct houses from individual grains of sand. They don’t normally design their own bricks. Designing basic building blocks is all too common among programmers. Sometimes it’s justified. Most of the time it’s not.
Mike’s onto something - that’s the scale, from the large to the small details. On my current project, I’ve designed the architecture. I’ve also implemented assembly code.
Meanwhile, as it happens, my new house is being constructed. They’re right on schedule, actually ahead I think. The architects of the house certainly did not have to develop particle board or paint in order to pull this off, which is essentially what I have to do (I’m working in an embedded environment with no OS or third-party libraries).
I think some of the readers are missing the point, it’s not that you it is not possible to manage a software project, you can. The point is that the degree of accuracy of determining the length of a COMPLEX project is lower then that of a SIMPLY project.
The analogy of building an airplane is a good one, an airplane is a complex thing, but each line worker has a relatively simple task, bolts, wiring, etc. Once you complete the first plane you are able to say how long it takes to complete the next one.
But if you are building a prop plane today, a jet tomorrow, a cargo plane after that. Can you really say you will know exactly how long each will take if you are building each one for the first time.
You read someones blog, you want to build that cool widget and someone says “IT WILL TAKE YOU 5 HOURS”. And when it takes you 8 hours, is it poor estimation or poor execution?
If God’s name is John Von Neumann, well maybe. Even so, you will obey his laws, because the computer you execute on does. It is more strict than Turing, which is Vapor.
Even engineering projects have the same problems - different root issues, but very similar. Check out “Critical Chain” by Eli Goldratt - David Anderson’s book “Agile Management” takes these concepts and ties them to software development. Both worthwhile reads.
When discussing building construction vs. software development, the key in both cases still comes down to the degree of planning that you are able to do prior to sitting down and writing the first line of code or digging the first hole. Both rely upon standardization, something that is increasingly true in the software arena as componentization continues to push its way up and down the abstraction stack.
Moreover, while there is still a certain level of art necessary to be a good software architect, its worth understanding that most software projects are going to be ranch houses or fairly standard office buildings, not Fallingwater or the Guggenheim museum.
I love these posts and find them very interesting - but when are we going to get back to some coding horrors? Seriously - can we have some code issues too please?
Software Engineering is like all other engineering, but all engineering is an art when it is young.
Imagine the first person to build a stone bridge with a support across a river. That person was probably an artist and visionary of immense talent. Many artists with extreme talent end up in engineering, but even so, almost all the people you encounter in “Engineering” today are simply copiers.
They take components and pieces designed by artists and place them in patterns that are copied from something an artist once made.
In software engineering, we are still creating the components and patterns. This means that without an artist on your team, you are very likely to fail. This will not always be true, but until artists create enough tools for programmers to use, the jobs will be difficult to schedule and results will be bug-ridden and sloppy.
Waterfall works pretty well if you have an artist on your team (which was more common when Waterfall was created, but less so as we moved through the 90’s and the tiny pool of artists diluted.)
Agile development is an acknowledgment that very few teams have artists on them and a set of methodologies to compensate.
Consider the difference between hiring Michaelangelo and having him pick talented young laborers to paint a Chapel roof vs going out yourself and hiring a bunch of house painters and saying “None of us really understand this process, so let’s allow an extra month in the schedule”.
The house painters don’t have a chance, and it’s this fact that causes the 80-95% of software projects that fail.
I guarantee there are at least a few at SUN, that beautiful, clean Java API didn’t write it self. No matter how many times you iterate, without the vision you will not get to beauty, you will just (barely) get to functional.
Also, My opinion is that like any other art, much of your aptitude is set before you are a teen. If you aren’t eating code for breakfast, lunch and dinner after your first 3 years of exposure to a PC, you’ll never be an artist.
Just like if you are not playing decent piano within the first few years of full-time exposure to one, you’ll never be a master. You can be good, you can play with a group, but you are not going to fill a concert hall alone.
And… folloowing last comment… once you have done a few easy buildings (easy software projects) you know more or less how much time is needed. Most of the project failures are due to lack of initial analysis and task breakdown.
A lot of people out there go straight to software arquitect after coding in a few big projects. Hence, they don’t have the required experience.
that was one of the reasons for the dot com boom, too many twats promoted without experience or knowledge. If you are doing rocket science, yes, it’s extremely difficult to be accurate on your estimations, if you are doing regular office building, you should know to certain extend how complicate is going to get…
It all comes down to the 90/10 rule. The last 10% takes 90% of the time.
Software fills the last 10% of the tricky logic problems. Solving the tough problems tends to unwind average designs. Changes force feed to a design that is failing, and the last 1% almost impossible.
Many systems (e.g. many construction problems) tolerate non-perfect results and can ignore the last n%. Tolerances in logic are slim.
Accurate estimation of a task involves
If you give the same problem to two different teams in the same company to solve, they’re most likely to come up with different solutions, depending on how they have understood the problem, and so will have different estimates of how long it will take.
With software, there are many ways to solve problems, and each solution has its own set of constraints.