Although I love reading programming books, I find software project management books to be some of the most mind-numbingly boring reading I've ever attempted. I suppose this means I probably shouldn't be a project manager. The bad news for the Stack Overflow team is that I effectively am one.
It’s almost scary how often people seem to forget the fact that in order to know when you are done you need to know what being done means. There is no way you can say you are 83% done if you don’t know what 100% done would be. That’s like asking a contractor how long it takes to build a house without showing them any plans or drawings. Also, if you’re not the only one working on the project, everyone else needs to know what 100% done means as well. The fact that the definition of being done tends to change during the duration of a project (new features, anyone?) doesn’t help either.
I think this is one of the well-known problems of software development that Extreme Programming, Scrum and other agile methodologies and techniques are trying to solve. Even if they can’t help you predict the future, at least they can help you see where you are, and hopefully where you are headed–or should be headed–in the near future.
Listing what is actually to be done, in detail, of course helps one plan better.
But I continue to be amazed, year after year, how managers think that writing down a list with times to completion actually makes it so.
I am likewise amazed that most managers don’t understand the inherent lower-bound bias in initial time estimates.
Jeff you are right about software project management books. EXTREMELY BORING almost all of them. I’ll check this book out next time I’m at the bookstore.
Some people at my company are perpetual We are 99% done people.
It seems after time and time and time again of being wrong, people would say to themselves hmm, I seem to be overly confident a lot, I should think about that the next time I give an estimate.
Of course, they don’t. They continue to try to please management (but really just frustrate them) by making estimates that are unrealistic and overconfident.
Never estimate based on the best case scenario.
Attempting to put software to a schedule is analogous to solving Turing’s halting problem for a given input.
Sometimes you can tell very quickly how long something will take. Other times it is so nearly impossible to plan ahead that you might as well just start programming. Sometimes it’s somewhere in the middle. Sometimes the schedule shifts midway from trivial to NP-Hard, or vice versa.
Actually it’s quite normal that there are more requirements/stuff to do then the team is able to finish before the deadline. SCRUM approach would be to ask client to prioritize the stories (tasks) and deliver those which are the most valuable. Projects are usually evolving, users ask for new features … it’s never 100% done from this perspective.
How are you supposed to know what to write on the list?
The only way you can write a detailed list is to design the system. And designing the system should take about half the time in the project.
That’s not to say that a list of tasks isn’t useful, but getting to the point where you can write a list of concrete, completable development tasks may just be the biggest part of the project.
I see really two threads to the responses here:
I do think that there’s some truth to the second, although I feel like a troll saying so. The problem is that a lot of time, when you’re writing exploratory software, when you sit down to write up a list, you really, honestly don’t know what needs to be done. And you can’t flesh that list out to tasks that you can clearly say are done before you start doing some things. The define the I/O behavior of module X can’t be drilled down, because it involves list items like design the Y data model, and you don’t yet know that you need a Y data structure.
I’m working on a project that’s trying to use scrum now, and I’d have to say that while it seemed really agile in the beginning, it’s now seeming like same old bad project management with generation of tons of tasks with ill-defined estimates for completion, etc. Precisely because we’re doing tasks that need more work before they can be turned into lists of actionable items.
I’m starting to think that this is one of the software engineering dilemmas for which, as Fred Brooks said, there is no silver bullet.
Hmmm…I’m surprised you didn’t link to
http://www.joelonsoftware.com/articles/fog0000000245.html
http://www.joelonsoftware.com/items/2007/10/26.html
given that they say many of the same things.
Huh? The article you cite in paragraph 4 was written by Mary Poppendieck, not Johanna Rothman…
I don’t understand why this is so difficult for most developers to do.
We’re all so logical, right? Lists!
On a related subject: You can improve your estimates by using 3-point-estimates (Warning: takes about 20 minutes to read):
http://www.visionarytools.com/decision-making/3-point-estimating.htm
But one estimate per task is of course still better than no estimate per task or (even worse) no list of tasks 
Or maybe to Steve McConnell:
This problem seems to be a really tough software development nut to crack. I wonder what it is about software developers (or our managers) that makes us so prone to being 90% done?
The other problem is analysis paralysis. So busy trying to make decisions that you never make any. Sometimes it is best to just get on with stuff and then use your understanding of the problem to do the estimate properly.
Jeff - have you looked at any of the Anti-patterns stuff? I read the book years ago and it made sense then, main thrust was to identify what kind of problem you suffer from and how to fix it. Worth a look.
I wonder what it is about software developers (or our managers) that makes us so prone to being 90% done?
I chalk it up to the desire to please, both ourselves and our managers. The way I see it, this will persist so long as the boss is more excited to hear it’s 90% done than she is to hear I’ve detailed the task and I estimate I’m really only 25% done. The later means your deadline is going to slip, and that’s a reality that a lot of people will resist.
It’s surely more immediately satisfying to convince yourself you’re 90% done than face the reality of the situation. I think for most people, getting past that rush will require having a manager that’s honestly appreciative of being presented reality, and displeased when presented a fantasy.
Create a list of tasks? You must be joking. I’ve got all this real work to do. 
Jeremy stacks are obviously most programmers favourite. You get paperwork dump it in a pile dump some more on top of it. I always tell my boss it’s date ordered.
How’s that list coming?
it’s 90% done… just the little things I haven’t thought of and don’t know about remain.
Have to agree. I only ever track tasks at 0% or 100% complete.
If anything more fine-grain is needed then the task can be broken down further.