In most real businesses, the idea of giving estimate ranges multiples of orders of magnitude wide to management are simply a fantasy.
That’s right. So the next logical question is, given the high level of uncertainty at the beginning of any software project, how do you eliminate or reduce that uncertainty?
It’s all about gathering data-- both as you go, and before you start, through research. Ideally, you’d have some historical project data from your own organization.
Just looking at the number of overbudget projects that continue to occur tells me that you have more chance of creating an accurate estimate using a dart board as you do using any of the estimation techniques used over the last 40+ years of software development.
You’re saying that software estimation is impossible. I don’t think that’s true. It’s a very hard problem, but it’s not unsolvable. I think the main problem is most organizations don’t gather enough data from their past and existing projects (bugs, bug fix rate, function points, etcetera), so they’re starting from a blank estimation slate every time they launch a new project.
The length of the pacific coastline is not well defined. Coast lines have a fractal dimension greater than one, and thus the measured length increases without limit as the measuring stick used grows smaller. Any value above 50000 miles is therefore correct for some stick-length.
The one other commenter who made the same point made the mistake of coupling the observation with the ridiculous and wrong statement that the volume of the great lakes is also not well defined.
Whatever reference produced that precise value for the Pacific coast length is either: a) teasing, or b) another instance of that special mixture of self confidence and ignorance that plagues so much of our world.
What is interesting about the study results are that we have a marked tendency to underestimate (if our estimation is 30% accurate is debatable). How many times have you estimated that a project will last less than what it actually did? My interpretation of this 90% vs. 30% anomaly, is that our estimations (excuses for generalization) are about a third of the real value, when we haven’t researched enough on the problem at hand, as it turns out to happen in the design of this quizz. That doesn’t mean, of course, that with a decent research we cannot approach more to the real value, and possibly that’s the core subject of the following chapters in the book.
Most of the comments here are interesting to read but a bit off the mark. I wonder how many of the people making the comments actually read the book. The “estimation” exercise was part of a bigger discussion of “estimation, targets and commitments”. Estimation, according to the author, is used to help the planners gauge the likelihood of having a successful project. Of course, estimates will be done by qualified individuals. Of course, the narrowness or breadth of the ranges matters. Of course, some people will know some of the answers. None of that is the point! The point is that a huge percentage of people who took the test were asked to get 9 out of 10 questions right yet the vast majority could only achieve 2-4 correct answers. Those of you who blame “management” need to look in the mirror and READ the test instructions.
The quiz is stupid. Most of the questions involve estimating things that are incomprehensibly huge; they’re just a number. There’s no sense in trying to ascertain the value of something when you have absolutely no conception of the scale at which you’re dealing with. The guy who got 2/2 right, 100%, has the right idea.
Scheduling programming, on the other hand, involves a directly tangible sense of days and months–at least, it should to an expert programmer. So it’s possible to make meaningful estimates, even if they are wrong.
Your quiz is unfair because it is designed to make people wrong. Scheduling programming is just disposed to hurting people who underestimate.
This quiz might be better if it asked for 50% confident estimates instead. Those who guess +/- infinity are actually estimating to 100% confidence, which is not the point of the quiz.
If I know Alexander’s birth-date to within a century, my idea of 90% confident is much narrower than if I only know it to within 5 centuries or have no clue at all. Because of this, the amount of knowledge you have on the subject doesn’t matter much.
90% confidence is difficult to distinguish against 100% confidence, so you get the impression that you should get them all right. But at 50% confidence, the best result is half right and half wrong, and 100% right is nearly as bad as 0 right.
That’s why I think 50% would be more meaningful, assuming I have interpreted the situation correctly.
I answer right 5. But I think Jeff is right, since my correct answers was due to my previous knowledge, and I fail almost all estimations (except for U.S. currency, with a factor of 8 range).
After know my results, I realize that I tend to narrow the range, without need it. With previous knowledge you can narrow it, but if you’re totally in uncertainy, the best practice would be to open the range and make clear that you don’t have any real data to narrow it.
As Ryan says, estimations without acuracy are useless, but so are estimations based on few data. You can estimate chances of dying in a car accident, but if you don’t know anything of the driver (frecuency of drive, drive times, country…) an acurate estimation is almost impossible.
Don’t make your ranges too narrow or too wide
Surely this influences the test taker to narrow their estimates? So although the conclusion that we believe that wide ranges make us appear ignorant or incompetent is correct, the test does not show this because it actually tells you not to use wide rangers.
I’m not saying the test itself is flawed but the conclusion is IMO, even if it is true. In fact it’s so true I would call it common sense, but that’s just me.
I got 4/10 btw but my estimate of 5-30 degrees for the latitude of shanghai is so close to the correct answer of 31 degrees that I’m going to say 5/10 from now on, just to avoid appearing incompetent 
Very early on in my programming career (1997), I was told, “the main difference - no - the only difference between an amateur and professional is the ability to estimate the time it will take to complete a project”. By that definition (and several others), I’m still an amateur. Of course, I try not to tell that to my customer.
And that’s the problem, in my opinion, with wide estimations for consultants. When you say, “I’ll deliver what you want, and it will take anywhere between two weeks and two months”, it sends a strong message that I don’t know what I’m doing. That I’m an amateur. I’m better off saying “three weeks, no more!”, sounding like a professional, and then extending the deadline with excuses, if need be. This sounds terrible, and it hurts the customer, but at least I got the job in the first place.
I think the best, most honest answer to the question, “How long will this project take you?” is the answer, “I don’t know, but within a week of research, I’ll be able to give you an educated guess”. Of course that week might turn into two.
I estimated 9 of 10 correct, I had no basis for the blue whale question and foolishly picked too low [40-140tons], and I knew some of the questions directly [the temp of the sun, alexander’s birth, lat of shanghai].
I think what Mr. McConnell is trying to say with the higher accuracy crowd being too narrow is that we feel we missed the question we missed because the answer was slightly outside of our range. Given the methodology on the previous questions, I felt my range was large [almost 400% my low], however on most other questions, my range high was always single or double digit percentages of my range low.
A narrow estimation range does not make my estimation skills wrong, it just means I have a lot more basis of expertese [or none at all] to answer the questions given with a high degree of accuracy.
Estimations without care for accuracy are useless. [ie: the time until the sun goes supernova, % chance of an asteroid hitting the earth, chances of dying in a fatal car accident]
I enjoyed this exercise, thank you. I took a peek at the findings in terms of the poor accuracy of estimates before taking the quiz (without seeing the answers). I got 8 out of 10 right (but probably would have gotten much less if I hadn’t been clued in to widening my estimates).
I agree with most of the authors conclusions about our tendencies and appreciate that he is encouraging us to acknowledge the uncertainty in our estimates and endeavoring to give a wide enough range to make then accurate instead of arbitrary. The only thing I didn’t agree with was some of the emphasis placed on no one getting 10 out of 10 correct. While taking the test, I continually tasked myself with being 90% “sure” of my answers. This should follow to establishing an average of 9 out of 10 answers correct. Yes, some probably should have gotten 10 correct, but that was not actually the goal. The goal was to be 90% sure, and some variation either way is understandable. Even I would have been capable of getting 10 our of 10 right with broad enough ranges (answer infinity for everything!!!), but all things considered, I am not particularly disappointed with my ultimate performance.
The answer for the Volume of Great Lakes in liters is incorrect - it does not match the number in cubic kilometers.