Defeating Optimism

In Extreme Programming Explained, Kent Beck notes that optimism is an occuplational hazard of programming. Excess optimism, in the guise of enthusiasm, is a serious pitfall for game developers in particular:


This is a companion discussion topic for the original blog entry at: http://www.codinghorror.com/blog/2005/05/defeating-optimism.html

The problem is that most managers want “can do” attitude because that suites THEM the most (they can pass on the good news to upper management) and makes them look good. If things start to go wrong, they can always blame you for promising what you can’t deliver.

The reason is that they are not interested in the success of the product, just their own well-being. If this project fails, they’ll just move on to the other. It’s extremely hard for an honest, pragmatic developer to get out of this situation.

In the end, developers start overpromising just to get managers off their back, and many, many software projects fail (sigh).

developers start overpromising just to get managers off their back

If you have poor management, your options are limited. You can try to nudge them in the right direction. Or, you can take control of the project and run it the right way internally, and put whatever public face on it you need to in the status meetings with the manager.

good words~!

I have translated it into Chinese and put in my Blog, I have put the track and your name on it also ~!

a href="http://blog.xuite.net/zealousage/ZEAX/4508420"http://blog.xuite.net/zealousage/ZEAX/4508420/a

This is a great post, Jeff, but… I have a different definition of optimism. I really REALLY agree with you about all the problems you point out–and how rampant and harmful they are–but I don’t agree that optimism and enthusiasm are usually the root cause.

Much of what you mention can be traced to one or more of the following: greed, fear/feeling trapped, or just plain stupidity.

The TRULY optimistic do believe in a positive outcome, but NOT one that’s out of touch with reality. As an optimist–and former game developer–my idea of an optimistic, positive outcome was that I would find a company committed to a practical view of software development even if that meant the company would lose jobs or clients to developers/companies who claim they can do it for less money and in less time.

Optimism is NOT about deluding yourself into thinking you can pull off heroics, it’s about believing that you don’t NEED heroics in order to be successful. Optimism is about having faith and hope that there ARE clients/projects/managers out there who care enough to set a smart, realistic schedule.

The companies/managers who care enough about the quality of work (nobody does their best work under extreme stress and sleep deprivation), and especially about the well-being of the developers, are not somehow more pessimistic or less enthusiastic… they are simply brave (willing to say no to a client or their boss), smart, and caring.

I do agree that “enthusiasm” can lead to problems with “pet features”. No doubt there, and the fixes aren’t always practical… although like so many other things, having developers spend more time interacting with real users (or better yet - doing a stint in tech support!!) can help.

No, I say that the true optimist believes that software development does NOT have to be so brutal and stressful, and believes that saying “no” to a client will not only NOT mean the end of your business, but will in fact lead to more long-term success. Most heroics and overpromising are simply greed and/or fear, masquerading as optimism.
But that’s just me ; )

One thing that goes hand-in-hand with the wishful thinking discussed in this article is poor memory. Allow me to explain. The first time wishful thinking kicks in, people are overly optimistic and think that a software release will get out the door on time, even if all indications are to the contrary. That’s understandable, once. But in many organizations, people don’t learn from that mistake, and then for the next release cycle, continue the wishful thinking, make all the same mistakes, and put out another late release. Then for the third release cycle, it happens again.

If people just remembered what had happened in all the previous releases, then they would at least have some ammunition to combat the optimism. However, in my experience this sort of organizational memory is frighteningly uncommon.

Most heroics and overpromising are simply greed and/or fear, masquerading as optimism.

I agree. Perhaps the better term is “selfish optimism”.

That said, software developers are, generally speaking, notoriously and almost comically optimistic. Developers should try to err on the side of cynicism and/or pragmatism to combat this propensity. It’s also a lot safer to work with a cynic. Not necessarily more fun, mind you, but the odds of total project destruction are significantly lower.

If people just remembered what had happened in all the previous releases, then they would at least have some ammunition to combat the optimism. However, in my experience this sort of organizational memory is frighteningly uncommon.

I’m not sure that’s optimism-- that’s just plain amnesia. It’s a huge problem. There are some key metrics that are relatively easy to collect, and that can help a lot:

http://www.codinghorror.com/blog/archives/000681.html

Without historical data of any kind, you’ll never make it off Gilligan’s Island:

http://www.codinghorror.com/blog/archives/000017.html

Optimism occurs because 90% of programmers think they are in the top 10% of their field. This isnt unique to IT. It is a common human condition. We are eager to prove that we really are better than average. Our own self worth holds higher value than the success of a project, and we will gladly risk the project to achieve it, in spite of the odds. And we subconciously avoid calculating the odds!