This is a great post, Jeff, but… I have a different definition of optimism. I really REALLY agree with you about all the problems you point out–and how rampant and harmful they are–but I don’t agree that optimism and enthusiasm are usually the root cause.
Much of what you mention can be traced to one or more of the following: greed, fear/feeling trapped, or just plain stupidity.
The TRULY optimistic do believe in a positive outcome, but NOT one that’s out of touch with reality. As an optimist–and former game developer–my idea of an optimistic, positive outcome was that I would find a company committed to a practical view of software development even if that meant the company would lose jobs or clients to developers/companies who claim they can do it for less money and in less time.
Optimism is NOT about deluding yourself into thinking you can pull off heroics, it’s about believing that you don’t NEED heroics in order to be successful. Optimism is about having faith and hope that there ARE clients/projects/managers out there who care enough to set a smart, realistic schedule.
The companies/managers who care enough about the quality of work (nobody does their best work under extreme stress and sleep deprivation), and especially about the well-being of the developers, are not somehow more pessimistic or less enthusiastic… they are simply brave (willing to say no to a client or their boss), smart, and caring.
I do agree that “enthusiasm” can lead to problems with “pet features”. No doubt there, and the fixes aren’t always practical… although like so many other things, having developers spend more time interacting with real users (or better yet - doing a stint in tech support!!) can help.
No, I say that the true optimist believes that software development does NOT have to be so brutal and stressful, and believes that saying “no” to a client will not only NOT mean the end of your business, but will in fact lead to more long-term success. Most heroics and overpromising are simply greed and/or fear, masquerading as optimism.
But that’s just me ; )