There definitely has to be a balance here - you obviouslly can’t release, say, flight control software for an airliner and wait for feedback.* And if you release something that is so buggy it is unusable you will not get any user feedback either.
But if you have a usable, stable product with some issues, like crashes in obscure cases, odd or inconsistent behavior, or unpolished behavior in some scenarios, you should try to release as soon as possible. The thing is, during development, you are guessing what bugs are important in your triages. You are guessing how customers use the software. You are guessing at the impact of a bug - if it is something that will effect all customers, some customers, or no customers. You could expend effect on getting these issues fixed. Or you can release, get feedback, and focus on things that actually impact customers, including things you never would have considered. No guessing at all - bug X is a crashing bug that you get 500 reports a month for. Bug Y prevents customers from deploying in a configuration you would never have tested it.
I had to exercise some self-control here to not make a tasteless pun on the word crash.
I think the point that got lost is not “release buggy crap”, but rather:
The engineering and internals of your 1.0 may not be where you want it
The visual design may not be where you want it
The feature set may not be as robust as you want it
etc.
Get a stable 1.0 and out to the customers. It should be high quality (as bug free as possible, like any software) and with at least the minimum viable feature set.
While that’s out in the world, the engineering can be firmed up, the internals can be changed, etc, for the next release. You’ll be receiving feedback from customers that will help shape the engineering, and help shape the UI, not to mention what features need improvement, what doesn’t work the way it is envisioned, what is missing etc.
@JeffGordon Assuming you have a refund policy, shouldn’t it be up to the user if the product is good enough for their needs? I think users will often buy a product not because of what it is, but because of what it may become. If the users can see your vision, or have their own and want to participate, it is worth it for them – it’s their choice, so let them choose… but be kind if it doesn’t work out.
@Portman Eric’s writings are wonderful. I too highly recommend it.
I think the main jest of what @codinghorror is talking about is new products. Products whose producers are trying to discover the exact need being met. For this target audience, it’s absolutely critical to get feedback early, and get it often. Steve Blank has a book /The Four Steps to the Epiphany/ that discusses this process in detail. If you’re writing software involving many unknowns, like many of us, it’s a must read.
…or perhaps we could do XP, be ready to release tested code at the end of every iteration, never slip the date, and adjust scope with the customer during the planning game.
We could release early and often, quickly respond to feedback, and not reduce quality.
Or we could make our paying customers into involuntary beta testers.
The best motivation for me to ship a better product is not to be embarrassed about the product I made. That’s motivation enough for me!!! An internal application in our company acutally will be used by tens of people daily and even all day long every day, and then it’s crucial that’s bugfree and free of most small annoyances! Reach for the stars and get to the treetops!! Good also!!
It’s often strange that this is how it works in the software world. We’re software engineers. In what other engineering world can we they release a “sucky” version 1?
If version 1 of my office building was buggy, it might collapse and kill me. Version 1 of the bridge I cross on the way to work better not suck either. And Version 1 of a space shuttle leads to Challenger-like instances.
Maybe my experience is tainted by the job I do (I design software for medical devices, so high reliability is the name of the game), but as a user, this kind of buggy early releases are enough to make me reject a product.
It doesn’t take a lot of crashes for me to lose faith in a product if it’s meant to serve any kind of useful role in my job (even if that role is, say, storing my knitting projects wish list. My time is precious.)
And I’ve seen my mom - a complete newbie - just give up on potentially useful applications because she just couldn’t get beyond the bugs. What I’d call sloppy UI and harmless but annoying bugs, she translates into “I guess I’m too dumb to use this application”.
And the less critical your application, the less it takes for users to abandon your product…
It is ironic that Jeff talks about shipping what your customers/users want, but when it comes to careers.stackoverflow he and Joel seem to refuse to listen to what one side of the user base wants.
Jeff’s not being consistent or honest with himself here.
Isn’t this why we have Beta, Alpha, and UAT releases of our products?
When a product is in development, it should be shipped to the customers BEFORE IT IS OFFICIALLY RELEASED and BEFORE ALL FUNCTIONALITY IS COMPLETED.
Of course, I’m not talking about web server enabled software like Gmail which can be updated without ever a customer knowing about it. But, standard here’s-the-CD-now-install-it-on-your-PC software is suppose to be released to a dedicated set of customers who test the stuffings out of it by using it every day.
If I buy a piece of software, even Release 1.0, and it contains some zingers of bugs, I don’t update to Release 1.0.1 or Release 2.0. In fact, I usually never buy a piece of software from that company ever again.
You should release to customers a beta copy that is usable. You should be working on one feature at a time, and making sure that feature works before you start on another one. You should be writing all of your unit tests before you write a lick of code so you know what each function has to be able to do before it works. You should be using CruiseControl or Hudson or whatever build automation tool you choose that builds and runs all of your unit testing on each check in.
If you do all of that, you’ll ship software that may not be 100% bug free, but is something users won’t regret buying.
Jeff’s advise is to be interpreted with care. First, as mentioned before, you cannot and should not apply this advise to commercial software development, software products and embedded software. It will ruin you business if v1 sucks.
As for the consumer web, clearly Jeff’s area of expertise, yes, you could consider the early release cycle methodology, but only if v1 does not “suck” as bad as to chase users away forever. Call me old fashioned, but what is wrong with doing a normal beta with limited users? What is so outdated about acceptance testing your software before releasing it?
I guess it depends on what you mean when you say v1 sucks. If your back-end code is not optimized and no user will notice, yes, you can release early. If the user experience or actual features suck, you’re in for a big risk, even on the consumer web.
All of this release early hype comes from the desire to have a short time to market. To be there before the competition. The result is that we now have TVs with bugs in them. Appearantly it is now normal practice to deliver crap on time and then fix it later.
You know who has the best software planning approach in the world? Blizzard. They do the exact opposite of what you do. They don’t care when they release, they release when they are ready. Everytime they do, they hit the jackpot first time. The whole rush of software releasing becomes irrelevant when you take your time and do it right from the start. Blizzard does it, and so does Google.
There is no perfect software @ version 1.0, the best tries I know have been in beta for ages and are lingering somewhere around 1.2, and they are all relatively small programs…
I started reading the article…felt was Interesting…kept reading until i reached the quote of “Donald Rumsfeld”!
I stopped reading. I don’t want to read someone who is quoting a guy like Donald Rumsfeld : this man is a war criminal (the butcher of Iraq).
If you want people to read you don’t quote problematic persons like this war criminal called Donald Rumsfeld!!
Jeff,
I have just experienced this issue from the customer side and have discovered that pushing it out works great.
There were bugs coming in all directions, so it has made the launch very stressful. But our customers are finding things that we never even dreamed of and I am pushing them back to the developers.
I am still in awe of how quickly these issues can be addressed by the programmers, once the system’s bones are in place.
I was very critical, and really pretty impatient with the development team to get everything perfect before going out. But now I see how well the customers partner with us in producing a better product.
Hummm… my comment the day this article came out is missing… It was brilliant, but I’m reduced to paraphrasing it, because it was but a brief moment of literary genius.
This is all well and good for the programming department, and we all know that nobody expects the sales department to actually tell you the truth, but this ‘policy’ REALLY screws over your support staff. BIG TIME. We’re the ones who deal with the angry customers. We’re the ones who have to make excuses and lie about why things don’t work. You pull this one too many times, and you start losing updates and support packages.
It’s always best to ship what works, and the new features that are still being polished get released in an update. This will keep people in a renewal plan, release only features that work properly, reduce pressure to do bad fixes, and gives a much better impression about your product. Ship 90% today, and the last 10% are updates/new features.
Do this, and you might actually be able to keep your support staff, long enough that they are actually good, or move on in the company. They are the only ones talking to users every day. They are the ones that keep the good feelings going when there’s problems. Here’s a quick test… if your support staff is happy, your customers are happy.
The problem with this approach as I see it, is that software is no longer as simple as it used to be. There are multiple layers full of contractual obligations to each-other. Rushing a first version and allowing crap will inevitable have negative consequences, i.e. the broken window syndrome.
We may think we can always come back to fix it, but we all know this rarely happens. Project managers don’t understand technical debt, and marketing people scream “ship it” as soon as they see more than a mock-up.
So I favor “Version 1 is delayed, but ship a feature-deprived preview” over “Version 1 Sucks, But Ship It Anyway”.
alot of people here are saying “oh he meant release as a beta.” he did not say beta, he said ship. ship means (to me) slap it on CD and put it in the stores.
THIS is the vile infliction upon software development today. Force the customers to betatest, CUSTOMERS are NOT beta testers, they are consumers. You do not give them crap and expect proper feedback. Give a proper alpha test and beta test, once you are sure you have fixed the major flaws and have polished up the shiny interface, THEN ship as v1.0.
Jeff you missed the mark something fierce. I agree with many here, your experience with web apps is clouding your judgement. In game design, or even basic software, there are people who buy it who never have access to the internet. You cant fix their problems, so they are stuck with them unless you release a fixed CD which costs even more money. Or if you force the customer to pay for that fixed CD as “version 2” your even more disgusting.
For any product there is a set of criteria which is “good enough”. Anything less will be suboptimal and anything above will be somewhat “wasted” in the sense that there could have been lost opportunity (to say code in another highly value-added feature or made a shipped feature more usable). Of course, defining “good enough” is an art and a science unto itself.
I think this is sort of naive navel gazing. At the heart of this is nothing more than a cliche. Sure, you shouldn’t keep polishing version 1.0 for years and years and never release. Ok, fine. But, you could can get that information from any and every source on building software. This is just kind of cliche.