One thing that continually frustrates me when working with dedicated test teams is that, well, they find too many bugs.
Another nice entry, Jeff. Man, when do you find the time, between gaming and coding, to do such a great job with regular updates?
Triage, is in fact, what my last boss used to call it when we got new user requests and bug reports. Itâs a gruesome necessity in a medical ward overwhelmed by patients and seems to me to naturally apply to the âcommon senseâ project management in any other field.
I didnât get into it in the post, but the âhow do you define bug?â question always comes up at some point.
I found this âWhen is a bug a bug?â discussion at the JOS forums very interesting:
http://discuss.fogcreek.com/joelonsoftware/default.asp?cmd=showixPost=72875
How many people have ever been killed by software bugs in modern airliners? Zero.
Iâm not so sure. Airbuses in particular seem to have quite a few glitches.
For example: a href="http://seattletimes.nwsource.com/html/opinion/2002078977_gaillard02.html"http://seattletimes.nwsource.com/html/opinion/2002078977_gaillard02.html/a
Those not-to-be-fixed-now bugs come in handy:
Jan
Jeff,
The problem you have with testers is the same problem testers have with Product Management and Customer/Support: Bad requirements and poor expectations. Your triages should not be as painful as you make them sound. True, QA people are not actual users, weâre smarter. If you and your QA people recieve proper req. doc. then the test cases should be pretty straight-forward as well as the defects. However, when we recieve the doc there are no prioritization lists so we test the main components and then work our way down to the minutae.
You have a right to be annoyed with the triage process and decisions made by management, but I donât think itâs fair that you give QA a hard time for doing itâs job. Rarely, do people come up to the QA team and say, âGood catch, nice work finding that bug.â Itâs more like, âWhy didnât you find this bug or that bug and what were you testing?â 90% of the time QA doesnât necessarily get to make those judgement calls about what MUST get fixed because business doesnât care if weâre not happy putting our names on a defective product.
All of my QA teams perform internal team reviews before we escalate to dev/PM triage and that liimits the time wasted in those meetings. We donât even want to be there, we have enough meetings about Priority/Severity/etc. Dev and PM need to hash out what needs to be fixed and by when. We just hope weâre given enough time to test fixes again before the product goes out.
Just one testers story.
Damien,
I donât mean to sound anti-tester; rather, I am pro-user. Testing isnât a single activity-- itâs a gauntlet of events your app has to make it through. A traditional test/qa is definitely an important section of that gauntlet.
But not the only section⌠and all decisions made during triage should be informed with real user data whenever possible.
Great article jeff, however I think more needs to be said about the âattitudeâ both lone and company developers take when it comes to handling bugs in their software. Lately it seems as if developers have more and more tolerance for bugs, relying on the old addage that, âItâs impossible to create bugless software!â And while this may be true to some extent, by relying on this as an excuse or reason more and more bugs are left unchallenged or at least until some future date/version! I see this a lot in third party component developers for Delphi and even more in popular shareware applications. Programmers need to understand that their prime directives need to include their making every attempt to prevent and solve bugs through the use of systematic and diligent testing methods, regression testing after each fix, and ensuring that releases can be patched or updated easily as the need arises so that the consumer, whether another developer or an end-user, doesnât have to wait forever for something to be done! The techniques used for finding and fixing bugs is obvious to all developers by now, so attitude is what needs changing!
Severity and frequency should be attributes of every bug the testers write.
WRT bugs submitted by users, many of them are actually requests for unscoped new functionality. Dont forget to quantity the cost and risk-to-project-plan of âfixingâ unscoped new functionality.
Cheers
Mike Tierney
My take on enhancements that testers call âbugsâ
Itâs like a person that wants a nose job walking into triageâŚ
That attitude of Team 2 (accepting bugs) really annoys me, not only as a programmer, but especially from a userâs perspective. I prefer having less features and less bugs, than somthing that comes up with errors when I use it. And we are talking only about reported bugs, not all the bugs. Would you tolerate any (even non-critical) bugs in your washing machine, or your refrigerator?
I think that this âusers can live with this bugâ-attitude is the reason that todays software is of such a poor quality. It is this attitude, which causes the software industry to be called immature. The users should demand more, but also pay more for the better quality.
If there is a bug, then should be asked, why it is there. If there are hundreds of minor not-fix-now-maybe-later bugs, then the real bug is not in the software. It is in the engineering process. And that bug should be fixed.
If there is a danger of creating more bugs by fixing one bug, then should be asked, why the danger exists. If programmers can not program any code, because they have to fear that more bugs start popping up from some mysterious shadows, then there surely is something wrong with the engineering process. And that is real critical and should be fixed.
Ahhh I must say this is truly the Developers perspective. Which is why we have testers and such things like user testing. More often than not testers think users are going to use a product in the logical manner that they would. However this is just not the case. Let me put it to you like this if a user can do it it will get done. So many times I hear âyah but the app should not be used like thatâ. If a tester is any good the bugs they take the time to assign to you or somehow bring up to you are important. We also know some bugs are crap. But then on the other hand at what point do you ignore crappie bugs before there is so many your product is crap. 100 crap bugs a thousand? Testers are here to help developers look good. We are trying to help you not annoy you. Also trust me you donât want our job. In the end a product that functions great is said to be the developers doing. So maybe next time a tester is showing you something try and trust that they have a reason.