Too Many Bugs

There is a time or two in every software engineer’s life when he experiences a deluge of open bug reports. I’ve seen times where there are ten new defect reports per day for two weeks straight against one module. At that point checking your “My Bugs” list in Bugzilla is a daunting task. You don’t want to see how far down the page scrolls. Worse, you don’t want to triage all those bugs; each one will take at least five minutes.

Right off the bat you have to read and comprehend the defect report. Not all reports are clear and concise Sometimes the problem still isn’t obvious just from reading the report and you’ve got to wait on clarification. Worse, sometimes the bug report is perfectly clear and you can’t determine whether the issue in question is a feature or a defect. You can expect to spend a lot of time going back and forth with dev. management, QE, customer support and product management discussing whether or not the functionality is correct, especially if you work at a large company. So let’s say you know that some functionality is not correct or at least not desired. Now you’ve got to think about the steps to reproduce it and what the root cause could be. Still, you’re just thinking about it; you haven’t yet dug into the unit tests searching for a missed failure condition. Despite not knowing the root cause for certain there has to be a business decision about this bug: is it worth our time to fix it? Is there a business case that says that this functionality must be corrected before it is released into the wild? Sometimes yes, sometimes no. More than five minutes has gone by in either case and you’re still not done. Someone in management is going to want to know the delivery time table. Now repeat this process for 74 more bugs. If you’re triaging by yourself and you work at near-100% efficiency you’re looking forward to at least one full work day of triaging and the bug reports are still incoming.

Realistically you’re not working by yourself and you don’t set aside an entire day to triage your bug list. In real life the situation is a lot worse because a long time ago somebody in management invented the bug triage meeting. Bug triage meetings can last for hours or even days where usually too many people get into a room, close the door and stay caged in there until they triage some percentage of the product or service’s entire bug list. Instead of spending one person-day on bug triage, these meetings make it possible to burn entire person-weeks in the blink of an eye by turning lots of little technical problems into one large social problem. The technical problems don’t matter here because, easy or hard, there is a fix for (almost) every bug. The social problem is a lot bigger: getting people to agree on the priority and severity of each bug in the list.

Remember the process to triage each bug? Now everyone in the room has to go through that process for each and every bug that comes up in the meeting. Unless your company is the Borg Collective (no, not Microsoft. I actually mean the Borg from ST:TNG) there are going to be questions that require in-depth answers and those answers will foster discussion to get everyone up to speed on the history of the bug, why someone thinks it’s high priority, why someone else thinks it’s low priority, until everyone agrees and the meeting moves on to the next bug. The bug triage meeting also introduces the notion that you’re not going to be dealing with only your own bugs. Other developers are going to be in the meeting and their bugs have to be triaged too. This means that a significant portion of the meeting time is going to be spent NOT talking about your own bugs which is a recipe for boredom even if you have the best intentions.

It would seem that I don’t have a very high opinion of bug triage meetings but that’s actually not the case. Bug triage meetings are vitally important to a project at the right time, with the right people and the right material to work with. Unless you’re into fooling yourself no product or service ever ships with zero bugs. Sure you can get all kinds of management involved to do requirements-lawyering and invalidate bugs in bulk but then the zero bug metric loses its meaning. I don’t know about you but I’d rather have a list of known risks going into release instead of sweeping them under the rug because company policy dictates that everything must ship with zero bugs and is completely perfect, absolutely flawless. One of them seems just a bit more realistic than the other too.

Anyway, because there are always bugs we need the triage meeting to decide which ones to fix right now. These are usually the bugs which will cause the most damage in the wild either to the customer or the company’s reputation. The rest should be put in the parking lot for consideration in the next release cycle. If you’ve done your engineering cycle right then a vast majority of the bugs will be put in the parking lot. If development is complete and you’re working on driving your bug list down for the release then 75% of the bugs in the list should not be classified as “must be fixed”. Maybe development isn’t complete after all. So if you know that most of the meeting discussion is about bugs that don’t need to be fixed right away then why spend so much time talking about them? It’s because people lack initiative. Do you want to be a rock star team player adored by your peers and management? There are a few things you can do to make bug triage more valuable to everyone involved and give your career a big boost by addressing the social issues before they become social problems.

Tomorrow we’ll start taking a look at specific things you can do to reduce your bug count, help QE file real bugs and make your contributions more valuable and more visible.