Too Many Bugs - Part 2

This is part two of a paper I’ve written on making bug triage meetings more manageable and becoming a leader in a cross functional team. You might want to check out part one before reading on.

The reason we have bug triage sessions in the first place is no one has bothered to talk about these bugs before dealing with the entire list became a high priority. Everyone pays the high price of brutally long bug triage sessions because the low price of communicating on a consistent basis hasn’t been a priority right along. In order to make triage sessions more valuable you have to make it your top priority to get to the meeting with the fewest, highest quality bugs possible. This doesn’t mean close all the small bugs before the meeting and hope no one notices or to assign them all to someone else. It means effectively dealing with bugs while they’re incoming. In order to do this right you’ll need to build a relationship with someone in QE. Yes I know QE and developers are supposed to be like cats and dogs or water and oil. In this case the relationship needs to be more like peanut butter and jelly. Or peanut butter and chocolate. Or peanut butter and whatever else you like since it goes well with just about everything. You need to be like peanut butter.

Part of the beauty in doing this is that your chosen QE co-conspirator doesn’t even need to know they are a co-conspirator, though eventually they will realize their job has become much, much easier to do effectively. One of your goals is to stop bugs from coming in by stopping them at the source, which means you need to get QE to reduce the number of bugs filed on your software. There are a few ways to go about this and not all of them are good. Some can even get you in trouble with HR. If you want to reduce the number of incoming bugs you have to first find out what software QE is testing. If you’re late to the game then you should start just after the code is released to QE, though starting any time is better than not starting at all. In order to find out what’s going on in QE you have to, gasp, talk to someone in QE! It’s true, they know how to communicate using spoken language, not just Bugzilla. You can ask your QE person, “hey, how’s it going?” and they will tell you everything is going fine, testing is going fine, day to day tasks are going fine and then move on to filing more bug reports on your software. Or you could ask, “hey <QE person>, have you had a chance to look at <my software module in question>? If you have a chance I’d like to go over the work I’ve done with you in order to answer any questions you have on it.” The second way always gets better results than the first and it’s surprising how very few software engineers are capable of doing this. Asking the question in this way sets up a discussion by showing the QE person that you are open to questions and feedback. It gives them assurance that their input is valuable because it is! If they’re testing software and they don’t understand what it’s doing they’re not going to be responsive to the first question of “how’s it going” unless they have an ax to grind. You’re not going to get anything useful out of that question because it gives no indication that questions or constructive criticism are welcome even if they are. Do you really want to reduce your bug count? Spend a half hour going over your work with them. Answer every question they have and go over the implementation. Draw diagrams, play charades, do everything you can to help them understand what you’ve unleashed. All of those bugs that are really just setup and configuration questions will be vaporized if you do this. Ditto for the “what does this software actually do” bugs that would have been filed. You have just taken the first step to reducing the number of low quality bugs you have to deal with.

I remember an iteration where I had a fairly high profile deliverable. I felt like I nailed the business requirement. All the features were complete and robust. The implementation was impeccable. This deliverable was a feature that had been lacking for a long time and I had a feeling that I’d be heralded as a hero for delivering just what everyone was begging to have for such a long time. I didn’t think anyone would be able to open even a single bug on my project. How could they? They would be too busy drooling over the amazing quality of what I just delivered.

Imagine my surprise when QE started opening up bug after bug after bug on what most people in development considered perfection. I watched in panic all day long as I saw my bug list get longer and longer. And almost everyone one of the bugs was shallow. The QE person didn’t dig very far at all for a reason to file a bug. If they didn’t understand what was supposed to happen they filed a bug. If they thought the software was supposed to do something and it didn’t then there was a bug. If they thought the software should have additional features they considered nice to have there was a bug. Needless to say that iteration’s acceptance process was very stressful for me. That’s when I realized the value in stopping bugs at the source.

Almost all of the bugs were in the class that would have gone away if I just communicated what was going to happen before the release. Fast forward to the next iteration. Once again I was working on high profile features that should be a slam dunk as far as quality goes. The week before it was released to QE I asked around to find out who would be testing my features. I scheduled a meeting with them to talk about my deliverable and give them a demo. In my meeting invite I sent a brief description of what I wanted to talk about and encouraged them to show up with questions or relevant topics to discuss. During the meeting we went over just about every facet of every feature in great detail. We went over how to configure the operating system environment, the topology and the application. I wanted to do everything in my power to make sure they had all the information they needed to meaningfully test my software. Meaningless bugs are a waste of everyone’s time. Once the software was officially released to QE I didn’t see a single bug filed on the first day of testing. One was filed on the second day of testing. None on the third. I had only a handful of bugs for that iteration’s release because I made sure to discuss my concerns with QE before they started testing it. I also had more time to concentrate on the meaningful bugs.

On Monday we’ll talk about what to do with the bugs that are already in your queue. If you found this worthwhile please forward it on to anyone else you think would find it helpful.