Too Many Bugs - Part 4

Today brings us to part four in the series of thoughts I’ve put together on reducing bug triage brutality by creating a more productive relationship with your QE team. You might want to check out part one, part two and part three before reading on.

There’s the old saying “when you’re in a hole stop digging”. So far we’ve talked about how to stop digging but we haven’t talked about how to get out of the hole yet. A bug list is a bit like Pandora’s box in that once the bugs are open they’re like all the sorrows let out of the box. You just can’t put the sorrows back in the box and you just can’t get rid of all the bugs. What happens to you when you’ve got a bug list a mile long? You try to avoid looking at it for one. But there are always people in management that subscribe to the mange-by-numbers theory who take bug metrics a little too seriously. Eventually you’re going to make it to the top of the naughty list when they’re looking at who has the most open bugs. This can be interpreted in a few ways and none of them are good. They could think you’re a terrible developer if you write so many bugs. They could think you’re lazy if you’ve got such a huge backlog. They could think you can’t prioritize well if they see something related to their pet topic is on your bug list and you’re working on something different. They could think any number of different things and whatever they come up with definitely will not benefit you one bit. You need to massage your bug list to stay off the slacker radar.

Start by picking off some bugs that are related somehow. They can be related on feature, topology specificity, platform specificity. Pick a few that you don’t understand well. Pick a few that you think are configuration error, user error, documentation error. Make up a list with these bugs and a summary of each one or print out each bug report. I hope you can guess what’s coming next. That’s right - go get your QE person or people and ask when they will have time to sit down with you to address your concerns. What you’re doing right here is proving to QE that they are valuable and their work is not in vain. You’re validating their presence in the organization by asking them to talk about their work. They need someone to listen to what they’re saying and it’s your job as a developer to do it. You’re making them feel good by making them feel important. This is a great way to start off a meeting or meeting request.

Let them know what bugs you want to discuss before the meeting starts to give them time to prepare too. This meeting is going to go very smoothly if everyone comes prepared. Your goal here is to find out enough information to resolve all of the bugs on your list you prepared. Every one of them must have some resolution before you leave this meeting. This is not a bug triage meeting because you know going into it that all the bugs must be resolved. This is a technical meeting that results in action items from all parties.

Start the meeting by saying “I understand you have some concerns about… <fill in the software module blank>. I was wondering if you could give me some insight into what you’re looking for from me”. Begin the discussion on each bug by asking them how they encountered it even if it’s already logged in the bug report. Ask questions about their process. A lot of extra information that wasn’t captured in Bugzilla will come out of this conversation.

Sometimes enough information will come out that you will all see that the bug is clearly due to incorrect expectations or incomplete documentation. These are easy bugs to resolve. Sometimes you’ll get that a bug is clearly a bug and through your conversation you will find out exactly what you need to do to fix it. These are also pretty easy to deal with. Another kind of bug that can end up here is one that turns out to be an enhancement request. I’ll talk in-depth about how to deal with enhancement requests another time, but if they come up in the meeting it’s best to just move on to the next bug. If you don’t you’ll end up at a mini-triage meeting trying to decide the priority, whether or not it’s an enhancement request and so on.

At the end of the meeting there should be a list of developer action items for you to take care of and a list of QE action items for them to take care of. Have these meetings as often as you can in order to drive down your list of bugs. Remember you’re trying to show up at the bug triage meeting with the shortest bug list possible and this will drive down your list very quickly.

I can derive another relevant example from the first iteration I wrote about in part two. Remember that QE had opened a huge number of bugs in a very short time period on my deliverable. I was overwhelmed by the incoming load and I couldn’t just mark all the bugs “won’t fix”. If QE wasn’t riled up already that would be the thing to do it. Most of the bugs came from two people in QE so I had to get them into a room with me, if not to talk to them then just to keep them away from their keyboards so they couldn’t open any more bugs for me.

I went in to the meeting with a print out of every bug I wanted to discuss. Each one was related to the feature that I had just delivered. I started the meeting by saying, “John, I understand you have some concerns about this feature. It seems like there are some things that are preventing you from testing further into the feature and I would like to know what I can do to help you get further along”. Notice how I didn’t say anything about “Please stop opening bugs or I’m going to quit my job” or “I hate you QE guys so much that I’ve written a thousand more bugs that you’ll never discover” or even something nicer like “There are an awful lot of bugs against this feature and I’m not sure some of them are valid”. Everything I said was phrased in a way that conveys my want to help QE. And I really did want to help them. Helping them helps me. There is no rule that says developers can’t help QE and if there is a rule that says that it’s wrong. Dev./QE is a two way street.

After that we went through each bug. We discovered that some of them were duplicates of each other. That gave QE the action item to close each duplicate bug. Some of the bugs were discovered to be due documentation that wasn’t explicit or detailed enough. Those bugs came into my action item list to fix the documentation. I asked the QE guys what documentation details would have helped them navigate through the steps safely so that I could use it in my resolution.

Some of the bugs were because the user tried to set up the software on an unsupported platform. You can’t use SuSE 7 and Perl 4 when the supported platform is Red Hat ES 4 and Perl 5. Those bugs become action items for QE; retest those bugs on supported platforms.

The most interesting bugs were real bugs caused by unexpected QE needs, but needs vital to their testing. Getting a firm grasp of what is causing these bugs is important and hopefully you can gain enough information that you can propose a fix in a relatively short period of time. Those bugs became action items for me to resolve.

In creating these action items it’s also important to provide a time table as to when QE can expect to see the resolution. Talking about the bug is one thing, but the goal is to drive down the bug list and the only way to do that is to actually fix the bugs. For my action items I told the QE guys that they could expect to see a fix in the next patch release they get from development and I made sure to deliver. Setting joint expectations and then driving toward them is a recurring pattern here and your consistency in doing so is important to your success.

Come back again tomorrow for the conclusion!