David Baron's Weblog

Back in October, Martin Best interviewed me (among others) to find out how we use Bugzilla. During the interview, while I was describing to him how I use it, I came up with some ideas on how I'd prefer Bugzilla to work.

One of the big problems I see with Bugzilla is that we don't have a good way to track what work has already been done for a bug and what work needs doing. For example, there are some bug reports filed against core Gecko code where the problem is clearly explained and reproducable, but it's not clear whether the behavior is correct. These bug reports would benefit from being looked at by somebody with knowledge of the relevant standards describing the correct behavior. There are other bug reports where the problem is clearly a bug (say, a crash), but it's not at all clear under what conditions the problem occurs. These bug reports would benefit from being looked at by somebody who can suggest conditions that might be relevant, so that the conditions that cause the crash can be found (so that the crash can be reproduce and debugged by a developer). But we don't have a way to direct these individuals to the right bug reports to spend their time on, which contributes to bug reports being ignored simply because dealing with bugs is an inefficient use of time.

One of the other problems with Bugzilla is that bug reports are often read frequently, but they're often read for a particular purpose, such as figuring out how serious the problem is in order to decide whether it needs to be fixed for a certain release, such as figuring out how to fix it, or such as figuring out whether a patch posted to the bug should be accepted (which depends on figuring out both whether and how the bug should be fixed). In both cases, there's a long linear stream of comments in the bug report, but the person reading the comments only needs to read a portion of the comments. This problem gets worse as the number of comments increases (which sometimes happens quite quickly).

I think it would help solve both the problem of directing people at the bug reports where they can be useful and the problem of filtering comments relevant to a given task, if Bugzilla's user interface were more oriented around these tasks.

I think the key goals that we'd want to separate things into are something roughly like the following:

Understand description

Do the developers or triagers reading the bug understand what the reporter is saying?

Agree it's a bug

Do the module owners or peers or other authority agree that the problem is a bug?

Can reproduce

Can others reproduce the problem described?

Why it's important

What makes this problem important or urgent to fix?

How to fix

What should be done to fix the problem?

Workarounds

What could be done to work around the problem if it can't be fixed now? (This discussion is currently discouraged because it gets in the way of other goals, but if it were separated it wouldn't need to be discouraged.)

Reports of intermittent test failures

Until we figure out a better way of tracking these than Bugzilla, this should probably be its own category. In fact, to some extent, our Bugzilla already has category UI for this.

Should this patch be committed (separate for each patch)

A discussion about the merits of and problems with each patch in the bug (with sequences of revision of the same patch being considered the same). In other words, code review.

One of the difficult aspects of designing something like this, however, is the tradeoff between the cost of maintaining metadata and the desire to get work done quickly. There are currently many bugs in Bugzilla that have a bunch of fields that are just left at their defaults (e.g., severity, priority), and in many cases that's fine because we don't have a need to maintain these fields. But once a bug gets complicated enough, it's useful to be able to keep the discussion organized. So it would be desirable to design the interface that allows this additional organization in a way that it's optional and that the organization can be added to data (e.g., comments) after those comments are created (for example, once there are too many comments to deal with in the current disorganized way). Doing this would also provide a transition mechanism for existing bugs.

One idea for how to build such a user interface, to give a vague idea of the sort of thing I'd like, though not one that I'm committed to, is to have separated areas of discussion (permanent comments) for the different goals we have (where comments could be recategorized after they were made, in case they were miscategorized or the bug was initially entirely uncategorized). These could, for example, be shown in tabs where the tab has an indication of the number of comments and the state. For many of the goals, in addition to the permanent comments (which could be moved to a different goal if miscategorized), we would also have a boolean state (a checkbox) to record if the goal has been met, and in some cases an editable description to record the summary of the discussion (e.g., an editable list of steps-to-reproduce that can be revised as they are better understood). I think these separate boolean states could replace much of the state currently stored in the status and resolution fields.