[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [Newsgroup Home]
[news.eclipse.platform] Re: ¡Ÿhow to obtain the detailed bug information¡¿
On 2/18/2009 1:44 AM, haihaoshen wrote: 
Hi,

Recently, I do some research work on developing an automatic bug-finding
tool. To validate the effectiveness of our tool, we would like to run
the real-world, well-tested projects, e.g., Eclipse. I have checked out
some source codes in Eclipse , however, I DO NOT know where there are
real bugs in source code because of the false positives reported in my
tool. Thus, I plan to compare the bug reported by our tool with the real
bugs stored in your bug database.

So, I am wondering whether you could do me a favor to tell me how to
know the existing bugs in some release of Eclipse, e.g., Eclipse 3.4M. I
want to know the detailed bug information, such as the version of
Eclipse, the package name, the class name, the method name, or the line
number of the bug.

This type of question has come up in the past, mostly from researchers like yourself. My response is always the same: in my opinion, bug reports are NOT a reliable or detailed enough source of data to support drawing conclusions in this kind of research. They can be useful for formulating theories or maybe even some trend analysis, but I, personally, would not respect any research that purported to draw conclusions.
The reasons are numerous, but the primary is that only a small percentage of bug reports actually maps directly to particular lines of code or even classes. Many bugs are cross-cutting and involve both design and implementation trade-offs or refactorings. Even if a bug is fixed with a specific code change, that information is not reported directly in the bugs, but rather as patch files. So any analysis of them would have to extract the patch attachments and analyze those. Not an easy task.
Aside from that, many more "bug" reports are actually feature requests or behavioral change requests. Bugzilla has fields to mark such things, but they are not used reliably enough to support drawing conclusions based on them.

The bottom line is that Bugzilla is a user-centric management tool, not a robust data gathering tool. It is therefore not appropriate for use as a primary source in a data-centric research effort.

Eric