Why users don’t bother to file bug reports
This has to be the saddest bug I have ever seen. Unfortunately, I see this type of response all too often, where the user, despite having a perfectly readable stack trace, dump or error message, is expected to either a) prove to the developers that the problem still exists in the latest nightly build or b) provide a reproducible test case.
Running the latest nightly build may be trivial for client software, but for server software running on busy, production servers, this is impractical and difficult, if not impossible. Furthermore, in a production environment, reproducibility is not an easy feat, as conditions are never the same, and accurately reproducing the load of hundreds of users is far from scientific.
Need more examples of saddness? Here’s another one. It’s a MySQL bug about corruption on a storage engine. This one is particularly bad, as the developers keep insisting on trying to reproduce the problem with various versions, despite several users (myself included) confirming the problem across many versions.
Of course, Eclipse servers are affected by both of those bugs — Apache won’t gracefully restart because of the PHP bug under certain conditions and Bugzilla searches fail because of a storage engine issues.
But wait - it gets even sadder. Here is how the above PHP bug is closed (comment by the reporter) :
Whatever. If you do not want bug reports, I will not post any. I thoughtyou welcome help and want to improve the product but it seems you careonly about having less work. Forget it. Let this bug be.
IMHO, that a fair statement.
The MySQL bug is closed with this automated message:
No feedback was provided for this bug for over a month, so it isbeing suspended automatically. If you are able to provide theinformation that was originally requested, please do so and changethe status of the bug back to "Open".
I understand that the developers’ time is precious and that good bug reports are required, but users are not intimate with the source code, and often cannot easily provide more than a crash dump or an error message. That doesn’t mean there is not a problem with the code, so relying on the user to do all the heavy lifting seems quite unfair, and a great way to convince your users to not report bugs.
Posted November 25th, 2008 by Denis Roy in category: Uncategorized
You can skip to the end and leave a response. Pinging is currently not allowed.
7 Responses to “Why users don’t bother to file bug reports”
Leave a Reply
You must be logged in using your Eclipse Bugzilla account to post a comment.
David Carver Says:
November 25th, 2008 at 10:32 am
What is just as bad as have a bad response, is having absolutely no response. I’ve seen this across multiple open source projects as well as on eclipse projects. Bugs that go unreviewed within a timely manner send the same message as bugs that get answered with the cavalier attitude in the bugs that you linked too.
Responding to users bug requests in a timely fashion is critical to making sure you get the appropriate information from them to help fix and reproduce and issue.
Maarten Says:
November 25th, 2008 at 2:33 pm
Hear hear!!
AlBlue Says:
November 25th, 2008 at 4:25 pm
Honestly, the attitude of the people who deal with the bug reports on the other end are a big influence on whether it’s worth coming back for. Case in point for me: bug 194978. Not only had I volunteered to fix it, I raised a subsequent bug because the instructions didn’t work (the documentation assumed a non-Mac OS X system).
After trying - and failing - to get some input on this issue, I tried to get myself dissociated from the bug which led to:
“Alex, if you continue to mis-use the bug system by taking actions that are not yours to take, I will be forced to ask the powers-that-be to revoke your bugzilla permissions. As I said before, this bug is not yours to WONTFIX.”
That’s right, not only do we not want your input, we don’t want your input on this or any future bugs on any other product co-located at this site.
Now, would I have got that response from anyone else but an Eclipse foundation member? Could you see that being an issue on any other project, in fact? It’s no wonder that Monkey died - it was a complete load of Monkey ****.
I’ve not had much to do with Eclipse lately; not specifically through choice, but not exactly going out of my way either. Bug reports are one of the few externally visible ways of involving users (yes, there are newsgroups/mailing lists, but not every casual user will be interested in reading them, and half the time the response is ‘file a bug report’ anyway).
I’ve also got to say - a vast amount of bugs in Eclipse will never be fixed. There just isn’t time. Some of them will get replaced when the next latest and greatest comes out (E4 will no doubt subsume many bug reports) or just plain out ignored (how many bugs, raised in the 2.0/2.1 days, are still open? 780)
Not only that, a vast amount of *user voted* bugs will never get done. Instead of focussing on what users want (shipping a decent Mac bundled app, word wrap in text editors, nested projects etc.) then you have to really ask yourself what the point of raising bugs is.
Now, as with any open-source project, there’s a dividing line between how much effort to put into fixing bugs, adding ‘my own features’ or addressing user concerns. In Eclipse projects, the ‘adding my own features’ seems to vastly outspend ‘fixing other bugs’, unless those bugs get in the way of the ‘my own features’ list. Even bugs with patches don’t get applied - I wrote a kerberized CVS implementaiton, and it’s still sitting around in the bug system with no hope of ever surfacing.
What’s more, once a bug is in a marooned state, then no hope of raising duplicate bugs will raise it from its grave. Indeed, I’ve advised not to raise enhancement requests in the latter stages of milestone releases because they peg the bug in a forever zombie state that is guaranteed to make sure it’s never implemented. And let’s forget the hellhole that was RESOLVED NEVER.
Humans learn from past behaviour, and it’s OK to make mistakes. Learning from mistakes is great; it means you can focus energy on things that will make a difference in the future.
Filing bugs on Eclipse.org? Really not worth anyone’s time.
Tiran Kenja Says:
November 26th, 2008 at 4:15 am
Some eclipse.org projects are pretty responsive to bugs (I have had great experiences with Subversive), while other are downright uncaring about what the users what, at least if it gets in the way of their “vision” (I’m looking at you P2). I suppose it is just a matter of figuring out which is which.
Only problem is that to get a good experience with Eclipse as an IDE you need components from a lot of projects, and you are bound to bang your head on a number that don’t really want your input
Denis Roy Says:
November 26th, 2008 at 11:15 am
For reference, I just saw this Architecture Council bug in my Inbox. FWIW, I didn’t ask anyone to open the bug, and I didn’t forward my post to anyone directly, so people are definitively listening to what we (you and I) have to say.
https://bugs.eclipse.org/bugs/show_bug.cgi?id=256660
Please be gentle with the poor, well-intentioned and attentive Architecture Council
Martin Oberhuber Says:
November 26th, 2008 at 11:51 am
Ha, you beat me posting bug 256660 here since I was cought in a phone call.
I fully agree that the stories heard here are sad, very sad indeed. We’ll not be able to change the rules of the game in terms of priorities for what gets fixed, since that’s guided by who’s ready to actually make contributions and who benefits from it. But I think there is one thing we can change at Eclipse as a Community:
We can make a difference in terms of communication culture. Nobody should have to feel mistreated, neither committers nor users. Technically sound requests should not remain unheard just because of communication challenges. For committers, there is the excellent Mylyn Contributor Reference. Not yet so for users, since they typically won’t find this.
Inventing some kind of grievance handling for users on bugzilla, that’s what bug 256660 is about. I do believe that this could make Eclipse as a whole a better place for all of us. Let’s see where the discussion goes on the bug.
Eric Clayberg - Instantiations Says:
November 30th, 2008 at 10:48 am
What is a support person to do when faced with insufficient information? A stack trace often tells you nothing about what caused a problem or how to fix it. Sure, if an NPE is being thrown, you could add a null check to that line of code, but that will often just shift the problem elsewhere. Reproducible test cases are often critically important to solving problems in complex systems.
When there are more problems being reported than there are people to track them down, any development team will triage incoming reports just like a MASH unit in a war zone. Bugs with their either a high probability of being fixable or those affecting the most people will get the most attention. That means bugs that come with solid info and test cases or bugs that can be easily reproduced with modest info get fixed first. Bugs without test cases where there is little information provided or offered are not going to be treated.
Asking a user to first try the latest build is a perfectly valid initial response in a case like this. No one wants to spend time tracking down a problem that may have already been fixed when faced with other issues that are known to still exist. Establishing that the problem still exists in the latest build is an important step to getting it fixed by the development team. That is even more important when no test case is provided as at least then the development team knows that it is a current issue that they should keep their eyes open for.
When using free, open source software like Eclipse, Apache, etc., the responsibility for tracking down and fixing bugs is *shared* by both users and developers. It is unfair to put the burden completely on the developers as seems to be the case with the referenced bug report. I thought dmitry’s response was unfair and a bit petty. He and his organization are getting the benefit of a huge stack of free, open source software. If something breaks, he should be willing to invest a bit of time in helping the development team reproduce the problem. Providing a test case and trying to reproduce the problem on the latest build are certainly reasonable things to ask of him (especially since the referenced issue only happens “time to time”). His final statement that “I thought you welcome help and want to improve the product but it seems you care only about having less work” was profoundly unfair and will only encourage the development team to completely ignore him in the future.