Community
Participate
Working Groups
(This bug was spun off of bug 124964) When an internal error occurs in Firefox, or in Microsoft applications, users have an easy way to send an automatically generated error report back to the vendor for quality feedback purposes. In some cases, the information is analyzed right away and a web page describing the issue and a possible resolution is opened. By entering this bug, I hope to start a discussion about whether we can implement something like this for Eclipse. Note that due to the open nature of Eclipse applications, error reports might be caused by third-party plug-ins. (It might still be good for Eclipse as a whole to accept those error reports.) There are many more issues that would have to be discussed, ranging from privacy issues, technicalities like who is paying for the bandwidth and storage, to the question whether this will generate enough value for the end user to warrant the costs involved. What do you think? For an answer to the obvious question: why don't we automatically generate bug reports in bugzilla, see bug 124964 comment #14.
Couple comments 1) Windows has link to support from add/remove program. Maybe we could use the help -> About and add hooks for plugin developer to add specific support info ? 2) When an error occurs, why don't we pass the error to the list of components/plugins in the stack trace and see if they want to handle it ? Each component could implement a 'handle error' method (providing the VM is not crashing :)
I find Christof's 2) suggestion very interesting. It's often quite hard to determine whether an error is caused by a plug-in or Eclipse itself, and even harder to reach the plug-in developer once it proves (yet again) not to be Eclipse's fault. Users and plug-in developers could be largely convenienced by such a functionality, reducing bug fix times.
Technically, an enhancement request.
Mylar provides this facility: from any Error Log entry the user can invoke "Report as Bug", which will create a bug report with the contents of the error event. If the event has a stack trace they are encouraged to click "Search for Duplicates", which will search the entire bug repository for matching bug reports and allow the match to be opened so that they can submit a comment instead. See: http://wiki.eclipse.org/index.php/Mylar_User_Guide#Report_Bugs_from_Error_Log http://wiki.eclipse.org/index.php/Mylar_User_Guide#Automatic_Duplicate_Detection Other than general duplicate detection improvements (Mylar bug 161866) There are are a couple of improvements that that would make this better for Europa: 1) Automatically corresponding plug-in IDs with Bugzilla products via an extension point (bug152420) 2) Integrate this with the error dialog. 3) Provide a screen capture facility (easy to do) and automatically add attachment to bug Any lightweight solution to this hits the problems discussed on bug 124964. However, I believe that most of the problems raised on that bug are addressed with Mylar's support for Task/bug repositories, and our users are making use of this feature. The duplicate detection keeps us from getting overwhelmed. So now I'm wondering if there is a simple way for us to make this available to Europa users who have Mylar installed without requiring Europa projects to couple to mylar. For (1) above I'm thinking that we could provide a default mapping for eclipse.org, not requiring Europa projects to couple to the Mylar extension point? For (2) I'm hoping that we get this by extending bug 154090 if it supports extensibility of the dialog?
Another option would be to log the stack dumps in a separate (not bugzilla) database and the projects could peruse that database and write bugzilla entries themselves. This would remove the user (who doesn't know what's going on) from the workflow of writing the bug. It would support the "1. Quality Feedback" notion in https://bugs.eclipse.org/bugs/show_bug.cgi?id=124964#c12
(In reply to comment #5) +1 from me for a special-purpose database. I think it would be great for all projects if the Eclipse Foundation hosted a database like that. Once we have a story for the server-side of this, we will have to look into how to do add the corresponding piece to the Eclipse client. The code should be in the Platform somewhere (Equinox & Platform UI?). It would be great though if we could get help from committers on other projects, or contributions from the greater community. Surely this kind of code has been written too many times already.
(In reply to comment #6) > I think it would be great for all > projects if the Eclipse Foundation hosted a database like that. Consider this comment #7 as an agreement in principle to host such a database at an eclipse.org server. Of course I have no idea exactly what the database should store just yet (you all will need to specify it) but we've got disk space (for the sake of cost, let's say that it's not going to be backed up) and we've got upload bandwidth (it's only the download bandwidth that gets maxed).
IBM is developing a set of technology based on Eclipse. They call it Jazz. http://jazz.net There is an enhancement in Jazz that also asks for feedback to the server. https://jazz.net/jazz/web/projects/Jazz%20Project#action=com.ibm.team.workitem.viewWorkItem&id=26784 Jazz has a server feature, which makes it easier for them. But they have the same questions: 1) what to send 2) Where to store it The way I see it right now is that we have a concept of a StatusHandler in Eclipse 3.3 that should take care of the the IStatus in the .log and in ErrorDialog. I also believe we can get the JobLog info. Of course this only works for UI issues. We were thinking in putting it into Equinox but it may be easire to listen to the ILog and then send it to the server. The StatusHandler could send the info to a server (to be configurable), using HTTP or other protocol. We have a Proof Of Concept that transforms a IStatus into an XMLMemento and sends it to the known servlet. What the servlet does is up to the server implementer (Store in database, in file locally, in remote file etc etc) So, what do we do ? I can develop the client side, but what contract will we have with the eclipse server ?
Sorry more food for thoughts 1) how extensible will it be ? Some companies do not have access to the internet and have drastic security policies. We need to allows companies to easily remove the feedback mechanism OR redirect it to their database 2) What will the UI be ? I am tired of clicking 'no' or 'yes' everytime I get an error. There should be a preference that states - always yes or always no 3) What is the expectation we give the user ? Is it : 'We will send the data back to the server but you will receive no confirmation' or is it ' We will ask the server if this is a known issue and we will let you know' - In the second case, we need to do better than other, as usually they only return valid info 10% of the time 4) What about 'asking the community' if I get an error. The use case would be, I have an error, I search, nothing comes back, I send to the server and then i 'publish to the community site (yet to be determine)'. When someone gives an answer we (the community) tag it, which then can be found easily during the search of the next user... Thoughts ? (Adding Krzystof so he can implement that)
(In reply to comment #8) > So, what do we do ? I can develop the client side, but what contract will we > have with the eclipse server ? I am willing to work with you (or any other contributor) to develop the server side for this. Basically we need to work together (you propose something first?) to define what this will look like on the server side. What does the database store? How is it indexed? What is the server-side API for uploading the data? Etc. (In reply to comment #9) > 1) how extensible will it be ? I agree that there needs to be a preference setting to turn this on and off and where the data gets sent. > 2) What will the UI be ? I am tired of clicking 'no' or 'yes' everytime I get > an error. There should be a preference that states - always yes or always no I agree. > 3) What is the expectation we give the user ? I think we should follow the de facto standard for this kind of thing and send the information and then show the user if this is a known problem. > 4) What about 'asking the community' if I get an error. It's probably easiest to tag the stack dump/error report in the database with a bugzilla number and then the response to the user is "known problem NNN, click here to read about what is being done to solve it". Something like that.
(In reply to comment #9) > 1) how extensible will it be ? IMHO there should be a preference option that must be set to turn it on (default would be off in general, could be set to on for the Eclipse IDE). > 2) What will the UI be ? I am tired of clicking 'no' or 'yes' everytime I get > an error. There should be a preference that states - always yes or always no From a scalability point of view (remember the Mylyn Bugzilla problem) I would opt for an asynchronous model (similar to what is in Vista), i.e. multiple problems could be collected, aggregated/summarized and then run through the server in a single batch once a week/once a day or one demand (if the problem is severe enough). > 3) What is the expectation we give the user ? Is it : 'We will send the data > back to the server but you will receive no confirmation' or is it ' We will ask > the server if this is a known issue and we will let you know' - In the second > case, we need to do better than other, as usually they only return valid info > 10% of the time That requires a smart server IMHO. But it could be an optional model, i.e. we send in the background and collect answers and only inform the users if we have a good match and a good solution. The solution could also be integrated in the usual error dialog. Thus, if the problem occurs a second time and we already have matching solutions locally we show them to the user. > 4) What about 'asking the community' if I get an error. The use case would be, > I have an error, I search, nothing comes back, I send to the server and then i > 'publish to the community site (yet to be determine)'. When someone gives an > answer we (the community) tag it, which then can be found easily during the > search of the next user... That looks like a nice model for an extension, i.e. communities can hook their own "solution provider" into the client (a plug-in). Maybe the Eclipse.org solution provider should be a plug-in that is bundled together with the Eclipse IDE but not RCP (by default).
(In reply to comment #0) > Note that due to the open nature of Eclipse applications, error reports might > be caused by third-party plug-ins. (It might still be good for Eclipse as a > whole to accept those error reports.) Not sure if everyone on this bug realizes the consequences - if we put a mechanism like this in place, errors might be sent to eclipse.org from all kinds of applications that are built on Eclipse. I don't think this is bad, in fact I think it would be great, but thought I should point this out. After all, the Eclipse member companies are building those applications, and they pay for the Foundation and its servers, bandwidth, etc. Perhaps this is an argument for restricting access to that database to committers and member companies' representatives? As for a smart server, and pointing to solutions - my suggestion would be to start with something very simple, such as opening a web page with the error details such as the error message, stack trace, Eclipse build ID etc, a link to a corresponding bugzilla search, and a link to the 'enter a bug' page where some of the fields would be filled out based on the error information. It should be possible to do this in a way that bugs entered in this way can be found by subsequent queries for the same kind of error. There are still lots of questions though, and that's why I asked for contributions from someone who already has experience with automated error reporting. It is not trivial, see also [1] and [2]. [1] http://kb.mozillazine.org/Mozilla_Quality_Feedback_Agent [2] http://mozillalinks.org/wp/2006/09/airbag-to-be-added-to-mozilla-applications/
When a company builds a product on top of Eclipse, stack traces from that product (which would not be open source) would also be sent to the server. I think it would be natural if companies who develop products on top of eclipse could specify a server URL where they want the traces to be sent. Naturally, the server technology would need to be Open Source and easy to set up for respective companies. Server URIs should be configurable on a per-plugin basis. Given a stack trace, the quality feedback client should walk down the stack frames until it finds a frame that it can associate with a plugin which has a feedback server URI associated.
(In reply to comment #13) > When a company builds a product on top of Eclipse, stack traces from that > product (which would not be open source) would also be sent to the server. -1 for sending these kind of stack traces to the Eclipse.org server. It would pollute the database with 3rd party bugs/problems. > Server URIs should be configurable on a per-plugin basis. Given a stack trace, > the quality feedback client should walk down the stack frames until it finds a > frame that it can associate with a plugin which has a feedback server URI > associated. I don't think that it can be this generic. Imagine the following scenario. A product may use the IResource API under the cover but does not expose any of the concepts like workspace, projects, folders & files in the UI. Even the views are not available. The UI just deals with business objects and business views. Resources are managed completely transparent in the background. They are just an implementation detail in the product, not a usage concept. Thus, any Eclipse.org solution for exception thrown by the IResource API (like out of sync, etc.) are completely out of scope. IMHO all stack traces and errors should be handled by a pluggable problem analyzer. The product vendor itself should be responsible for defining the analyzation strategy. There would be a Eclipse.org analyzer which runs problems through an Eclipse.org server. Adopters may choose to use this analyzer together with their own analyzer (talking to their own servers). However, in this case the UI should make it clear that some solutions come directly from Eclipse.org and may not apply to the current product.
(In reply to comment #12) > if we put a > mechanism like this in place, errors might be sent to eclipse.org from all > kinds of applications that are built on Eclipse. Yes. * Companies that build products on top of Eclipse would either redirect this or turn it off. * Perhaps we could have a per-plug-in filter (supplied by the plug-in) that would obfuscate the stack trace for that plug-in, e.g., replace Foo >> bar(x,y) with **** >> ****. So then secrets would not be revealed. Hmmm. > Perhaps this is an argument for > restricting access to that database to committers and member companies' > representatives? Do you mean restricting the posting of error traces to just members? I think that would limit the usefulness of the database. Or do you mean restricting the read access to the error traces? That would make it harder to a non-committer to become a contributor to a project, especially when projects start to say "N% of our traces (see database) have X problem". Restricting the ability to join a project would not be very 'open source'. > As for a smart server, and pointing to solutions - my suggestion would be to > start with something very simple, such as opening a web page ... > a link to the 'enter a bug' page where > some of the fields would be filled out based on the error information. My thought was even simpler: we just log the stack traces and don't involve the user in filing a bug. Then the project teams can query the stack trace database (perhaps automatically with scripts?). When the team starts get lots of traces that show a particular problem, someone *on the team* enters the bug. (In reply to comment #13) > Naturally, > the server technology would need to be Open Source and easy to set up for > respective companies. Not only open source, but EPL. (In reply to comment #14) > It would > pollute the database with 3rd party bugs/problems. Hence my proposal that these not file bugs but rather store traces in a separate database and let committers (experts) take the information from the trace-database and write bugs. > However, in > this case the UI should make it clear that some solutions come directly from > Eclipse.org and may not apply to the current product Or the company could disable the connection to the eclipse.org servers for their product.
(In reply to comment #14) > -1 for sending these kind of stack traces to the Eclipse.org server. It would > pollute the database with 3rd party bugs/problems. I don't think you can avoid this. For example, JFace - being near the bottom of the software stack - throws exceptions when illegal arguments are passed to one of its methods. Almost always, these kinds of problems are a 3rd party bug. Imagine a user who has the Eclipse SDK, WTP, and Adobe's closed source Javascript editor. An exception is thrown in JFace. This might be due to a bug in JFace, the SDK, WTP, or Adobe's code, and there is no automatic way to tell who caused it. So all you could do is try to prevent *some* of those 'foreign' stack traces to be sent to the eclipse.org server, but pollution will always be something you have to deal with. IMO it is more efficient to think about how to make best use of the polluted data than to try to avoid some of the pollution.
(In reply to comment #15) > Or do you mean restricting the read access to the error traces? That would make > it harder to a non-committer to become a contributor to a project, especially > when projects start to say "N% of our traces (see database) have X problem". > Restricting the ability to join a project would not be very 'open source'. Yes, that's true. I didn't really explain what I had in mind, and perhaps I was too pessimistic: I thought about a non-Eclipse member company that builds a product using Eclipse technology and then uses the error database to monitor problems in its own (closed source) code. > Hence my proposal that these not file bugs but rather store traces in a > separate database and let committers (experts) take the information from the > trace-database and write bugs. Yes, makes sense, however: When uploading error data, you need to ask the user for permission. It would be great if there was a direct benefit to those who agree to share their data with eclipse.org. > * Perhaps we could have a per-plug-in filter (supplied by the plug-in) that > would obfuscate the stack trace for that plug-in, e.g., replace Foo >> bar(x,y) > with **** >> ****. So then secrets would not be revealed. Hmmm. Agree that something like this should be part of the story.
In reply to comment #10) >I am willing to work with you (or any other contributor) to develop the server >side for this. Basically we need to work together (you propose something >first?) to define what this will look like on the server side. What does the >database store? How is it indexed? What is the server-side API for uploading >the data? Etc. Thanks, I think I will do a very simple proof of concept that will serialize a IStatus and log it on a server log. Then we can use the ErrorView to look at it. Will be M1 (In reply to comment #10) >I agree that there needs to be a preference setting to turn this on and off and >where the data gets sent. Should be in M2 (In reply to comment #10) >I think we should follow the de facto standard for this kind of thing and send >the information and then show the user if this is a known problem. M2 - M2 will be UI (In reply to comment #10) >It's probably easiest to tag the stack dump/error report in the database with a >bugzilla number and then the response to the user is "known problem NNN, click >here to read about what is being done to solve it". Something like that. M4 or 5 - Tagging/FingerPrinting is something interesting. Retrieveing the exact match of a fingerprint is challenging. (In reply to comment #11) >From a scalability point of view (remember the Mylyn Bugzilla problem) I would >opt for an asynchronous model (similar to what is in Vista), i.e. multiple >problems could be collected, aggregated/summarized and then run through the >server in a single batch once a week/once a day or one demand (if the problem >is severe enough). Ok, so the use case will be: Never send never ask again, Send always and do not ask or Ask all the time Ask all the time should be non-pervasive - small, hidden yet quickly available ? Then there is also a 'send batch daily- weekly' which could be independently turned on or off right ? (In reply to comment #11) Point #3 and #4 - Yep, we can do that later, after we have the basic framework working right ? :) **************** Me and Krzystof will try to look at a simple yet extensible transport mechanism We should rebase the UI Feedback plugin to it We will also do a Proof of concept (basic) to show how to send error messages The extensions of the ErrorDialog/StatusHandler already exist so companies can develop their own components. - Filter sending/obfuscating - is it really a need ? (I do not know so I ask the question :) - Show response in ErrorDialog or other (extensions already exists) - find already known issue (need smart server - Companies can extend the ErrorDialog with their branding) - Securing the data in eclipse for companies that pay eclipse... ummm - I let eclipse.org decide ! :) - Analysis of the stack trace by extension points - IMHO a huge pitfall :) Or at least not *that* easy. I agree with Boris, let's have the 'banded' product decide -
I know I'm 8 years late. But I'd consider this as fixed for Eclipse Mars [1]. [1] https://dev.eclipse.org/recommenders/community/confess/#/about
