Bug 437435 - Provide a Logback-to-Bugzilla Appender
Summary: Provide a Logback-to-Bugzilla Appender
Status: ASSIGNED
Alias: None
Product: z_Archived
Classification: Eclipse Foundation
Component: Recommenders.incubator (show other bugs)
Version: unspecified   Edit
Hardware: PC Mac OS X
: P2 normal (vote)
Target Milestone: ---   Edit
Assignee: Ashwin Jha CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2014-06-14 05:21 EDT by Marcel Bruch CLA
Modified: 2019-07-24 14:36 EDT (History)
3 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Marcel Bruch CLA 2014-06-14 05:21:16 EDT
It would be great to have an appender that logs errors directly to a bugzilla component. What I have in mind is something along these lines:

Whenever an error is logged and the plug-in id starts with "org.eclipse.recommenders.*", a browser should be opened that opens a bugzilla URL and pre fills the fields with the error message, the stack trace etc. An example of such an URL is [1]. This could be extended to support more projects later.

For usability reasons, this appender needs a little UI that (i) asks a user whether he want’s to report that error, (ii) allows him to ignore similar error messages etc. 


The appender could be placed in [2] and make use of the Java 7 Desktop class [3]


[1] https://bugs.eclipse.org/bugs/enter_bug.cgi?product=Recommenders&version=2.0.7&bug_severity=critical&short_desc=Foo&comment=Bar&component=Core&rep_platform=All&op_sys=Linux

[2] https://git.eclipse.org/c/recommenders.incubator/org.eclipse.recommenders.logging.git/tree/plugins/org.eclipse.recommenders.logging.ext

[3] http://docs.oracle.com/javase/7/docs/api/java/awt/Desktop.html#browse(java.net.URI)
Comment 1 Ashwin Jha CLA 2014-06-21 11:17:16 EDT
Marcel,

I have implemented a simple appender that checks whether the logger name starts with a given keyword or not.
If yes, then the bugzilla url is opened in a browser with pre-filled values.

The parameters like search key, bugzilla url, component, product and version are configured in the config file itself.

Now, I have two queries:

1. This will work only, if a valid bugzilla session is present (i.e. the user is currently logged into bugzilla).

2. What should be the strategy in case of a burst of error logs?

Also, can you give me some relevant resources for learning about the UI part. I have seen some links but
couldn't get a clear understanding of it.
Comment 2 Marcel Bruch CLA 2014-06-22 01:37:35 EDT
(In reply to Ashwin Jha from comment #1)
> The parameters like search key, bugzilla url, component, product and version
> are configured in the config file itself.

That sounds quite smart to me - except the version which is constantly changing and thus shouldn't be hardcoded in a hard to change config file.

Another idea: could this appender be registered by a plugin at runtime? E.g. as soon as the logging system it up, some code could register his own appender? Then this idea would become a "proof-of-concept" for dynamically configuring the logging system.

> Now, I have two queries:
> 
> 1. This will work only, if a valid bugzilla session is present (i.e. the
> user is currently logged into bugzilla).

Yes, if the user is not logged in it may not work. I could live with that right now. 

Note: In a long run, we'll likely not log into bugzilla directly but have a (very small) web service (e.g. using spring boot and restful web service api) on the server side that would log the event only if it's new/unknown and otherwise increase the votes or something along these lines.

> 2. What should be the strategy in case of a burst of error logs?


Yes, that is tricky.

First, the logger would have to remember which messages it already logged (e.g. by remembering the exception "type+message+topmost stackframe").

Second, if it gets flooded with events, it should stop opening windows for a while. I'd say (without thinking too much about it), the appender should only send one event every 5 minutes. 

Before the first event is logged, the plugin needs to ask the user whether it should create prefilled bugzilla entries for errors. In that dialog you may ask the user about how many errors it log per hour (may default to 10 or so).

Does that make sense?


An additional idea:
You plug-in may contribute a popup menu for Eclipse Log Events that, whenever you open the context menu (right mouse click) in the error log view on a log event whether it should prepare a bugzilla bug report for this event.

 
> Also, can you give me some relevant resources for learning about the UI
> part. I have seen some links but
> couldn't get a clear understanding of it.


I'm not sure what you are looking for. The simplest dialog I can think of is a Jface MessageDialog. See [1] for reference and let me know what else you need. To open such a dialog from the appender you likely need either Display.asyncExec(Runnable) or use a UIJob. I'd prefer the latter. See [2] for a starting point. HTH





[1] http://www.vogella.com/tutorials/EclipseDialogs/article.html
[2] http://stackoverflow.com/questions/1836025/best-practice-for-passing-values-from-a-non-ui-thread-to-a-ui-thread-in-an-eclip
Comment 3 Andreas Sewe CLA 2014-06-23 04:00:22 EDT
(In reply to Marcel Bruch from comment #2)
> Another idea: could this appender be registered by a plugin at runtime? E.g.
> as soon as the logging system it up, some code could register his own
> appender? Then this idea would become a "proof-of-concept" for dynamically
> configuring the logging system.

I think that's a good idea. In particular, we don't want to hard-code the plugin-ID pattern to "org.eclipse.recommenders.*"; each plugin should be able to register its own pattern.
Comment 4 Ashwin Jha CLA 2014-06-23 05:10:24 EDT
(In reply to Andreas Sewe from comment #3)
> (In reply to Marcel Bruch from comment #2)
> > Another idea: could this appender be registered by a plugin at runtime? E.g.
> > as soon as the logging system it up, some code could register his own
> > appender? Then this idea would become a "proof-of-concept" for dynamically
> > configuring the logging system.

I think this facility is already available in slf4j/logback. I have to check it though. I will post the updates.

> 
> I think that's a good idea. In particular, we don't want to hard-code the
> plugin-ID pattern to "org.eclipse.recommenders.*"; each plugin should be
> able to register its own pattern.

This is in my TODO's list. Basically, each plugin will provide the following parameters to the appender:
1. Array of plugin-ID patterns to search.
2. Required bugzilla parameters like url, product, technology, version.

Please advice any changes or addition.
Comment 5 Ashwin Jha CLA 2014-06-28 11:44:40 EDT
Hi Andreas & Marcel,

Some updates:

- Basic logback-bugzilla appender implemented.
- Dialog box for user permission added.
- User can also choose to ignore all such exceptions.
- For now, I have set a fixed limit of 1 bug per 5 min.

Next on my list:

- Regarding the runtime addition of appender. This can be done from 
  the client plugins as well as a service provided by us. I am thinking 
  of using the later one as it will serve two purposes:
  1. It will provide a basis for the runtime configuration of the logging
     framework (as suggested by Marcel).
  2. This will enable addition of appenders in case of plugins using frameworks
     other than slf4j.
  Implementation: 
  a) A service that will provide an interface between the logback configuration
     and client plugins.
  b) Clients can use this service to change their log level, add/update/delete
     appenders etc.

- With regard to the context menu option in log view. Apparently, both bugzilla
  appender and the pop up menu will be opening a bugzilla url. So, I was
  thinking of moving the relevant piece of code into another service. This will
  also make sense, if in future we may want to extend this to other facilities
  like emailing where the client may want to send the logged exception in an
  email. 

What is your opinion on these points?
Comment 6 Andreas Sewe CLA 2014-06-30 05:46:20 EDT
(In reply to Ashwin Jha from comment #5)
> - Regarding the runtime addition of appender. This can be done from 
>   the client plugins as well as a service provided by us. I am thinking 
>   of using the later one as it will serve two purposes:
>   1. It will provide a basis for the runtime configuration of the logging
>      framework (as suggested by Marcel).
>   2. This will enable addition of appenders in case of plugins using
> frameworks
>      other than slf4j.
>   Implementation: 
>   a) A service that will provide an interface between the logback
> configuration
>      and client plugins.
>   b) Clients can use this service to change their log level,
> add/update/delete
>      appenders etc.

Having a service (e4-injectable?) to change your configuration at runtime sounds great. It would also be great if a plugin need not (if no runtime-changes are desired) write code to configure its appender. In that case, simply declaring your desired configuration in your plugin.xml should be enough. The o.e.r.logging.rcp bundle would then go through all plugin.xmls, read the configuration therein, and perform the necessary service calls. That way, everything goes through the service API but clients who don't want the full flexibility can fall back onto a simple, declarative approach. 

> - With regard to the context menu option in log view. Apparently, both
> bugzilla
>   appender and the pop up menu will be opening a bugzilla url. So, I was
>   thinking of moving the relevant piece of code into another service. This
> will
>   also make sense, if in future we may want to extend this to other
> facilities
>   like emailing where the client may want to send the logged exception in an
>   email. 
> 
> What is your opinion on these points?

I think that's a good idea in general. But can you please sketch the services interface so that I have a clearer idea what we are talking about specifically? Thank you.
Comment 7 Ashwin Jha CLA 2014-06-30 07:37:14 EDT
(In reply to Andreas Sewe from comment #6)
> (In reply to Ashwin Jha from comment #5)
> > - Regarding the runtime addition of appender. This can be done from 
> >   the client plugins as well as a service provided by us. I am thinking 
> >   of using the later one as it will serve two purposes:
> >   1. It will provide a basis for the runtime configuration of the logging
> >      framework (as suggested by Marcel).
> >   2. This will enable addition of appenders in case of plugins using
> > frameworks
> >      other than slf4j.
> >   Implementation: 
> >   a) A service that will provide an interface between the logback
> > configuration
> >      and client plugins.
> >   b) Clients can use this service to change their log level,
> > add/update/delete
> >      appenders etc.
> 
> Having a service (e4-injectable?) to change your configuration at runtime
> sounds great. It would also be great if a plugin need not (if no
> runtime-changes are desired) write code to configure its appender. In that
> case, simply declaring your desired configuration in your plugin.xml should
> be enough. The o.e.r.logging.rcp bundle would then go through all
> plugin.xmls, read the configuration therein, and perform the necessary
> service calls. That way, everything goes through the service API but clients
> who don't want the full flexibility can fall back onto a simple, declarative
> approach. 
> 

Thanks for this nice suggestion Andreas. This will surely help the users a lot.

> > - With regard to the context menu option in log view. Apparently, both
> > bugzilla
> >   appender and the pop up menu will be opening a bugzilla url. So, I was
> >   thinking of moving the relevant piece of code into another service. This
> > will
> >   also make sense, if in future we may want to extend this to other
> > facilities
> >   like emailing where the client may want to send the logged exception in an
> >   email. 
> > 
> > What is your opinion on these points?
> 
> I think that's a good idea in general. But can you please sketch the
> services interface so that I have a clearer idea what we are talking about
> specifically? Thank you.

Okay, I will try to implement a basic version of both these services. From there we can add on.
Comment 8 Marcel Bruch CLA 2014-07-02 08:52:42 EDT
Please keep this bug focused on discussions about the bugzilla appender. Thx.

Regarding the current implementation (with the modal popup dialog) two ideas I'd like to get your feedback on:

1. how about using more lightweight popups similar to Mylyn which also disappear if the user ignores them? See [1, 2] for code and examples (the dependency to mylyn shouldn't be pulled in later - copying the class for now would be okay IMHO). 

2. Since most plugins *currently* log to the EclipseLog rather than using Logback, should we register a LogListener to the Eclipse log instead of / in addition to defining a Logback Appender? When listening to the Eclipse Log we'd also get errors of other Eclipse Plugins (which I'd be interested for Mars Milestone builds). 

3. A context menu in the Eclipse Error Log View "Report this error to Bugzilla" would be sweet.

Marcel



[1] org.eclipse.mylyn.commons.ui.dialogs.AbstractNotificationPopup
[2] http://krishnanmohan.wordpress.com/category/mylyn-notifications/
Comment 9 Andreas Sewe CLA 2014-07-02 09:03:50 EDT
(In reply to Marcel Bruch from comment #8)
> Regarding the current implementation (with the modal popup dialog) two ideas
> I'd like to get your feedback on:
> 
> 1. how about using more lightweight popups similar to Mylyn which also
> disappear if the user ignores them? See [1, 2] for code and examples (the
> dependency to mylyn shouldn't be pulled in later - copying the class for now
> would be okay IMHO). 

That would be worth a try as it's less obtrusive. It would certainly need some tweaking w.r.t. to the time the popup is shown and whether multiple popups can accumulate or are even aggregated ("There were 3 errors". Click here to file bugs for them?"), as it's pretty random with the current implementation for which error a popup is shown and for which none is shown.

> 2. Since most plugins *currently* log to the EclipseLog rather than using
> Logback, should we register a LogListener to the Eclipse log instead of / in
> addition to defining a Logback Appender? When listening to the Eclipse Log
> we'd also get errors of other Eclipse Plugins (which I'd be interested for
> Mars Milestone builds). 

If all logback messages end up in the Error Log, then registering only for EclipseLog messages makes sense, as that's the final sink of the log event stream. The logback part would then essentially be our event collector, collecting entries from the different low-level logging frameworks (SLF4J, JUL, etc.) and feeding that into the higher-level logging framework (EclipseLog). Having too much functionality at this routing level (BugzillaAppender) feels wrong to me.

> 3. A context menu in the Eclipse Error Log View "Report this error to
> Bugzilla" would be sweet.

Agreed.
Comment 10 Marcel Bruch CLA 2014-07-02 09:07:12 EDT
(In reply to Marcel Bruch from comment #8)
> 1. how about using more lightweight popups similar to Mylyn [...]

Worth looking at: org.eclipse.jface.dialogs.PopupDialog
The "New updates are available" popup is subclassing this one (org.eclipse.equinox.internal.p2.ui.sdk.scheduler.AutomaticUpdatesPopup)
Comment 11 Ashwin Jha CLA 2014-07-06 03:22:15 EDT
To summarize all the thoughts:

1. Look at the PopUp dialog implementation. See if this can be used in a way that provides better performance than the current dialog implementation. 

2. Log listener for Eclipse log.

3. Runtime addition of appender or read from plugin.xml.

4. Re-look implementation of Message Filtering.
Comment 12 Ashwin Jha CLA 2014-07-07 05:41:21 EDT
(In reply to Ashwin Jha from comment #11)
 To summarize all the thoughts:
 
 1. Look at the PopUp dialog implementation. See if this can be used in a way
 that provides better performance than the current dialog implementation. 
 
 2. Log listener for Eclipse log.
 
 3. Runtime addition of appender or read from plugin.xml.
 
 4. Re-look implementation of Message Filtering.

 5. Add context menu.
Comment 13 Ashwin Jha CLA 2014-07-07 05:43:41 EDT
(In reply to Ashwin Jha from comment #12)
> (In reply to Ashwin Jha from comment #11)
>  To summarize all the thoughts:
>  
>  1. Look at the PopUp dialog implementation. See if this can be used in a way
>  that provides better performance than the current dialog implementation. 
>  
>  2. Log listener for Eclipse log.
>  
>  3. Runtime addition of appender or read from plugin.xml.
>  
>  4. Re-look implementation of Message Filtering.
I have changed the implementation such that one appender will handle only one keyword.
> 
>  5. Add context menu.
Comment 14 Ashwin Jha CLA 2014-07-07 06:12:33 EDT
Regarding burst errors, I would like to propose the following solution:

1. Keep a queue of error events that will store all the errors generated during an interval of time (say 5 min.). While adding, if an "Identical" event is present current event will be dropped.

2. At the end of the given interval, show a popup dialog asking for user's permission to create bugs for these errors.

3. If the user selects to drop these or does not respond within the fade away time then the queue will be flushed out. After the addition of context menu, user can anyway create bugs for errors that went unnoticed due to the automatic fading of dialog.

4. If the user selects bug creation then start a job that will create these bugs.

"Identical": For now, we can use the combination of log message + top most element of the stack trace.(As suggested by Marcel in an earlier comment)

What is your opinion on this?
Comment 15 Ashwin Jha CLA 2014-07-09 04:11:58 EDT
(In reply to Marcel Bruch from comment #10)
> (In reply to Marcel Bruch from comment #8)
> > 1. how about using more lightweight popups similar to Mylyn [...]
> 
> Worth looking at: org.eclipse.jface.dialogs.PopupDialog
> The "New updates are available" popup is subclassing this one
> (org.eclipse.equinox.internal.p2.ui.sdk.scheduler.AutomaticUpdatesPopup)

I think PopupDialog will be good in our case. We can use it to create a custom
dialog that will have an additional feature of accumulating errors related to a particular keyword.