Community
Participate
Working Groups
API analysis is great thing, but it costs time. I think the builder can run in parallel, after the Java builder, ideally it can "hide" analysis time if main build still runs for other projects. I have POC, and it seems to work, the time for my clean workspace build reduces from ~2 minutes to ~1.5. I will push a patch for discussion.
New Gerrit change created: https://git.eclipse.org/r/c/pde/eclipse.pde.ui/+/186655
Gerrit change https://git.eclipse.org/r/c/pde/eclipse.pde.ui/+/186655 was merged to [master]. Commit: http://git.eclipse.org/c/pde/eclipse.pde.ui.git/commit/?id=3c1c48b4e9caf1ce57eecb6f863c0ec8f9369604
Created attachment 287808 [details] new option added (disabled by default)
IMO, this new option should be turned on by default, or even no option should be given.
(In reply to Mickael Istria from comment #4) > IMO, this new option should be turned on by default, or even no option > should be given. Consider current state as experimental. I will switch option next week, if no major issues with new code will be reported. Similar, if we will get no reports later we may hide it completely from UI or evem remove.
(In reply to Andrey Loskutov from comment #5) > Consider current state as experimental. That's why I suggest forcing this experiment on from now on, to make sure people actually try it ;) But I'm fine with your plan as it is now.
New Gerrit change created: https://git.eclipse.org/r/c/pde/eclipse.pde.ui/+/189484
New Gerrit change created: https://git.eclipse.org/r/c/pde/eclipse.pde.ui/+/189485
Gerrit change https://git.eclipse.org/r/c/pde/eclipse.pde.ui/+/189484 was merged to [master]. Commit: http://git.eclipse.org/c/pde/eclipse.pde.ui.git/commit/?id=07d96b0d31bfebc3417922bfd8117d1afb6e981a
Overall very nice, Thanks! Possible improvements I notice: 1. In my test with workspace clean, when multiple jobs are spanned, it would be good if 'build' job stays on top(and not getting moved below as it started first) 2. Since we run multiple jobs(for API analysis) we can update test option as: "Run API analysis builder as multiple jobs"
Created attachment 287822 [details] In one of the attempts, things seemed to get stuck Steps: 1) Take all pde plugins ( except test plugins and nested projects) 2) Set no baseline 3) Missing baseline ->Error 4) Set the option of "Run API analysis as a job" as on. 5) Clean->Build All Note: When no baseline is set, API analysis still does analysis like API leaks ( for which it needs no baseline). Also it outputs "Missing baseline" problems in all API enabled projects. When I tried to recreate, this didn't reoccur.
(In reply to Vikas Chandra from comment #11) > Created attachment 287822 [details] > In one of the attempts, things seemed to get stuck Please next time create jstack thread dump from Eclipse. Could be a deadlock but could be some error happened - btw, any errors in the log?
(In reply to Andrey Loskutov from comment #12) > (In reply to Vikas Chandra from comment #11) > > Created attachment 287822 [details] > > In one of the attempts, things seemed to get stuck > > Please next time create jstack thread dump from Eclipse. Could be a deadlock > but could be some error happened - btw, any errors in the log? Nothing in the error log !
Created attachment 287824 [details] Another error I re-launched eclipse into workspace with PDE plugins. "Run API analysis as job" option was on.
!MESSAGE File not found: C:\Users\VIKASCHANDRA\git\eclipse.pde.ui\ds\org.eclipse.pde.ds.ui\bin\org\eclipse\pde\internal\ds\ui\wizards\DSFileWizardPage$2.class. !STACK 0 java.lang.Exception: File not found: C:\Users\VIKASCHANDRA\git\eclipse.pde.ui\ds\org.eclipse.pde.ds.ui\bin\org\eclipse\pde\internal\ds\ui\wizards\DSFileWizardPage$2.class. at org.eclipse.core.internal.resources.ResourceException.provideStackTrace(ResourceException.java:42) at org.eclipse.core.internal.resources.ResourceException.<init>(ResourceException.java:38) at org.eclipse.core.internal.localstore.FileSystemResourceManager.read(FileSystemResourceManager.java:835) at org.eclipse.core.internal.resources.File.getContents(File.java:275) at org.eclipse.pde.api.tools.internal.model.ResourceApiTypeRoot.getContents(ResourceApiTypeRoot.java:62) at org.eclipse.pde.api.tools.internal.model.AbstractApiTypeRoot.getStructure(AbstractApiTypeRoot.java:62)
(In reply to Niraj Modi from comment #10) > Possible improvements I notice: > 1. In my test with workspace clean, when multiple jobs are spanned, it would > be good if 'build' job stays on top(and not getting moved below as it > started first) I *guess* this is because of the "A" in the job name that gets this entry sorted before the "B" in the build job. What about "Performing API Analysis" instead of "API Analysis Builder"? > 2. Since we run multiple jobs(for API analysis) we can update test option as: > "Run API analysis builder as multiple jobs" What about this: "Run API analysis parallel to the build job" ?
(In reply to Vikas Chandra from comment #15) > !MESSAGE File not found: > C:\Users\VIKASCHANDRA\git\eclipse.pde.ui\ds\org.eclipse.pde.ds. > ui\bin\org\eclipse\pde\internal\ds\ui\wizards\DSFileWizardPage$2.class. This is most likely because we don't hold workspace lock anymore while API build is runnning, and once the JDT build decides to build project again *while* previously started API analysis is still running, we might see this. I didn't wanted to take workspace lock for a simple reason - I believe JDT build still uses workspace root as build rule, so all API jobs would need to run *after* the main build for all projects is done (so we would not use idling CPU cores) *and* they will still block the IDE if running. I wonder if we check for that and restart the job on such errors? Schouldn't happen very often, and because they would still not hold workspace locks, they should not be blocking for other tasks.
(In reply to Andrey Loskutov from comment #16) > (In reply to Niraj Modi from comment #10) > > Possible improvements I notice: > > 1. In my test with workspace clean, when multiple jobs are spanned, it would > > be good if 'build' job stays on top(and not getting moved below as it > > started first) > > I *guess* this is because of the "A" in the job name that gets this entry > sorted before the "B" in the build job. What about "Performing API Analysis" > instead of "API Analysis Builder"? > > > 2. Since we run multiple jobs(for API analysis) we can update test option as: > > "Run API analysis builder as multiple jobs" > > What about this: > > "Run API analysis parallel to the build job" ? May be it is better if we club all API analysis jobs in a single group during display in progress view? May be setProgressGroup can help. But as of now, it is not that important. Just a name change in preference could suffice as of now.
New Gerrit change created: https://git.eclipse.org/r/c/pde/eclipse.pde.ui/+/189550
New Gerrit change created: https://git.eclipse.org/r/c/pde/eclipse.pde.ui/+/189551
Gerrit change https://git.eclipse.org/r/c/pde/eclipse.pde.ui/+/189550 was merged to [master]. Commit: http://git.eclipse.org/c/pde/eclipse.pde.ui.git/commit/?id=ada4b66bc21c70fa62e8af3ab8cd6786f928574a
Gerrit change https://git.eclipse.org/r/c/pde/eclipse.pde.ui/+/189551 was merged to [master]. Commit: http://git.eclipse.org/c/pde/eclipse.pde.ui.git/commit/?id=0bdda06181d91e6ba636dabc23d9aeab22e4dc52
New Gerrit change created: https://git.eclipse.org/r/c/pde/eclipse.pde.ui/+/189559
Gerrit change https://git.eclipse.org/r/c/pde/eclipse.pde.ui/+/189559 was merged to [master]. Commit: http://git.eclipse.org/c/pde/eclipse.pde.ui.git/commit/?id=18a26604c2697fdf6d9d71cdd0838099eb7d634d
(In reply to Andrey Loskutov from comment #16) > (In reply to Niraj Modi from comment #10) > > Possible improvements I notice: > > 1. In my test with workspace clean, when multiple jobs are spanned, it would > > be good if 'build' job stays on top(and not getting moved below as it > > started first) > > I *guess* this is because of the "A" in the job name that gets this entry > sorted before the "B" in the build job. Nope. Changed the job name, but still the "build" job doesn't stay on top. Must be something else there, I don't see a way how Progress view can be instructed to keep the "build" entry pinned on top. (In reply to Vikas Chandra from comment #18) > May be it is better if we club all API analysis jobs in a single group > during display in progress view? May be setProgressGroup can help. Because the project builder code doesn't know anything outside the project scope of the build (how many other jobs are triggered on which projects), there is no simple solution for that - a job group should be created / updated on one central place. One could probably add some global "Analysis" job group in PDE UI plugin that could be reused by all analysis jobs for all builds.
New Gerrit change created: https://git.eclipse.org/r/c/pde/eclipse.pde.ui/+/189596
(In reply to Eclipse Genie from comment #26) > New Gerrit change created: > https://git.eclipse.org/r/c/pde/eclipse.pde.ui/+/189596 Ensuring ApiAnalysisJob to have a priority of DECORATE ensures that build is always on top. Snippet from doc " Decoration jobs generally compute extra information that the user may be interested in seeing" This looks good for API analysis. @Andrey what do you think?
(In reply to Vikas Chandra from comment #27) > This looks good for API analysis. > > @Andrey what do you think? Sure, great idea, I was not aware the entries were sorted by priority, and the low prio is absolutely OK here.
Gerrit change https://git.eclipse.org/r/c/pde/eclipse.pde.ui/+/189596 was merged to [master]. Commit: http://git.eclipse.org/c/pde/eclipse.pde.ui.git/commit/?id=133980b4600aa07407aa3d8de2391990a54b74c3
Just found a new issue with the jobs - livelock in OverflowingLRUCache, see bug 578204.
New Gerrit change created: https://git.eclipse.org/r/c/pde/eclipse.pde.ui/+/189673
Created attachment 287840 [details] pre and post gerrit 189673
Gerrit change https://git.eclipse.org/r/c/pde/eclipse.pde.ui/+/189673 was merged to [master]. Commit: http://git.eclipse.org/c/pde/eclipse.pde.ui.git/commit/?id=aaf8f712a5c1d51c3a75346a8631019ae2278ccd
New Gerrit change created: https://git.eclipse.org/r/c/pde/eclipse.pde.ui/+/189674
Gerrit change https://git.eclipse.org/r/c/pde/eclipse.pde.ui/+/189674 was merged to [master]. Commit: http://git.eclipse.org/c/pde/eclipse.pde.ui.git/commit/?id=947f2f5e9d0df97bc6a607a024d746ac85150c37
Gerrit change https://git.eclipse.org/r/c/pde/eclipse.pde.ui/+/189485 was merged to [master]. Commit: http://git.eclipse.org/c/pde/eclipse.pde.ui.git/commit/?id=08a998d8bb4d3fc13255c4c01365711274cca880
If this preference has to stay for few releases, we can consider clubbing the 2 API analysis preference in a group ( similar to Test plug-in detection)
Andrey, can this be resolved? We probably need a N&N item for this.
New Gerrit change created: https://git.eclipse.org/r/c/www.eclipse.org/eclipse/news/+/190055
Gerrit change https://git.eclipse.org/r/c/www.eclipse.org/eclipse/news/+/190055 was merged to [master]. Commit: http://git.eclipse.org/c/www.eclipse.org/eclipse/news.git/commit/?id=9b29109a17b3a5a7bb71f255ae2f21f93fb523a5
(In reply to Vikas Chandra from comment #38) > Andrey, can this be resolved? I've just pushed a fix for bug 578386, will see tomorrow if there are more issues or regressions and resolve otherwise. > We probably need a N&N item for this. Done
