Bug 432695 - [patch][performance] All project API descriptions get saved for every project close
Summary: [patch][performance] All project API descriptions get saved for every project...
Status: RESOLVED FIXED
Alias: None
Product: PDE
Classification: Eclipse Project
Component: API Tools (show other bugs)
Version: 4.4   Edit
Hardware: All All
: P3 normal (vote)
Target Milestone: 4.5 M4   Edit
Assignee: Marc-André Laperle CLA
QA Contact:
URL:
Whiteboard:
Keywords: performance
Depends on:
Blocks:
 
Reported: 2014-04-13 13:49 EDT by Marc-André Laperle CLA
Modified: 2015-05-21 13:50 EDT (History)
3 users (show)

See Also:


Attachments
New patch (2.33 KB, patch)
2014-11-18 04:07 EST, Vikas Chandra CLA
no flags Details | Diff
Patch that also ignores non-java projects (2.75 KB, patch)
2014-11-19 13:03 EST, Marc-André Laperle CLA
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Marc-André Laperle CLA 2014-04-13 13:49:44 EDT
Using Eclipse 4.4-I20140408-1645.

The method ApiDescriptionManager.saving saves all modified project API descriptions even for a save context with the kind ISaveContext.PROJECT_SAVE. To illustrate this problem, one can follow those steps:

1. Import a large number of projects with API tooling enabled. For example, all projects from git://git.eclipse.org/gitroot/cdt/org.eclipse.cdt.git
2. Make sure all the projects are in a "modified" state by doing a "Clean all projects" and waiting for the build to complete
3. Insert a breakpoint on the line Util.saveFile in ApiDescriptionManager.saving or add a print statement.
4. Select all the project and close them.

Result: For every project that gets closed, all project API descriptions get saved. Instead, I think only the project API description for the project in the ISaveContext should be saved when the saving is of kind ISaveContext.PROJECT_SAVE.

Here are the results before and after the change for the scenario described above:
time before handling PROJECT_SAVE: 37173 ms
time after: 29682 ms

By extension, if a General Project (non-java, no API tooling) in the same workspace gets closed, a measurable difference can also be seen:
time before handling PROJECT_SAVE: 120 ms
time after: 33 ms

The times are measured be comparing the time after and before the WorkspaceJob in WorkspaceAction.
Comment 1 Marc-André Laperle CLA 2014-04-13 13:54:29 EDT
Patch:
https://git.eclipse.org/r/#/c/24917/
Comment 2 Vikas Chandra CLA 2014-04-22 06:59:51 EDT
I am looking at the proposed fix.
Comment 3 Curtis Windatt CLA 2014-04-22 17:11:03 EDT
Mike, do you have any thoughts on this change?  It looks quite reasonable to me and Darin's original code hasn't been modified (we have never considered what is in the save context).
Comment 4 Curtis Windatt CLA 2014-04-23 13:46:29 EDT
The more interesting issue is why the api descriptions are marked as modified, but can never be unmodified.  If the api descriptions are modified, they should be saved in the save participant, but not repeatedly.

A better solution would be to move the save/write code into ProjectApiDescription and have it clear the modified flag.

Inspecting the save context might provide some additional time savings, but it could lead to odd behaviour where dependent projects are not saved.  Perhaps it wouldn't cause any issue, but can't spend a lot of time testing it for the minimal performance boost.
Comment 5 Curtis Windatt CLA 2014-05-13 16:04:12 EDT
Moving to 4.5. I haven't had time to investigate and this change would have too much risk for the RC builds.
Comment 6 Vikas Chandra CLA 2014-11-18 04:07:49 EST
Created attachment 248717 [details]
New patch

With the proposed fix, api descriptors of org.eclipse.cdt.debug.ui.memory.transport etc are never saved.

On the lines of comment#4 from Curtis, this is another fix that can solve the performance issue. Once saved, the modify flag should be cleared and further save of the same file should happen only when the modify flag is set to true.
Comment 7 Marc-André Laperle CLA 2014-11-19 11:24:33 EST
(In reply to Vikas Chandra from comment #6)
> Created attachment 248717 [details]
> New patch
> 
> With the proposed fix, api descriptors of
> org.eclipse.cdt.debug.ui.memory.transport etc are never saved.
> 
> On the lines of comment#4 from Curtis, this is another fix that can solve
> the performance issue. Once saved, the modify flag should be cleared and
> further save of the same file should happen only when the modify flag is set
> to true.

Thanks Vikas, it looks good.

(In reply to Curtis Windatt from comment #4)
> Inspecting the save context might provide some additional time savings, but
> it could lead to odd behaviour where dependent projects are not saved. 
> Perhaps it wouldn't cause any issue, but can't spend a lot of time testing
> it for the minimal performance boost.

Could we at least check if the project in the save context is a Java project? That should be safe I would think. That way, other kinds of projects won't get a performance penalty because of the presence of PDE projects.
Comment 8 Marc-André Laperle CLA 2014-11-19 13:03:28 EST
Created attachment 248765 [details]
Patch that also ignores non-java projects
Comment 9 Vikas Chandra CLA 2014-11-22 05:24:11 EST
Thanks Marc-Andre

Tests looks fine. Initial scenario works fine. Duplicate saves are not happening. There is a performance improvement without changing the behavior.

Committed  "Patch that also ignores non-java projects"  in masters ( 4.5) via 

http://git.eclipse.org/c/pde/eclipse.pde.ui.git/commit/?id=404cfaf8830753778d2c2463428cbc0b006e479e
Comment 10 Marc-André Laperle CLA 2014-11-24 11:15:46 EST
(In reply to Vikas Chandra from comment #9)
> Committed  "Patch that also ignores non-java projects"  in masters ( 4.5)
> via 

Great, thank you for looking into this!