Bug 274834 - Investigate performance improvements for tests
Summary: Investigate performance improvements for tests
Status: NEW
Alias: None
Product: PDE
Classification: Eclipse Project
Component: Build (show other bugs)
Version: 3.5   Edit
Hardware: PC Windows XP
: P3 normal with 2 votes (vote)
Target Milestone: ---   Edit
Assignee: pde-build-inbox CLA
QA Contact:
URL:
Whiteboard: stalebug
Keywords: performance
Depends on:
Blocks:
 
Reported: 2009-05-04 10:12 EDT by Andrew Niefer CLA
Modified: 2022-11-14 11:41 EST (History)
3 users (show)

See Also:
pascal: review+


Attachments
turn off resolution problem reporting (4.07 KB, patch)
2009-05-04 10:18 EDT, Andrew Niefer CLA
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Andrew Niefer CLA 2009-05-04 10:12:44 EDT
The pde.build test suite is starting to take longer and is cumbersome to run on each change.

This bug is to investigate some changes that could be made to the tests to reduce the amount of time they take.

This may or may not result in performance improvements for regular headless builds.
Comment 1 Andrew Niefer CLA 2009-05-04 10:18:13 EDT
Created attachment 134242 [details]
turn off resolution problem reporting

Patch turns off reporting of resolution errors during tests.  Errors are reported for all bundles in the target that failed to resolve, with p2 it is common to have multiple versions of some bundles, and hence have "another singleton version is selected" warnings.

This saves approximately 10%
Comment 2 Andrew Niefer CLA 2009-05-11 14:07:27 EDT
Attached patch is released, any more improvements will have to wait
Comment 3 Mattias Andersson CLA 2010-11-27 09:26:57 EST
I think this patch is in place in 3.6 but does not work when the eclipse.generateFeature target has the "verify" parameter set to true. reportResolutionErrors is set to the value of "verify" argument in FeatureGenerator.

In productBuild.xml the generateFeature is called twice, first without the verify argument and secondly with the argument. I removed the first call and gained a significant build time enhancement (~30%). My build failed if I set the verify argument to false.

I also removed the line in FeatureGenerator that set reportResolutionErrors according to the verify argument and got rid of all the "Another singleton version selected:" warnings. 

Unfortunately I have no idea what the verify flag does and why the eclipse.generateFeature task is called twice in productBuild.xml?
Comment 4 Andrew Niefer CLA 2010-11-29 11:32:21 EST
eclipse.generateFeature is called twice because we need to have the feature created in order to fetch all the plugins from SCM that are contained in it.  However, we need to have all the plugins that are being included in order to properly generate the os/ws/arch filters on the plugin entries as well as to guess the value for the unpack attribute.

So, the first time, we run with verify=false, which simply creates the feature with no attempt at resolving the plug-ins to get the filters or the unpack attribute.  Then, after the fetch, we run again with verify=true, and this time we resolve the state with all the plug-ins and generate the proper platform filters, as well we will also generate an error if one of the included plugins does not resolve in the state because it is missing dependencies, or if it is missing completely because it didn't get fetched.
Comment 5 Mattias Andersson CLA 2010-11-30 10:22:09 EST
(In reply to comment #4)

Ok, thanks for the info. So I guess I could safely live without the first call since we do not fetch plugins from an external repository.
Comment 6 Andrew Niefer CLA 2010-11-30 11:13:51 EST
If your product is based on features instead of plugins, then the verify=true is not strictly needed, and using verify=false would be faster.

We really only need the second verify=true for plugin based products, for features we don't need filters/unpack and it just throws an error if one of your features is missing and this would happen later anyway.
Comment 7 Lars Vogel CLA 2018-12-03 09:10:44 EST
Currently we are not actively enhancing PDE build anymore. Therefore, I close this bug as WONTFIX. 

Please reopen, if you plan to provide a fix.
Comment 8 Lars Vogel CLA 2018-12-03 09:11:24 EST
Currently we are not actively enhancing PDE build anymore. Therefore, I close this bug as WONTFIX. 

Please reopen, if you plan to provide a fix.
Comment 9 Eclipse Genie CLA 2020-11-23 00:48:32 EST
This bug hasn't had any activity in quite some time. Maybe the problem got resolved, was a duplicate of something else, or became less pressing for some reason - or maybe it's still relevant but just hasn't been looked at yet.

If you have further information on the current state of the bug, please add it. The information can be, for example, that the problem still occurs, that you still want the feature, that more information is needed, or that the bug is (for whatever reason) no longer relevant.

--
The automated Eclipse Genie.
Comment 10 Eclipse Genie CLA 2022-11-14 11:41:38 EST
This bug hasn't had any activity in quite some time. Maybe the problem got resolved, was a duplicate of something else, or became less pressing for some reason - or maybe it's still relevant but just hasn't been looked at yet.

If you have further information on the current state of the bug, please add it. The information can be, for example, that the problem still occurs, that you still want the feature, that more information is needed, or that the bug is (for whatever reason) no longer relevant.

--
The automated Eclipse Genie.