Bug 199209 - poor performances managing and saving build infos
Summary: poor performances managing and saving build infos
Status: ASSIGNED
Alias: None
Product: CDT
Classification: Tools
Component: cdt-build-managed (show other bugs)
Version: 4.0   Edit
Hardware: PC Windows XP
: P3 normal (vote)
Target Milestone: ---   Edit
Assignee: Project Inbox CLA
QA Contact: Jonah Graham CLA
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2007-08-08 03:29 EDT by Tiziano Leidi CLA
Modified: 2020-09-04 15:20 EDT (History)
3 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Tiziano Leidi CLA 2007-08-08 03:29:19 EDT
Build ID: I20070625-1500

Steps To Reproduce:
1. create a project programmatically
2. add some Resource Configurations with exclusions
3. save build infos


More information:
I moved our plugins to Eclipse 3.3-CDT 4.0 last week and noticed a 
regression in performance when calling following methods:

Configuration.createResourceConfiguration(file);
ManagedBuildManager.saveBuildInfo(project, true);

I use both of them when I programmatically create CProjects in order to set 
file build-exclusions in an automatic way.

What I further noticed is that at the end of project creation the ".cproject" file is 1.5MB big.

The performance for the whole operation of a loop to se exclusions and 
saveBuildInfos changed from 30 seconds to 5 minutes for the creation of a 
project with medium size (100 - 150 files) and 10 different configurations.
Comment 1 Mikhail Sennikovsky CLA 2007-08-08 07:28:45 EDT
I'll have a look..
Comment 2 Mikhail Sennikovsky CLA 2007-08-09 11:28:04 EDT
Hi,

First of all I'd like to mention that if you need to set exclusions/source information only you do not need to create resource configurations.
Instead you could use the IConfiguration.get/setSourceEntries() or ICConfigurationDescription.get/setSourceEntries(). This will significantly speed up your algorithm.

Second of all, to figure out what the bottle neck is I need some more detailed info on your project settings and on how you are using the IManagedBuildInfo API, namely:
1. how many resource configurations are you creating?
2. are you calling the saveBuildInfo only once or each time you add a resource configuration?
3. some test-case or at leas pseudo-code that demonstrates the performance degradation would be very useful.

Thanks,
Mikhail
Comment 3 Mikhail Sennikovsky CLA 2007-09-17 11:49:03 EDT
According to our conversation with Tiziano and my investigations, the slowdown seems to be caused by the Scanner discovery mechanism which gets invoked for each of project's resource configuration for the projects containing hundreds of resource configurations. 
 Since the resource configurations are created for the source exclusion purposes only, their usage should be replaced with the source entries functionality which should solve the problem, so I don't think it's critical for the 4.0.1 knowing that there's only one week left until release. 
 More than that, instead of making additional "hacks" to the scanner discovery functionality I would prefer to have the scanner discovery be re-worked in general.
 
Re-targeting to Future for now..
Comment 4 James Blackburn CLA 2008-06-11 11:47:26 EDT
I'm having a similar problem with both CDT4.0.x and today's HEAD of CDT 5.

Using CDT 5 I've created a new Makefile Project (from some existing sources) using the Linux GCC toolchain and disabled both the scanner config builder, and disabled 'Discovery Options > Automate discovery of paths and symbols' for both the Per Language and Configuration-wide Discovery profiles scope of the Project.

I then create approximates 400 ICResourceDescriptions adding includes and macros to them (common includes / macros are set on parent resourceDesc, however CDT seems to duplicate these macro/include settings when creating children resource descriptions, apparently negating any space saving...).  Performance is really very bad:
Time taken to create and setSettingEntries on 400 items in the project is 42s

Time taken to persist the resource configuration with CoreModel.getDefault().setProjectDescription(project, cfgDesc.getProjectDescription())  is another 94s.

Afterwards the size of the .cproject file has grown from 357 lines to 59690 lines with ~42k lines in the scannerConfiguration storageModule... 

Given that I've disabled both the 'Scanner Configuration Builder' and Automated Discovery of Paths and Symbols, this is pretty bad.

Any pointers on how to disable scanner discovery properly?

Comment 5 Chris Recoskie CLA 2009-02-20 13:16:04 EST
Mikhail isn't working on CDT any more, and I assume he's not working on this, so I'm taking the bug for now as a part of the scalability work.
Comment 6 Chris Recoskie CLA 2009-06-05 10:52:30 EDT
We have reached code freeze on 6.0, and cannot afford to risk stability at this point, so the earliest we will work on this again is for 6.0.1