Bug 543161 - Build is taking too long - Comparison 4.2.2 Vs 4.6.3
Summary: Build is taking too long - Comparison 4.2.2 Vs 4.6.3
Status: NEW
Alias: None
Product: JDT
Classification: Eclipse Project
Component: Core (show other bugs)
Version: 4.6   Edit
Hardware: PC Windows All
: P3 normal (vote)
Target Milestone: ---   Edit
Assignee: JDT-Core-Inbox CLA
QA Contact:
URL:
Whiteboard: stalebug
Keywords:
Depends on:
Blocks:
 
Reported: 2019-01-03 19:07 EST by David Christensen CLA
Modified: 2022-12-23 12:10 EST (History)
4 users (show)

See Also:


Attachments
trace.options (1.45 KB, text/plain)
2019-01-03 19:10 EST, David Christensen CLA
no flags Details
output.txt (169.33 KB, text/plain)
2019-01-03 19:14 EST, David Christensen CLA
no flags Details
Build trace for 4.2 on same workspace (191.67 KB, text/plain)
2019-01-08 20:57 EST, David Christensen CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description David Christensen CLA 2019-01-03 19:07:49 EST
Migrating from 4.2.2 to 4.6.3, using same projects and same settings, the workspace build process time has been increased dramatically.

In 4.6.3, Clean All projects is taking ~10 minutes. Developer cannot do any work on already
checked out files while this is going on. Additionally, build All projects is taking 15-16
minutes, and developer cannot do any work on already checked out files while this is going
on.

So Clean+Build in total takes at least ~25 minutes in Eclipse 4.6 during which time, the developer
cannot do any work even on the already checked out files.


In Eclipse 4.2.2, the developer could continue to work on already checked-out files while Clean/Build
All was running.

Also, the building of individual project was almost instantaneous in 4.2; where as in 4.6,
it takes ~60 seconds.


To track where the time goes, trace was enabled on build with 

# turn on debugging for the org.eclipse.core.resources plugin.
org.eclipse.core.resources/debug=true

# reports the start and end of all builder invocations
org.eclipse.core.resources/build/invoking=true


(complete trace options attached)


Resulting in the output.txt (attached)


Few time stamps to note:

 15:37:41 – the first clean build happens

About 10 minutes later, there is the first “Build All”. This will be the first full build manually triggered.

At approximately, 16:49, 16:51 & 16:52 3 different incremental builds were triggered. First one was by Cntrl+B, the last 2 were by setting auto-build flag on and trying to save a file.
Comment 1 David Christensen CLA 2019-01-03 19:10:15 EST
Created attachment 277060 [details]
trace.options
Comment 2 David Christensen CLA 2019-01-03 19:14:21 EST
Created attachment 277061 [details]
output.txt
Comment 3 Andrey Loskutov CLA 2019-01-04 01:47:10 EST
David, please consider to check 4.10. 4.6 is too old an will not be patched anymore.

Looking on the log, you seem to use a lot of web technologies and builders. Are they *all* needed, or other way: can you reduce your problem to a small reproducible example *without* all those web technologies builders?
Comment 4 David Christensen CLA 2019-01-08 20:54:29 EST
(In reply to Andrey Loskutov from comment #3)
> David, please consider to check 4.10. 4.6 is too old an will not be patched
> anymore.
> 
> Looking on the log, you seem to use a lot of web technologies and builders.
> Are they *all* needed, or other way: can you reduce your problem to a small
> reproducible example *without* all those web technologies builders?


This is for a customer using our product that is based on 4.6. His source code should remain private and unfortunately I am still not able to reproduce the slowness in my environment. Migration to a newer version is not possible at this moment.

I am working on the shadows and clues registered in the logs hoping to construct a diagnose out of there. Started reviewing Bug 292126 ('Building workspace' takes horrifyingly long), the enabled trace should be a beginning to track where the time escapes and a better picture of what's going on.

I think technologies used in the projects are orthogonal to the build algorithm. I am not finding or stating a bug on the build process. The aim here is on perf aspects, on what could slows the build. The projects and its technologies are the payloads that remains unchanged between Eclipse versions.

The intuitive knowledge given by reading doc 'Incremental project builders'[1], 
gives a floor for interpreting the trace. Nevertheless it is still blur what's going on. I see Full Build takes too long and Incremental Build is detecting too many deltas. Could you take a look on that aspects? 

Also I am attaching a log that is derived from the same trace.options but this time for 4.2. for comparison purposes.

[1] Incremental project builders - https://help.eclipse.org/neon/topic/org.eclipse.platform.doc.isv/guide/resAdv_builders.htm?cp=2_0_11_1
Comment 5 David Christensen CLA 2019-01-08 20:57:41 EST
Created attachment 277111 [details]
Build trace for 4.2 on same workspace
Comment 6 Eclipse Genie CLA 2020-12-29 05:35:01 EST
This bug hasn't had any activity in quite some time. Maybe the problem got resolved, was a duplicate of something else, or became less pressing for some reason - or maybe it's still relevant but just hasn't been looked at yet.

If you have further information on the current state of the bug, please add it. The information can be, for example, that the problem still occurs, that you still want the feature, that more information is needed, or that the bug is (for whatever reason) no longer relevant.

--
The automated Eclipse Genie.
Comment 7 Eclipse Genie CLA 2022-12-23 12:10:26 EST
This bug hasn't had any activity in quite some time. Maybe the problem got resolved, was a duplicate of something else, or became less pressing for some reason - or maybe it's still relevant but just hasn't been looked at yet.

If you have further information on the current state of the bug, please add it. The information can be, for example, that the problem still occurs, that you still want the feature, that more information is needed, or that the bug is (for whatever reason) no longer relevant.

--
The automated Eclipse Genie.