Community
Participate
Working Groups
I am currently using M5 we have a fairly large project approx 7000 source files split into about 250 packages plus about 20 jar files containing about 3MB of code. I am running on windows NT Service pack 6 with a 2Ghz processor and 1GB of RAM with vm params =-Xmx640m -Xms640m. Eclipse is running using JDK1.4.2_02. The problem I am facing is that build performance seems to have drastically slowed down in the 3.x builds compared to a 2.1 build. The entire UI and the machine seems to come to a stop when a build is occurring or when updating from a cvs sync. Keeping the progress view open I notice that there seems to be (too frequently) updates to the decorations during a build and the decoration update seems to take a very long time before the next task becomes active in the progress view. I notice that this also occurs when doing long operations like organize imports on an entire source tree. Maybe the decoration update could be delayed till the long operation is complete. The package explorer also seems to have slowed down. I have a package with about 80 sub packages each of which have 4 sub packages themselves. Just clicking on the expand of the package node of this package takes more than 35 secs.
Here are some numbers: All tests run with the exact same workspace contents on windows NT Service pack 6 with a 2Ghz processor and 1GB of RAM with vm params =-Xmx640m -Xms640m. Eclipse is running using JDK1.4.2_02. R2.1.2 : Project RebuildAll : 3mins 15secs 3.0 M5 : Project RebuildAll : 6 mins 15 secs R2.1.2 : Expand Package Explorer (80 packages Hierarchical layout) : 25 secs 3.0 M5 : Expand Package Explorer (80 packages Hierarchical layout) : 45 secs
CC'ing JohnA who may be interested by these numbers. John: isn't the build process atomic ? If so, how are decorators involved during the build ?
Sanjay - how do numbers look like if turning off decorators ? (Preferences>Workbench>Label decorations)
Phillipe, I just noticed that my earlier number comparison was not a fair comparison since on the 2.1.2 image I had the CVS label decoration turned off. I redid the numbers and the numbers are as follows. *** With CVS and Linked Resources label decorations ENABLED.*** R2.1.2 : Project RebuildAll : 4mins 15secs 3.0 M5 : Project RebuildAll : 6 mins 15 secs R2.1.2 : Expand Package Explorer (80 packages Hierarchical layout) : 25 secs 3.0 M5 : Expand Package Explorer (80 packages Hierarchical layout) : 45 secs *** With CVS and Linked Resources label decorations DISABLED.*** R2.1.2 : Project RebuildAll : 2mins 15secs 3.0 M5 : Project RebuildAll : 3 mins 5 secs R2.1.2 : Expand Package Explorer (80 packages Hierarchical layout) : 25 secs 3.0 M5 : Expand Package Explorer (80 packages Hierarchical layout) : 30 secs So it does look like the label decorations is playing a significant role in the slowdown and I would suspect it is the CVS label decoration that is the culprit here.
I just want to be clear about your build test case, since the behaviour of build has changed slightly. How many projects are in your workspace? Is autobuild turned on or off during this test? The changed behaviour for 3.0 M5 is that "Rebuild Project" will now also cause dependent projects to be built as well if autobuild is turned on. In your test case you say "Project Rebuild All"... Can you be more precise? The two build actions in the menu are "Rebuild All" and "Rebuild Project". We are very sensitive to regressions in build performance so please stick with us and we'll get this figured out.
Sanjay, one other question: do you have the tasks or problems views open during this test? There is currently a major performance problem in these views, especially when filters are applied (see bug 44443). If so, please try again without these views open. From your numbers it looks like the decorators are to blame, but I just want to eliminate other variables from the test case.
John, I have 7 projects in the workspace with one of the projects being the main project and referencing the others. My test cases were using the Rebuild All. Regarding the problems and Tasks view being open. Yes. I always leave those open in my custom Java perspective and I do have filters enabled (to eliminate deprecated warnings) so those numbers would reflect any problem there as well. Since I am at home now I will run the tests again tomorrow at work with the problems and tasks view closed and post the numbers here.
I have also noticed a significant performance degradation between M4 and M5 when doing something as simple as scrolling through a java source file. My workspace contains 5 projects, with a total of 4k or so java files. In M5 I opened a java file, placed the cursor at the top of the file, and held the down arrow until I hit the bottom of the file... Every 10 lines or so when the cursor ends up on a type there is a pause where the cursor doesn't move, and then it jumps back to life. This is quite annoying when trying to move around in the file and get stuff done. I notice in the progress view when the pauses occur the task "Java AST creation" is running and when the pause stops the tasks is finished (or its marked as cancled). Doing the same thing with M4 (same workspace, so same settings) results in no noticable pauses. M4 seems to be quite zippy. Note: I have not used M4 all that much to say with 100% confidence this is the case (I will use M4 for the rest of the day and see how it goes), but I have used M3 for quite a while and it definitly does not have this performance problem. Let me know if this warrants opening a seperate defect.
Does the slowing still occur if you disable all 'smart' features in Java editor preferences ?
Ryan, please open a separate defect against JDT Text so we can keep this one focused on builder performance. If you export your preferences (Windows > Preferences > Export) and attach the exported preference file to the bug report, then the editor developers will be able to reproduce your exact setup. This sounds similar to bug 44864, but that bug existed already in M4.
I cannot reproduce a general regression in java builder performance between 2.1.2 and 3.0 M4 or M5. My test case was 17 Eclipse projects (all core projects plus tests), for a total of 4715 java files. Results: 2.1.2 Rebuild All: 1 m 9 s 3.0 M4 Rebuild All: 1 m 10 s 3.0 M5 Rebuild All: 1 m 12 s M5 with CVS decorators on: 1 m 12 s M5 with problems view open: 1 m 12 s There must be something special about Sanjay's environment that is causing this regression.
John, I have opened a defect for my M5 performance issue. It is number 47474.
Moving to CVS decorator owner for further analysis.
Its strange when I ran the tests again this morning I got completely different results, actions that took 6 mins yesterday took only 3.5-4mins today. Keeping the Problems and Tasks view open does not change the numbers significantly. I compared my results with other machines on the network and others are also sporadically reporting slowdowns with CVS related activity. Trying to narrow the problem down I stumbled on the following issue. In M5 selecting a single java source file from the package explorer and then using the popup menu and choosing Team->Synchronize with repository takes over 35 seconds during which time the ide and machine is non responsive. Choosing the same file and choosing "Show in resource history" or "Team->compare with latest from head" is less than a second. Doing any of the above actions in R2.1.2 on the same machine takes less than a second. This is reproduceable.
I am not able to reproduce this behavior. Does it take 35 seconds each time you choose Team>Synchronize or just the first time? It could be Team/CVS plugin initialization that is taking the time.
It takes 35 seconds each time I choose Team->Synchronze with Repository
I decided to create a new workspace to see if it was somehow a workspace corruption. The problem still occurs after building a new workspace. Strange thing is if I choose my "main" project then any file in this project takes a long tome when doing a "Team->Sync with repository" choosing any of the other projects in the same workspace and repeating the action is nearly instantaneous. My main project is the largest project in the workspace and has a very large number of packages and classes. Im not sure if that would have any effect on the Sync operation. All the projects are connected to the same cvs repository.
Even more strange is that another developer in my team is having the same problem with Sync with repository but in his case it is for any file in any project. He is using the 200311050952 build whereas I am using the M5 build
How about other operations? Specifically, can you try Replace with Latest from Repository (assuming that you don't have local changes you need to keep)?
I have tried Team>Synchronize on a large project on a 500MHz PC running WinNT and it was instantaneous in launching the background refresh. The operation itself took a while (as expected) but the UI was responsive during that time. Is there anything in your log file (found inside the .metadata directory of your workspace)?
I cant see a pattern either. I removed all the plugins that I am currently using and tried with a vanilla M5 image. Same result. There does not seem to be any significant errors in the .log file. There are some errors on missing wsad.* builders since the projects in question are often loaded into a WSAD5 workspace but that has been the case for a long time and I doubt it has any bearing on the problem. In my vanilla image with a fresh workspace it no longer takes 35 secs for a Sync but still about 5-6 secs during which time the UI is blocked the CPU usage is at 99% for the javaw process for my main project whereas other projects in the same workspace are instantaneous.
Other operations are instantaneous. Specifically Replace from repository takes less than a second.
Additionally noticed that the delay in doing the "Team-Synchronize with Repository" only occurs if the Package Explorer has the Project expanded. Repeating the same action on the same source file in the same image but from the Navigator view while keeping the Package explorer collapsed on the Project is instantaneous. Repeating the action again from the Navigator view but with the Package Explorer expanded on the project shows the 5-6 secs delay.
I have further isolated the problem on the Package explorer and I think it now explains all the anomalies that I have been seeing with the disparate timings reported during testing. As I mentioned earlier the project in question has an extremely large number of packages. Actual count is 383. I had not taken notice of the actual Package explorer state when doing my tests and hence sometimes the Package explorer was expanded and sometimes it was closed during my tests and hence I was seeing very different numbers. After I noticed that it was Package explorer related I tried switching the layout from Hierarchical (which is my favorite layout) to flat and immediately the issues with the non responsive UI went away. Based on this I would guess that: 1) The Package explorer is being updated even when it is not visible 2) The Hierarchical layout on the package explorer is causing the slow down when the number of packages is very large. This is reproduceable on a number of 3.0 builds that I have tested and
Do you have the CVS decorators turned on?
It appears that the Package Explorer is refreshing the entire view when the contents of ANY working set changes. This could be causing the problem. The view should probably only refresh if the contenst of it's active working set change. Moving to JDT UI.
Sanjay, thanks for your persistence in tracking down this problem.
Thanks, for tracking this down.
you are welcome.. I guess it would be good if the package explorer is marked as dirty and only updates when it becomes active. This would allow a zero cost when the package explorer is not the active view. Also it seems contra intuitive that the hierarchical layout costs more than the flat layout. If anything the hierarchical layout is displaying less elements than the flat layout so at first guess it should be faster since collapsed nodes do not have to be updated. Based on the numbers it looks like all the nodes are being updated all the time.
The hierarchical mode isn't isomorph to the underlying Java model, which makes the updating more complicated than the flat model. Erich, can you please investigate.
This needs to be investigated, but different test cases are reported, so I'd like to narrow the test cases down. Michael can you please provide the steps for the problem described in comment #26. >It appears that the Package Explorer is refreshing the entire view when the >contents of ANY working set changes.
Steps to reproduce: 1) Turn on CVS decorations (not part of the problem but they provide a visual indication of what is going on) 2) Open the packages explorer, Navigator and sync view as regular views (i.e. not fast views) and make sure they are non-overlapping. 3) Configure the sync operation to stay in the same perspective (i.e. on the Team preference page select None as the perspective). 4) In the navigator, perform a Team>Synchronize on a project mapped to CVS. This modifies a working set and shows the sync view. 5) Notice that the decorators in the package view flash. This is because the package view picks up the working set change and refreshes the entire view. Note that fixes in the works to remove the setting of the working set on sync and remove the decorator flash (no exact date for these yet). However, it seems like it should be an easy fix to check whether the changed working set matches the active working set in the packages explorer. As for the other reported problems (slow build, etc), they may have been present initially due to problem markers or some other factor that effected te Package Explorer redraw time (Can you comment on this possibility Sanjay?).
I believe/guess the slow build was also related to the Package Explorer state/cvs decorators. I think if you have a very large project and keep the package explorer in hierarchical layout mode and expand a node that has a large number of sub nodes (packages) and then perform a Rebuild All you should see the same behavior. I did not repeat the tests to verify this but will do so on monday and post the results.
fixed problem pointed out in comment #26 keeping bug open until Sanjay has repeated his scenario.
The general build performance does seem to still be slower on 3.0 builds. Its difficult to test since each run I am getting slightly different numbers so I did the following. 1) Closed all perspectives so that any view related issues will hopefully not have any bearing on the results 2) Turned off Label decorations. 3) Turned off the option to switch to problems view when problems exist. 4) Turned off build automatically on workspace modification 5) Turned automatic workspace save to 15mins so it does not have any bearing on the results. 6) Cold start of the ide before each test. 7) run each test twice to take average time (cold start each time). The numbers I am now getting are: 3.0 M5: 4mins 45secs 2.1.2 build: 3mins 20secs
Do you have the exact same compiler settings ? Wondering if one of our new diagnosis would be so expensive... though we did some stats, and did not find anything obvious.
I also repeated the same tests but with the Package explorer open and expanded in hierarchical layout mode on the package that has the largest number of sub packages. The times were (with package explorer open): 3.0 M5: 5mins 30secs 2.1.2: 3mins 55secs Regarding the compiler settings: i have left everything at the default settings for each build. (2.1.2 has its defaults and 3.0 M5 has its defaults) I have not consciously changed any of these settings and I doubt I changed them accidently either since i used fresh workspaces for these tests.
Sanjay - how does the topology of your expanded package explorer look like, i.e., how many children at each level. There are now more label provider changes during a build and the processing should be proportional to the number of visible lines. I couldn't reproduce the same amount of slowdown yet.
Erich, I am on vacation till the first week of jan so I cannot give you accurate numbers but from my memory... The project in question uses a lot of generated service code each of which is generated into its own package. The total number of packages was 383 for the entire project. The number of nodes visible was for the generated package which has probably about 80+ nodes visible with each node having additionally 4 sub nodes/packages (but in a collapsed state).
Moving to M7. Waiting for input from reporter.
Tested I20040203 using a full development workspace. org.eclipse.jdt.ui (JDT UI) alone has 1843 files (1396 CUs) and a nesting of 8. Settings: - don't show members in Package Explorer [??? is this what you have ???] - auto-build on - CVS, binary and linked resource decorators enabled I could not see a big difference depending on the flat/hierarchical mode. Inital decoration might take a but longer since there are many more nodes. Looking at your figures and comparing flat (3.0 M5: 4mins 45secs,2 1.2 build: 3mins 20secs) to hierarchical (3.0 M5: 5mins 30secs, 2.1.2: 3mins 55secs) it's not that a difference given the fact that - the sub-packages need to be computed and aren't provided by the Java Model - the decoration computing has to be doen (at least for problem markers which you can't turn off) - there are many more nodes (at least twice as much) Since your figures are based on a cold start your figures show that your scenario has slowed down comparing 2.1.2 and 3.0 M5 but it does not proof that the hierachical view is really the problem here. I rebuilt projects in both (states, fully expanded) and building took the same. Could you give I20040203 a try and tell us whether this build works better now for your scenario. Note that several things have changed since 2.1 which affect your scenario especially because it is based on a cold start: - we are now OSGi based (totally different runtime story) - RCP introduced new plug-in (loading can take longer since there are more JARs involved) - decoration is done using jobs
There might be a regression because in I20040203 I still see the problem reported by Michael in comment 26. Since this is not directly related to this PR I filed bug 51168.
Here are some new numbers: I20040205 build: - Package explorer open (84 sub packages, hierarchical layout): 7mins 10secs - Package explorer collapsed (hierarchical layout): 6mins 45secs - Package explorer (flat layout) : 6mins 30secs R212 build: - Package explorer open (84 sub packages, hierarchical layout): 5mins 50secs - Package explorer collapsed (hierarchical layout): 5mins 50secs - Package explorer (flat layout) : 5mins 05 secs
closing - no more action planned
