Community
Participate
Working Groups
A month ago it took about a 10 seconds. As of about two weeks ago it has begun taking 60 seconds. This is impacting the ability of folks to debug plugins. The problem appears to be the writing of the .registry file. The objectArray has over 7000 elements in it and is implemented as an ArrayList. Additionally the process of saving calls indexOf (which is implemented linearly) against the array *many* times (70000 times maybe?) which of course creates O(n**2) situation. Additionally, the implementation of ArrayList.indexOf() uses the equals() method on the objects, not the == operator. As you might imagine... and can easily measure, this slows things down a lot. I understand why an ArrayList is being used, but you need to use some other structure to do the semantic equivalent of the "indexOf" operation when writing out the .registry file. To reproduce this, start the workbench in a debugger, quit the workbench, and stop the code a few times after the workbench window disappears. You should find that most of the times you stop the code you'll be in either the equals() method or the indexOf() method.
Will investigate.
build F3 J9 VM 2GHz machine 427 plug-ins reading registry: 1045ms writing registry: 19450ms
Currently we always write out the .registry file on shutdown. A quick optimization would be to only write it out if it is dirty from last startup. - startup - read registry cache - if cache is valid then continue - otherwise delete cache from disk and parse plug-ins ... - on shutdown - does cache file exist? - if true do nothing - otherwise write out the registry We still need to investigate the use of ArrayLists.
The quick optimization would be a good thing also, but using a better structure than ArrayList (perhaps a HashMap) is the real heart of the problem.
Thanks for looking into this so quickly! This is quite an important bug for us.
That alternative data structure probably only needs to exist during the save process.
Thought I'd mention this may not be relevent to only shutdown, per se, but with the timed "save workspace" option in F3, there's been a few times that my F3 workbench has appeared to hang for several minutes. I don't know if the delay is all related to writing the registry, but something takes several minutes (not seconds). (This is on a 800 GHz machine, 1 Gig of ram, with several hundred plugin projects) It doesn't seem to consistently take that much time, but thought I'm make this observation here.
David, I don't know if this is related to your problem or not, but the one place where I have seen long delays, is if the Eclipse process gets swapped out by Windows. Swapping it back in then takes a very long time. During one of these delays if you open the Task Manager, you can tell if this is the cause of the delay by watching the memory usage of the eclipse process. If it is slowing growing (a couple of Meg a second) then thats most likely the cause.
Have fixed this by writing out the registry cache only if it has changed (as per DJ's comments on 2002-06-14 17:27). If the cache has changed, we delete the cache file. The issue of using ArrayLists has been documented as bug 20480 and will be investigated post 2.0.
In the original comment, you mention that it used to take 10 seconds to write out the registry and as of late it has gotten considerably worse. The registry cache serialization code hasn't changed in months so this increase in time is curious. Has your number of plug-ins significantly increased in this time?
You wrote: "A quick optimization would be to only write it out if it is dirty from last startup." Do you have a feel for how many times the quick optimization would apply? What are some of the end user actions that would cause the quick optimization not to apply?
The registry cache will not be written out in a normal scenerio. It will be written out via the same operations that would have invalidated the registry cache before. Essentially modifying the plugins directory. (installing new plugins, removing plugins, editing plugin.xml files, etc)
DJ_Houghton asked if we've increased the number of plugins recently because he doesn't think the code in question has changed recently. Unfortuntately I don't know. I don't normally work at the plugin level and I no longer have any builds from last month with which to do a comparison. I do think that's a good question though. I will clarify something though. I said 10 seconds vs 60 seconds in my original posting. The 60 second figure was accurately measured. 10 seconds was just my recollection. I think that might have been as high as 25 seconds, but it clearly wasn't as long as I'm currently seeing... and it's clearly too long.
The number of plugins in WSAD has increased since release 4. In release 4 we had 196 plugins (68 were from eclipse). We currently have 418 (70 from eclipse). But I didn't think that the number had gone up recently, but I am not sure.
