Community
Participate
Working Groups
When i touch the build.properties file in a plugin then eclipse takes a minute to "Updating plug-in dependencies" in a big workspace. As far as i have sampled there are two reasons why it takes so long: 1. eclipse reads every dependend .jar file. This currently happens in JavaModelManager.verifyArchiveContent(IPath) calling getZipFile. Even though the jar files have not been changed. Thousends in my case. Can this please avoided by caching all .jar files in the workplace resources and only updated when "refresh" of resources is triggered? 2. RequiredPluginsClasspathContainer.getRule(StateHelper, BundleDescription, ExportPackageDescription) name.replaceAll("\\.", "/") alone takes some seconds. Note that replaceAll takes a Regexp and all that is needed there is to substitute single characters: name.replace(".", "/") This may also help on bug https://bugs.eclipse.org/bugs/show_bug.cgi?id=568157 even though that usecase seems to have other root reasons.
For me , "Updating plug-in dependencies" is almost instantaneous. Howmany projects do you have in your workspace?
We have > 500 Projects with > 2500 .jar and > 50_000 .java files in workspace.
(In reply to Jörg Kubitz from comment #2) > We have > 500 Projects with > 2500 .jar and > 50_000 .java files in > workspace. OK. I will simulate a similar workspace.
(In reply to Jörg Kubitz from comment #0) Jens, you seem to have this fixed already? If so can you push a Gerrit?
(In reply to Wim Jongman from comment #4) > (In reply to Jörg Kubitz from comment #0) > > Jens, you seem to have this fixed already? If so can you push a Gerrit? I mean, Jörg. :)
I will work on this. I will propose a patch for the easy (second) part first.
New Gerrit change created: https://git.eclipse.org/r/c/jdt/eclipse.jdt.ui/+/175935
New Gerrit change created: https://git.eclipse.org/r/c/pde/eclipse.pde.ui/+/175936
New Gerrit change created: https://git.eclipse.org/r/c/jdt/eclipse.jdt.core/+/175937
ok i got an advanced question where i could need help or adivse: I figured out, that the zipfile would be opened in request of Bug 229042 "[buildpath] could create build path error in case of invalid external JAR format" So i will need ClasspathTests.testBug229042() to still work. And i would like to add an assertion that the zip will be opened exactly once (currently it will be opened more then once!). I tried to use jdk.internal.perf.PerfCounter for this: diff --git a/org.eclipse.jdt.core.tests.model/src/org/eclipse/jdt/core/tests/model/ClasspathTests.java b/org.eclipse.jdt.core.tests.model/src/org/eclipse/jdt/core/tests/model/ClasspathTests.java index 91a8723..bc7f43f 100644 --- a/org.eclipse.jdt.core.tests.model/src/org/eclipse/jdt/core/tests/model/ClasspathTests.java +++ b/org.eclipse.jdt.core.tests.model/src/org/eclipse/jdt/core/tests/model/ClasspathTests.java @@ -80,6 +80,8 @@ import org.eclipse.jdt.internal.core.builder.State; import org.eclipse.team.core.RepositoryProvider; +import jdk.internal.perf.PerfCounter; + @SuppressWarnings({"rawtypes", "unchecked"}) public class ClasspathTests extends ModifyingResourceTests { private static final IClasspathAttribute ATTR_IGNORE_OPTIONAL_PROBLEMS_TRUE = JavaCore.newClasspathAttribute(IClasspathAttribute.IGNORE_OPTIONAL_PROBLEMS, "true"); @@ -6916,7 +6918,10 @@ try { IJavaProject p = createJavaProject("P"); createFile("/P/library.jar", ""); + long zipFileCount0 = PerfCounter.getZipFileCount().get(); setClasspath(p, new IClasspathEntry[] { JavaCore.newLibraryEntry(new Path("/P/library.jar"), null,null)}); + long zipFileCount1 = PerfCounter.getZipFileCount().get(); + assertEquals("Zip File was not opened exactly 1 time",1,zipFileCount1-zipFileCount0); assertBuildPathMarkers("Expected marker", "Archive for required library: \'library.jar\' in project \'P\' cannot be read or is not a valid ZIP file", p); setClasspath(p, new IClasspathEntry[0]); But access to that class is denied. Is there a way to open it up / am i on the wrong track / do you know a better way?
(In reply to Jörg Kubitz from comment #10) > ok i got an advanced question where i could need help or advise: What are you trying to achieve with PerfCounter?
> What are you trying to achieve with PerfCounter? Measure how often the zip was opened.
Gerrit change https://git.eclipse.org/r/c/pde/eclipse.pde.ui/+/175936 was merged to [master]. Commit: http://git.eclipse.org/c/pde/eclipse.pde.ui.git/commit/?id=8db2927adb362bd785e40ff164dead2551fb637e
Jörg, is the PerfCounter issue still pending because all changes seem to be merged.
the JavaModelManager.verifyArchiveContent is still unsolved. i keep working on it.
When a single JAR is added to a Project it is opened up in JDT to 4 times: 1. synchronous JavaModelManager.verifyArchiveContent to check wether its a valid Jar 2. async ResourceChangeEvent -> JavaModelManager.verifyArchiveContent to check wether its a valid Jar (this was already done!) 3. async IndexManager -> AddJarFileToIndex.execute to index all filenames within and extract "module-info" file 4. async IndexManager -> AutomaticModuleNaming.determineAutomaticModuleName to extract "Automatic-Module-Name" see Stacktraces below. I would prefer a single synchronous indexing (reading the filenames and extracting two small files is cheap) and when the JAR is used again in the workspace to use that indexed information from JavaModelManager.getIndexManager().getIndex(...) I need to understand why there are further caches which can hold Jar information: a) threadlocal ZipCache in JavaModelManager.getZipFile mostly used to extract files b) JavaModelCache JavaModelManager.cache c) threadlocal JavaModelManager.temporaryCache Stacktraces: Thread [main] (Suspended (breakpoint at line 228 in ZipFile)) owns: RunnableLock (id=1161) ZipFile.<init>(File) line: 191 JavaModelManager.getZipFile(IPath, boolean) line: 2937 JavaModelManager.getZipFile(IPath) line: 2906 JavaModelManager.verifyArchiveContent(IPath) line: 2887 ClasspathEntry.validateLibraryContents(IPath, IJavaProject, String) line: 2533 ClasspathEntry.validateLibraryEntry(IPath, IJavaProject, String, IPath, String, boolean) line: 2459 ClasspathEntry.validateClasspathEntry(IJavaProject, IClasspathEntry, IClasspathContainer, boolean, boolean) line: 2322 ClasspathEntry.validateClasspathEntry(IJavaProject, IClasspathEntry, boolean, boolean) line: 2193 ClasspathValidation.validate() line: 75 SetClasspathOperation(ChangeClasspathOperation).classpathChanged(ClasspathChange, boolean) line: 55 SetClasspathOperation.executeOperation() line: 78 SetClasspathOperation(JavaModelOperation).run(IProgressMonitor) line: 740 Workspace.run(ICoreRunnable, ISchedulingRule, int, IProgressMonitor) line: 2292 Workspace.run(IWorkspaceRunnable, ISchedulingRule, int, IProgressMonitor) line: 2317 SetClasspathOperation(JavaModelOperation).runOperation(IProgressMonitor) line: 811 JavaProject.setRawClasspath(IClasspathEntry[], IClasspathEntry[], IPath, boolean, IProgressMonitor) line: 3608 JavaProject.setRawClasspath(IClasspathEntry[], IPath, boolean, IProgressMonitor) line: 3568 JavaProject.setRawClasspath(IClasspathEntry[], IProgressMonitor) line: 3624 ClasspathTests(AbstractJavaModelTests).setClasspath(IJavaProject, IClasspathEntry[]) line: 3102 ClasspathTests.testBug229042() line: 6920 Thread [Worker-12: Building] (Suspended (breakpoint at line 228 in ZipFile)) ZipFile.<init>(File) line: 191 JavaModelManager.getZipFile(IPath, boolean) line: 2937 JavaModelManager.getZipFile(IPath) line: 2906 JavaModelManager.verifyArchiveContent(IPath) line: 2887 ClasspathEntry.validateLibraryContents(IPath, IJavaProject, String) line: 2533 ClasspathEntry.validateLibraryEntry(IPath, IJavaProject, String, IPath, String, boolean) line: 2459 ClasspathEntry.validateClasspathEntry(IJavaProject, IClasspathEntry, IClasspathContainer, boolean, boolean) line: 2322 ClasspathEntry.validateClasspathEntry(IJavaProject, IClasspathEntry, boolean, boolean) line: 2193 ClasspathValidation.validate() line: 75 DeltaProcessor.resourceChanged(IResourceChangeEvent) line: 2199 DeltaProcessingState.resourceChanged(IResourceChangeEvent) line: 477 NotificationManager$1.run() line: 305 SafeRunner.run(ISafeRunnable) line: 45 NotificationManager.notify(ResourceChangeListenerList$ListenerEntry[], ResourceChangeEvent, boolean) line: 295 NotificationManager.broadcastChanges(ElementTree, ResourceChangeEvent, boolean) line: 158 Workspace.broadcastBuildEvent(Object, int, int) line: 366 AutoBuildJob.doBuild(IProgressMonitor) line: 150 AutoBuildJob.run(IProgressMonitor) line: 244 Worker.run() line: 63 Daemon Thread [Java indexing] (Suspended (breakpoint at line 228 in ZipFile)) ZipFile.<init>(File) line: 191 AddJarFileToIndex.execute(IProgressMonitor) line: 150 IndexManager(JobManager).run() line: 414 Thread.run() line: 834 Daemon Thread [Java indexing] (Suspended (breakpoint at line 228 in ZipFile)) JarFile(ZipFile).<init>(File, int, Charset) line: 228 JarFile(ZipFile).<init>(File, int) line: 177 JarFile.<init>(File, boolean, int, Runtime$Version) line: 348 JarFile.<init>(File, boolean, int) line: 319 JarFile.<init>(String) line: 258 AutomaticModuleNaming.determineAutomaticModuleName(String) line: 33 AddJarFileToIndex.execute(IProgressMonitor) line: 252 IndexManager(JobManager).run() line: 414 Thread.run() line: 834
"// TODO (kent) we could just remove the in-memory index and have the indexing check for timestamps" (ClasspathChange:534)
(In reply to Jörg Kubitz from comment #17) > "// TODO (kent) we could just remove the in-memory index and have the > indexing check for timestamps" (ClasspathChange:534) LOL. But everything worked fine with < 100 projects, so that didn't make it...
Jörg, looks to me that all PDE changes are merged, therefore I close this bug for PDE. Unfortunately we have the limitation that a bug can only be assigned to one component. Thanks for your work. FYI - for recent JDT cleanups (https://www.vogella.com/tutorials/Eclipse/article.html#eclipse-code-checks-and-cleanup) Fabrice added a regex clean for this work. So you could do a mass check / change for other Eclipse projects if you desire. Please note that the multiple selection is currently only supported via the package explorer (https://bugs.eclipse.org/bugs/show_bug.cgi?id=548265). Also please note that two changes for this cleanup are currently pending: https://git.eclipse.org/r/c/jdt/eclipse.jdt.ui/+/176477 https://git.eclipse.org/r/c/jdt/eclipse.jdt.ui/+/176427 Could you open a new bug for the remaining JDT work? Or if you still have pending PDE issues, please reopen.
