Community
Participate
Working Groups
JDT appears to be causing JAR locking on Windows. "JAR locking" is where a file on a window file system has one or more file handles open. This stops Windows from allowing the file to be deleted. This does not occur on Unix systems as those systems allow a file unlink() when if a file handle is open to it. JDT causes JAR locking on Windows systems, must exit eclipse to force file handle close, to allow deletion of files. JDT need to have a button in preferences to force it to close all possible file handles. My guess is that file handles are being cached in some misguided attempt to be efficient. Where as it would be more useful for eclipse to open a JAR index the data it needs into some other location and then immediately close the JAR. The scenario is that the m2e-wtp plugin is installed, during project reconfiguration and project build operatation it modifies ClassPath objects to point at JAR files that are copied into target/** subdirectory. It appears that JDT randomly opens some of these JARs. I have no idea why. This means that any "mvn clean" operation on the project will always fail from this point on. Even closing the project in Eclipse does not force the open file handles to be closed by Eclipse, even though those files are inside that projects target/** folder. The only way to be able to remove these files is to Exit Eclipse. Once Eclipse is shutdown you can either manually delete the JARs with your operating system UI or command-line. Or if you disabled auto-build before Exiting eclipse 80% of the time you can use Eclipse UI to delete the JAR files or run "mvn clean" successfully. Obviously having to Exit Eclipse in order to continue working is a major productivity destroyer. An example is a WAR project where often 5 JARs out of 40 JARs get locked. These 5 JARs often have the Jar File icon next to them when shown in project explorer, while the rest of the standard file icon (no idea what this means). Sometimes if you randomly change the Build Path (like adding JUnit 4) you can get it to release another 3 or 4 handles. But for me is never releases all handles to allow the "target" subdir to be deleted.
Are you sure it's JDT and not WTP or it's components? Do you see this happening with plain Java projects too?
This issue is related to https://bugs.eclipse.org/bugs/show_bug.cgi?id=406173 Something in eclipse (that is not M2E-WTP) is causing JAR locking to occur on entries in the Build ClassPath. If you take a look at this code: https://github.com/dlmiles/m2e.wtp/blob/916077b6debefaa2c864c556c71a0c485ec7df0b/org.eclipse.m2e.wtp/src/org/eclipse/m2e/wtp/WebProjectConfiguratorDelegate.java Notice how the IClasspathEntryDescriptor are setup, this causes something outside of M2E-WTP module to JAR lock the file. Why does this file need to be opened and kept open ? Not even a "Close Project" can fix it.
I've had a quick scan through the m2e-core project and the org.eclipse.m2e.jdt.* code and can not find anything in there that directly opens a JAR filehandle. If you know of any JDT API calls to look at in this code base that would be helpful. Or if you can provide the details of an open source Java agent that can perform diagnostic on windows by providing stack traces for every open/closed file handle. It would be possible to get the stack trace of the problem code that is holding file handles open (even after "Close Project").
(In reply to comment #3) > If you know of any JDT API calls to look at in this code base that would be > helpful. JDT/Core internally uses JavaModelManager#getZipFile(path) and #closeZipFile(path) to handle zip and jar files. The call to the former must always have a corresponding closeZipFile(). I just verified that all getZip calls have the closeZipFile. May be you can look for calls to this method and see if the zip is left open somehow.
FWIW, a JAR file can be "locked" on Windows if the JAR was loaded by a JRE process and the process is still running. Similar case is described here: http://www-01.ibm.com/support/docview.wss?uid=swg21496444 Making long story short: if the JAR was loaded by Eclipse process, you cannot delete the JAR on Windows without closing the process first. In that case JDT has nothing to do with it. I think you should check what is "locking" the JAR in your case instead of making any assumptions.
A grep over the GIT trees for m2e-core and m2e-wtp does not indicate this API #getZipFile() is used (directly). * We still have not confirmed the stack trace (the caller) that opens the problematic file handles. * The m2e code maybe indirectly causing the locking by using other API that calls getZipFile(). What APIs might be used to allow the file to be opened in one time frame and then closed at another timeframe. * It is also not ruled out that maybe some other plugin that is installed listens for Build classpath changes and is reacting to this. * I believe that marking the classpath entires as derived and hidden reduces this problem substantially. A recent patch to m2e-wtp cause the derived/hidden flags to be set and that did appear to fix the problem, except for some cases where I believe information was being obtained Java indexing. * I believe some element of the Java indexing maybe at work because it is possible to get the JARs locked before any build has chance to run after a restart of Eclipse (with Automatic Build disabled). Some parts of m2e/m2e-wtp do get to run, even through no Workspace/Project build should have occurred yet. The reason for this thought is that in the cases where JAR locking still occurred I had deleted the JAR file from the target/**/*.jar directory (while Eclipse was shutdown) yet upon restarting eclipse the IDE still thinks the JAR file is there as it is displayed in a node under the Project Explorer tree. In a scenario where I did not delete the JAR while Eclipse was shutdown, being indexed in this was was enough to cause the JAR locking to re-occur because the first build had a chance to run on restart (again as Automatic Build was disabled).
(In reply to comment #5) > FWIW, a JAR file can be "locked" on Windows if the JAR was loaded by a JRE > process and the process is still running. Similar case is described here: > http://www-01.ibm.com/support/docview.wss?uid=swg21496444 > > Making long story short: if the JAR was loaded by Eclipse process, you > cannot delete the JAR on Windows without closing the process first. In that > case JDT has nothing to do with it. > > I think you should check what is "locking" the JAR in your case instead of > making any assumptions. One has to assume some aspects of unknown information in order to try to test an angle on the problem when trying to find a solution. Yes hence why my previous comments on a debug agent to provide evidence via a full stack trace of all APIs that open all files while the process is loaded. Maybe someone can cite an open-source project or maybe someone has access to a close-source solution to provide stack traces. If this information was available then we can remove a number of assumptions that have to be made in light of difficult to obtain information that would be helpful to understanding the cause of the problem. Re use of ANT. I am not aware of any ANT use inside the projects I am testing with, nor any other projects in the workspace at the time. FWIW I am also using Oracle Java7 and some issues like JarURLConnection may have been improved. From my point of view, there is no reason to "assume" the article you reference has any bearing on the problem reported here. It reads as a known issue for certain versions of Eclipse/JRE when using ANT. The problem of JAR locking itself is known and understood from the point of view of Windows operating system calls. May I suggest to users of the ANT JAR locking problem, try to use ANT out-of-process from Eclipse. Maybe selecting some other JRE still do the trick. This way the JRE that is caching will be torn down after every ANT use and its cache destroyed in the process as the process no longer exists. Now to come back to this bug report. Yes Maven can also use an alternative JRE and implementation of Maven, however I have not found any aspect of the problem that connects Maven itself to be the cause of the JAR locking problem described in this ticket. The problem appears to be triggered by Build Classpath changes and Eclipse project configuration performed by m2e-wtp/m2e-core and this is a purely Eclipse Java language matter. I did cite in a previous comment the code to be looking at what I see is code that configured things JDT operates on. This doesn't necessarily mean JDT itself is a fault. Maybe I can add some kind of logging to the JavaModelManager#getZipFile(path) and #closeZipFile(path) ??? Get information dumped to Eclipse logfile ? This will at least prove those APIs calls are implicated. Then the next task is to narrow down the callers/users of those API calls.
(In reply to comment #7) > Maybe I can add some kind of logging to the > JavaModelManager#getZipFile(path) and #closeZipFile(path) ??? > > Get information dumped to Eclipse logfile ? > > This will at least prove those APIs calls are implicated. Then the next > task is to narrow down the callers/users of those API calls. I would say, it's worth a try. If the JAR is opened during the classpath resolution of a project in the workspace, we don't expect the handle to be kept open and if it does occurs, possibly one of the callers of getZipFile() is not following the protocol, which would be a bug. But we also have to rule out that the JAR is not opened by the class loader or any other process.
I have the same issue on last Eclipse Luna with webAppProject. My lib *.jars all are in WEB-INF/lib/ My workaround to use ant build (updateRefLibs.xml in my case) to update referenced libs in WEB-INF/lib/. Ant somehow removing lock from jar and updates it. May be this helps.
to previous comment: updateRefLibs.xml: (as example) <project name="updateRefLibs" default="all" basedir="."> <property name="project.path" value="${basedir}" /> <property name="libs.dir" value="${project.path}/WebContent/WEB-INF/lib/" /> <property name="refProj1" value="rov-gwt-utils" /> <property name="refProj1.jar" value="${refProj1}-gwt.jar" /> <property name="refProj1.jar.path" value="../${refProj1}/dist/${refProj1.jar}" /> <target name="checkBuildAll"> <available file="../${refProj1}/buildAll.xml" property="refProj1.buildAll.xml.exists" /> </target> <target name="execBuildAll1" depends="checkBuildAll" if="refProj1.buildAll.xml.exists"> <echo message="processing ../${refProj1}/buildAll.xml" /> <ant antfile="../${refProj1}/buildAll.xml" useNativeBasedir = "true"/> </target> <target name="buildAllRefLibs" depends="execBuildAll1" /> <echo message="processing ref libs:" /> <echo message="${refProj1.jar.path}" /> <path id="refLibs"> <pathelement location="${refProj1.jar.path}" /> </path> <target name="copyRefLibs" depends=""> <copy toDir="${libs.dir}" flatten="true" verbose="true" overwrite="true"> <path refid="refLibs" /> </copy> </target> <target name="all" depends="buildAllRefLibs,copyRefLibs" /> </project>
The issue is still happening on latest eclipse 4.7 (m4) I use git in git-for-windows bash prompt to switch between branches, and it always failed to unlink JAR modified files when Eclipse was running. I have to restart my eclipse everything I hit the issue. It is interesting that the JAR file is still locked, even if the project was already closed in Eclipse and no "progress" are running. And is it a signal of possible memory leak?
I'm experiencing the same issue on Windows 7 with JDK 1.8.0u121 and Eclipse Neon. We have projects referencing external archives which are built via ant scripts at the command line. When trying to delete the external archives eclipse has a lock on the files after running a launch configuration. As a result, the only reasonable course of action is closing eclipse and re-opening which is a time consuming and inefficient. This is the same as was reportedly fixed in this bug https://bugs.eclipse.org/bugs/show_bug.cgi?id=29753, it appears to be broken again.
I do see something similar. We update jar files through SVN and SVN is unable to update particular files. SVN reports svn: Can't move 'D:\dev\workspaceTrunk\aquintos.mm.di\.svn\tmp\svn-193A5147' to 'D:\dev\workspaceTrunk\aquintos.mm.di\lib\DI.jar': Permission Denied Checking with process explorer I do see that jar files are being kept open by eclipse. Using file leak detector I do see numerous files being held by JDT Indexing while none of the Jobs are running. Maybe the JDT indexing has crashed somehow. #319 D:\dev\workspaceTrunk\aquintos.mm.di\lib\DI.jar by thread:Java indexing on Tue May 02 08:57:20 CEST 2017 at java.util.zip.ZipFile.<init>(ZipFile.java:150) at java.util.zip.ZipFile.<init>(ZipFile.java:163) at org.eclipse.jdt.internal.core.JavaModelManager.getZipFile(JavaModelManager.java:2749) at org.eclipse.jdt.internal.core.JavaModelManager.getZipFile(JavaModelManager.java:2715) at org.eclipse.jdt.internal.core.search.matching.JavaSearchNameEnvironment.mapToClassPathLocation(JavaSearchNameEnvironment.java:104) at org.eclipse.jdt.internal.core.search.matching.JavaSearchNameEnvironment.computeClasspathLocations(JavaSearchNameEnvironment.java:92) at org.eclipse.jdt.internal.core.search.matching.JavaSearchNameEnvironment.<init>(JavaSearchNameEnvironment.java:51) at org.eclipse.jdt.internal.core.search.indexing.SourceIndexer.resolveDocument(SourceIndexer.java:159) at org.eclipse.jdt.internal.core.search.JavaSearchParticipant.resolveDocument(JavaSearchParticipant.java:102) at org.eclipse.jdt.internal.core.search.indexing.IndexManager.indexResolvedDocument(IndexManager.java:510) at org.eclipse.jdt.internal.core.search.indexing.IndexManager$1.execute(IndexManager.java:989) at org.eclipse.jdt.internal.core.search.processing.JobManager.run(JobManager.java:394) at java.lang.Thread.run(Thread.java:745)
Created attachment 268104 [details] Eclipse Log File
Created attachment 268105 [details] List of Eclispe File Leaks
Manoj, please look at comment #13 in particular. We should implement a fail-safe mechanism to release the Jar files in the event of index crashing. Long term, we could look at using java.nio APIs to read the Jar file if it guareentes the same performance or better.
Interesting, some files are contained multiple times in the opened descriptors list, for example D:\dev\workspaceTrunk\vi.mm.rm\lib\m2mrulemodel850.jar. org.eclipse.jdt.internal.core.JavaModelManager.zipFiles is a threadLocal - if it is accessed from multiple threads, the problem with too many opened zip files is by design.
New Gerrit change created: https://git.eclipse.org/r/97007
(In reply to Eclipse Genie from comment #18) > New Gerrit change created: https://git.eclipse.org/r/97007 This is not a fix in any way, this is for better tracing and error logging zip file access in JDT.
New Gerrit change created: https://git.eclipse.org/r/99141
(In reply to Eclipse Genie from comment #20) > New Gerrit change created: https://git.eclipse.org/r/99141 Manoj, not sure if this will help. I think this will merely caches the zip files and release them. Somewhere, we open a Jar and don't call closeZipFile() at the end.
Is there a way to just simply disable the broken cache? Or it is a problem of lost reference?
Answering multiple queries here: (In reply to Jens Kuebler from comment #13) First of all, thanks Jens for spending time and effort on this issue. This comment gave a headstart to this issue. > Using file leak detector I do see numerous files being held by JDT Indexing Files opened are not closed at this juncture and remain open. A similar behavior on file leaks was observed albeit with a different threshold (as this is a variable). (In reply to Jay Arthanareeswaran from comment #21) > (In reply to Eclipse Genie from comment #20) > > New Gerrit change created: https://git.eclipse.org/r/99141 > > Manoj, not sure if this will help. I think this will merely caches the zip > files and release them. Somewhere, we open a Jar and don't call > closeZipFile() at the end. Patch uses the model cache to group together the open fds and then do a close finally. [And that somewhere is found to be here]. Instead of calling the closeZipFile() on each, we can do a final flushZipFile() [at finally] which would close the zip file. [Note: patch is under test] And on running this [as mentioned earlire - threshold of around 250], without the patch the max file threshold was breached while with the patch, that problem did not surface. (In reply to Dennis Cheung from comment #22) > Is there a way to just simply disable the broken cache? doesn't look like there is a broken cache - Let me know if you have some data that point to a broken cache. > Or it is a problem of lost reference? yes, it is a problem of lost reference. Noting that the patch is under test now.
Gerrit change https://git.eclipse.org/r/99141 was merged to [master]. Commit: http://git.eclipse.org/c/jdt/eclipse.jdt.core.git/commit/?id=1c9be7804c5f24d8d2857bf04bda58d61ad462eb
(In reply to Eclipse Genie from comment #24) > Gerrit change https://git.eclipse.org/r/99141 was merged to [master]. > Commit: > http://git.eclipse.org/c/jdt/eclipse.jdt.core.git/commit/?id=1c9be7804c5f24d8d2857bf04bda58d61ad462eb > @Jens: Could you please try tomorrow's I-Build (that will have this change) and checkout whether the issue is resolved/improved at your end? TIA.
We did a backport today for 4.6.3. Unfortunately this problem rarely appeared so it may take some time to get some feedback.
Gerrit change https://git.eclipse.org/r/97007 was merged to [master]. Commit: http://git.eclipse.org/c/jdt/eclipse.jdt.core.git/commit/?id=de4760f66363d73ec203ed3af242baabfad7daa2
(In reply to Eclipse Genie from comment #27) > Gerrit change https://git.eclipse.org/r/97007 was merged to [master]. > Commit: > http://git.eclipse.org/c/jdt/eclipse.jdt.core.git/commit/?id=de4760f66363d73ec203ed3af242baabfad7daa2 > committed the logging info addition code by Andrey Loskutov anyway - ref comment 19 - Thanks Andrey!
(In reply to Manoj Palat from comment #25) > (In reply to Eclipse Genie from comment #24) > > Gerrit change https://git.eclipse.org/r/99141 was merged to [master]. > > Commit: > > http://git.eclipse.org/c/jdt/eclipse.jdt.core.git/commit/?id=1c9be7804c5f24d8d2857bf04bda58d61ad462eb > > > > @Jens: Could you please try tomorrow's I-Build (that will have this change) > and checkout whether the issue is resolved/improved at your end? TIA. We tried the fix and it seems that the above change improved the issue but couldn't resolve it. I'm going to provide more logging information.
Created attachment 269351 [details] Eclipse Logfile
(In reply to Thomas Eger from comment #29) > (In reply to Manoj Palat from comment #25) > > > > @Jens: Could you please try tomorrow's I-Build (that will have this change) > > and checkout whether the issue is resolved/improved at your end? TIA. > > We tried the fix and it seems that the above change improved the issue but > couldn't resolve it. I'm going to provide more logging information. Moving this out of 4.8M1 [practically to post Java 9] - however, the current code committed would stay on as it is reported to have improved the issue.
(In reply to Manoj Palat from comment #31) > (In reply to Thomas Eger from comment #29) > > (In reply to Manoj Palat from comment #25) > > > > > > @Jens: Could you please try tomorrow's I-Build (that will have this change) > > > and checkout whether the issue is resolved/improved at your end? TIA. > > > > We tried the fix and it seems that the above change improved the issue but > > couldn't resolve it. I'm going to provide more logging information. > > Moving this out of 4.8M1 [practically to post Java 9] - however, the current > code committed would stay on as it is reported to have improved the issue. Today I encountered the issue again. Therefore I'm able to provide additional information: My use case is to update the jars via subclipse svn team provider. That didn't work because the jars were locked. Restarting Eclipse and even restarting the pc didn't help. Finally the following workaround fixed the issue temporarily: Close the affected project via context menu in package explorer. Reopen it. After that I was able to perform the svn update.
I am actually having this problem in bndtools consistently for a customer. Doing a file leak I find hundreds of references from one place. (They are not garbage collected because I used string file references.) #XXX D:\projects\iguana_ld\de.sma.iguana.framework\de.sma.api\generated\de.sma.api.jar by thread:Java indexing on Thu Sep 28 08:51:46 CEST 2017 at java.util.zip.ZipFile.<init>(ZipFile.java:150) at java.util.zip.ZipFile.<init>(ZipFile.java:163) at org.eclipse.jdt.internal.core.JavaModelManager.getZipFile(JavaModelManager.java:2843) at org.eclipse.jdt.internal.core.JavaModelManager.getZipFile(JavaModelManager.java:2812) at org.eclipse.jdt.internal.core.search.matching.JavaSearchNameEnvironment.mapToClassPathLocation(JavaSearchNameEnvironment.java:113) at org.eclipse.jdt.internal.core.search.matching.JavaSearchNameEnvironment.computeClasspathLocations(JavaSearchNameEnvironment.java:101) at org.eclipse.jdt.internal.core.search.matching.JavaSearchNameEnvironment.<init>(JavaSearchNameEnvironment.java:54) at org.eclipse.jdt.internal.core.search.matching.IndexBasedJavaSearchEnvironment.create(IndexBasedJavaSearchEnvironment.java:325) at org.eclipse.jdt.internal.core.search.indexing.SourceIndexer.resolveDocument(SourceIndexer.java:162) at org.eclipse.jdt.internal.core.search.JavaSearchParticipant.resolveDocument(JavaSearchParticipant.java:102) at org.eclipse.jdt.internal.core.search.indexing.IndexManager.indexResolvedDocument(IndexManager.java:553) at org.eclipse.jdt.internal.core.search.indexing.IndexManager$1.execute(IndexManager.java:1045) at org.eclipse.jdt.internal.core.search.processing.JobManager.run(JobManager.java:396) I just did a check of the code and I could not find where the getZipFile() was closed. In the constructor of JavaSearchNameEnvironment the computeClasspathLocations is called for the locationSet field. This is a LinkedHashSet<ClasspathLocation>. To calculate each path it calls mapToClassPathLocation, which opens the zip file and stores it in the ClassPathLocation. In the JavaSearchNameEnvironment.cleanup method this list is cleared without clearing the entries itself. I.e. the file remains open and since the reference is gone this can no longer be closed. That said, this might minimise my problem but make it disappear because I also tried to do a GC before I try to delete that file and that did not work. I have therefore some work left. However, this seems a bug in the Eclipse code.
New Gerrit change created: https://git.eclipse.org/r/106041
(In reply to Peter Kriens from comment #33) > In the constructor of JavaSearchNameEnvironment the > computeClasspathLocations is called for the locationSet field. This is a > LinkedHashSet<ClasspathLocation>. To calculate each path it calls > mapToClassPathLocation, which opens the zip file and stores it in the > ClassPathLocation. > > In the JavaSearchNameEnvironment.cleanup method this list is cleared without > clearing the entries itself. I.e. the file remains open and since the > reference is gone this can no longer be closed. I've just pushed the patch, see https://git.eclipse.org/r/106041. I honestly have no idea if this fixes the problem or creates more and different other problems, but may be this is exact what was missed all the time?
(In reply to Andrey Loskutov from comment #35) > (In reply to Peter Kriens from comment #33) > > In the constructor of JavaSearchNameEnvironment the > > computeClasspathLocations is called for the locationSet field. This is a > > LinkedHashSet<ClasspathLocation>. To calculate each path it calls > > mapToClassPathLocation, which opens the zip file and stores it in the > > ClassPathLocation. > > > > In the JavaSearchNameEnvironment.cleanup method this list is cleared without > > clearing the entries itself. I.e. the file remains open and since the > > reference is gone this can no longer be closed. > > I've just pushed the patch, see https://git.eclipse.org/r/106041. > I honestly have no idea if this fixes the problem or creates more and > different other problems, but may be this is exact what was missed all the > time? Nope, it caused ~240 test failures, see https://hudson.eclipse.org/platform/job/eclipse.jdt.core-Gerrit/3389/#showFailuresLink They all fail with something like this: java.lang.IllegalStateException: zip file closed at java.util.zip.ZipFile.ensureOpen(ZipFile.java:669) at java.util.zip.ZipFile.getEntry(ZipFile.java:309) at org.eclipse.jdt.internal.core.AbstractClassFile.getClassFileContent(AbstractClassFile.java:197) at org.eclipse.jdt.internal.core.ModularClassFile.getJarBinaryModuleInfo(ModularClassFile.java:167) at org.eclipse.jdt.internal.core.ModularClassFile.getBinaryModuleInfo(ModularClassFile.java:132) at org.eclipse.jdt.internal.core.ModularClassFile.buildStructure(ModularClassFile.java:53) at org.eclipse.jdt.internal.core.Openable.generateInfos(Openable.java:259) at org.eclipse.jdt.internal.core.JavaElement.openWhenClosed(JavaElement.java:583) at org.eclipse.jdt.internal.core.JavaElement.getElementInfo(JavaElement.java:320) at org.eclipse.jdt.internal.core.JavaElement.getElementInfo(JavaElement.java:306) at org.eclipse.jdt.internal.core.JavaElement.exists(JavaElement.java:224) at org.eclipse.jdt.internal.core.Openable.exists(Openable.java:203) at org.eclipse.jdt.internal.core.PackageFragmentRoot.getModuleDescription(PackageFragmentRoot.java:872) at org.eclipse.jdt.internal.core.search.matching.JavaSearchNameEnvironment.mapToClassPathLocation(JavaSearchNameEnvironment.java:184) at org.eclipse.jdt.internal.core.search.matching.JavaSearchNameEnvironment.computeClasspathLocations(JavaSearchNameEnvironment.java:132) at org.eclipse.jdt.internal.core.search.matching.JavaSearchNameEnvironment.<init>(JavaSearchNameEnvironment.java:76) at org.eclipse.jdt.internal.core.search.matching.IndexBasedJavaSearchEnvironment.create(IndexBasedJavaSearchEnvironment.java:327) at org.eclipse.jdt.internal.core.search.matching.MatchLocator.initialize(MatchLocator.java:1209) at org.eclipse.jdt.internal.core.search.matching.MatchLocator.locateMatches(MatchLocator.java:1239) at org.eclipse.jdt.internal.core.search.matching.MatchLocator.locateMatches(MatchLocator.java:1355) at org.eclipse.jdt.internal.core.search.matching.MatchLocator.locateMatches(MatchLocator.java:1487) at org.eclipse.jdt.internal.core.search.JavaSearchParticipant.locateMatches(JavaSearchParticipant.java:122) at org.eclipse.jdt.internal.core.search.BasicSearchEngine.findMatches(BasicSearchEngine.java:236) at org.eclipse.jdt.internal.core.search.BasicSearchEngine.search(BasicSearchEngine.java:583) at org.eclipse.jdt.core.search.SearchEngine.search(SearchEngine.java:600)
Looks like we need a flushing of cache at ML.init as well similar to the earlier commit - looking into this now
(In reply to Manoj Palat from comment #37) > Looks like we need a flushing of cache at ML.init as well similar to the > earlier commit - looking into this now This does not seem to be the issue as the issue is at indexing. (In reply to Peter Kriens from comment #33) > I just did a check of the code and I could not find where the getZipFile() > was closed. These are cached at the JMM zip cache (SI:160 - initiates) and these zipfiles will be closed inside the finally block (174) during flush. So the remaining possibility is by some chance the ZipCache retrieval/adding fails - investigating in those lines. Just checking- is there any test workspace which you can send in that exhibits this problem frequently?
(In reply to Peter Kriens from comment #33) > > Doing a file leak I find hundreds of references from one place. (They are > not garbage collected because I used string file references.) > One more thing to check here : Was the file leak detector configured to provide an output when the number of open file desc reached a particular number? If that is the case, please increase the limit to see whether that reduces - based on the premise that the indexing is on-going and is yet to finish. If it is yet to finish, then I expect the flush to close these descriptors bringing down the number of open descriptors.
bulk move out of 4.8
This bug hasn't had any activity in quite some time. Maybe the problem got resolved, was a duplicate of something else, or became less pressing for some reason - or maybe it's still relevant but just hasn't been looked at yet. If you have further information on the current state of the bug, please add it. The information can be, for example, that the problem still occurs, that you still want the feature, that more information is needed, or that the bug is (for whatever reason) no longer relevant. -- The automated Eclipse Genie.
Problem is still present in latest release 2020-03. Some how the file handle is not closed. I do not exactly know when the problem occures, but it always happens after some time working with eclipse rcp ide.
New Gerrit change created: https://git.eclipse.org/r/c/jdt/eclipse.jdt.core/+/170353
(In reply to Eclipse Genie from comment #43) > New Gerrit change created: > https://git.eclipse.org/r/c/jdt/eclipse.jdt.core/+/170353 The stacktrace for where such a JAR file is opened first is irrelevant to some degree, as it turns out that java.util.zip.ZipFile does open the JAR filesystem handle just once for multiple instances of ZipFile, and does count references instead. The ZipFile handle leak in that one ClasspathMultiReleaseJar constructor seems obvious to me, but probably there are more such leaks around, as JAR files still seem to remain open.
(In reply to Michael Haubenwallner from comment #44) > (In reply to Eclipse Genie from comment #43) > > New Gerrit change created: > > https://git.eclipse.org/r/c/jdt/eclipse.jdt.core/+/170353 > > The stacktrace for where such a JAR file is opened first is irrelevant to > some degree, as it turns out that java.util.zip.ZipFile does open the JAR > filesystem handle just once for multiple instances of ZipFile, and does > count references instead. > > The ZipFile handle leak in that one ClasspathMultiReleaseJar constructor > seems obvious to me, but probably there are more such leaks around, as JAR > files still seem to remain open. Michael, could you please create a dedicated bug for ClasspathMultiReleaseJar, because the current one is too old / has nothing to do with ClasspathMultiReleaseJar that was introduced recently. Also please provide steps to reproduce on the new bug. (In reply to Stefan Prieschl from comment #42) > Problem is still present in latest release 2020-03. > Some how the file handle is not closed. > I do not exactly know when the problem occures, but it always happens after > some time working with eclipse rcp ide. Also here: if you have steps to reproduce, please provide them. Just saying that "jars are locked after some work" is not a bug, because jars could be opened for reading (and not closed yet) for a good reason.
As far as I can see, when executed on Windows, lots of the existing test cases already do run into a problem like this during their teardown, when they are about to remove their jar files. A console output example: > WARNING in test: org.eclipse.jdt.core.tests.compiler.regression.DeprecatedTest - tearDown > - problems occured while deleting C:\Users\haubi\AppData\Local\Temp\comptest\run.1604042499404\lib\p008\x.jar > - stack trace: > -> org.eclipse.jdt.core.tests.util.Util.waitUntilFileDeleted(Util.java:1318) > -> org.eclipse.jdt.core.tests.util.Util.delete(Util.java:516) > -> org.eclipse.jdt.core.tests.util.Util.flushDirectoryContent(Util.java:773) > -> org.eclipse.jdt.core.tests.util.Util.delete(Util.java:509) > -> org.eclipse.jdt.core.tests.util.Util.flushDirectoryContent(Util.java:773) > -> org.eclipse.jdt.core.tests.compiler.regression.AbstractRegressionTest.tearDown(AbstractRegressionTest.java:4057) > -> org.eclipse.jdt.core.tests.compiler.regression.DeprecatedTest.tearDown(DeprecatedTest.java:51) > - p008 file info: read, write, exist, dir > + children: > - x.jar file info: read, write, exist, file > > !!! ERROR: C:\Users\haubi\AppData\Local\Temp\comptest\run.1604042499404\lib\p008\x.jar was never deleted even after having waited 10000ms!!!
For the records: Here, we have improved the situation a lot, yet not completely fixed, with two changes to the code that does potentially modify jar files in the workspace: Close handles to jar file providing annotations: change https://git.eclipse.org/r/166704, bug#565436 Stop Indexer to release jar file handles and run GC to close them: change https://git.eclipse.org/r/172532, bug#568965
