Bug 550786 - Files in output directories cannot be deleted during SVN update
Summary: Files in output directories cannot be deleted during SVN update
Status: NEW
Alias: None
Product: JDT
Classification: Eclipse Project
Component: Core (show other bugs)
Version: 4.13   Edit
Hardware: PC Windows 8
: P3 normal (vote)
Target Milestone: ---   Edit
Assignee: JDT-Core-Inbox CLA
QA Contact:
URL:
Whiteboard: stalebug
Keywords:
Depends on:
Blocks:
 
Reported: 2019-09-05 03:49 EDT by Alexander Veit CLA
Modified: 2023-09-06 16:09 EDT (History)
1 user (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Alexander Veit CLA 2019-09-05 03:49:05 EDT
OS: Windows 8.1

Eclipse IDE for Enterprise Java Developers.
Version: 2019-09 M3 (4.13.0M3)
Build id: 20190829-1310

Subclipse	Subversion Revision Graph  4.2.0.1	org.tigris.subversion.subclipse.graph.feature
Subclipse	SVN Team Provider Core  4.3.0.201901172050	org.tigris.subversion.subclipse
Subclipse	SVNKit Client Adapter  1.8.12.3	org.tigris.subversion.clientadapter.svnkit.feature

Running SVN update with Subclipse frequently produces errors due to non-deletable files in the configured output directories (probably Windows file locks).

Usually many projects in the workspace are updated simultaneously. "Project/Build Automatically" is enabled.

No virus scanner is running. Up to now I could not detect any sign indicating that a process external to Eclipse could have locked the files.


Error occurred while deleting: /push-protocol/target/test-classes/de

org.eclipse.core.internal.resources.ResourceException: Problems encountered while deleting resources.
	at org.eclipse.core.internal.resources.Resource.delete(Resource.java:767)
	at org.eclipse.jdt.internal.core.builder.BatchImageBuilder.cleanOutputFolders(BatchImageBuilder.java:140)
	at org.eclipse.jdt.internal.core.builder.BatchImageBuilder.build(BatchImageBuilder.java:65)
	at org.eclipse.jdt.internal.core.builder.JavaBuilder.buildAll(JavaBuilder.java:278)
	at org.eclipse.jdt.internal.core.builder.JavaBuilder.build(JavaBuilder.java:187)
	at org.eclipse.core.internal.events.BuildManager$2.run(BuildManager.java:833)
	at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:45)
	at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:220)
	at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:263)
	at org.eclipse.core.internal.events.BuildManager$1.run(BuildManager.java:316)
	at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:45)
	at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:319)
	at org.eclipse.core.internal.events.BuildManager.basicBuildLoop(BuildManager.java:371)
	at org.eclipse.core.internal.events.BuildManager.build(BuildManager.java:392)
	at org.eclipse.core.internal.events.AutoBuildJob.doBuild(AutoBuildJob.java:154)
	at org.eclipse.core.internal.events.AutoBuildJob.run(AutoBuildJob.java:244)
	at org.eclipse.core.internal.jobs.Worker.run(Worker.java:63)
Contains: Could not delete '/push-protocol/target/test-classes/de'.
org.eclipse.core.internal.resources.ResourceException: Problems encountered while deleting resources.
	at org.eclipse.core.internal.localstore.FileSystemResourceManager.delete(FileSystemResourceManager.java:367)
	at org.eclipse.core.internal.resources.ResourceTree.internalDeleteFolder(ResourceTree.java:361)
	at org.eclipse.core.internal.resources.ResourceTree.standardDeleteFolder(ResourceTree.java:813)
	at org.eclipse.core.internal.resources.Resource.unprotectedDelete(Resource.java:1773)
	at org.eclipse.core.internal.resources.Resource.delete(Resource.java:754)
	at org.eclipse.jdt.internal.core.builder.BatchImageBuilder.cleanOutputFolders(BatchImageBuilder.java:140)
	at org.eclipse.jdt.internal.core.builder.BatchImageBuilder.build(BatchImageBuilder.java:65)
	at org.eclipse.jdt.internal.core.builder.JavaBuilder.buildAll(JavaBuilder.java:278)
	at org.eclipse.jdt.internal.core.builder.JavaBuilder.build(JavaBuilder.java:187)
	at org.eclipse.core.internal.events.BuildManager$2.run(BuildManager.java:833)
	at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:45)
	at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:220)
	at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:263)
	at org.eclipse.core.internal.events.BuildManager$1.run(BuildManager.java:316)
	at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:45)
	at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:319)
	at org.eclipse.core.internal.events.BuildManager.basicBuildLoop(BuildManager.java:371)
	at org.eclipse.core.internal.events.BuildManager.build(BuildManager.java:392)
	at org.eclipse.core.internal.events.AutoBuildJob.doBuild(AutoBuildJob.java:154)
	at org.eclipse.core.internal.events.AutoBuildJob.run(AutoBuildJob.java:244)
	at org.eclipse.core.internal.jobs.Worker.run(Worker.java:63)
Contains: Problems encountered while deleting files.
Contains: Could not delete: D:\prj\trunk\modules\push-protocol\target\test-classes\de\uplanet\pushnotification.
java.nio.file.DirectoryNotEmptyException: D:\prj\trunk\modules\push-protocol\target\test-classes\de\uplanet\pushnotification
	at sun.nio.fs.WindowsFileSystemProvider.implDelete(WindowsFileSystemProvider.java:266)
	at sun.nio.fs.AbstractFileSystemProvider.deleteIfExists(AbstractFileSystemProvider.java:108)
	at java.nio.file.Files.deleteIfExists(Files.java:1165)
	at org.eclipse.core.internal.filesystem.local.LocalFile.internalDelete(LocalFile.java:256)
	at org.eclipse.core.internal.filesystem.local.LocalFile.internalDelete(LocalFile.java:251)
	at org.eclipse.core.internal.filesystem.local.LocalFile.internalDelete(LocalFile.java:251)
	at org.eclipse.core.internal.filesystem.local.LocalFile.delete(LocalFile.java:140)
	at org.eclipse.core.internal.localstore.DeleteVisitor.delete(DeleteVisitor.java:66)
	at org.eclipse.core.internal.localstore.DeleteVisitor.visit(DeleteVisitor.java:154)
	at org.eclipse.core.internal.localstore.UnifiedTree.accept(UnifiedTree.java:118)
	at org.eclipse.core.internal.localstore.FileSystemResourceManager.delete(FileSystemResourceManager.java:364)
	at org.eclipse.core.internal.resources.ResourceTree.internalDeleteFolder(ResourceTree.java:361)
	at org.eclipse.core.internal.resources.ResourceTree.standardDeleteFolder(ResourceTree.java:813)
	at org.eclipse.core.internal.resources.Resource.unprotectedDelete(Resource.java:1773)
	at org.eclipse.core.internal.resources.Resource.delete(Resource.java:754)
	at org.eclipse.jdt.internal.core.builder.BatchImageBuilder.cleanOutputFolders(BatchImageBuilder.java:140)
	at org.eclipse.jdt.internal.core.builder.BatchImageBuilder.build(BatchImageBuilder.java:65)
	at org.eclipse.jdt.internal.core.builder.JavaBuilder.buildAll(JavaBuilder.java:278)
	at org.eclipse.jdt.internal.core.builder.JavaBuilder.build(JavaBuilder.java:187)
	at org.eclipse.core.internal.events.BuildManager$2.run(BuildManager.java:833)
	at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:45)
	at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:220)
	at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:263)
	at org.eclipse.core.internal.events.BuildManager$1.run(BuildManager.java:316)
	at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:45)
	at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:319)
	at org.eclipse.core.internal.events.BuildManager.basicBuildLoop(BuildManager.java:371)
	at org.eclipse.core.internal.events.BuildManager.build(BuildManager.java:392)
	at org.eclipse.core.internal.events.AutoBuildJob.doBuild(AutoBuildJob.java:154)
	at org.eclipse.core.internal.events.AutoBuildJob.run(AutoBuildJob.java:244)
	at org.eclipse.core.internal.jobs.Worker.run(Worker.java:63)
Contains: Could not delete: D:\prj\trunk\modules\push-protocol\target\test-classes\de\uplanet.
Contains: Could not delete: D:\prj\trunk\modules\push-protocol\target\test-classes\de.

eclipse.buildId=4.13.0.I20190821-1800
java.version=1.8.0_202
java.vendor=Oracle Corporation
BootLoader constants: OS=win32, ARCH=x86_64, WS=win32, NL=de_DE
Framework arguments:  -product org.eclipse.epp.package.jee.product
Command-line arguments:  -os win32 -ws win32 -arch x86_64 -product org.eclipse.epp.package.jee.product -data D:\prj\trunk\workspace
Comment 1 Alexander Veit CLA 2019-09-24 03:10:24 EDT
The problem persists with Eclipse 2019-09

Eclipse IDE for Enterprise Java Developers.
Version: 2019-09 R (4.13.0)
Build id: 20190917-1200

Again SVN update fails when new versions of libraries should replace existing ones.

E.g. this is one of the error messages observed:

org.apache.subversion.javahl.ClientException: Problem running log
svn: Failed to run the WC DB work queue associated with 'D:\prj\modules\common\lib', work item 25375 (file-remove lib/bcpkix-jdk15on-1.62.jar)
Der Prozess kann nicht auf die Datei zugreifen, da sie von einem anderen Prozess verwendet wird.  
svn: Can't remove file 'D:\prj\modules\common\lib\bcpkix-jdk15on-1.62.jar': Der Prozess kann nicht auf die Datei zugreifen, da sie von einem anderen Prozess verwendet wird.  

org.apache.subversion.javahl.ClientException: Problem running log
svn: Failed to run the WC DB work queue associated with 'D:\prj\modules\common\lib', work item 25375 (file-remove lib/bcpkix-jdk15on-1.62.jar)
Der Prozess kann nicht auf die Datei zugreifen, da sie von einem anderen Prozess verwendet wird.  
svn: Can't remove file 'D:\prj\modules\common\lib\bcpkix-jdk15on-1.62.jar': Der Prozess kann nicht auf die Datei zugreifen, da sie von einem anderen Prozess verwendet wird. 

A dump of open file handles (http://file-leak-detector.kohsuke.org/) shows 273 open handles to different JAR files. All are opened with the same stack, e.g.:

#155 D:\prj\modules\common\lib\bcpkix-jdk15on-1.62.jar by thread:Worker-8: Building workspace on Tue Sep 24 08:47:22 CEST 2019
	at java.util.zip.ZipFile.<init>(ZipFile.java:156)
	at java.util.zip.ZipFile.<init>(ZipFile.java:126)
	at org.eclipse.jdt.internal.core.builder.ClasspathMultiReleaseJar.initializeVersions(ClasspathMultiReleaseJar.java:96)
	at org.eclipse.jdt.internal.core.builder.ClasspathMultiReleaseJar.<init>(ClasspathMultiReleaseJar.java:36)
	at org.eclipse.jdt.internal.core.builder.ClasspathLocation.forLibrary(ClasspathLocation.java:159)
	at org.eclipse.jdt.internal.core.builder.NameEnvironment.computeClasspathLocations(NameEnvironment.java:274)
	at org.eclipse.jdt.internal.core.builder.NameEnvironment.<init>(NameEnvironment.java:62)
	at org.eclipse.jdt.internal.core.builder.JavaBuilder.initializeBuilder(JavaBuilder.java:651)
	at org.eclipse.jdt.internal.core.builder.JavaBuilder.build(JavaBuilder.java:181)
	at org.eclipse.core.internal.events.BuildManager$2.run(BuildManager.java:833)
	at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:45)
	at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:220)
	at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:263)
	at org.eclipse.core.internal.events.BuildManager$1.run(BuildManager.java:316)
	at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:45)
	at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:319)
	at org.eclipse.core.internal.events.BuildManager.basicBuildLoop(BuildManager.java:371)
	at org.eclipse.core.internal.events.BuildManager.build(BuildManager.java:392)
	at org.eclipse.core.internal.events.AutoBuildJob.doBuild(AutoBuildJob.java:154)
	at org.eclipse.core.internal.events.AutoBuildJob.run(AutoBuildJob.java:244)
	at org.eclipse.core.internal.jobs.Worker.run(Worker.java:63)

where bcpkix-jdk15on-1.62.jar is the JAR file that should be replaced with the newer version of the library that comes in with the SVN update.
Comment 2 Andrey Loskutov CLA 2019-09-24 07:15:56 EDT
Alexander, can you please provide a simple standalone project that can be used to reproduce the problem, and the steps to reproduce using that simple project.

Note: we don't need SVN to detect file leaks (opened but not closed files). On Windows one can try simply to remove them and this should fail, on Linux we have "lsof".
Comment 3 Alexander Veit CLA 2019-09-24 07:58:58 EDT
I'll try to provide a standalone project.

However, from the attached stack that has been dumped with File Leak Detector, you can see that the ClasspathMultiReleaseJar seems not to be closed correctly.

The same open file handles can be observed during a lenghty build triggered by Project/Clean. In this case the handles are being closed after the build completed successfully.

So the cause of the problem might be the error caused by trying to replace/delete a temporarily open file, and possibly a missing finally-block for closing the file handles held by ClasspathMultiReleaseJar, or an uncleared object reference that prevents these objects from being garbage-collected.
Comment 4 Andrey Loskutov CLA 2019-09-24 08:04:06 EDT
(In reply to Alexander Veit from comment #3)
> I'll try to provide a standalone project.
> 
> However, from the attached stack that has been dumped with File Leak
> Detector, you can see that the ClasspathMultiReleaseJar seems not to be
> closed correctly.

This was clear, however the life cycle of the opened object is what causes trouble, and having a clear example to reproduce will simplify / speed up the possible patch.

Please note, if you observe the opened files *during build*, we might have no real solution for this, because we *must* read libraries during the build. So if SVN tries to delete files ony-by-one *without* locking workspace (and that triggers the build), I fear the solution would be that SVN should lock the workspace during update => not a JDT bug.
Comment 5 Eclipse Genie CLA 2021-09-14 16:26:55 EDT
This bug hasn't had any activity in quite some time. Maybe the problem got resolved, was a duplicate of something else, or became less pressing for some reason - or maybe it's still relevant but just hasn't been looked at yet.

If you have further information on the current state of the bug, please add it. The information can be, for example, that the problem still occurs, that you still want the feature, that more information is needed, or that the bug is (for whatever reason) no longer relevant.

--
The automated Eclipse Genie.
Comment 6 Eclipse Genie CLA 2023-09-06 16:09:10 EDT
This bug hasn't had any activity in quite some time. Maybe the problem got resolved, was a duplicate of something else, or became less pressing for some reason - or maybe it's still relevant but just hasn't been looked at yet.

If you have further information on the current state of the bug, please add it. The information can be, for example, that the problem still occurs, that you still want the feature, that more information is needed, or that the bug is (for whatever reason) no longer relevant.

--
The automated Eclipse Genie.