Bug 154693 - project clean & build sometimes copies subversion .svn folders to bin tree
Summary: project clean & build sometimes copies subversion .svn folders to bin tree
Status: VERIFIED FIXED
Alias: None
Product: JDT
Classification: Eclipse Project
Component: Core (show other bugs)
Version: 3.2   Edit
Hardware: PC Windows XP
: P3 normal with 1 vote (vote)
Target Milestone: 3.3 RC1   Edit
Assignee: Kent Johnson CLA
QA Contact:
URL:
Whiteboard:
Keywords:
: 176005 (view as bug list)
Depends on:
Blocks:
 
Reported: 2006-08-22 09:17 EDT by Ray H CLA
Modified: 2009-06-09 11:42 EDT (History)
7 users (show)

See Also:
Olivier_Thomann: review+


Attachments
Proposed patch (1.06 KB, patch)
2007-05-04 15:46 EDT, Kent Johnson CLA
no flags Details | Diff
Regression test (1.44 KB, patch)
2007-05-07 15:06 EDT, Olivier Thomann CLA
no flags Details | Diff
Released patch (2.93 KB, patch)
2007-05-07 15:26 EDT, Kent Johnson CLA
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Ray H CLA 2006-08-22 09:17:44 EDT
+++ This bug was initially created as a clone of Bug #111368 +++

When I checkout a working project folder from subversion, the source tree has
".svn" administrative folders throughout the tree.  When I import the project
into my workspace, or whenever I clean the project, the automatic build process
ends up copying the .svn folders into the bin tree.  If I run an update over the
top-level project folder, subversion/TortoiseSVN gets confused and "restores"
copies of source files into the bin tree.  This led to much confusion until we
figured out what was happening.
Comment 1 Ray H CLA 2006-08-22 09:21:43 EDT
Hello Eclipse Team,

is still see this bug sometimes.

I use the resource filter option: *.launch, .svn
But sometimes it still copies the .svn folder to the bin ( or classes ) folder.

My problem is, I still havn't nailed it when it happens... as I have
not found a way to reproduce it yet...
I have a feeling it happens when I use the "clean..." option.

Comment 2 Olivier Thomann CLA 2006-08-23 20:28:55 EDT
I'll try to reproduce.
Comment 3 Alberto Lovato CLA 2006-09-13 04:50:47 EDT
(In reply to comment #2)
> I'll try to reproduce.
> 

It's simple to reproduce: refresh the project (F5), and the .svn dirs
are copied (recursively) from source folders to classes folders.
I can't understand the logic behind it - should not eclipse only
place compiled files in bin folders?
Comment 4 Ray H CLA 2006-09-14 04:36:15 EDT
No, javac and eclipse copy all files, except java files to the bin folder.
That is correct and expected behaviour.
But it should ignore my filtered recources, which
yesterday, again, didn't work. From the time stamp I could tell
it happened when I started working, so maybe my eclipse start, or an
initial refresh...
Comment 5 Ray H CLA 2006-09-14 05:37:51 EDT
Okay, It just happened again.

Here is what I did:

Before I started eclipse I deleted all my files in my classes folder.
( Which is my bin target folder )

It had a com folder and a erroneous .svn folder on the first level.

Then I started ecplise, which is set to auto build.

Nothing happened.

Then I right clicked on the project and said refresh.

Then they came, all folders and subfolders where created toghether
with their .svn counterpart.

Interestingly no class files where created...

After I clicked "Clean..." the class files where all there and the .svn
folders where gone...


Comment 6 Ray H CLA 2006-09-14 05:38:59 EDT
Please note, that I again cannot reproduce this behavior.
I assume it is connected to some internal state...
Comment 7 Jaeseok Park CLA 2006-11-24 00:32:47 EST
I think filtering resources option doesn't work on folders.
So I filtered '*.svn' files and avoided this problem.
(Still '.svn' folders are copied into output folder. but, subversion works well.)
Comment 8 Andrea Aime CLA 2007-01-02 06:11:11 EST
I'm having the same problem. 
Eclipse 3.2.1/Windows keeps on copying the .svn folders even if my resource filter is:
*.launch, .svn/, .svn, *.svn, .svn*
(tried adding extra filters out of desperation, does not work).
Comment 9 Frederic Fusier CLA 2007-03-01 06:34:54 EST
*** Bug 176005 has been marked as a duplicate of this bug. ***
Comment 10 Issa Gorissen CLA 2007-03-02 12:16:59 EST
I think I found the problem.

Eclipse team, please review and confirm.

It seems that Path class gives segment with separator stripped (/ is not returned). But comparison is being done inside JavaBuilder.filterExtraResource with values stored inside array extraResourceFolderFilters. Values in this array contains separator.

The fix is in method JavaBuilder.initializeBuilder, during the construction of array extraResourceFolderFilters. Please find my fix below.


.
.
.
if (forBuild) {
	String filterSequence = javaProject.getOption(JavaCore.CORE_JAVA_BUILD_RESOURCE_COPY_FILTER, true);
	char[][] filters = filterSequence != null && filterSequence.length() > 0
		? CharOperation.splitAndTrimOn(',', filterSequence.toCharArray())
		: null;
	if (filters == null) {
		this.extraResourceFileFilters = null;
		this.extraResourceFolderFilters = null;
	} else {
		int fileCount = 0, folderCount = 0;
		for (int i = 0, l = filters.length; i < l; i++) {
			char[] f = filters[i];
			if (f.length == 0) continue;
			if (f[f.length - 1] != '/') {
				fileCount++;
			} else if (f.length >= 2) {
				folderCount++;
			}
		}
		this.extraResourceFileFilters = new char[fileCount][];
		this.extraResourceFolderFilters = new String[folderCount];
		for (int i = 0, l = filters.length; i < l; i++) {
			char[] f = filters[i];
			if (f.length == 0) continue;
			if (f[f.length - 1] != '/') {
				extraResourceFileFilters[--fileCount] = f;
			} else if (f.length >= 2) {
				extraResourceFolderFilters[--folderCount] = new String(f, 0, f.length - 2);
			}
		}
	}
}
.
.
.

From the implementation I saw, I can add the comment that a more precise description of what to enter should be added below that Preferences textbox. Because folders must be entered with a trailing / and this is not obvious.
Comment 11 Kent Johnson CLA 2007-03-05 16:12:25 EST
Sorry, but I must be missing something.

We're already removing the trailing '/' when its a folder:

if (f[f.length - 1] == '/')
  extraResourceFolderFilters[--folderCount] = new String(f, 0, f.length - 1);

Why would we remove the trailing '/' and the last character?
Comment 12 Issa Gorissen CLA 2007-03-12 08:19:50 EDT
Indeed, I have checked again, and my previous report is irrelevant. The problem must lie somewhere in the code which copies the resources to the output folder.
Comment 13 Kent Johnson CLA 2007-03-23 15:19:28 EDT
We need to gather some information from everyone who is experiencing this problem.

1. Which exact version of eclipse are you using? Also list any other plugin versions that are relevant.

2. As John asked in bug 111368, are any of using the subversion plugin? And if not, why not? Is anyone using the plugin also seeing this problem?

3. If there is some reason you cannot use it, then what are your resource filters?

4. Now delete the contents of the bin directory
 - then startup eclipse & turn off Auto-build
 - perform a simple refresh of your project
 - is there anything in the bin folder?
 - now perform a clean... on your project, BUT not a build
 - now, do any .svn folders exist in the bin folder?
 - finally perform a build using crtl-B
 - do .svn folders appear now?
Comment 14 Issa Gorissen CLA 2007-03-26 03:56:46 EDT
Answer for comment 13

1. Eclipse  build id: M20070212-1330

2. Not using subversion plugin

3. Not willing to use subversion plugin

4. I must say that I am very confuse now - I did all the steps and I can no more see any remaining .svn folders!
Comment 15 Kent Johnson CLA 2007-03-26 09:56:45 EDT
If none of the steps in part 4 result in .svn folders appearing in the output folder, then this problem is likely not related to the JavaBuilder.

Its possible you have another builder which could be copying resources & folders, or there is another task that is the culprit.

For now, please pay close attention to when the .svn folders appear. Then if you could do a clean/full build & retry the last few things you did prior to the folders appearing.

Hopefully we'll get a reproduceable case.
Comment 16 Kent Johnson CLA 2007-03-28 12:25:50 EDT
So no one else is seeing this problem anymore?

And no one has a reproduceable case to help us track it down?

If you do, please respond to comment #13
Comment 17 Issa Gorissen CLA 2007-04-04 11:51:29 EDT
Answer for comment 13

Sorry for the confusion.

1. Eclipse  build id: M20070212-1330

2. Not using subversion plugin

3. Not willing to use subversion plugin

4. I still see the folders .svn inside bin folder. Sorry for the confusion with my comment 14 - but the reason I did not see them previously was that someone else in my big project changed the .classpath file and added an exclusion rule like this one : **/.svn/* - I removed this exclusion, but left the global parameter like this /instance/org.eclipse.jdt.core/org.eclipse.jdt.core.builder.resourceCopyExclusionFilter=*.launch, .svn/

So, this is a no go. I can reproduce this problem. How can I help, what should I test to give you more input?
Comment 18 Kent Johnson CLA 2007-04-04 12:01:37 EDT
Perform the steps in part 4 of comment 13 & tell us when exactly the .svn folders first appear.
Comment 19 Issa Gorissen CLA 2007-04-05 04:59:04 EDT
The .svn folders appear inside the bin folder after I perform a build of the project by using Ctrl-B.

.classpath content is

<?xml version="1.0" encoding="UTF-8"?>
<classpath>
	<classpathentry kind="src" path="src"/>
	<classpathentry kind="con" path="org.eclipse.jdt.launching.JRE_CONTAINER"/>
	<classpathentry combineaccessrules="false" kind="src" path="/p_common_libs"/>
	<classpathentry kind="lib" path="/p_common_libs/libs/commons-codec.jar"/>
	<classpathentry kind="output" path="bin"/>
</classpath>




.project content is

<?xml version="1.0" encoding="UTF-8"?>
<projectDescription>
	<name>MD5Gui</name>
	<comment></comment>
	<projects>
	</projects>
	<buildSpec>
		<buildCommand>
			<name>org.eclipse.jdt.core.javabuilder</name>
			<arguments>
			</arguments>
		</buildCommand>
	</buildSpec>
	<natures>
		<nature>org.eclipse.jem.workbench.JavaEMFNature</nature>
		<nature>org.eclipse.jdt.core.javanature</nature>
		<nature>org.eclipse.jem.beaninfo.BeanInfoNature</nature>
	</natures>
</projectDescription>
Comment 20 Isaac Wilcox CLA 2007-04-26 12:04:43 EDT
(In reply to comment #13)
> We need to gather some information from everyone who is experiencing this
> problem.
> 
> 1. Which exact version of eclipse are you using? Also list any other plugin
> versions that are relevant.


Version: 3.2.2
Build id: M20070212-1330

I'm running Eclipse on Mac OS X 10.4.9.

I'm new to Eclipse (imported my project into it yesterday) and have no plugins installed yet.


> 2. As John asked in bug 111368, are any of using the subversion plugin? And if
> not, why not? Is anyone using the plugin also seeing this problem?


I'm not using the subversion plugin because I prefer to keep version control and development separate.  It's not really relevant, though, because it's not subversion-specific.  Any directory in the source tree (name .svn or anything else) sneaks through the filter, whatever filter expressions I try (see below).

 
> 3. If there is some reason you cannot use it, then what are your resource
> filters?


While trying to narrow the problem down I added a directory called "parent" to my source tree, and tried the following filters (each time following exactly the steps in question 4 below).  I tried them all in isolation, so the text you see is the only text that was in the "Filtered resources" field.

parent
parent/
*parent*/
*parent

None of these filters worked - "parent" always gets copied to my output directory.

 
> 4. Now delete the contents of the bin directory
>  - then startup eclipse & turn off Auto-build
>  - perform a simple refresh of your project
>  - is there anything in the bin folder?
>  - now perform a clean... on your project, BUT not a build
>  - now, do any .svn folders exist in the bin folder?
>  - finally perform a build using crtl-B
>  - do .svn folders appear now?


The folders (parent, .svn, whatever) appear only after I "finally perform a build".


As I don't have much invested in Eclipse just yet, I'm quite happy to completely wipe it (and all its preferences, settings etc) from my system and start afresh in order to try to come up with a set of reproduction steps absolutely from scratch (i.e. including constructing some made-up Java project).  Would that help?

For me, the main reason this messes up my day is that my output directory is under version control (for no good reason; a hangover from my previous Makefile/vim setup).  If I just take the output directory out of Subversion then 'svn' won't look in there for a .svn directory, and won't get confused by the overwriting of the original.  That's my workaround for now, FWIW, but ideally I'd like to be able to produce a completely clean output directory.
Comment 21 Kent Johnson CLA 2007-04-26 12:19:32 EDT
Just to be clear... you said:

>None of these filters worked - "parent" always gets copied to my output
>directory.

So do the files in the parent folder also get copied or just the folder itself?
Comment 22 Isaac Wilcox CLA 2007-04-26 12:53:00 EDT
(In reply to comment #21)
> Just to be clear... you said:
> 
> >None of these filters worked - "parent" always gets copied to my output
> >directory.
> 
> So do the files in the parent folder also get copied or just the folder itself? 

Couldn't remember for sure, so retested with the following tree created (and refreshed) under src:
src/thisisatestdir/
src/thisisatestdir/thisisatestfile
src/thisisatestdir/thisisanothertestdir/
src/thisisatestdir/thisisanothertestdir/thisisanothertestfile

Then I ran through all the steps again four times, once with each resource filter...

Filters:
  *thisisatestdir*/
  (an empty filter, i.e. "")
both gave me exactly what I had in src/.  Not sure that's what I'd expect with "*thisisatestdir*/", even with the bug!

Filters:
  thisisatestdir
  *thisisatestdir
  thisisatestdir/
all gave me the same thing:
  <outputdir>/thisisatestdir/
  <outputdir>/thisisatestdir/thisisanothertestdir/
  (plus all my real sources, obviously)
i.e. no thisisa*file files, but the directory tree was preserved.  This is what I remember happening to my .svn directories, now I think about it.
Comment 23 Kent Johnson CLA 2007-05-04 15:46:57 EDT
Created attachment 65956 [details]
Proposed patch

Believe it or not, we finally have a 'fix' for this problem.

We currently create package folders in the output folder while we're collecting source files. The resource filter is not used to decide if a folder has source files so even tho we won't copy extra resources from the folder, we will walk it to find .java files & as a result, create it in the output folder. With this change, we'll only create it if it can be a valid package name.

If you had excluded the folder entirely, then we wouldn't have walked it to find .java files or extra resources (see the Excluded list of each source folder on a project's build path).
Comment 24 Isaac Wilcox CLA 2007-05-04 16:07:13 EDT
(In reply to comment #23)
> Believe it or not, we finally have a 'fix' for this problem.
[...]
> If you had excluded the folder entirely, then we wouldn't have walked it to
> find .java files or extra resources (see the Excluded list of each source
> folder on a project's build path).

Patch looks like it'll DWIM - nice work.

I suspect that people will still get caught out trying to use "Filtered Resources" to filter directories *without* dots in the names, though, and end up trawling bugzilla.  Using the per-source exclusion list never occurred to me - though to be fair to Eclipse, I am a newbie.  To newbie-proof the problem, I'd strongly suggest adding a few words to the docs for the "Filtered resources" field that make clear exactly what the filter will and won't apply to, and mention the per-source exclusion list.

Just my $0.02.  Thanks for your efforts!
Comment 25 Olivier Thomann CLA 2007-05-07 15:06:19 EDT
Created attachment 66190 [details]
Regression test
Comment 26 Kent Johnson CLA 2007-05-07 15:26:53 EDT
Created attachment 66196 [details]
Released patch

Replaced package test with call to JavaConventions to validate name
Comment 27 Kent Johnson CLA 2007-05-07 15:28:39 EDT
Released into HEAD for 3.3RC1
Comment 28 Olivier Thomann CLA 2007-05-07 15:33:24 EDT
+1
Comment 29 David Audel CLA 2007-05-15 07:14:56 EDT
Verified for 3.3RC1 using I20070515-0010

The problem is fixed for .svn but when a filtered directory has a valid java name then the directory is copied in the bin folder.
Kent - this behavior is correct ?
Comment 30 Kent Johnson CLA 2007-05-15 10:07:18 EDT
Yes

We must create all package folders with .java files in the output folder, even if the extra resources are being filtered.

But we do not want to do it for folder names that are not valid packages.
Comment 31 Cyrille CLA 2007-07-16 21:39:48 EDT
(In reply to comment #27)
> Released into HEAD for 3.3RC1
> 

Hello

I've updated eclipse to 3.3.0 build I20070621-1340
and still have the bug.
Should I report my configuration like told in comment #13 ?

cheers, cyrille
Comment 32 Cyrille CLA 2007-07-20 08:48:44 EDT
(In reply to comment #31)
> (In reply to comment #27)
> > Released into HEAD for 3.3RC1
> > 
> 
> Hello
> 
> I've updated eclipse to 3.3.0 build I20070621-1340
> and still have the bug.
> Should I report my configuration like told in comment #13 ?

Sorry but this bug seems to be not fixed.

I'm using "Eclipse 3.3.0 build I20070621-1340" and .svn folders are copied in the build folder.

Best regards
cyrille.
Comment 33 Kent Johnson CLA 2007-07-23 09:44:04 EDT
Please open a new bug with a reproduceable testcase since your case must be different than the other users.
Comment 34 Ron Smith CLA 2009-06-08 16:20:45 EDT
Did anyone open a new bug? There is no reference to it and I can't find it with a search. 

This *is* still an issue in version 3.4.2 and the testcase is very simple: Open a project that has .svn files in the source tree and do a clean build. The .svn files get copied into the bin directory.

I also tried installing subversive hoping it might convince eclipse to ignore the .svn folders but didn't help.
Comment 35 Kent Johnson CLA 2009-06-09 11:42:44 EDT
Ron - I tried a simple case again with .svn added to the resource filter & it worked fine.


Please open a new bug with as much detail as possible.

See comment #13