Bug 320927 - By default, .svn files should not be copied to bin folder
Summary: By default, .svn files should not be copied to bin folder
Status: NEW
Alias: None
Product: JDT
Classification: Eclipse Project
Component: Core (show other bugs)
Version: 3.5.2   Edit
Hardware: All All
: P3 enhancement with 1 vote (vote)
Target Milestone: ---   Edit
Assignee: JDT-Core-Inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2010-07-26 12:33 EDT by Nick Dunn CLA
Modified: 2012-05-25 16:50 EDT (History)
8 users (show)

See Also:
william.furr: review?


Attachments
add common source management tools to default exclusion list (2.15 KB, patch)
2012-05-23 17:10 EDT, William CLA
no flags Details | Diff
add common source management tools to default exclusion list (2.15 KB, patch)
2012-05-24 11:00 EDT, William CLA
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Nick Dunn CLA 2010-07-26 12:33:16 EDT
Build Identifier: M20100211-1343

When projects are built, the contents of src are copied into bin.  This is usually not a problem.  However, if you are using SVN, then it is a huge problem.  In SVN, each folder has a .svn folder embedded inside.  That means that each of these .svn folders is copied to the bin folder.  That means that when you use your svn, suddenly you notice that bin has all sorts of duplicate information.  Which means you go to remove bin from the svn repository, and instead, due to the fact that the .svn files are copies, the src folder is deleted instead.  See http://stackoverflow.com/questions/3336576/svn-how-does-removing-a-bin-folder-cause-the-src-folder-to-be-removed-instead/3336623#3336623 for more details

The solution is to go into Preferences->Java->Compiler->Building and add "*.svn" as a filtered resource.  Due to the aforementioned havoc it wreaks with source control, I request that that is a default option.

I only use eclipse for Android development so I know it's an issue there; I don't know whether it affects standard java development but I would bet it does.


Reproducible: Always
Comment 1 Remy Suen CLA 2010-07-26 17:07:42 EDT
If you have a Subversion plug-in installed and it is hooked up with the project, this shouldn't be a problem.
Comment 2 Ayushman Jain CLA 2010-07-27 05:43:49 EDT
Fix for 154693 should've taken care of this actually.
JavaCore.CORE_JAVA_BUILD_RESOURCE_COPY_FILTER is the option for this.
org.eclipse.jdt.internal.core.JavaCorePreferencesInitializer sets the default value for this.
Comment 3 Jay Arthanareeswaran CLA 2010-07-27 06:29:32 EDT
(In reply to comment #2)
> Fix for 154693 should've taken care of this actually.
> JavaCore.CORE_JAVA_BUILD_RESOURCE_COPY_FILTER is the option for this.
> org.eclipse.jdt.internal.core.JavaCorePreferencesInitializer sets the default
> value for this.

I don't see JavaCorePreferenceInitializer including "*.svn" as default. As Remy said, the Subversion plug-in should already take care of this.
Comment 4 Nick Dunn CLA 2010-07-27 08:16:09 EDT
(In reply to comment #3)
> (In reply to comment #2)
> > Fix for 154693 should've taken care of this actually.
> > JavaCore.CORE_JAVA_BUILD_RESOURCE_COPY_FILTER is the option for this.
> > org.eclipse.jdt.internal.core.JavaCorePreferencesInitializer sets the default
> > value for this.
> 
> I don't see JavaCorePreferenceInitializer including "*.svn" as default. As Remy
> said, the Subversion plug-in should already take care of this.

Perhaps I'm not making myself clear.  My understanding of how the mechanism works is that by default *ALL* of the files within the src folder are copied to the bin directory.  The filter is those that are NOT copied.  Thus if you don't see the *.svn entry there, that's *exactly* where I propose it be entered.

So my suggestion is that by default the field for "filtered resources" should have the "*.svn" entry added like so:

http://grab.by/grabs/552e113d8ad854e128dae56cf0a4a775.png


I don't think the SVN plugin has anything to do with solving the problem.  1) I try to install as few plugins as possible.  I've had a bad plugin install ruin my entire eclipse install before.  2) Just because I'm using SVN doesn't mean I want to be using it from within Eclipse. 3) Users should not have to install a plugin just to avoid a crippling deficiency like this one.
Comment 5 Nick Dunn CLA 2010-07-27 08:17:25 EDT
(In reply to comment #4)
> (In reply to comment #3)
> > (In reply to comment #2)
> > > Fix for 154693 should've taken care of this actually.
> > > JavaCore.CORE_JAVA_BUILD_RESOURCE_COPY_FILTER is the option for this.
> > > org.eclipse.jdt.internal.core.JavaCorePreferencesInitializer sets the default
> > > value for this.
> > 
> > I don't see JavaCorePreferenceInitializer including "*.svn" as default. As Remy
> > said, the Subversion plug-in should already take care of this.
> 
> Perhaps I'm not making myself clear.  My understanding of how the mechanism
> works is that by default *ALL* of the files within the src folder are copied to
> the bin directory.  The filter is those that are NOT copied.  Thus if you don't
> see the *.svn entry there, that's *exactly* where I propose it be entered.
> 
> So my suggestion is that by default the field for "filtered resources" should
> have the "*.svn" entry added like so:
> 
> http://grab.by/grabs/552e113d8ad854e128dae56cf0a4a775.png
> 
> 

Ah I didn't read the comment in context.  But my reasoning still stands for why this is not a plugin issue.
> I don't think the SVN plugin has anything to do with solving the problem.  1) I
> try to install as few plugins as possible.  I've had a bad plugin install ruin
> my entire eclipse install before.  2) Just because I'm using SVN doesn't mean I
> want to be using it from within Eclipse. 3) Users should not have to install a
> plugin just to avoid a crippling deficiency like this one.
Comment 6 Jay Arthanareeswaran CLA 2010-07-28 07:43:54 EDT
When I said it should work, I assumed that the Subversion plug-in takes care of adding "*.svn" to the filter. Otherwise, I don't see that as part of the default filter. 

Remy, is how things work with Subversion plug-in? Any idea?
Comment 7 Remy Suen CLA 2010-07-28 07:50:35 EDT
(In reply to comment #6)
> Remy, is how things work with Subversion plug-in? Any idea?

To clarify, I meant I assumed a Subversion plug-in would tag Subversion metadata (these forenamed .svn resources) as team ignored/whatever and then the Java builder wouldn't touch them.

See bug 111368.
Comment 8 Ayushman Jain CLA 2010-07-28 07:54:55 EDT
(In reply to comment #6)
> When I said it should work, I assumed that the Subversion plug-in takes care of
> adding "*.svn" to the filter. Otherwise, I don't see that as part of the
> default filter. 

Jay, is bug 154693 completely different? I guessed it should've fixed this issue. Maybe this is a special case and the fix for bug 154693 can be improved?
Comment 9 Jay Arthanareeswaran CLA 2010-07-28 08:18:08 EDT
(In reply to comment #8)
> (In reply to comment #6)
> > When I said it should work, I assumed that the Subversion plug-in takes care of
> > adding "*.svn" to the filter. Otherwise, I don't see that as part of the
> > default filter. 
> 
> Jay, is bug 154693 completely different? I guessed it should've fixed this
> issue. Maybe this is a special case and the fix for bug 154693 can be improved?

They are different. Bug 154693 fixed the case which was not working when "*.svn" was included as a resource filter. What Nick wants here is to have "*.svn" as part of the default filter value. And Remy is suggesting that it should not matter when Subversion plug-in is installed.
Comment 10 Nick Dunn CLA 2010-07-28 08:43:17 EDT
(In reply to comment #9)
> 
> They are different. Bug 154693 fixed the case which was not working when
> "*.svn" was included as a resource filter. What Nick wants here is to have
> "*.svn" as part of the default filter value. And Remy is suggesting that it
> should not matter when Subversion plug-in is installed.


Could someone explain why the files are copied from src into bin in the first place?  It doesn't make sense, in my mind, for source code to end up in a bin folder.  And obviously I can create a filter to prevent that from happening, but in my opinion an end user of Eclipse shouldn't have to dig down into 4 menus (Preferences->Java->Compiler->Building) to change this stuff.  That's why I think the default behavior of copying everything is such a problem, because it forces users to have to know about this implementation detail of how Eclipse is working, when it has no bearing on what he's intending to do.
Comment 11 Stephan Herrmann CLA 2010-07-29 08:57:20 EDT
(In reply to comment #10)
> Could someone explain why the files are copied from src into bin in the first
> place? 

From my understanding this is about putting non-Java resources on the 
classpath so that the ClassLoader.getResource family of methods finds 
resources like, e.g., images, property files etc.
Comment 12 Stephan Herrmann CLA 2010-07-29 09:13:17 EDT
(In reply to comment #4)
> I don't think the SVN plugin has anything to do with solving the problem.

But how can the JDT know about the directory layout of all possible
source control systems? JDT provides configurability and that's already a big
step. Now *someone* needs to tell the JDT what *are* internal files/dirs
of your source control system. Any name used by one source control system
could hypothetically be a name for "real" resources in another project.

Installing an SVN plugin should be *one* way: let the SVN plugin tell the
JDT what files/directories to exclude. *Another* way of doing it is
manually setting the filter.

My feelings are that everything beyond that comes close to trying to read
your mind what you consider actual resources vs. internal files that are
not part of the program.

my 0.02 €
Comment 13 Nick Dunn CLA 2010-07-29 10:06:58 EDT
(In reply to comment #12)
> (In reply to comment #4)
> > I don't think the SVN plugin has anything to do with solving the problem.
> 
> But how can the JDT know about the directory layout of all possible
> source control systems? JDT provides configurability and that's already a big
> step. Now *someone* needs to tell the JDT what *are* internal files/dirs
> of your source control system. Any name used by one source control system
> could hypothetically be a name for "real" resources in another project.
> 
> Installing an SVN plugin should be *one* way: let the SVN plugin tell the
> JDT what files/directories to exclude. *Another* way of doing it is
> manually setting the filter.
> 
> My feelings are that everything beyond that comes close to trying to read
> your mind what you consider actual resources vs. internal files that are
> not part of the program.
> 
> my 0.02 €

You make a compelling argument against not filtering out .svn alone - then you get into a slippery slope of having to know about the file formats of all of the source control programs.

However, perhaps this ticket should be more generally stated as, "hidden directories/files should not be copied to the bin folder from the src folder", e.g. the filter would be .* instead of *.svn by default (recalling that in Unix, a dot in front of a file or folder implies that it is hidden).  I do not buy the argument that people would intentionally put resources they're trying to access into hidden folders and referencing it from the code that way; if they are then they probably should have to jump through an extra hoop to explicitly allow that behavior.

That would cover svn and git (though git doesn't really suffer from this problem, because it has a single .git file at the root of where you initialized the repository, unlike eclipse which has one .svn folder per folder.), and any other source control program that assumes that dot prepended files are going to be hidden and used by it alone.
Comment 14 Stephan Herrmann CLA 2010-07-29 11:39:39 EDT
(In reply to comment #13)

Just some quick selected comments:

> [...] (recalling that in
> Unix, a dot in front of a file or folder implies that it is hidden).

Hidden, indeed from the user who types "ls" without the "-a" switch, yes.
Still it is little more than a convention. Nothing that is *specified* by
the filesystem etc.

> [...]  and any
> other source control program that assumes that dot prepended files are going to
> be hidden and used by it alone.

"Hidden" is not the same as "used by it alone". I *do* want svn to store my
hidden .project, .classpath, .settings/ etc. Just one example, where it makes
perfect sense that one tool can see the "hidden" files/folders of another.

This doen't directly say whether or not a default filter of .* is a good idea.
Let's see what others think.
Comment 15 Oleg Atamanenko CLA 2010-09-29 09:04:54 EDT
I met with this bug today. The issue is also that Eclipse SVN Plugins tries to commit content of the bin folder too and it leads to some confusion, like conflicts, because it commited code from the bin directory, the source is not updated, then it shows you conflict.
Comment 16 Mike Schrag CLA 2010-10-03 20:50:44 EDT
None of these mechanisms have worked for me -- .svn, *.svn, .*

There's definitely a race condition when you have automatic refreshing with an fsevents-based refresh (which I have an implementation of on OS X) where the workspace is refreshed and the .svn folders appear before the team provider gets a chance to mark them as a team derived resource. Under that race condition, they end up copying into the bin folder.

It would be nice if JDT provided an extension point (maybe you do already?) where the team providers can add an extension that specifies their additional custom filters?

In the meantime, am I doing something wrong with the filter? This is on 3.6.1 under Java=>Compiler=>Building=>Filtered resources, which I set to "*.launch,.svn" (along with the above variations).
Comment 17 William CLA 2012-05-22 17:24:17 EDT
I challenge anyone to present a use case where copying .svn, .git, .bzr, .hg, or .cvs folders to the build directory is a reasonable thing to do.

It is also unreasonable to say that because you are adding some *popular* source control programs to the default exclusion list you must somehow "read the user's mind" to do so.

It is a short list, updated infrequently, and easy to do.
Comment 18 William CLA 2012-05-23 17:10:13 EDT
Created attachment 216160 [details]
add common source management tools to default exclusion list

I would love to build and test this, but maven complains about a missing parent pom that I can't figure out how to get or where to put.  The JDT/Core build documents leave much to be desired.
Comment 19 Stephan Herrmann CLA 2012-05-23 18:53:17 EDT
(In reply to comment #18)
> I would love to build and test this, but maven complains about a missing parent
> pom that I can't figure out how to get or where to put.  The JDT/Core build
> documents leave much to be desired.

For building JDT/Core developers entirely rely on PDE.
No-one here uses maven for building JDT/Core.

The pom.xml you may be seing has just been committed yesterday, so please consider this as work in progress.
Comment 20 William CLA 2012-05-24 11:00:40 EDT
Created attachment 216198 [details]
add common source management tools to default exclusion list

Fixed a typo with the NLS flag.

Trying to figure out how to set up and run this thing with the Eclipse PDE is a huge amount of work for what's essentially a one-line change.
Comment 21 Olivier Thomann CLA 2012-05-24 11:11:10 EDT
(In reply to comment #20)
> Trying to figure out how to set up and run this thing with the Eclipse PDE is a
> huge amount of work for what's essentially a one-line change.
You just need to export the org.eclipse.jdt.core bundle into a folder and patch your current install with the one you exported in order to test it.
The easiest way to do this is to export using the same qualifier as the one of your install.
Comment 22 Jay Arthanareeswaran CLA 2012-05-25 01:46:32 EDT
(In reply to comment #18)
> I would love to build and test this, but maven complains about a missing parent
> pom that I can't figure out how to get or where to put.  The JDT/Core build
> documents leave much to be desired.

The page here has some information on this:

http://wiki.eclipse.org/CBI/Eclipse_Platform_Build

I couldn't get it working either as I didn't spend lot of time. But you could give it a try. But again, it's quite possible that there is still some work to be done with respect to the pom files.
Comment 23 Stephan Herrmann CLA 2012-05-25 16:50:54 EDT
(In reply to comment #20)
> Trying to figure out how to set up and run this thing with the Eclipse PDE is a
> huge amount of work for what's essentially a one-line change.

I'm frequently using various options, none of which don't require very much knowledge about the build:
- launch a runtime workbench
- export into host 
  (thus updating the Eclipse install you're currently running)
- export and manually replace a plug-in (see comment 21)
- create a patch feature, export it and install from local update site
  (well this is a *bit* more involved, requires you to write a feature.xml)

For development and experimenting with a patch these options should suffice, no?

BTW: are you sure you got all the names right? E.g., what about .cvs vs. CVS??
I don't know about all of the other systems.

These comments shouldn't say that I'm really convinced that the JDT is the suitable place for fixing this. But if s.o. thouroughly tests it against all version control systems I won't fight against.