Bug 332222 - Unable to open java files in excluded Java Build Path
Summary: Unable to open java files in excluded Java Build Path
Status: NEW
Alias: None
Product: JDT
Classification: Eclipse Project
Component: Core (show other bugs)
Version: 3.6.1   Edit
Hardware: PC Windows XP
: P3 enhancement (vote)
Target Milestone: ---   Edit
Assignee: JDT-Core-Inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2010-12-09 11:26 EST by Mikel Waxler CLA
Modified: 2013-01-29 09:34 EST (History)
5 users (show)

See Also:


Attachments
.project (380 bytes, text/plain)
2010-12-09 16:18 EST, Mikel Waxler CLA
no flags Details
.classpath (733 bytes, text/plain)
2010-12-09 16:18 EST, Mikel Waxler CLA
no flags Details
Opening via Type (14.89 KB, image/png)
2010-12-09 16:21 EST, Mikel Waxler CLA
no flags Details
Opening via Resource (12.93 KB, image/png)
2010-12-09 16:21 EST, Mikel Waxler CLA
no flags Details
Source tree setup (46.91 KB, image/png)
2010-12-09 16:22 EST, Mikel Waxler CLA
no flags Details
Screenshots (60.47 KB, image/jpeg)
2010-12-10 00:50 EST, Jay Arthanareeswaran CLA
no flags Details
Junk project - example of bug (28.14 KB, image/png)
2010-12-10 15:13 EST, Mikel Waxler CLA
no flags Details
Example of broken test project. (26.39 KB, image/png)
2010-12-13 14:41 EST, Mikel Waxler CLA
no flags Details
Log file (489.57 KB, text/plain)
2011-05-02 13:06 EDT, Mikel Waxler CLA
no flags Details
log backup 0 (1021.35 KB, text/plain)
2011-05-02 13:07 EDT, Mikel Waxler CLA
no flags Details
Proposed patch (2.86 KB, patch)
2011-05-13 09:44 EDT, Satyam Kandula CLA
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Mikel Waxler CLA 2010-12-09 11:26:54 EST
Setup:
1) main source is located at c:\p4\main\JavaApplications\
2) Extra source is located at c:\p4\main\JavaApplications\test\

Project properties has both folders set as source and "test/" is excluded source #1. This exclusion was added automatically when I added the second folder. 

When I try to open a class by type (ctrl-shift-T) which is located at "c:\p4\main\JavaApplications\test\devtool\chat\ChatTools.java" it cannot find the file. It thinks the file is located at "test.devtool.chat.ChatTools" instead of "devtool.chat.ChatTools". This is true for all files contained in source #2. 

I have tried exiting eclipse.
I tried refreshing the workspace. 
I have tried removing folder #2 from sources, exiting the project properties window, then re-adding it and waiting for the refresh to finish. This does not fix the issue.
I have not tried deleting the project and recreating it. 



-- Configuration Details --
Product: Eclipse 1.3.1.20100913-1228 (org.eclipse.epp.package.java.product)
Installed Features:
 org.eclipse.jdt 3.6.1.r361_v20100714-0800-7z8XFUSFLFlmgLc5z-Bvrt8-HVkH
Comment 1 Walter Harley CLA 2010-12-09 12:06:33 EST
I suspect you are going to have some pain as long as you have overlapping source folders.  Not saying this isn't a bug, just saying that you're on thin ice.
Comment 2 Olivier Thomann CLA 2010-12-09 12:10:15 EST
Jay, please investigate.
Comment 3 Mikel Waxler CLA 2010-12-09 12:16:28 EST
(In reply to comment #1)
> I suspect you are going to have some pain as long as you have overlapping
> source folders.  Not saying this isn't a bug, just saying that you're on thin
> ice.

How is this "on thin ice"? This is a feature that has existed in eclipse for a long time. The UI explicitly supports this using the Excluded list. 

Also, my entire company uses this feature and it worked fine for over a year on my previous desktop.
Comment 4 Jay Arthanareeswaran CLA 2010-12-09 14:05:29 EST
I am unable to reproduce this on HEAD. Are these source folders inside the java project or external? It would help if you can attach the affected project, the .classpath and the folder structure in particular.
Comment 5 Mikel Waxler CLA 2010-12-09 16:18:00 EST
Created attachment 184899 [details]
.project
Comment 6 Mikel Waxler CLA 2010-12-09 16:18:17 EST
Created attachment 184900 [details]
.classpath
Comment 7 Mikel Waxler CLA 2010-12-09 16:20:22 EST
folder structure

c:\p4\main is the project root

here is the general structure:
 JavaApplications\com\example\package
 JavaApplications\com\example\application
 JavaApplications\test\com\example\package
 JavaApplications\test\devtool\
Comment 8 Mikel Waxler CLA 2010-12-09 16:21:05 EST
Created attachment 184901 [details]
Opening via Type

note that the package is incorrect
Comment 9 Mikel Waxler CLA 2010-12-09 16:21:44 EST
Created attachment 184902 [details]
Opening via Resource
Comment 10 Mikel Waxler CLA 2010-12-09 16:22:39 EST
Created attachment 184903 [details]
Source tree setup
Comment 11 Jay Arthanareeswaran CLA 2010-12-10 00:50:07 EST
Created attachment 184918 [details]
Screenshots

Even with the given files, I am not able to reproduce the bug. I am attaching a couple of screen shots that show that the correct source folder is picked up. I tried this with HEAD as well as 3.6. Did I miss anything or something wrong with the set-up?

Can you please try one of the newer builds? If you still find the issue, try isolating the issue in a sample workspace/project and please share it.
Comment 12 Mikel Waxler CLA 2010-12-10 15:06:31 EST
I am using Eclipse version: " Helios Service Release 1, Build id: 20100917-0705". When I check for updates, it tells me there is nothing to be done. 

My co-worker suggested the a potential cause. When creating a new project, you have the option to set roots. His theory is that we are setting up "JavaApplications" as the only root, and letting it parse the source. We then later go back in and add test as a root. 

Something then goes wrong such that it never completes the refresh and updates the package for the files contained in test/.
Comment 13 Mikel Waxler CLA 2010-12-10 15:13:19 EST
Created attachment 184992 [details]
Junk project - example of bug
Comment 14 Mikel Waxler CLA 2010-12-10 15:16:34 EST
Aha, that comment may be the issue. Here are the rough steps:
Create a new java project and add a single java file in src.
create a folder inside src and put a java file in that. 
go into project settings and add the new folder as a root. 
Add a new class inside that folder (whee.java)
Do not set a package on that class, since it will be at the top of a root.
 (it should automatically update the Excluded list, but for me it did not)
Save the settings and refresh the project
Open by type and type the class name.

Notice that the class shows up in both the old and new locations.
Comment 15 Olivier Thomann CLA 2010-12-10 16:26:27 EST
What happens if you close/reopen the project ?
Comment 16 Mikel Waxler CLA 2010-12-13 14:41:20 EST
Created attachment 185089 [details]
Example of broken test project.
Comment 17 Mikel Waxler CLA 2010-12-13 14:44:39 EST
For the small project, no it goes away after re opening the new project. It
does do a re-parse before open by type does anything though. 
I just reproduced the "cannot find file" error on the test project. I have
attached a screen shot. The steps are the same that I mentioned. 

This bug has been occurring on my primary project for a while now and exists
across closing eclipse, closing the project, and refreshing the project. 

This may be related to the large size of my primary project. There are a LOT of
files in it, 27,000 java files. Is it possible that the refresh is never
actually completing and just silently failing?
Comment 18 Mikel Waxler CLA 2011-03-02 12:51:01 EST
Any progress on this, it still occurs 100% of the time for my team.
Comment 19 Olivier Thomann CLA 2011-03-02 12:58:04 EST
Jay, could you please investigate what is going on based on the latest comments ?
If this requires more info, update the bug report accordingly.

Thanks.
Comment 20 John Gaquin CLA 2011-03-02 12:59:33 EST
Just want to chime in that I see this frequently as well. I am a coworker of
Mikel's. After reading through these comments, I don't think it was explicitly
accepted that this is not 100% reproducible on our end either. That is to say
that we cannot force this bug to happen at will, but it *will* eventually
happen 100% of the time. Typically the project will start out in the correct
state but at some point after a period of use it will revert to the incorrect
structure.

Given a project:

Source#1 - /Java (exclude /test)
Source#2 - /Java/test

Once it is created and refreshed properly, a file may be located at:

com.my.package.SomeClass.java (/Java/test)

This will work correctly for a while (days, weeks, hours?), but eventually it
will begin appearing in search results as

test.com.my.package.SomeClass.java (/Java)

Once that happens, selecting it from the search results gives a "file cannot be
opened" error, because the file clearly doesn't exist at that path. It is very
difficult to fix this state, usually requiring completely closing the project
and Eclipse and then letting it refresh everything upon re-opening it.
Comment 21 Mikel Waxler CLA 2011-04-28 11:24:42 EDT
Any update on this? This still occurs on all my active projects. My team members also have the same issue on their machines.

I recently had my desktop completely replaced and did a fresh install of the entire system and the issue comes right back.
Comment 22 Olivier Thomann CLA 2011-04-28 11:53:39 EDT
Can you give it a try using latest I-build ?
For now, my understanding is that you reported the issue against 3.6.1. Did you try a recent Indigo build ?

Jay, please update with what build you tried to reproduce.

On comment 3, you mentioned that this worked. Could you please be more specific? What version did you use where this worked as expected ?
Comment 23 Mikel Waxler CLA 2011-04-28 12:29:39 EDT
Gleefully installing indigo. 

Sadly, I don't know what version I was using, that was several years and 2 machines ago :).
Comment 24 Mikel Waxler CLA 2011-04-28 16:28:10 EDT
Nope, still fails. 

Here are the steps I took:

Install indigo
open existing workspace
wait for refresh to finish
Attempt to open file in sub-root.
   Worked successfully, opened net/foo/bar.java inside test/

Sync my local source with source control using external tool
Wait for refresh to finish
Attempt to open file in sub-root.
  Failed with same error, 
   could not find "test/net/foo/bar.java"
Comment 25 Olivier Thomann CLA 2011-04-29 10:17:19 EDT
Jay, could you please work with Mikel to make sure you can reproduce the issue and understand what is going on?
Thanks.
Mikel, please make sure you talk with Jay so that he gets enough information to reproduce this issue.
Thanks.
Comment 26 Mikel Waxler CLA 2011-04-29 10:41:09 EDT
One of my co workers thought of something that might be helpful. We do not ever use Eclipse to build the java files. We use a command line script to build. We actually turn off Build Automatically because it takes a long, long time to build all 27,000 java files.

Apparently some people suggest doing a build all to fix this issue, that is not something we would ever do. 

Also, after we build externally, eclipse has to refresh the project because of all the file changes.

Not sure if those two items effect anything, but I thought it would be good to let you know.
Comment 27 Jay Arthanareeswaran CLA 2011-05-02 01:48:13 EDT
I can reproduce this issue with I20110428-0848.  It happens on two scenarios -
a) When I add the nested folder to the source and add an exclusion in the
parent source folder accordingly b) When nested source folder is removed from
the list of source folders but the exclusion filter remains. In both cases, I
see two entries and when I select the wrong ones, I get the error.

However, things start working when I refresh the project.

Copying Satyam for inputs.

(In reply to comment #24)
> Wait for refresh to finish

Can you please check if there are any errors in the log file that indicate an
indexing or refresh failure?
Comment 28 Mikel Waxler CLA 2011-05-02 09:29:12 EDT
Where is the log file you want me to check?
Comment 29 Jay Arthanareeswaran CLA 2011-05-02 09:49:36 EDT
(In reply to comment #28)
> Where is the log file you want me to check?

The logs get stored at: ${workspace_loc}/.metadata/.log
Comment 30 Mikel Waxler CLA 2011-05-02 13:06:43 EDT
Created attachment 194513 [details]
Log file
Comment 31 Mikel Waxler CLA 2011-05-02 13:07:07 EDT
Created attachment 194514 [details]
log backup 0
Comment 32 Jay Arthanareeswaran CLA 2011-05-02 13:29:33 EDT
(In reply to comment #30)
> Created attachment 194513 [details]
> Log file

Thanks for the log. I see the following errors in the log. 

!MESSAGE Background Indexer Crash Recovery
....
!ENTRY org.eclipse.core.jobs 4 2 2011-04-28 12:50:27.529
!MESSAGE An internal error occurred during: "Java indexing... 479 files to index (/main)".
java.lang.OutOfMemoryError: Java heap space

The stack trace of the failing indexing job is available in the log and this seems to confirm my initial thoughts. You mentioned your workspace is big. What is your JVM heap parameters? Could you try with a higher value?
Comment 33 Mikel Waxler CLA 2011-05-02 13:34:34 EDT
That is a false positive. When I setup indigo and opened for the first time I got an obvious heap space error. I had not adjusted from the default .ini settings. I closed and set the .ini to -Xmx900m and no error was shown.
Comment 34 Mikel Waxler CLA 2011-05-12 16:12:08 EDT
These /might/ be steps to reproduce:

Add java source tree 
add sub folder as a source
refresh project
(...guessing here)
(refresh parses the first tree)
(refresh STARTS parsing the second tree)
(STOP the refresh)

A file in the sub tree will have been incorrectly detected in the first parse and thus show up as part of the first tree.
Comment 35 Satyam Kandula CLA 2011-05-13 09:39:32 EDT
I am able to reproduce the problem. This is similar to bug 312436, just a smaller problem than that and I do have a patch for this problem.
Comment 36 Mikel Waxler CLA 2011-05-13 09:41:59 EDT
You are my hero :)
Comment 37 Satyam Kandula CLA 2011-05-13 09:44:10 EDT
Created attachment 195590 [details]
Proposed patch

This patch should fix the problem. However, I think this could wait for 3.7.1.
Comment 38 Mikel Waxler CLA 2011-06-28 13:31:16 EDT
Is there a 3.7.1 beta or nightly that includes this patch? I would be happy to test it.
Comment 39 Satyam Kandula CLA 2011-07-04 02:51:08 EDT
(In reply to comment #38)
> Is there a 3.7.1 beta or nightly that includes this patch? I would be happy to
> test it.
This is not yet released. I will add a test and then try to get it released on the 3.7.1 branch.