Bug 266771 - NameLookup.findPackageFragment returns very incorrect package fragments
Summary: NameLookup.findPackageFragment returns very incorrect package fragments
Status: VERIFIED FIXED
Alias: None
Product: JDT
Classification: Eclipse Project
Component: Core (show other bugs)
Version: 3.4.1   Edit
Hardware: PC Linux
: P3 major (vote)
Target Milestone: 3.5 M7   Edit
Assignee: Jerome Lanneluc CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2009-03-02 19:21 EST by Robert Konigsberg CLA
Modified: 2009-04-28 12:32 EDT (History)
2 users (show)

See Also:


Attachments
minimal .jar file to be used in test case (1.40 KB, application/octet-stream)
2009-04-20 07:30 EDT, Robert Konigsberg CLA
no flags Details
Proposed fix and regression test (2.58 KB, patch)
2009-04-20 13:26 EDT, Jerome Lanneluc CLA
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Robert Konigsberg CLA 2009-03-02 19:21:17 EST
Build ID: I20080617-2000

Steps To Reproduce:
I'd love to give you something more concrete, but unfortunately, this has to be a bit prosaic.

Names have been changed.

I have a fully qualified path
IPath: /path/to/source/com/foo/project/facet/Source.java

It resolves to an IFile in the workspace:
IFile F/project/src_0/source/com/foo/project/facet/Source.java (verified with a call to IWorkspaceRoot.findFilesForLocation()


Calling project.findPackageFragment() eventually calls NameLookup.findPackageFragment(IPath). Here's the code for findPackageFragment as I see it:

     /**
      * Returns the package fragment whose path matches the given
      * (absolute) path, or <code>null</code> if none exist. The domain of
      * the search is bounded by the classpath of the <code>IJavaProject</code>
      * this <code>NameLookup</code> was obtained from.
      * The path can be:
      *   - internal to the workbench: "/Project/src"
      *  - external to the workbench: "c:/jdk/classes.zip/java/lang"
      */
[0]  public IPackageFragment findPackageFragment(IPath path) {
       if (!path.isAbsolute()) {
         throw new IllegalArgumentException(Messages.path_mustBeAbsolute); 
       }
      /*
       * TODO (jerome) this code should rather use the package fragment map to find the    candidate package, then
       * check if the respective enclosing root maps to the one on this given IPath.
       */    
       IResource possibleFragment = ResourcesPlugin.getWorkspace().getRoot().findMember(path);
       if (possibleFragment == null) {
         //external jar
         for (int i = 0; i < this.packageFragmentRoots.length; i++) {
[1]        IPackageFragmentRoot root = this.packageFragmentRoots[i];
           if (!root.isExternal()) {
             continue;
           }
[2]        IPath rootPath = root.getPath();
[3]        int matchingCount = rootPath.matchingFirstSegments(path);
           if (matchingCount != 0) {
[4]          String name = path.toOSString();
             // + 1 is for the File.separatorChar
[5]          name = name.substring(rootPath.toOSString().length() + 1, name.length());
[6]          name = name.replace(File.separatorChar, '.');
             IJavaElement[] list = null;
             try {
[7]            list = root.getChildren();
             } catch (JavaModelException npe) {
               continue; // the package fragment root is not present;
             }
             int elementCount = list.length;
             for (int j = 0; j < elementCount; j++) {
[8]            IPackageFragment packageFragment = (IPackageFragment) list[j];
[9]            if (nameMatches(name, packageFragment, false)) {
                 return packageFragment;
               }
             }
           }
         }
       } else {
         ...
       }
       return null;
     }

Let's look at the progression:

[0]: path is "/path/to/source/com/foo/project/facet/Source.java"
[1]: After navigating through 12 of the 14 entries, it comes across a .jar file stored at
/path/to/externaljars/p_service/v2_0/service.jar. It should be obvious that this jar file should not be the returned package fragment. But we will see that it is, in fact, returned.

Its toString representation is

   /path/to/externaljars/p_service/v2_0/service.jar
     <default> (not open)
     com.p_service (not open)
     com.p_service.util (not open)
     com (not open)

[2]: rootPath is "/path/to/externaljars/p_service/v2_0/service.jar"

[3]: matchingCount is obviously a value greater than 0.
[4]: name is "/path/to/source/com/foo/project/facet/Source.java"
[5]: Now take a look at the paths: the length of rootPath.toOSString() is 1 less than the length of name. Go ahead and count it, it _matters_.

This means that
name = name.substring(rootPath.toOSString().length() + 1, name.length());
is the same as
name = name.substring(name.length(), name.length()).

And name becomes "".

[6]: name stays ""
[7]: This succeeds and returns a four-element array:
   <default> [in /path/to/externaljars/p_service/v2_0/service.jar],
   com.p_service (not open) [in /path/to/externaljars/p_service/v2_0/service.jar],
   com.p_service.util (not open) [in /path/to/externaljars/p_service/v2_0/service.jar],
   com (not open) [in /path/to/externaljars/p_service/v2_0/service.jar]

[8]: The first run through this loop, packageFragment is <default> [in /path/to/externaljars/p_service/v2_0/service.jar].
[9]: because nameMatches passes false for the partial match parameter, this is functionally equivalent to

   packageFragment.getElementName().equals(name);

Both packageFragment.getElementName() and name are the 0-length string.

An invalid package fragment is returned, QED.

This happens *Repeatedly* throughout my project as I analyze the source.

More information:
Comment 1 Robert Konigsberg CLA 2009-03-07 00:17:06 EST
I have been able to create a test case that verifies this nasty, nasty behavior.

I created a plug-in project and used the Hello, World Command template as a way to create a trigger.

In the handler, I added this code:

IProject project = ResourcesPlugin.getWorkspace().getRoot().getProject("asdf");
IJavaProject jp = JavaCore.create(project);
IPath path = Path.fromOSString("/Users/kberg/Desktop/myf.txt");
IPackageFragment fragment = jp.findPackageFragment(path);
System.out.println(fragment);

I created a .jar file and stored it at "/Users/kberg/Desktop/my.jar". Again, notice that the size of /Users/kberg/Desktop/my.jar" is one less than path.toOSString().

Then I launched the Eclipse application. I created a java project called "asdf". I configured the project by adding the "my.jar" file to the build path as an external library.

Then I triggered the command. Again, it returned my.jar.

This was on my mac.

I would try to fix this but I really don't know what you're going for here.
Comment 2 Robert Konigsberg CLA 2009-03-09 14:16:02 EDT
Can you perhaps explain to me what this method should be doing? Or even, is there a good way for me to check after-the-fact that my IPath isn't actually part of the package fragment? I could see using IPath.isPrefixOf(IPath) but since I don't know what should really be happening, I'm afraid this will cause a different set of errors.
Comment 3 Robert Konigsberg CLA 2009-03-11 18:32:34 EDT
Is there something I can do to expedite this?
Comment 4 Chris Aniszczyk CLA 2009-03-30 10:52:12 EDT
Jerome, have you looked at this yet?
Comment 5 Jerome Lanneluc CLA 2009-03-31 04:33:14 EDT
I'll try to have a look next week. 

In the meantime, 
1. Can you please try with a more recent build? Does it still happen in 3.4.2? in 3.5M6?
2. Does it happen if the jar is internal to the project?
3. Can you attach /Users/kberg/Desktop/my.jar to this bug? (or a simplified version that reproduces the problem)
Comment 6 Robert Konigsberg CLA 2009-03-31 11:08:18 EDT
1. Can you please try with a more recent build? Does it still happen in 3.4.2?
in 3.5M6?

Reconfirmed with 3.4.2 on Linux.

2. Does it happen if the jar is internal to the project?

I do not know what you mean.

3. Can you attach /Users/kberg/Desktop/my.jar to this bug? (or a simplified
version that reproduces the problem)

Really, the .jar has nothing in it. Just create a jar with any ol class in it, because the jar's contents are not relevant to this algorithm. But, I'll attach it.
Comment 7 Robert Konigsberg CLA 2009-03-31 11:09:54 EDT
Yeah I already deleted the .jar, so I cannot attach it. Seriously, build a jar with one class in it. Or none. It still shouldn't return that .jar.
Comment 8 Robert Konigsberg CLA 2009-04-17 22:02:22 EDT
Ping! Seriously, this is causing a real problem for us, and I am beginning to worry that this will not make it in to 3.5. I am happy to take a stab at this, as I've said, but I do not know what this method is supposed to do. Additionally, I will be happy with a workaround, but either way, I need some guidance, or at least a reply.

And hey, I understand that you're busy and have plenty to do: if someone can point me in a direction that works around this then hey, I'll be more than happy to do that and reduce the severity.
Comment 9 Jerome Lanneluc CLA 2009-04-20 05:01:41 EDT
Let's start with getting all the information first.

1.I asked if the problem was reproducible in 3.4.2, you said yes and that's good. But I also asked if the problem was reproducable in 3.5M6, you didn't respond.

2. Then I asked if the problem happened if the jar is internal to the project?
This is a jar located inside the project's folder (as opposed to a jar located in another project's folder, or an external jar which is located outside the workspace's folder).

3. I asked you to attach the jar. You replied that I can built it myself. This is not helping.

A good start would be for you to provide the JUnit test that shows the problem. This test should be located in /org.eclipse.jdt.core.tests.model/src/org/eclipse/jdt/core/tests/model/NameLookupTests2.java

To get the JDT/Core code, see http://wiki.eclipse.org/JDT_Core_Committer_FAQ#Where_is_the_JDT.2FCore_code.3F
Comment 10 Robert Konigsberg CLA 2009-04-20 07:30:10 EDT
Hello Jerome,

Thanks for your reply. It's good to see what is in the way. I'll try to address all of your comments here.

It was unclear to me by your question that what you wanted was a verification on both versions. You gotta admit, "3.5M6?", as a sentence fragment, can be interpreted more than one way. In fact, I thought you were just looking for me to try either version. I will try to reproduce this with 3.5M6.

Regarding whether the jar is internal or external, I am waiting on here -- I told you I did not know what you mean. Is comment #1 insufficient where I say" "I configured the project by adding the "my.jar" file to the build path as an external library." This doesn't mean I know the answer; I really do not have a good understanding of classpath internals.

While I did, at first, say that you can just create any ol' jar to this project, that it didn't matter what the jar had, I admit that consequently I  told you I would attach one, which I failed to do. Attaching in subsequent comment.

I know that writing an actual test case would be good, but I'm really hoping that I can ask you to help out here: the truth is that it's become very difficult to try to figure out where source code is in the CVS repository. By that I mean that everything has been refactored in CVS, and the amount of time it takes for me to figure this stuff out is a huge pain in the butt. If you can help me out here by telling me the right way to get the latest and correct code from CVS, then I'll see what I can do. As you know, the amount of local knowledge to get started with Eclipse is not low.

But more important than that, I have described reproducing the test case using manual actions because I don't have any idea how to create a test project that actually attaches the jar in that way. Or even how to create a dummy .jar file. I TOTALLY recognize that you're busy, but this is your domain of expertise, I know how to attach a jar using the  UI, but not through the IDE, and if I do it incorrectly, I run the risk of not writing the test properly, and as such, perhaps it becomes marked as "working as intended", for instance. So really, your help is what I need here.

When someone replies with enumeration like the one in 9, I suspect they're being defensive. I suspect it now, but I also hope I'm wrong. Clearly, you and I are going to need to work together to fix this, or at least, as a minimum, find a workaround. I can help out when there is communication, and I can't make progress without your shared experience.
Comment 11 Robert Konigsberg CLA 2009-04-20 07:30:46 EDT
Created attachment 132400 [details]
minimal .jar file to be used in test case
Comment 12 Robert Konigsberg CLA 2009-04-20 07:39:45 EDT
Jerome, this still occurs with 3.5m6.
Comment 13 Robert Konigsberg CLA 2009-04-20 07:50:14 EDT
Okay, before I head off to work, let me make sure I say: I've given all the information I can at this point, and am waiting to hear from you. I'm hoping this will be enough. I would also love to do a test case, but as I mentioned in the opening description, (and because of my limited knowledge of JDT and the framework) steps to reproduce have to be mostly prosaic and only partly code. But I assure you if you follow those steps (,and of course if my description of those steps were correct,) you should see the problem.

So now I will wait to hear from you. I hope you have enough to go on. Let me know what you need!

Thanks, Robert
Comment 14 Jerome Lanneluc CLA 2009-04-20 13:26:33 EDT
Created attachment 132460 [details]
Proposed fix and regression test
Comment 15 Jerome Lanneluc CLA 2009-04-20 16:40:01 EDT
Fix and test released for 3.5M7
Comment 16 Kent Johnson CLA 2009-04-28 12:32:13 EDT
Verified for 3.5M7 using attached regression test