Bug 206391 - [DOM] Binding Resolutions for projects outside of Eclipse workspace
Summary: [DOM] Binding Resolutions for projects outside of Eclipse workspace
Status: VERIFIED FIXED
Alias: None
Product: JDT
Classification: Eclipse Project
Component: Core (show other bugs)
Version: 3.4   Edit
Hardware: All All
: P3 enhancement with 2 votes (vote)
Target Milestone: 3.6 M6   Edit
Assignee: Olivier Thomann CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2007-10-15 17:29 EDT by Alexei Svitkine CLA
Modified: 2010-03-10 01:06 EST (History)
8 users (show)

See Also:


Attachments
Proposed patch using INameEnvironment (7.98 KB, patch)
2007-10-15 17:36 EDT, Alexei Svitkine CLA
no flags Details | Diff
Proposed fix + regression test (32.29 KB, patch)
2010-02-04 17:47 EST, Olivier Thomann CLA
no flags Details | Diff
Proposed fix + regression tests (44.79 KB, patch)
2010-02-04 21:54 EST, Olivier Thomann CLA
no flags Details | Diff
Proposed fix + regression tests (104.09 KB, patch)
2010-02-17 15:45 EST, Olivier Thomann CLA
no flags Details | Diff
Updated test (989 bytes, patch)
2010-02-22 02:40 EST, Jay Arthanareeswaran CLA
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Alexei Svitkine CLA 2007-10-15 17:29:22 EDT
The ASTParser class provides the ability to resolve bindings, using parser.setResolveBindings(true);

However, no bindings are resolved if this is performed outside of an Eclipse context (such that there is no IProject available).



Various users have expressed their desire for this ability to be present, as seen from many mailing list and forum posts returned by Google. For example:

http://dev.eclipse.org/newslists/news.eclipse.tools.jdt/msg12746.html

Also, some additional discussion of this was posted on this bug:

https://bugs.eclipse.org/bugs/show_bug.cgi?id=87852
Comment 1 Alexei Svitkine CLA 2007-10-15 17:36:04 EDT
Created attachment 80387 [details]
Proposed patch using INameEnvironment

As I have posted (in more detail), on: https://bugs.eclipse.org/bugs/show_bug.cgi?id=87852

One solution is to be able to set an INameEnvironment on the ASTParser, that will then be used to generate the bindings.

Then, the environment can be created without needing an IProject, for example with
org.eclipse.jdt.internal.compiler.batch.FileSystem, and set on the ASTParser.

I have successfully modified the code with the above changes, and have been able to reach the desired results (ie: a solution to the problem of not being able to resolve bindings outside of an Eclipse workspace).

The patch is attached.
Comment 2 Jerome Lanneluc CLA 2007-10-16 05:06:01 EDT
Unfortunately INameEnvironment is internal (as its package name indicates) and it cannot be exposed in an API. The fix should propose a new API that wraps INameEnvironment. It will also need regression tests.
Comment 3 Alexei Svitkine CLA 2007-10-16 09:17:33 EDT
Other than the location of INameEnvironment (its package), is there any other reason why it can't be exposed as an API?
Comment 4 Jerome Lanneluc CLA 2007-10-16 10:30:21 EDT
INameEnvironment is an implementation detail of the compiler (source folder 'compiler') and we want to be able to change it without breaking clients (e.g. add a new method). This is why I suggested to wrap this implementation detail in another API.
Comment 5 Alexei Svitkine CLA 2007-10-16 10:51:30 EDT
I see your point.

It is important to note that the resolving mechanism,

org.eclipse.jdt.core.dom.CompilationUnitResolver

already acts on a INameEnvironment environment already - it is one of the parameters to its constructor.

This is why I decided to base my solution on INameEnvironment.


The existing static resolve() methods just create a CancelableNameEnvironment (which implements INameEnvironment) from the JavaProject and WorkingCopyOwner that are passed to it.


So one solution is to create a new interface, IResolutionEnvironment (or some more appropriate environment), that will be part of org.eclipse.jdt.core, as a copy of the _current_ INameEnvironment interface. Then, INameEnvironment can be modified to extend IResolutionEnvironment.

Then, CompilationUnitResolver can be modified to work with IResolutionEnvironments rather than INameEnvironments like currently.

This will address your concern of changes made to INameEnvironment (at least additions of methods - it will still need to keep the old ones to be able to extend IResolutionEnvironment).

Obviously, if it's desired to add more levels of abstraction to this stuff, more changes are needed. But I went with INameEnvironment initially because it was already used in the APIs of CompilationUnitResolver - which was not internal.

(It also resulted in a minimum set of changes to the existing code.)
Comment 6 Alexei Svitkine CLA 2007-10-16 10:59:06 EDT
Actually, on a further examination, the above solution doesn't seem to be possible because CompilationUnitResolver extends org.eclipse.jdt.internal.compiler.Compiler - which expects to work with INameEnvironments (they're both from the internal.compiler package).

This means that it's not possible to make a new API, since whatever is passed in must be able to map to a INameEnvironment, since that's what's expected.

If we make a clone of INameEnvironment... like say a IPublicNameEnvironment (just an example), then CompilationUnitResolver would have to create an INameEnvironment out of it anyway.

I don't see how that would make things better, since if new things were added to INameEnvironment, they would have to also be present on the IPublicNameEnvironment, else how would CompilationUnitResolver be able to convert from one to the other?

Unless the new things are just convenience methods, in which case we can just make an NameEnvironmentAdapter or similar class that comes with the convenience methods already, and you have to implement the core methods...

Do you see a better way to solve this problem?

Comment 7 Jerome Lanneluc CLA 2007-10-16 11:17:06 EDT
Actually CompilationUnitResolver is not API since it is not a public class (see http://www.eclipse.org/articles/article.php?file=Article-API-Use/index.html for details).

Otherwise I would use an abstract class ResolutionEnvironment that clients must subclass instead of an interface. This way methods can be added/removed more easily.
Comment 8 Olivier Thomann CLA 2007-10-16 11:22:00 EDT
Whatever solution is chosen, this is a very interesting feature. The binding resolution is too restrictive right now by requiring a java project to be there.
I saw many messages on the newsgroup to propose a solution that would work outside of the workbench. Of course in this context the bridge between ASTNode and IJavaElement would not work.
Comment 9 Alexei Svitkine CLA 2007-10-16 11:47:49 EDT
> Otherwise I would use an abstract class ResolutionEnvironment that clients must
> subclass instead of an interface. This way methods can be added/removed more
> easily.
> 

That is one solution. The problem is the current static resolve() methods on CompilationUnitResolver create a CancelableNameEnvironment (extends SearchableEnvironment). All of this would have to be changed to be subclasses of ResolutionEnvironment in that scenario...
Comment 10 Samuel Gaiffe CLA 2008-02-15 05:56:34 EST
Hi,

I just want to know if a solution is going to be given to that bug. I face this problem and reading the javadoc, the solution to resolve bindings using a char[] source is to provide a IJavaProject which can be created by providing a IProjet which ... can't be created in fact, in a reasonable way.

So, in fact, the oarser can't still be used outside of an eclipse context because without bindings resolution, the useness is very poor.

Sam
Comment 11 Jerome Lanneluc CLA 2008-02-15 09:39:20 EST
This is currently not on the plan.
Comment 12 Alexei Svitkine CLA 2008-02-15 11:36:08 EST
If you really need the functionality, you can try applying my patch to the code checked out from CVS, and building your own version of the core jdt .jar to use in your project.

I know it's a pain, but that's what I did for my project. (Well, I also actually had to implement the patch too... so it should be easier for you.)

With my patch, simply set your own INameEnvironment on the ASTParser, and it will work.

The INameEnvironment can then be an instance of org.eclipse.jdt.internal.compiler.batch.FileSystem, which doesn't depend on IJavaProject or anything like that.
Comment 13 Samuel Gaiffe CLA 2008-02-15 13:47:19 EST
Thank you, i'll apply your patch and i'll let you know how I feel with it. I had thought finding bindings through reflection in the compiled classes but if I can deal with your patch, it's better to me.
Comment 14 Samuel Gaiffe CLA 2008-02-18 04:07:30 EST
I applied your patch but the bindings don't seem to work at all. I haven't find documentation on how to instanciate a FileSystem object. Can you tell me if I go the right way by doing this :

String[] cp = new String[]{"d:\\dev\\test", "d:\\dev\\test\\heavenize-kernel.jar", "D:\\dev\\jdks\\jdk1.6.0\\jre\\lib"};
String[] fn = new String[]{"*.class", "*.jar"};
FileSystem fs = new FileSystem(cp, fn, null);

The class I want to parse is in the test directory without package specified. It depends on too other classes also located in the same directory bit the parser donn't find them.

I added the jre lib because a String can not be resolved. But nothing works correctly.

Could you please help me ?

Samuel
Comment 15 Olivier Thomann CLA 2010-02-04 17:47:26 EST
Created attachment 158245 [details]
Proposed fix + regression test
Comment 16 Olivier Thomann CLA 2010-02-04 17:48:38 EST
Alexei, could you please try this patch and let me know if you find it usable ?
Thanks.
Comment 17 Olivier Thomann CLA 2010-02-04 21:54:06 EST
Created attachment 158252 [details]
Proposed fix + regression tests
Comment 18 Olivier Thomann CLA 2010-02-08 13:06:45 EST
Should we add the bootclasspath inside the API ?
The bootclasspath can be set using the classpath and put it in at the beginning. This would end up with the same result.

Should we allow a default bootclasspath detection using the running VM ? We have that mechanism in the batch compiler where the bootclasspath from the running VM is added to the classpath if no bootclasspath is specified.

Any comments?
Comment 19 Alexei Svitkine CLA 2010-02-10 23:28:18 EST
Unfortunately, I am no longer working on the project where I needed this, so I can't test it out in that context.

However, I've looked over the patch, and the changes look good to me.

As for the default boot classpath detection, I suggest making it optional - i.e. not add it by default, but have a method addRunningVMBootClasspath() or something similar to add those entries in.
Comment 20 Olivier Thomann CLA 2010-02-12 15:13:54 EST
(In reply to comment #19)
> Unfortunately, I am no longer working on the project where I needed this, so I
> can't test it out in that context.
Too bad. Sorry it took so long to look into it.
 
> However, I've looked over the patch, and the changes look good to me.
Ok, good to know.

> As for the default boot classpath detection, I suggest making it optional -
> i.e. not add it by default, but have a method addRunningVMBootClasspath() or
> something similar to add those entries in.
In this case I would propose a boolean in the API

So we could have:
public void setEnvironment(String[] classpathEntries, String[] sourcepathEntries, boolean includeRunningVMBootclasspath)

If the boolean is true, the default running vm bootclasspath would be prepended to the environment classpath. No need to have a default. It will be up to the user to specify if they want it or not when invoking the setEnvironment(..) method.
Comment 21 Alexei Svitkine CLA 2010-02-13 01:37:22 EST
Sounds good to me.
Comment 22 Markus Keller CLA 2010-02-17 05:12:58 EST
(In reply to comment #17)
> Created an attachment (id=158252) [details] [diff]
> Proposed fix + regression tests

Javadocs in ASTParser need to be finished:
- setEnvironment(..) needs to specify the base path relative to which the sourcepath and classpath entries are resolved
- other Javadocs (e.g. setResolveBindings(boolean) should mention the new method

Will ASTParser#createASTs(ICompilationUnit[], String[], ASTRequestor, IProgressMonitor) also allow to resolve bindings from binding keys when setEnvironment(..) has been used?

Passing CUs to ASTParser#createASTs(..) will not work if there's no project, so clients can only create a single AST with bindings at a time. I think many of the use cases that need bindings without a project will do batch compilations, so having only setSource(char[]) and createAST(..) will be quite a bottleneck.
To support this use case, you could add new APIs like
ASTParser#createAST(String[] sourceFilePaths, String[] bindingKeys, FileASTRequestor requestor, IProgressMonitor monitor) and a new FileASTRequestor with #acceptAST(String sourceFilePath, CompilationUnit ast).
Comment 23 Olivier Thomann CLA 2010-02-17 15:45:21 EST
Created attachment 159374 [details]
Proposed fix + regression tests

This should cover most cases reported in comment 22.

>Will ASTParser#createASTs(ICompilationUnit[], String[], ASTRequestor,
>IProgressMonitor) also allow to resolve bindings from binding keys when
>setEnvironment(..) has been used?
No. The documentation clearly states that setProject(..) must be used to set up the context for this call.
Comment 24 Olivier Thomann CLA 2010-02-20 22:39:29 EST
Released for 3.6M6.
Regression tests added in:
org.eclipse.jdt.core.tests.dom.StandAloneASTParserTest
Comment 25 Jay Arthanareeswaran CLA 2010-02-22 02:40:09 EST
Created attachment 159743 [details]
Updated test

This is required for the tests to pass in Windows machines.
Comment 26 Jay Arthanareeswaran CLA 2010-02-22 02:48:03 EST
Released the updated tests for 3.6M6.
Comment 27 Jay Arthanareeswaran CLA 2010-03-10 01:06:35 EST
Verified for 3.6M6 using build I20100305-1011.