Bug 400905 - [1.8][search] Search engine skips functional interface types.
Summary: [1.8][search] Search engine skips functional interface types.
Status: VERIFIED FIXED
Alias: None
Product: JDT
Classification: Eclipse Project
Component: Core (show other bugs)
Version: 4.3   Edit
Hardware: All All
: P3 normal (vote)
Target Milestone: BETA J8   Edit
Assignee: Srikanth Sankaran CLA
QA Contact:
URL:
Whiteboard:
Keywords:
: 400904 (view as bug list)
Depends on: 425134
Blocks: 423123 400899 428739
  Show dependency tree
 
Reported: 2013-02-15 05:19 EST by Srikanth Sankaran CLA
Modified: 2023-03-13 15:12 EDT (History)
7 users (show)

See Also:


Attachments
Proposed Patch - Part 1 (74.04 KB, patch)
2013-12-14 20:12 EST, Manoj N Palat CLA
no flags Details | Diff
Proposed Patch - Part 1 (74.76 KB, patch)
2014-01-08 08:55 EST, Manoj N Palat CLA
no flags Details | Diff
Proposed Patch (72.81 KB, patch)
2014-02-05 19:18 EST, Manoj N Palat CLA
no flags Details | Diff
Patch that applies properly. (72.05 KB, patch)
2014-02-19 20:08 EST, Srikanth Sankaran CLA
no flags Details | Diff
Pending changes to be reviewed. (53.95 KB, patch)
2014-02-23 10:41 EST, Srikanth Sankaran CLA
no flags Details | Diff
Pending changes to be reviewed. (66.36 KB, patch)
2014-02-23 22:40 EST, Srikanth Sankaran CLA
no flags Details | Diff
Mostly works - still some rough edges. (19.72 KB, patch)
2014-02-24 15:48 EST, Srikanth Sankaran CLA
no flags Details | Diff
Support for find declarations in hierarchy (43.25 KB, patch)
2014-02-25 18:04 EST, Srikanth Sankaran CLA
no flags Details | Diff
Latest patch (63.25 KB, patch)
2014-02-26 12:26 EST, Srikanth Sankaran CLA
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Srikanth Sankaran CLA 2013-02-15 05:19:23 EST
BETA_JAVA8:

Attempt to search for the declarations of an interface method fails
to find lambda methods:

// ----
interface I {
	void doit();
}

class X {
	static void foo() {}
	I i = () -> {};
	I i2 = new I() {
		void doit() {
			
		}
	};
}

Search for implementations of I#doit, brings up the one inside the
anonymous class, but not the anonymous lambda method.
Comment 1 Srikanth Sankaran CLA 2013-02-15 11:55:01 EST
This may be due to https://bugs.eclipse.org/bugs/show_bug.cgi?id=384750
still being open and the bindings not reflecting on the AST node.
Comment 2 Srikanth Sankaran CLA 2013-02-15 11:55:36 EST
(In reply to comment #1)
> This may be due to https://bugs.eclipse.org/bugs/show_bug.cgi?id=384750
> still being open and the bindings not reflecting on the AST node.

Please ignore this comment, this was meant for some other bug.
Comment 3 Srikanth Sankaran CLA 2013-02-16 23:33:16 EST
Dani & Markus, what would you like to see here ? 

The search engine/index is word based and by virtue of being anonymous
the indexed document won't be seen to be referring to I#doit.

In other contexts where something elides (in <>, inferred type arguments
to a generic method call etc) we don't include the inferred entities:
see https://bugs.eclipse.org/bugs/show_bug.cgi?id=112461.

But this is a case that could have impacts in refactoring: If the interface
method signature is changed and the lambda is not a part of search result set
refactoring would succeed but the code won't compile.

This raises some interesting challenges for the indexer. The word based
approach hooks into parse tree construction, if we want to show the lambda
we will need hooks into resolution phases also - solvable.
Comment 4 Markus Keller CLA 2013-02-18 12:53:24 EST
Yup, lambda expressions and their implicit references to functional interfaces implementations of abstract methods have one major difference to the examples mentioned in bug 112461 comment 8: The functional interfaces can be user-defined and hence they can be modified.

For the things in bug 112461, the references are to APIs from the JRE. Since those cannot be modified, refactorings and code analysis tools don't need special support to find references.

Unfortunately, not even a search for references to the functional interface would find the lambda's enclosing CU all the time. The lambda could be an argument to another method, and that method can be declared outside of the CU. This problem is similar to bug 102279 comment 11, but I'm afraid we really need a solution this time.

I don't think a syntax-based index can be good enough in a world with lambdas. If a project is fully built, then we could use the build state to find potential dependencies (if we do it on the granularity of a type, not per project as done in bug 102279). But without that, the best a a syntax-based index can do is remember which files contain at least one lambda expression. That will soon be a big percentage.
Comment 5 Srikanth Sankaran CLA 2013-12-03 22:06:15 EST
(In reply to Markus Keller from comment #4)

> I don't think a syntax-based index can be good enough in a world with
> lambdas. If a project is fully built, then we could use the build state to
> find potential dependencies (if we do it on the granularity of a type, not
> per project as done in bug 102279). But without that, the best a a
> syntax-based index can do is remember which files contain at least one
> lambda expression. That will soon be a big percentage.

Yep, tantamounting to measuring a with micrometer, marking with a chalk and 
cutting with an axe :(

Manoj, we need to trigger resolution if we see a lambda/method reference during
parse tree construction so as to gather additional index entries. Please work 
on this for EA2. Thanks.
Comment 6 Manoj N Palat CLA 2013-12-14 20:12:51 EST
Created attachment 238356 [details]
Proposed Patch - Part 1

This patch includes solutions for LambdaExpression and bug 400904 (ReferenceExpression) searches for with support for source level. 

Todo : Binary level
Comment 7 Manoj N Palat CLA 2014-01-08 08:55:14 EST
Created attachment 238778 [details]
Proposed Patch - Part 1

Updated to the latest ToT
Comment 8 Manoj N Palat CLA 2014-02-05 19:18:42 EST
Created attachment 239683 [details]
Proposed Patch

Updated patch to the latest ToT
Comment 9 Srikanth Sankaran CLA 2014-02-06 19:03:41 EST
*** Bug 400904 has been marked as a duplicate of this bug. ***
Comment 10 Srikanth Sankaran CLA 2014-02-19 20:08:30 EST
Created attachment 240127 [details]
Patch that applies properly.

Posted patch does not apply. Here it is.
Comment 11 Srikanth Sankaran CLA 2014-02-23 10:39:35 EST
Review comments: I'll categorize these into two: One set requiring follow
up and another where I have cleaned up things myself.

Comments NOT requiring follow up:

(1) I eliminated the various overrides for ReferenceExpression handling in
MatchLocatorParser - instead simply overriding a new method I introduced
in Parser : Parser.consumeReferenceExpression which gets called for all the
different forms.

(2) Likewise in IndexingParser

(3) Same type has methods resolveIndexed() and indexResolved()

Comments requiring follow up:

1. If I comment out the following methods in IndexingParser namely:

    consumeLambdaExpression,
    consumeReferenceExpressionTypeForm,
    consumeReferenceExpressionPrimaryForm,
    consumeReferenceExpressionSuperForm
    consumeReferenceExpressionGenericTypeForm

All the new tests still pass. This shows that all new test cases that exercise 
reference expression searches, probably have a single source file which also
defines the functional interface type/method and mentions them in other 
explicit contexts

(Last 4 of these have been eliminated per earlier, but same observation holds)

(2) Given

// -- I.java
public interface I {
	void foo(); 
}

// -- X.java
public class X {
	public static void main(String[] args) {
		Y.foo(()->{});
		Y.foo(Y::goo);
	}
}
// -- Y.java
public class Y {
	static void foo(I i) {}
	static void goo() {}
}

searching for reference to Y.goo brings up nothing. It is referenced in X.java

(3) Looking at MatchLocatorParser.consumeReferenceExpression:

The mention of this.patternFineGrain & IJavaSearchConstants.THIS_REFERENCE
looks completely bogus ? It will inhibit our finding super::foo with
fine grained pattern set to SUPER_REFERENCE ? Could you explain why you
are using THIS_REFERENCE here ? 

Please also add some search tests that *do* specify fine grained pattern.
For example, if I take JavaSearchBugs8Tests.testBug400904_0001(), paste
it in IDE and search for Y.foo with match location set to super references
it finds nothing.

(4) Many of the tests are badly formatted :-( As one example out of numerous
cases, why would testBug400904_0002 be formatted the way it has been ? 

(5) Given the example below:

interface I {
    X foo(int x);
}
public class X  {
	X(int x) {
		
	}
    void foo() {
        I i = X::new; 
    }
}

searching for the references to X's constructor does not bring up any matches

(6) Test testBug400904_0015 passes in the test suite, when I try it in IDE
I don't see any matches. There is something wrong here.

(7) Find declarations in hierarchy is completely broken - nothing seems to
work here: 

Given the comment#0 test case fixed to compile:

interface I {
	void doit();
}

class X {
	static void foo() {}
	I i = () -> {};
	I i2 = new I() {
		public void doit() {
			
		}
	};
}

Searching for I.foo's declarations in hierarchy, brings up the anonymous
classes doit method, but does not pull up the lambda method. The lambda
is implementing I and so the lambda body is an implementation of the SAM.
This will break refactoring - i.e change method signature.

For that reason we should also see how method references should be handled.
It may be the right thing to do leave the method's reference's implementation
method's signature alone.

(8) We need some searches that exercise other scopes: See the subtypes
of AbstractSearchScope: In the UI you would have seen, workspace, project,
hierarchy in some menus and project, required libraries, JRE libraries etc
in other contexts.

I released some infrastructure changes to support functional types in
search via these commits:

http://git.eclipse.org/c/jdt/eclipse.jdt.core.git/commit/?h=BETA_JAVA8&id=96bf1a9e2ee14d6938d23f375dc557eb131baf1b

http://git.eclipse.org/c/jdt/eclipse.jdt.core.git/commit/?h=BETA_JAVA8&id=11204112fa5839fef68c4b511b3aece9ba6627e4

I'll post a patch of the remaining pieces and continue with their review.
Comment 12 Srikanth Sankaran CLA 2014-02-23 10:41:30 EST
Created attachment 240237 [details]
Pending changes to be reviewed.

I cleaned up a bunch of things, eliminated a good bit of indirection - the net
effect is much more localized changes. These 7 files are under review.
Comment 13 Srikanth Sankaran CLA 2014-02-23 11:28:33 EST
The tests needs major rewrite. The tests cannot be all working copy based.
If I completely comment out the resolve + index code, every single new test
passes :(

Please create tests based on actual files. See org.eclipse.jdt.core.tests.model.JavaSearchBugsTests.testBug181981() for example.
Comment 14 Srikanth Sankaran CLA 2014-02-23 12:28:28 EST
Adding a new test that actually works of files instead of working copies exposes
some serious problems - even basic searches don't work - this is going to take
a bit of rewrite - I'll look into this this week.
Comment 15 Srikanth Sankaran CLA 2014-02-23 22:40:23 EST
Created attachment 240240 [details]
Pending changes to be reviewed.

Search works better with non-working copies with this patch - still unreviewed.
Comment 16 Srikanth Sankaran CLA 2014-02-24 07:14:23 EST
More review comments:

   (9) ConstructorLocator.match(ReferenceExpression, MatchingNodeSet)
should first be checking if the ReferenceExpression is a constructor reference ?
Also possibly other filters ? 

   (10) We should not use ReferenceExpression.binding - this could be a bridge
method - we saw this problem already on DOM/AST side ? RE.getMethodBinding is
what we should use.

   (11) Without a comment, the change in MemberDeclarationVisitor is impossible
to understand. Please document why exactly this change is required and what
exactly it is doing.

   (12) MethodLocator.match(LambdaExpression, MatchingNodeSet) has far fewer
checks than MethodLocator.match(MethodDeclaration, MatchingNodeSet) just below
it - why ? 

   (13) Likewise MethodLocator.match(ReferenceExpression, MatchingNodeSet)
needs more checks along the lines of MethodLocator.match(MessageSend, MatchingNodeSet) ? This piece is tricky (see RE.receiverPrecedesParameters)

   (14) With no comment, it is impossible to understand the change in 
org.eclipse.jdt.internal.core.search.matching.MethodLocator.matchContainer()

   (15) The method MatchLocator.createHandle(LambdaExpression, IJavaElement)
looks highly suspect in its handling of synthetic arguments. LE.arguments()
does not include synthetic arguments, but you are offseting by synthetic
arguments. I think there is potential for AIOOB here.
Comment 17 Srikanth Sankaran CLA 2014-02-24 07:22:48 EST
(In reply to Srikanth Sankaran from comment #16)

>    (15) The method MatchLocator.createHandle(LambdaExpression, IJavaElement)
> looks highly suspect in its handling of synthetic arguments. LE.arguments()
> does not include synthetic arguments, but you are offseting by synthetic
> arguments. I think there is potential for AIOOB here.

    (16) Not a single new test reaches this method :(
Comment 18 Srikanth Sankaran CLA 2014-02-24 07:57:08 EST
I completed one round of review cleaning up a bunch of things along the way.
Still many loose ends.

The implementation is much more compact now: Here:
http://git.eclipse.org/c/jdt/eclipse.jdt.core.git/commit/?h=BETA_JAVA8&id=89e8879d5dd978e5f2535f9d8b7bac080dd4f73b

The following files need to reviewed again:
ConstructorLocator.java
MatchLocator.java
MemberDeclarationVisitor.java
MethodLocator.java


I'll leave this bug open - this needs much more testing before we can
conclude we are done.

Here is the summarized list of comments that need follow up:

(1) Many more tests are needed. Lambdas in method invocations, constructor
invocations, lambda that capture outer locals, different search scopes,
working copy vs disk files, fine grained search patterns are all themes
and we need tests that exercise different combinations.

Likewise, constructor references, method references, generic methods,
static vs instance methods, array constructor references etc should
feature in tests.

Existing tests pass even if comment out critical sections of the code and
some big chunks of code are not entered at all.

(2) Looking at MatchLocatorParser.consumeReferenceExpression:

The mention of this.patternFineGrain & IJavaSearchConstants.THIS_REFERENCE
looks completely bogus ? It will inhibit our finding super::foo with
fine grained pattern set to SUPER_REFERENCE ? Could you explain why you
are using THIS_REFERENCE here ? 

Please also add some search tests that *do* specify fine grained pattern.
For example, if I take JavaSearchBugs8Tests.testBug400904_0001(), paste
it in IDE and search for Y.foo with match location set to super references
it finds nothing.

(3) Many of the tests are badly formatted :-( As one example out of numerous
cases, why would testBug400904_0002 be formatted the way it has been ?

(4) Test testBug400904_0015 passes in the test suite, when I try it in IDE
I don't see any matches. There is something wrong here.

(5) Find declarations in hierarchy is completely broken - nothing seems to
work here: 

Given the comment#0 test case fixed to compile:

interface I {
	void doit();
}

class X {
	static void foo() {}
	I i = () -> {};
	I i2 = new I() {
		public void doit() {
			
		}
	};
}

Searching for I.foo's declarations in hierarchy, brings up the anonymous
classes doit method, (Now it does not, something is broken) but does not 
pull up the lambda method. The lambda is implementing I and so the lambda 
body is an implementation of the SAM.
This will break refactoring - i.e change method signature.

For that reason we should also see how method references should be handled.
It may be the right thing to do leave the method's reference's implementation
method's signature alone.

(further thought: I think we should not treat the method reference as 
implementing the interface, but simply manufacturing an object that does)


 (6) ConstructorLocator.match(ReferenceExpression, MatchingNodeSet)
should first be checking if the ReferenceExpression is a constructor reference ?
Also possibly other filters ? 

   (7) We should not use ReferenceExpression.binding - this could be a bridge
method - we saw this problem already on DOM/AST side ? RE.getMethodBinding is
what we should use.

   (8) Without a comment, the change in MemberDeclarationVisitor is impossible
to understand. Please document why exactly this change is required and what
exactly it is doing.

   (9) MethodLocator.match(LambdaExpression, MatchingNodeSet) has far fewer
checks than MethodLocator.match(MethodDeclaration, MatchingNodeSet) just below
it - why ? 

   (10) Likewise MethodLocator.match(ReferenceExpression, MatchingNodeSet)
needs more checks along the lines of MethodLocator.match(MessageSend, MatchingNodeSet) ? This piece is tricky (see RE.receiverPrecedesParameters)

   (11) With no comment, it is impossible to understand the change in 
org.eclipse.jdt.internal.core.search.matching.MethodLocator.matchContainer()

   (12) The method MatchLocator.createHandle(LambdaExpression, IJavaElement)
looks highly suspect in its handling of synthetic arguments. LE.arguments()
does not include synthetic arguments, but you are offseting by synthetic
arguments. I think there is potential for AIOOB here.

   (13) No test reaches MatchLocator.createHandle(LambdaExpression, 
IJavaElement)

I'll follow up why/how the anonymous class declaration search broke and
fix it ((5) above)
Comment 19 Srikanth Sankaran CLA 2014-02-24 08:04:32 EST
(In reply to Srikanth Sankaran from comment #18)
 
> I'll follow up why/how the anonymous class declaration search broke and
> fix it ((5) above)

I misspoke, this does work alright.

(14) There may be something wrong in the way search results are presented.
When you change the interface method's signature by adding an int parameter
via refactoring in this test case:

interface I {
	void doit();
}

class X {
	static void foo() {}
	I i = () -> {};
	I i2 = new I() {
		public void doit() {
			
		}
	};
}

I get:

Root exception:
java.lang.ClassCastException: org.eclipse.jdt.internal.core.ResolvedSourceField cannot be cast to org.eclipse.jdt.core.IMethod
	at org.eclipse.jdt.internal.corext.refactoring.rename.RippleMethodFinder2$1MethodRequestor.acceptSearchMatch(RippleMethodFinder2.java:337)
	at org.eclipse.jdt.internal.core.search.matching.MatchLocator.report(MatchLocator.java:1958)
	at org.eclipse.jdt.internal.core.search.matching.MatchLocator.reportMatching(MatchLocator.java:2273)
	at org.eclipse.jdt.internal.core.search.matching.MemberDeclarationVisitor.visit(MemberDeclarationVisitor.java:213)
	at org.eclipse.jdt.internal.compiler.ast.LambdaExpression.traverse(LambdaExpression.java:631)
	at org.eclipse.jdt.internal.compiler.ast.FieldDeclaration.traverse(FieldDeclaration.java:343)

Certainly UI may not be prepared yet to handle this refactoring, but there
could be something wrong on Core side also.
Comment 20 Srikanth Sankaran CLA 2014-02-24 08:52:47 EST
In commit, http://git.eclipse.org/c/jdt/eclipse.jdt.core.git/commit/?h=BETA_JAVA8&id=a3ffbd4eada0192ebad239b0062039adb5f51f2f I short circuited
the lambda indexing code by commenting out the one line that triggers it - as
it stands, I am noticing some issues in indexing JRE8 as a source project -
there are some NPEs thrown with descriptor coming as null - I'll address these
and then enable the new functionality.
Comment 21 Srikanth Sankaran CLA 2014-02-24 10:58:46 EST
(In reply to Srikanth Sankaran from comment #20)
> In commit,
> http://git.eclipse.org/c/jdt/eclipse.jdt.core.git/commit/
> ?h=BETA_JAVA8&id=a3ffbd4eada0192ebad239b0062039adb5f51f2f I short circuited
> the lambda indexing code by commenting out the one line that triggers it - as
> it stands, I am noticing some issues in indexing JRE8 as a source project -
> there are some NPEs thrown with descriptor coming as null - I'll address
> these
> and then enable the new functionality.

Resolution of documents is backed up by model based name environment - this
will not work as the model is not fully built and answers missing types for
files not seen so far. I'll switch the name environment and see how that fairs.
Comment 22 Jay Arthanareeswaran CLA 2014-02-24 11:20:39 EST
(In reply to Srikanth Sankaran from comment #18)
> The implementation is much more compact now: Here:
> http://git.eclipse.org/c/jdt/eclipse.jdt.core.git/commit/
> ?h=BETA_JAVA8&id=89e8879d5dd978e5f2535f9d8b7bac080dd4f73b

There are couple of API issues due to this commit: SearchParticipant#indexResolvedDocument() and SearchDocument#shouldIndexResolvedDocument need @since tags, if the latter is intended to be public.
Comment 23 Srikanth Sankaran CLA 2014-02-24 11:37:15 EST
(In reply to Jayaprakash Arthanareeswaran from comment #22)
> (In reply to Srikanth Sankaran from comment #18)
> > The implementation is much more compact now: Here:
> > http://git.eclipse.org/c/jdt/eclipse.jdt.core.git/commit/
> > ?h=BETA_JAVA8&id=89e8879d5dd978e5f2535f9d8b7bac080dd4f73b
> 
> There are couple of API issues due to this commit:
> SearchParticipant#indexResolvedDocument() and
> SearchDocument#shouldIndexResolvedDocument need @since tags, if the latter
> is intended to be public.

Thanks, these are going away - the whole functionality is undergoing a rewrite.
Comment 24 Srikanth Sankaran CLA 2014-02-24 15:48:46 EST
Created attachment 240277 [details]
Mostly works - still some rough edges.
Comment 25 Srikanth Sankaran CLA 2014-02-25 03:29:48 EST
The indexing side of story is more or less complete with this commit:
http://git.eclipse.org/c/jdt/eclipse.jdt.core.git/commit/?h=BETA_JAVA8&id=124641ac651bbea2bd062a9605817d39b473bf0e.

I'll work on the search side now,

(In reply to Jayaprakash Arthanareeswaran from comment #22)
> There are couple of API issues due to this commit:
> SearchParticipant#indexResolvedDocument() and
> SearchDocument#shouldIndexResolvedDocument need @since tags, if the latter
> is intended to be public.

These are taken care of now.
Comment 26 Srikanth Sankaran CLA 2014-02-25 18:04:13 EST
Created attachment 240309 [details]
Support for find declarations in hierarchy

Much clean up needed, but this works at the junit level. UI still has issues due
to incomplete APIs triggering JMEs, NPEs etc.
Comment 27 Srikanth Sankaran CLA 2014-02-26 12:26:09 EST
Created attachment 240337 [details]
Latest patch

Indexing, searching and hierarchy works reasonably well, need more testing.

2 Model classes have been introduced, one borrowed from Jay'a patch and
modified somewhat.

Jay, please start adopting this for your work on code select and completion.

I'll write the 2 more classes (LambdaMethod and LambdaMethodElementInfo)
tomorrow.

The 4 new classes will require thorough study. At the moment, I am throwing
JME in a bunch of places, returning null or other invalid sentinels at other
place - the whole thing needs to be relooked at pounded into shape.

Anybody willing to test and report bugs ? Welcome. 

You should expect to see the following work:

    - Reference search for a method will/should show its usage in method/constructor references.

    - Declaration search in project, workspace and hierarchy scope should
work.

    - F4 type hierarchy should show ()-{}'s as implementations of the
functional interface.

Expect a bunch of JME, NPEs etc since as I said I chose to do that in
most every API that at first glance was not meaningful for lambdas. Careful
contemplation may well show otherwise.
Comment 28 Srikanth Sankaran CLA 2014-02-26 12:32:36 EST
Once the functionality is in place, we should carefully study the
performance impact and make suitable changes. This problem could call
resolving lots of files. We may want to devise a solution that purges
the contents of methods that don't have any functional types for example
before resolving.

This also calls for model -> cud to build more nodes.

A problem I encountered is that the handles sometimes do not have the
relevant information and we need to go to the element. This defeats the
purpose in some situations.

Example: The code in STC that checks for the magic > 10 annotations or any
type annotation to decide whether to convert, diet parse or parse - this
decision cannot be made by looking at the handle. To ask the element means
risking rebuilding the element - for which cost we could have parsed the file
straightaway.
Comment 29 Srikanth Sankaran CLA 2014-02-26 21:38:06 EST
In http://git.eclipse.org/c/jdt/eclipse.jdt.core.git/commit/?h=BETA_JAVA8&id=0d09a2966f3cebfb8d6601000e2b64259fed7dd8 I released the
core implementation. 

We are able to index JRE8 source project - I played around with a bit and
things look better - still seeing issues though. For example hierarchy of
Function does not show the lambdas that implement it though debug print
statements at the time of indexing do record the correct entries.


Follow up tasks: 

(15) Remove the kludges in Member.java and SourceMethod.java
(16) Find out why many lambdas don't show up as implementors of Function
Comment 30 Srikanth Sankaran CLA 2014-02-26 21:40:30 EST
Jay, please start adopting for selection, completion and debug support.

org.eclipse.jdt.internal.core.LambdaExpression and
org.eclipse.jdt.internal.core.LambdaTypeElementInfo are the new classes.

Equivalent ones for methods should come in later today.

org.eclipse.jdt.internal.core.util.HandleFactory.createLambdaTypeElement(LambdaExpression, ICompilationUnit, HashSet, HashMap) and

org.eclipse.jdt.internal.core.util.HandleFactory.createElement(Scope, int, ICompilationUnit, HashSet, HashMap)

are from util package.
Comment 31 Srikanth Sankaran CLA 2014-02-27 03:14:00 EST
(In reply to Srikanth Sankaran from comment #29)

> We are able to index JRE8 source project - I played around with a bit and
> things look better - still seeing issues though. For example hierarchy of
> Function does not show the lambdas that implement it though debug print
> statements at the time of indexing do record the correct entries.

The lambda indexing code was not emitted special signatures in the index files 
required to recognize CUs with local types. Fixed here:

http://git.eclipse.org/c/jdt/eclipse.jdt.core.git/commit/?h=BETA_JAVA8&id=2b3acbf04ca04c733316af44132ec605de080f66
Comment 32 Srikanth Sankaran CLA 2014-02-27 03:18:31 EST
UI team: Please start testing search + hierarchy - you should see Java elements
for search results where required.

My basic testing with JRE8 passed - so this functionality should be considered
ready for integration though there could be rough edges both on the presentation
side as well implementation side (see earlier comment about deliberate JMEs and returning nulls etc. Code review is under way to define precisely what APIs
can be inherited by the new classes and what they must override - this should
not stop initiation of adoption from your side.

Basically from UI side, you should just expect to work an IType and an IMethod
that answer all the relevant queries - that they are implementation artifacts
for modelling a lambda shouldn't matter too much.
Comment 33 Jay Arthanareeswaran CLA 2014-02-27 05:53:10 EST
Looks like we need a null check in match(LambdaExpression ...) for this.pattern.parameterSimpleNames. Searching for declarartion with "lambda" throws an NPE in the following case:

interface FunctionalInterface {
	int thrice(int x);
}
Comment 34 Srikanth Sankaran CLA 2014-02-27 07:16:22 EST
(In reply to Jayaprakash Arthanareeswaran from comment #33)
> Looks like we need a null check in match(LambdaExpression ...) for
> this.pattern.parameterSimpleNames. Searching for declarartion with "lambda"
> throws an NPE in the following case:
> 
> interface FunctionalInterface {
> 	int thrice(int x);
> }

Can you post a full test case and describe what exactly I should do ? As is
I am not able to reproduce.
Comment 35 Srikanth Sankaran CLA 2014-02-27 21:23:03 EST
(In reply to Jayaprakash Arthanareeswaran from comment #33)
> Looks like we need a null check in match(LambdaExpression ...) for
> this.pattern.parameterSimpleNames. Searching for declarartion with "lambda"
> throws an NPE in the following case:
> 
> interface FunctionalInterface {
> 	int thrice(int x);
> }

Full test case from Jay:

interface FunctionalInterface {
	int thrice(int x);
}
interface J {
	int twice(int x);
}
public class X {
	FunctionalInterface i = (x) -> {return x * 3;}; 
	X x = null;
	static void goo(FunctionalInterface i) {} 
} 

One needs to actually open the search dialog and mention thrice as the
search key. If you right click on thrice and search for references, NPE
does not occur.
Comment 36 Srikanth Sankaran CLA 2014-02-27 21:48:43 EST
(In reply to Jayaprakash Arthanareeswaran from comment #33)
> Looks like we need a null check in match(LambdaExpression ...) for
> this.pattern.parameterSimpleNames. Searching for declarartion with "lambda"
> throws an NPE in the following case:
> 
> interface FunctionalInterface {
> 	int thrice(int x);
> }

Thanks, fixed here: http://git.eclipse.org/c/jdt/eclipse.jdt.core.git/commit/?h=BETA_JAVA8&id=c0674ec415ef7e8e2f540d4ac4caafa60e454663
Comment 37 Srikanth Sankaran CLA 2014-03-01 17:19:15 EST
(In reply to Srikanth Sankaran from comment #18)

> The following files need to reviewed again:
> ConstructorLocator.java
> MatchLocator.java
> MemberDeclarationVisitor.java
> MethodLocator.java


> Here is the summarized list of comments that need follow up:
> 
> (1) Many more tests are needed. Lambdas in method invocations, constructor
> invocations, lambda that capture outer locals, different search scopes,
> working copy vs disk files, fine grained search patterns are all themes
> and we need tests that exercise different combinations.

Added many more tests.

> (2) Looking at MatchLocatorParser.consumeReferenceExpression:
> 
> The mention of this.patternFineGrain & IJavaSearchConstants.THIS_REFERENCE
> looks completely bogus ? It will inhibit our finding super::foo with
> fine grained pattern set to SUPER_REFERENCE ? Could you explain why you
> are using THIS_REFERENCE here ? 

Fixed.

> Please also add some search tests that *do* specify fine grained pattern.
> For example, if I take JavaSearchBugs8Tests.testBug400904_0001(), paste
> it in IDE and search for Y.foo with match location set to super references
> it finds nothing.

Done. This test passes now.

> (3) Many of the tests are badly formatted :-( As one example out of numerous
> cases, why would testBug400904_0002 be formatted the way it has been ?

Ignored. Please ensure proper formatting in future.

> (4) Test testBug400904_0015 passes in the test suite, when I try it in IDE
> I don't see any matches. There is something wrong here.

This is a corner case issue: Raised https://bugs.eclipse.org/bugs/show_bug.cgi?id=429388. Problem predates 1.8 work.

> (5) Find declarations in hierarchy is completely broken - nothing seems to
> work here: 

Fixed, many tests have been created.

>  (6) ConstructorLocator.match(ReferenceExpression, MatchingNodeSet)
> should first be checking if the ReferenceExpression is a constructor
> reference ?

Done.

>    (7) We should not use ReferenceExpression.binding - this could be a bridge
> method - we saw this problem already on DOM/AST side ? RE.getMethodBinding is
> what we should use.

Done.

> 
>    (8) Without a comment, the change in MemberDeclarationVisitor is
> impossible
> to understand. Please document why exactly this change is required and what
> exactly it is doing.

This piece required a good bit of rework. Without this, nested lambda searches
won't show up properly.

>    (9) MethodLocator.match(LambdaExpression, MatchingNodeSet) has far fewer
> checks than MethodLocator.match(MethodDeclaration, MatchingNodeSet) just
> below
> it - why ? 

This part is OK, lambda parameters must match for a binding to be valid.

>    (10) Likewise MethodLocator.match(ReferenceExpression, MatchingNodeSet)
> needs more checks along the lines of MethodLocator.match(MessageSend,
> MatchingNodeSet) ? This piece is tricky (see RE.receiverPrecedesParameters)

This looks OK too.

> 
>    (11) With no comment, it is impossible to understand the change in 
> org.eclipse.jdt.internal.core.search.matching.MethodLocator.matchContainer()

I have backed out this change as it looks plain wrong - no tests are impacted.

> 
>    (12) The method MatchLocator.createHandle(LambdaExpression, IJavaElement)
> looks highly suspect in its handling of synthetic arguments. LE.arguments()
> does not include synthetic arguments, but you are offseting by synthetic
> arguments. I think there is potential for AIOOB here.

This has undergone major rewrite.

>    (13) No test reaches MatchLocator.createHandle(LambdaExpression, 
> IJavaElement)

Now they do, many of them.

(In reply to Srikanth Sankaran from comment #19)
> (In reply to Srikanth Sankaran from comment #18)
>  
> > I'll follow up why/how the anonymous class declaration search broke and
> > fix it ((5) above)
> 
> I misspoke, this does work alright.
> 
> (14) There may be something wrong in the way search results are presented.
> When you change the interface method's signature by adding an int parameter
> via refactoring in this test case:
> 
> interface I {
> 	void doit();
> }
> 
> class X {
> 	static void foo() {}
> 	I i = () -> {};
> 	I i2 = new I() {
> 		public void doit() {
> 			
> 		}
> 	};
> }
> 
> I get:
> 
> Root exception:
> java.lang.ClassCastException:
> org.eclipse.jdt.internal.core.ResolvedSourceField cannot be cast to
> org.eclipse.jdt.core.IMethod

Now I get:

Java Model Exception: Java Model Status [lambda$1() [in <lambda> [in i [in X [in [Working copy] X.java [in <default> [in src [in P]]]]]]] does not exist]
	at org.eclipse.jdt.internal.core.JavaElement.newNotPresentException(JavaElement.java:499)
	at org.eclipse.jdt.internal.core.JavaElement.openWhenClosed(JavaElement.java:533)
	at org.eclipse.jdt.internal.core.JavaElement.getElementInfo(JavaElement.java:259)
	at org.eclipse.jdt.internal.core.JavaElement.getElementInfo(JavaElement.java:245)
	at org.eclipse.jdt.internal.core.SourceMethod.getParameterNames(SourceMethod.java:118)
	at org.eclipse.jdt.internal.corext.refactoring.structure.ChangeSignatureProcessor.checkParameterNamesInRippleMethods(ChangeSignatureProcessor.java:1140)


This will be addressed in https://bugs.eclipse.org/bugs/show_bug.cgi?id=425134.
I added a note there: https://bugs.eclipse.org/bugs/show_bug.cgi?id=425134#c29

(In reply to Srikanth Sankaran from comment #29)

> (15) Remove the kludges in Member.java and SourceMethod.java

This will happen in https://bugs.eclipse.org/bugs/show_bug.cgi?id=425134

> (16) Find out why many lambdas don't show up as implementors of Function

I have raised https://bugs.eclipse.org/bugs/show_bug.cgi?id=429375.

I think this is ready for release - I'll just review once more and commit.
Comment 38 Srikanth Sankaran CLA 2014-03-01 17:38:34 EST
Final installment released here: http://git.eclipse.org/c/jdt/eclipse.jdt.core.git/commit/?h=BETA_JAVA8&id=ca349eb1728a54f5c4569863d8f64340b099c4ce

With this search + indexing + hierarchy support for functional types should
be considered complete and ready for next round of testing.
Comment 39 shankha banerjee CLA 2014-03-07 00:45:26 EST
Without the fix search for I#doit() does not show up (Lambda):
() -> {}

With the fix it does show the lambda.
Comment 40 shankha banerjee CLA 2014-03-07 00:46:40 EST
Verified as working for Eclipse + Java 8 RC2 using Kepler SR2 +   
Eclipse Java Development Tools Patch for Java 8 Support (BETA)
1.0.0.v20140306-1935
Comment 41 Srikanth Sankaran CLA 2014-03-07 00:49:16 EST
Thanks Shankha