Bug 573632 - [content assist] Content assist is not working anymore after "if"
Summary: [content assist] Content assist is not working anymore after "if"
Status: VERIFIED FIXED
Alias: None
Product: JDT
Classification: Eclipse Project
Component: Core (show other bugs)
Version: 4.20   Edit
Hardware: PC Linux
: P2 blocker (vote)
Target Milestone: 4.20 RC1   Edit
Assignee: Stephan Herrmann CLA
QA Contact:
URL:
Whiteboard:
Keywords: regression
: 573694 (view as bug list)
Depends on:
Blocks:
 
Reported: 2021-05-19 05:46 EDT by Andrey Loskutov CLA
Modified: 2021-10-28 08:53 EDT (History)
9 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Andrey Loskutov CLA 2021-05-19 05:46:44 EDT
Using I20210518-1800.

With /org.eclipse.ui.ide/extensions/org/eclipse/ui/dialogs/WizardNewFolderMainPage.java

go to handleAdvancedButtonSelect() method, to the first "if (linkedResourceComposite != null) {" and try to auto-complete any existing field, e.g. 
linkedResourceComposite.|

There are no proposals shown. Nada.
Try to do same in the same method, but *before* the "if (linkedResourceComposite != null)" - content assist works.

This is highly frustrating, IMHO it is a blocker issue for content assist.
Comment 1 Andrey Loskutov CLA 2021-05-19 05:47:57 EDT
Stephan, I know you did something in the content assist area - is this may be related?
Comment 2 Jay Arthanareeswaran CLA 2021-05-20 07:03:54 EDT
I am on I20210519-1800 and I see this problem. Interestingly, If you have a semi colon after your invocation point, then proposals are offered. 

Andrey, are you seeing that too?
Comment 3 Andrey Loskutov CLA 2021-05-20 07:38:58 EDT
(In reply to Jay Arthanareeswaran from comment #2)
> Andrey, are you seeing that too?

Exact. I will try bisect where it is coming from later today. For me this *must* be in 4.20.
Comment 4 Jay Arthanareeswaran CLA 2021-05-20 07:56:51 EDT
Okay, I see this is most likely caused by 

https://git.eclipse.org/c/jdt/eclipse.jdt.core.git/commit/?id=e3517ddc41f3c9536a29ef9be4e7dd3104993ab2

I see that in the broken case, we end up creating a TypeReference (instead of NameReference). Interestingly, we also end up parsing the complete "then" block. Previously the then statement only had the completion node.
Comment 5 Jay Arthanareeswaran CLA 2021-05-20 08:13:43 EDT
Okay, something weird going on when we create CompletionQTypeReference. This is the point in paring at which we create the completion node:


linkedResourceComposite.
			linkedResourceComposite 
===============================
Starts here -->=<-- Ends here
===============================
 null;

Note that we are already past the completion node in question and onto the next statement. And this is the stack:

CompletionOnQualifiedTypeReference.<init>(char[][], char[], long[]) line: 45	
	CompletionParser.checkAndCreateModuleQualifiedAssistTypeReference(char[][], char[], long[]) line: 5008	
	CompletionParser.createQualifiedAssistTypeReference(char[][], char[], long[]) line: 5034	
	CompletionParser(AssistParser).getTypeReference(int) line: 1492	
	CompletionParser(Parser).consumeEnterVariable() line: 3668

How did we even get to consumeEnterVariable() with a statement that has "."?

linkedResourceComposite.
			linkedResourceComposite = null;
Comment 6 Stephan Herrmann CLA 2021-05-20 09:55:36 EDT
(In reply to Jay Arthanareeswaran from comment #4)
> Okay, I see this is most likely caused by 
> 
> https://git.eclipse.org/c/jdt/eclipse.jdt.core.git/commit/
> ?id=e3517ddc41f3c9536a29ef9be4e7dd3104993ab2
> 
> I see that in the broken case, we end up creating a TypeReference (instead
> of NameReference). Interestingly, we also end up parsing the complete "then"
> block. Previously the then statement only had the completion node.

Right, since Bug 539685 we parse to the end of the expression to provide more context for resolution / type inference.

Given that I broke it, I will take a look later today.

BTW, regression: yes. High priority: yes. Blocker: 

"the bug blocks development or testing of the build (for which there is no work-around)" [1]

no way! :)


[1] https://wiki.eclipse.org/Bug_Reporting_FAQ#What_is_the_difference_between_Severity_and_Priority.3F
Comment 7 Andrey Loskutov CLA 2021-05-20 10:03:13 EDT
(In reply to Stephan Herrmann from comment #6) 
> Given that I broke it, I will take a look later today.

Thanks.
 
> BTW, regression: yes. High priority: yes. Blocker: 
> 
> "the bug blocks development or testing of the build (for which there is no
> work-around)" [1]
> 
> no way! :)

Given the fact that the major IDE functionality (editing/content assist) is "randomly" broken for the user, this is a blocker. This is a trivial thing may be, and you still can compile, debug etc, but from user perspective IDE is not usable anymore for daily work. With this bug one can go directly to vi for editing sources.

Therefore releasing with this bug is a "no go" from my POV, and so it is a blocker.
Comment 8 Stephan Herrmann CLA 2021-05-20 12:50:23 EDT
(In reply to Andrey Loskutov from comment #7)
> (In reply to Stephan Herrmann from comment #6) 
> > Given that I broke it, I will take a look later today.
> 
> Thanks.
>  
> > BTW, regression: yes. High priority: yes. Blocker: 
> > 
> > "the bug blocks development or testing of the build (for which there is no
> > work-around)" [1]
> > 
> > no way! :)
> 
> Given the fact that the major IDE functionality (editing/content assist) is
> "randomly" broken for the user, this is a blocker. This is a trivial thing
> may be, and you still can compile, debug etc, but from user perspective IDE
> is not usable anymore for daily work. With this bug one can go directly to
> vi for editing sources.
> 
> Therefore releasing with this bug is a "no go" from my POV, and so it is a
> blocker.

Are you saying the documentation is no longer up-to-date / used / relevant: https://wiki.eclipse.org/Bug_Reporting_FAQ#What_is_the_difference_between_Severity_and_Priority.3F
?

What you describe exactly fits to the description of "major", if you ask me.
Comment 9 Stephan Herrmann CLA 2021-05-20 12:53:27 EDT
BTW, for me bug 569668 is a true blocker according to your description. If somebody looks into that, it would certainly increase my enthusiasm for Eclipse :)
Comment 10 Andrey Loskutov CLA 2021-05-20 13:26:46 EDT
(In reply to Stephan Herrmann from comment #8)
> Are you saying the documentation is no longer up-to-date / used / relevant:
> https://wiki.eclipse.org/
> Bug_Reporting_FAQ#What_is_the_difference_between_Severity_and_Priority.3F
> ?

Yes. Never seen "P" flags used in last few years, and also the rest is very developer-focused, not user-focused. The impact of this bug on the average IDE user is *very* high, even if there is no data loss and no crashes etc. I guess "usability" was not that prominent at time the wiki was written.

Also please note "But overall, it is up to each project do decide how they triage/handle bugs so some variability from project to project will occur." 

> What you describe exactly fits to the description of "major", if you ask me.

:-) I was beaten by our customers so many times for so much smaller bugs in Eclipse, I don't want to imagine how they would complain about non functional content assist => blocker for me / my users.
Comment 11 Stephan Herrmann CLA 2021-05-20 14:31:21 EDT
(In reply to Jay Arthanareeswaran from comment #5)
> Note that we are already past the completion node in question and onto the
> next statement. And this is the stack:
> 
> CompletionOnQualifiedTypeReference.<init>(char[][], char[], long[]) line: 45	
> 	CompletionParser.checkAndCreateModuleQualifiedAssistTypeReference(char[][],
> char[], long[]) line: 5008	
> 	CompletionParser.createQualifiedAssistTypeReference(char[][], char[],
> long[]) line: 5034	

Clearly checkAndCreateModuleQualifiedAssistTypeReference does more than what its name suggests. It contains the old pre-module code as well.
Comment 12 Stephan Herrmann CLA 2021-05-20 15:10:42 EDT
(In reply to Jay Arthanareeswaran from comment #5)
> How did we even get to consumeEnterVariable() with a statement that has "."?
> 
> linkedResourceComposite.
> 			linkedResourceComposite = null;

Due to the greater parse range, the parser will see linkedResourceComposite.linkedResourceComposite as a qualified type reference that starts a variable declaration.

Inside fetchNextToken() we have some heuristics when to fall back to the old strategy (reduced parse range) by signaling an early EOF. One of rules is to check if any expressions are pending on expressionStack. While the current block doesn't have any pending expressions, the heuristic is tricked by the if-condition that still lingers on the expression stack, but shouldn't influence that heuristic.

Upcoming patch will refine the heuristic to detect that the expression is "out of scope".

All this probably implies that inside a lambda that same problem persists, because with lambdas involved, the old strategy is no good.
Comment 13 Eclipse Genie CLA 2021-05-20 16:30:30 EDT
New Gerrit change created: https://git.eclipse.org/r/c/jdt/eclipse.jdt.core/+/180822
Comment 14 Stephan Herrmann CLA 2021-05-20 17:42:47 EDT
(In reply to Eclipse Genie from comment #13)
> New Gerrit change created:
> https://git.eclipse.org/r/c/jdt/eclipse.jdt.core/+/180822

Intermediate test failures should be fixed in patch set #2

Jay, Andrey, can you take it to completion from here (incl. any required approvals)?
Comment 15 Stephan Herrmann CLA 2021-05-21 02:57:06 EDT
I've seen the failure in CompletionTests.testCompletionKeywordContinue5.
I can look into it after business hours.
Comment 16 Jay Arthanareeswaran CLA 2021-05-21 05:20:34 EDT
(In reply to Stephan Herrmann from comment #15)
> I've seen the failure in CompletionTests.testCompletionKeywordContinue5.
> I can look into it after business hours.

If I make a small change to the following statement, then the failing test pass. But I guess this could have side effects.

pushOnElementStack(K_BLOCK_DELIMITER, IF, this.expressionStack[this.expressionPtr]); // Change IF to FOR/WHILE

Wonder if we should push respective infos in all case statements?
Comment 17 Jay Arthanareeswaran CLA 2021-05-21 05:35:53 EDT
(In reply to Jay Arthanareeswaran from comment #16)
> Wonder if we should push respective infos in all case statements?

Amended the patch to this effect. Let's see how the tests go.

Stephan, please take a look.
Comment 18 Andrey Loskutov CLA 2021-05-21 12:26:53 EDT
See also bug 573702 and bug 573694. Probably they are also related here.
Comment 19 Andrey Loskutov CLA 2021-05-21 12:57:59 EDT
*** Bug 573694 has been marked as a duplicate of this bug. ***
Comment 20 Stephan Herrmann CLA 2021-05-22 08:19:40 EDT
For those not following on gerrit: we have a green build.

Over to the people who decide about inclusion in RC1 :)
Comment 21 Gayan Perera CLA 2021-05-22 13:21:49 EDT
@Stephan i have tested your patches and it works perfect now. I added few comments on the tests though, but we can decide when and how to handle them with the given time frame.
Comment 22 Gayan Perera CLA 2021-05-22 14:22:15 EDT
(In reply to Gayan Perera from comment #21)
> @Stephan i have tested your patches and it works perfect now. I added few
> comments on the tests though, but we can decide when and how to handle them
> with the given time frame.

I test this issue as well, but the comments are for other issue.
Comment 24 Sravan Kumar Lakkimsetti CLA 2021-05-25 03:52:51 EDT
Verified in
Eclipse SDK
Version: 2021-06 (4.20)
Build id: I20210524-1800
OS: Windows 10, v.10.0, x86_64 / win32
Java vendor: AdoptOpenJDK
Java runtime version: 11.0.10+9
Java version: 11.0.10
Comment 25 Luke Usherwood CLA 2021-10-28 08:53:02 EDT
Just to say, there may still be failing cases: bug 576934