Bug 563711 - ASTConverter throws IAE on code with syntax errors
Summary: ASTConverter throws IAE on code with syntax errors
Status: NEW
Alias: None
Product: JDT
Classification: Eclipse Project
Component: Core (show other bugs)
Version: 4.16   Edit
Hardware: PC Windows 10
: P3 normal with 1 vote (vote)
Target Milestone: ---   Edit
Assignee: JDT-Core-Inbox CLA
QA Contact: Sarika Sinha CLA
URL:
Whiteboard: stalebug
Keywords: needinfo
Depends on:
Blocks:
 
Reported: 2020-05-29 06:55 EDT by Ed Willink CLA
Modified: 2023-03-10 10:37 EST (History)
4 users (show)

See Also:


Attachments
log file (1017.50 KB, application/octet-stream)
2020-05-29 06:55 EDT, Ed Willink CLA
no flags Details
First log file (1.02 MB, text/plain)
2020-05-30 03:11 EDT, Ed Willink CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Ed Willink CLA 2020-05-29 06:55:13 EDT
Created attachment 283079 [details]
log file

M3: While checking on a QVTo error log entry from a couple of hours ago, I find that the error log is full of JDT logs padded out with complete source text.

The attached log file conains a variety of issues such as:

java.lang.IllegalArgumentException
	at org.eclipse.jdt.core.dom.InfixExpression.setOperator(InfixExpression.java:383)
	at org.eclipse.jdt.core.dom.ASTConverter.convert(ASTConverter.java:1276)
	at org.eclipse.jdt.core.dom.ASTConverter.convert(ASTConverter.java:2020)
	at org.eclipse.jdt.core.dom.ASTConverter.convertToParenthesizedExpression(ASTConverter.java:3750)
	at org.eclipse.jdt.core.dom.ASTConverter.convert(ASTConverter.java:1932)
	at org.eclipse.jdt.core.dom.ASTConverter.convert(ASTConverter.java:1204)
	at org.eclipse.jdt.core.dom.ASTConverter.convert(ASTConverter.java:1963)
	at org.eclipse.jdt.core.dom.ASTConverter.convert(ASTConverter.java:3023)
	at org.eclipse.jdt.core.dom.ASTConverter.convert(ASTConverter.java:751)
	at org.eclipse.jdt.core.dom.ASTConverter.buildBodyDeclarations(ASTConverter.java:208)
	at org.eclipse.jdt.core.dom.ASTConverter.convert(ASTConverter.java:3302)
	at org.eclipse.jdt.core.dom.ASTConverter.convert(ASTConverter.java:1513)
	at org.eclipse.jdt.core.dom.CompilationUnitResolver.convert(CompilationUnitResolver.java:323)
	at org.eclipse.jdt.core.dom.ASTParser.internalCreateAST(ASTParser.java:1231)
	at org.eclipse.jdt.core.dom.ASTParser.createAST(ASTParser.java:820)
	at org.eclipse.jdt.core.manipulation.CoreASTProvider$1.run(CoreASTProvider.java:271)
	at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:45)
	at org.eclipse.jdt.core.manipulation.CoreASTProvider.createAST(CoreASTProvider.java:263)
	at org.eclipse.jdt.core.manipulation.CoreASTProvider.getAST(CoreASTProvider.java:196)
	at org.eclipse.jdt.core.manipulation.SharedASTProviderCore.getAST(SharedASTProviderCore.java:138)
	at org.eclipse.jdt.internal.ui.text.java.hover.NLSStringHover.internalGetHoverInfo(NLSStringHover.java:139)
	at org.eclipse.jdt.internal.ui.text.java.hover.NLSStringHover.getHoverInfo2(NLSStringHover.java:118)
	at org.eclipse.jdt.internal.ui.text.java.hover.BestMatchHover.getHoverInfo2(BestMatchHover.java:163)
	at org.eclipse.jdt.internal.ui.text.java.hover.BestMatchHover.getHoverInfo2(BestMatchHover.java:130)
	at org.eclipse.jdt.internal.ui.text.java.hover.JavaEditorTextHoverProxy.getHoverInfo2(JavaEditorTextHoverProxy.java:89)
	at org.eclipse.jface.text.TextViewerHoverManager$1.run(TextViewerHoverManager.java:155)
Comment 1 Thomas Wolf CLA 2020-05-29 07:18:21 EDT
Looks like the JDT parser is badly thrown off by the merge conflict markers
<<<<<<, ======, >>>>>> . Shouldn't fail, though.
Comment 2 Stephan Herrmann CLA 2020-05-29 10:25:56 EDT
(In reply to Thomas Wolf from comment #1)
> Looks like the JDT parser is badly thrown off by the merge conflict markers
> <<<<<<, ======, >>>>>> . Shouldn't fail, though.

Apparently InfixExpression.setOperator() didn't expect these "operators" :)
Comment 3 Ed Willink CLA 2020-05-29 12:03:17 EDT
Looks like this is a major regression since GIT has been creating merged *.java files without problems for years.
Comment 4 Stephan Herrmann CLA 2020-05-29 12:51:38 EDT
(In reply to Ed Willink from comment #3)
> Looks like this is a major regression since GIT has been creating merged
> *.java files without problems for years.

Every source file is different, mostly: VERY different.
Comment 5 Ed Willink CLA 2020-05-29 15:26:06 EDT
Not sure what your comment means.

The JDT parser has a Problems View in which the prolific corollaries of GIT merge operators can be diagnosed. Using the Error Log instead/as well is the regression.
Comment 6 Stephan Herrmann CLA 2020-05-29 18:14:20 EDT
(In reply to Ed Willink from comment #5)
> Not sure what your comment means.
> 
> The JDT parser has a Problems View in which the prolific corollaries of GIT
> merge operators can be diagnosed. Using the Error Log instead/as well is the
> regression.

My comment only meant that I'm not convinced that this is a regression. Having worked on various files with similar syntax errors proves nothing, unless the same file with the same syntax errors has been used in different versions of Eclipse.

Bug it is, but whether or not a regression is unknown, OK?

Unfortunately, even the source quoted in the error log doesn't allow me to reproduce the error.


Some possible reasons:

- you hovered over a specific location, and I'm unable to guess which

- the error only occurs when all dependencies are resolved

So I added these jars as dependencies:

org.eclipse.emf.common-2.16.0.v20190625-1131.jar
org.eclipse.jdt.annotation_2.2.600.v20200408-1511.jar
org.eclipse.ocl.pivot_1.11.0.v20200309-0848.jar

Then I hovered here and there, clicked around, but no Error.

All we know from the stack: we were about to convert (from private to public AST) a binary expression that had an unexpected ID, so getOperatorFor(expressionOperatorID) returned null.
Well, infact the binary expression appeared as the LHS of an assignment, ooops? 

Structure similar to
 (a + b) = c;
Here '+' represents the operator with unknown ID.
Wow, when I hover over ewillink in the second "<<<<<<< ewillink/520440", JDT tells me:
"The left-hand side of an assignment must be a variable"

Close! But no cigar!!


The issue about the operator ID rings a bell as the change from Bug 562392 is very near by (having changed the values of ASTNode.OperatorMASK & OperatorSHIFT).


@Jay, any idea what might have happened? If you had .class files of jdt.core mixed from different versions, then they would be working with different values of the same constants, but this is hardly possible (and would blow up much harder).

@Ed, I'm afraid, we do need steps for reproducing.
Comment 7 Ed Willink CLA 2020-05-30 00:55:00 EDT
As a minimum I would say that a fault is:

Never copy more than 10 lines/2500 bytes of source text to the Error Log.

Have you looked at 'all' the logs. There seemed to be a large diversity of different entries causing four log files to be filled in short order.

As far as repros are concerned very hard. I was chopping and changing between two commits to track down my own divergence. As I have commented many times before, when GIT changes the workspace, the tooling responds long before a stable state has been established, hence JDT gets to report errors for truly crazy and unstable 'workspace contents'.
Comment 8 Ed Willink CLA 2020-05-30 03:11:29 EDT
Created attachment 283099 [details]
First log file

(In reply to Ed Willink from comment #7)
> As far as repros are concerned very hard. I was chopping and changing
> between two commits to track down my own divergence.

The GIT RefLog shows that at

9:45:05 checkout: moving from ...
9:45:35 'commidid' Updating HEAD

The error log shows a UI freeze at 9:39, and then

!ENTRY org.eclipse.jdt.core 4 4 2020-05-29 09:45:55.033
!MESSAGE Exception occurred during compilation unit conversion:

followed by the full source text of the challenging file. I've attached the relevant log file. (The previous log file was four log turns later.)

Presumably the full source allows you to see where setOperator is unhappy. Superficially there just appear to be two total function body replacements; rather normal. But the tail comment is 

">>>>>>> 43063ff No need for OclSelf dispatch if validating"

Perhaps the "for" or the "if" has confused the parser.
Comment 9 Stephan Herrmann CLA 2020-05-30 07:05:44 EDT
(In reply to Ed Willink from comment #7)
> As a minimum I would say that a fault is:
> 
> Never copy more than 10 lines/2500 bytes of source text to the Error Log.

There's a trade-off: if only one in ten cases provides a real repro by logging the full source, then IMHO this behavior has already paid off. Perhaps comment 8 marks this one successful case :).
But I also see the annoyance of huge log files.

 
> As far as repros are concerned very hard. I was chopping and changing
> between two commits to track down my own divergence. As I have commented
> many times before, when GIT changes the workspace, the tooling responds long
> before a stable state has been established, hence JDT gets to report errors
> for truly crazy and unstable 'workspace contents'.

JDT knows nothing about GIT.



(In reply to Ed Willink from comment #8)
> Created attachment 283099 [details]
> First log file

Thanks, I'll take a look soonish.
Comment 10 Markus Oley CLA 2021-02-01 08:25:07 EST
Hi, any changes with this bug? 
We have the same issue, which leads to UI thread freezing some minutes, which is very annoying. 

Best regards
Markus
Comment 11 Eclipse Genie CLA 2023-03-10 10:37:39 EST
This bug hasn't had any activity in quite some time. Maybe the problem got resolved, was a duplicate of something else, or became less pressing for some reason - or maybe it's still relevant but just hasn't been looked at yet.

If you have further information on the current state of the bug, please add it. The information can be, for example, that the problem still occurs, that you still want the feature, that more information is needed, or that the bug is (for whatever reason) no longer relevant.

--
The automated Eclipse Genie.