Bug 157570 - Bug in ASTParser
Summary: Bug in ASTParser
Status: VERIFIED FIXED
Alias: None
Product: JDT
Classification: Eclipse Project
Component: Core (show other bugs)
Version: 3.2   Edit
Hardware: PC Windows XP
: P3 normal (vote)
Target Milestone: 3.3 M4   Edit
Assignee: Olivier Thomann CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2006-09-16 14:12 EDT by Vimil Saju CLA
Modified: 2006-12-12 13:25 EST (History)
2 users (show)

See Also:


Attachments
Fix for recovery (7.97 KB, patch)
2006-11-16 12:47 EST, David Audel CLA
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Vimil Saju CLA 2006-09-16 14:12:52 EDT
1) Create a new ASTParser using ASTParser.newParser()
2) Set the 'kind' to K_STATEMENTS
3) Set source to the following code snippet
"   public static void m1()
    {
    	int a;
    	int b;
    }
    
    public static void m2()
    {
    	int c;
    	int d;
    }
"

4)Create AST by invoking the createAST method

only partial AST is returned. ie. AST for the following code fragment is only returned
"{
  int c;
  int d;
}"

When I looked at the source of ASTParser I found the following problem

In the method private ASTNode internalCreateASTForKind()
at line 963 shown below
<--
ConstructorDeclaration constructorDeclaration = codeSnippetParsingUtil.parseStatements(this.rawSource, this.sourceOffset, this.sourceLength, this.compilerOptions, true, true);
-->

the constructorDeclaration object is initialized and at the next line the 'compilationResult' field of the 'constructorDeclaration' is accessed.  Therefore the assumption is  constructorDeclaration object will never be null.

However at line 978 there is an 'if else statement' where the body of the if statement executes only if 'constructorDeclaration' is not null and the body of the else statement executes if 'constructorDeclaration' is null. Since 'constructorDeclaration' object can never be null, as per the above assumption, the body of the else statement is never executed. 

The eclipse version used was Version: 3.2.0
Build id: M20060628-1325
Comment 1 Frederic Fusier CLA 2006-09-17 07:38:24 EDT
You simply do not use the right kind of ASTParser as your code is not a sequence of statements but a sequence of class body declarations instead. So, changing K_STATEMENTS to K_CLASS_BODY_DECLARATIONS will fix your issue.

Here's the snippet I wrote using your example:
    ASTParser parser = ASTParser.newParser(AST.JLS3);
    parser.setKind(ASTParser.K_CLASS_BODY_DECLARATIONS);
    String source =
        "public static void m1()\n" + 
        "{\n" + 
        "	int a;\n" + 
        "int b;\n" + 
        "}\n" + 
        "public static void m2()\n" + 
        "{\n" + 
        "	int c;\n" + 
        "	int d;\n" + 
        "}\n";
    parser.setSource(source.toCharArray());
    ASTNode node = parser.createAST(null);
    System.out.println(node.toString());

And I get following (expected) output:
class MISSING {
  public static void m1(){
    int a;
    int b;
  }
  public static void m2(){
    int c;
    int d;
  }
}
Comment 2 Vimil Saju CLA 2006-09-17 12:08:20 EDT
(In reply to comment #1)
But shouldn't it mark the source as malformed. i.e if 'aN' is the ASTNode returned after parsing then shouldn't
'aN.getFlags() == ASTNode.MALFORMED' return true. It currently does not.

Could you please look at the else block on lines 998-1006 of the method internalCreateASTForKind() in org.eclipse.jdt.core.dom.ASTParser.class
I think the else block was intended to get executed if there were any parse errors, but it does not as the variable 'constructorDeclaration' is never null. 

> You simply do not use the right kind of ASTParser as your code is not a
> sequence of statements but a sequence of class body declarations instead. So,
> changing K_STATEMENTS to K_CLASS_BODY_DECLARATIONS will fix your issue.
> Here's the snippet I wrote using your example:
>     ASTParser parser = ASTParser.newParser(AST.JLS3);
>     parser.setKind(ASTParser.K_CLASS_BODY_DECLARATIONS);
>     String source =
>         "public static void m1()\n" + 
>         "{\n" + 
>         "       int a;\n" + 
>         "int b;\n" + 
>         "}\n" + 
>         "public static void m2()\n" + 
>         "{\n" + 
>         "       int c;\n" + 
>         "       int d;\n" + 
>         "}\n";
>     parser.setSource(source.toCharArray());
>     ASTNode node = parser.createAST(null);
>     System.out.println(node.toString());
> And I get following (expected) output:
> class MISSING {
>   public static void m1(){
>     int a;
>     int b;
>   }
>   public static void m2(){
>     int c;
>     int d;
>   }
> }

Comment 3 Olivier Thomann CLA 2006-11-13 16:58:39 EST
Several problems here:
1) The statement recovery should only be triggered if the method:
ASTParser.setStatementRecovey(boolean) is called with true.
Otherwise the list of statements of the returned block should be empty.
2) The statements should indeed be flagged as MALFORMED and/or RECOVERED.
Comment 4 Olivier Thomann CLA 2006-11-14 11:29:42 EST
I would expect the statements to be flagged as RECOVERED and not as MALFORMED.
The are well formed, but recovered.
Comment 5 Olivier Thomann CLA 2006-11-15 22:53:38 EST
(In reply to comment #3)
> 1) The statement recovery should only be triggered if the method:
> ASTParser.setStatementRecovey(boolean) is called with true.
> Otherwise the list of statements of the returned block should be empty.
Done.

> 2) The statements should indeed be flagged as MALFORMED and/or RECOVERED.
I am not so sure about that. First the two statements are well formed and not recovered. We might need to flag the block as recovered and you can always get the error messages from the root compilation unit.

David, any idea what would be the best way to give a feedback to the user about the fact that the original source code was not just statements.
Comment 6 Olivier Thomann CLA 2006-11-15 23:13:22 EST
Released for 3.3M4.
David, if you have a better idea, reopen it.
Comment 7 David Audel CLA 2006-11-16 12:47:08 EST
Your fix is not complete, there is still a bug in recovery. Once the recovery will be fixed, there will be a better way to flag recovered nodes.
Comment 8 David Audel CLA 2006-11-16 12:47:40 EST
Created attachment 54001 [details]
Fix for recovery
Comment 9 David Audel CLA 2006-11-16 12:49:12 EST
Fix released and test updated
Comment 10 Frederic Fusier CLA 2006-12-12 13:25:28 EST
Verified for 3.3 M4 using build I20061212-0010.