Community
Participate
Working Groups
Build ID: 3.4 - I20070911-0833 For double literal "0x0.0000000000001P-1022", as in public static final double VAR = 0x0.0000000000001P-1022; the lenght of the AST NumberLiteral is set to 3. After little debugging I found that it it initially set to the correct length of 23, and then set to 3 by the ASTConverter.removeLeadingAndTrailingCommentsFromLiteral() method.
I cannot reproduce with HEAD. Are you sure that your project settings are 1.5 ?
I tried this test case and the number literal positions look good. ====================================== public class X { public static final double VAR = 0x0.0000000000001P-1022; } ====================================== I also tried with comments before and after the field, without more success.
Added disabled test org.eclipse.jdt.core.tests.dom.ASTConverter15Test#_test0284. Please provide a patch with a test case that fails.
If I get a test case in time, this could potentially be fixed for 3.4M2.
ok, one moment. I'll try to reproduce it programmatically.
private void testAST() { ASTParser parser = ASTParser.newParser(AST.JLS3); String code= "\r\n" + "public class ThisClass {\r\n" + "\r\n" + " public static final long VAL = 100L + \r\n" + " 20L" + " public static final double XO = 0x0.0000000000001P-1022; \r\n" + "}\r\n" + ""; parser.setSource(code.toCharArray()); CompilationUnit u = (CompilationUnit) parser.createAST(null); ASTNode nd = ASTNodeSearchUtil.getAstNode(u, code.indexOf("0x0"), 23); System.out.println(nd.getLength()); }
And without ASTNodeSearchUtil: ASTParser parser = ASTParser.newParser(AST.JLS3); String code= "\r\n" + "public class ThisClass {\r\n" + "\r\n" + " public static final long VAL = 100L + \r\n" + " 20L" + " public static final double XO = 0x0.0000000000001P-1022; \r\n" + "}\r\n" + ""; parser.setSource(code.toCharArray()); CompilationUnit u = (CompilationUnit) parser.createAST(null); FieldDeclaration field = (FieldDeclaration) ((org.eclipse.jdt.core.dom.AbstractTypeDeclaration) u.types().get(0)).bodyDeclarations().get(1); VariableDeclarationFragment fr = (VariableDeclarationFragment)field.fragments().get(0); Expression e = fr.getInitializer(); System.out.println(e.getLength()); -- That prints 3 for me.
Isn't this related to the missing ';' at the end of the first field declaration? I'll investigate.
Missed the semicolon in the test case. But that doesn't change the result.
The problem comes from the fact that the AST parser is not set to use 1.5 options. If you add this code: Map options = new HashMap(); options.put(JavaCore.COMPILER_CODEGEN_TARGET_PLATFORM, JavaCore.VERSION_1_5); options.put(JavaCore.COMPILER_COMPLIANCE, JavaCore.VERSION_1_5); options.put(JavaCore.COMPILER_SOURCE, JavaCore.VERSION_1_5); parser.setCompilerOptions(options); it works as expected. Note that with our code, you do have an error on the field you are looking at. The ASTParser is scanning the code using a 1.4 scanner which doesn't know how to read hexadecimal floating point literals. Ok to close?
Ok, it works for me with the compiler options (isn't JLS3 enough?). So now it is probably an enhancement request: But if you manage to parse the source code, why do you need to forcefully chop the source of the value? If there is an error, the user won't mind about getting the full initialization, right? With the semicolon in place, I also tried to check compilation unit's messages and problems and both returned arrays were empty.
With this code: ASTParser parser = ASTParser.newParser(AST.JLS3); String code= "\r\n" + "public class ThisClass {\r\n" + "\r\n" + " public static final long VAL = 100L +\r\n" + " 20L;" + " public static final double XO = 0x0.0000000000001P-1022; \r\n" + "}\r\n" + ""; parser.setSource(code.toCharArray()); /* Map options = new HashMap(); options.put(JavaCore.COMPILER_CODEGEN_TARGET_PLATFORM, JavaCore.VERSION_1_5); options.put(JavaCore.COMPILER_COMPLIANCE, JavaCore.VERSION_1_5); options.put(JavaCore.COMPILER_SOURCE, JavaCore.VERSION_1_5); parser.setCompilerOptions(options);*/ CompilationUnit u = (CompilationUnit) parser.createAST(null); System.out.println("" + u.getProblems().length + " problems"); FieldDeclaration field = (FieldDeclaration) ((org.eclipse.jdt.core.dom.AbstractTypeDeclaration) u.types().get(0)).bodyDeclarations().get(1); VariableDeclarationFragment fr = (VariableDeclarationFragment)field.fragments().get(0); Expression e = fr.getInitializer(); System.out.println(e.getLength()); I do get a problem in the compilation problems array. Also the javadoc of ASTParser.setCompilerOptions(Map) is pretty clear about the defaults. JLS3 is more about the shape of the resulting tree.
I think I found the difference. If the workbench settings are Java6.0 and the default complience box is checked, I get the behavior I've described (and no errors). If I uncheck the "use defaults" box it and set the source code level to 5.0 I get the correct behavior. If the default compliance level is 5.0, parsing works ok too. If the workbench compliance level is set to 1.4 I get the error which is also ok.
(In reply to comment #13) > I think I found the difference. If the workbench settings are Java6.0 and the > default complience box is checked, I get the behavior I've described (and no > errors). This would be a bug. I'll try to reproduce. > If I uncheck the "use defaults" box it and set the source code level to 5.0 I > get the correct behavior. If the default compliance level is 5.0, parsing works > ok too. If the workbench compliance level is set to 1.4 I get the error which > is also ok. So this seems to be acceptable. Let me know if I misunderstood, but only the first case should be investigated. Right?
Reproduced the first case. I am investigating.
Yes, only the first case is problematic. In the real code (patch for bug 85382) I have a compilation unit which should carry the appropriate compiler options. Maybe the documentation of ASTParser can be improved to stress that the compiler options affect the parsing. For example, the top level comment says <code> ASTParser parser = ASTParser.newParser(AST.JLS3); // handles JDK 1.0, 1.1, 1.2, 1.3, 1.4, 1.5, 1.6 </code> and it can be understood that using JLS3 is enough to parse 5.0 syntax. Another place where the need for compiler options could be documented is in the newParser() method.
The problem comes from the inner scanner. It was not set to "scan" in 1.6 mode. So it missed part of the floating point literal. Fix is under testing. Thanks for the test case and the time you spent on this issue. You are right about the documentation. I'll see what I can do for this.
I added this comment in the header of ASTParser: This is especially important to use if the parsing/scanning of the source code requires a different version than the default of the workspace. For example, the workspace defaults are 1.4 and you want to create an AST for a source code that is using 1.5 constructs. Hopefully this is clear enough now. Tests are running. As soon as they passed, I'll release the fix.
Created attachment 78435 [details] Proposed fix
Released for 3.4M2. Regression test added in org.eclipse.jdt.core.tests.dom.ASTConverter16Test#test0001
Thanks for the quick fix!
This was well deserved after all the help you provided. If you could test the 3.4M2 warmup build to provide feedback on this one, it would be great.
Verified for 3.4M2 using build I20070918-0010
Works for me too.
