Description Jonas Konrad 2020-02-23 07:46:38 EST

This is a regression from a previous JDT version. The code runs without exception in 3.15.0 but crashes in 3.20.0. Bisecting shows that this bug was probably introduced in commit 03980ed8a170fb1a3db7e242fe3e2831224a91ec .

When compiling the 'share' and 'macosx' *combined* platforms of the java.base module of the jdk9u head, the following exception (line numbers from jdt 3.20.0):

Exception in thread "main" java.lang.NullPointerException
	at org.eclipse.jdt.internal.compiler.ast.ReferenceExpression.generateCode(ReferenceExpression.java:350)
	at org.eclipse.jdt.internal.compiler.ast.Statement.generateArguments(Statement.java:420)
	at org.eclipse.jdt.internal.compiler.ast.MessageSend.generateCode(MessageSend.java:501)
	at org.eclipse.jdt.internal.compiler.ast.ReturnStatement.generateCode(ReturnStatement.java:203)
	at org.eclipse.jdt.internal.compiler.ast.Block.generateCode(Block.java:83)
	at org.eclipse.jdt.internal.compiler.ast.IfStatement.generateCode(IfStatement.java:204)
	at org.eclipse.jdt.internal.compiler.ast.AbstractMethodDeclaration.generateCode(AbstractMethodDeclaration.java:344)
	at org.eclipse.jdt.internal.compiler.ast.AbstractMethodDeclaration.generateCode(AbstractMethodDeclaration.java:281)
	at org.eclipse.jdt.internal.compiler.ast.TypeDeclaration.generateCode(TypeDeclaration.java:579)
	at org.eclipse.jdt.internal.compiler.ast.TypeDeclaration.generateCode(TypeDeclaration.java:649)
	at org.eclipse.jdt.internal.compiler.ast.CompilationUnitDeclaration.generateCode(CompilationUnitDeclaration.java:412)
	at org.eclipse.jdt.internal.compiler.Compiler.process(Compiler.java:912)
	at org.eclipse.jdt.core.dom.CompilationUnitResolver.resolve(CompilationUnitResolver.java:1044)
	at org.eclipse.jdt.core.dom.CompilationUnitResolver.resolve(CompilationUnitResolver.java:662)
	at org.eclipse.jdt.core.dom.ASTParser.createASTs(ASTParser.java:1013)
	at at.yawk.javabrowser.generator.TestCase.main(TestCase.java:36)

The exact steps to reproduce this are as follows:

- `cd /var/tmp`
- Download the jdk9u repo: https://hg.openjdk.java.net/jdk-updates/jdk9u/jdk/archive/tip.tar.bz2 (according to the website, 17464:d54486c189e5 is the latest revision at the time I'm writing this)
- `tar xf tip.tar.bz2`
- `mkdir merged`
- `cp -r jdk-d54486c189e5/src/java.base/{share,macosx}/classes/* merged/`
- Build the test case from git@github.com:yawkat/eclipse-jdt-testcase-0.git . It contains the java code listed below, and a basic maven pom to build with the latest jdt.
- Run the test case. This can be done by building the classpath using `mvn dependency:build-classpath`, appending `target/testcase-1.0-SNAPSHOT.jar`

The test case in the repo is:

import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.Comparator;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.stream.Collectors;
import org.eclipse.jdt.core.JavaCore;
import org.eclipse.jdt.core.dom.AST;
import org.eclipse.jdt.core.dom.ASTParser;
import org.eclipse.jdt.core.dom.FileASTRequestor;

public class TestCase {
    public static void main(String[] args) throws IOException {
        ASTParser parser = ASTParser.newParser(AST.JLS9);
        Map<String, String> options = new HashMap<>();
        options.put(JavaCore.COMPILER_SOURCE, JavaCore.VERSION_10);
        options.put(JavaCore.CORE_ENCODING, "UTF-8");
        options.put(JavaCore.COMPILER_DOC_COMMENT_SUPPORT, JavaCore.ENABLED);
        parser.setCompilerOptions(options);
        parser.setResolveBindings(true);
        parser.setKind(ASTParser.K_COMPILATION_UNIT);
        String src = "/var/tmp/merged";
        parser.setEnvironment(
                new String[0],
                new String[]{ src },
                new String[]{ "UTF-8" },
                false
        );
        List<Path> files = Files.walk(Paths.get(src))
                .filter(p -> p.toString().endsWith(".java") && !Files.isDirectory(p))
                .sorted(Comparator.reverseOrder())
                .collect(Collectors.toList());
        parser.createASTs(
                files.stream().map(Path::toString).toArray(String[]::new),
                files.stream().map(f -> "UTF-8").toArray(String[]::new),
                new String[0],
                new FileASTRequestor() {
                },
                null
        );
    }
}

What I have discovered about this bug so far:
- The bug is triggered because for some reason, during compilation, the type for java.lang.Object in the package java.lang is missing, even though it had previously been visited. This leads to the NPE in the codegen because this "unknown" class has no `getClass` method.
- Automatic bisecting (`mvn -Pbuild-individual-bundles -pl org.eclipse.jdt.core -am clean package` and then running the test case with the new jdt.core) shows that this bug first occurs in 03980ed8a170fb1a3db7e242fe3e2831224a91ec .
- This bug does not happen when instead of merging the two source directories they are compiled as two separate source locations, i.e. by passing two strings to setEnvironment.
- This bug relies on file order, so the `.sorted(Comparator.reverseOrder())` in the test case above is absolutely essential.
- This bug does not happen with other platform source code, only with macosx.

Since this bug appears to be extremely dependent on the input it is possible that I have missed some factor in my environment that triggers this bug that I failed to mention in the reproduction steps. If you have trouble reproducing, I will try to narrow it down further.