Bug 530147 - aspectJ with Java 9 and JAXB
Summary: aspectJ with Java 9 and JAXB
Status: RESOLVED FIXED
Alias: None
Product: AspectJ
Classification: Tools
Component: Compiler (show other bugs)
Version: DEVELOPMENT   Edit
Hardware: PC Windows 10
: P3 normal (vote)
Target Milestone: 1.9.1   Edit
Assignee: aspectj inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2018-01-22 17:54 EST by James Bryant CLA
Modified: 2018-10-11 14:58 EDT (History)
2 users (show)

See Also:


Attachments
Simple class that replicates the issue (1.16 KB, application/x-zip-compressed)
2018-01-26 18:58 EST, James Bryant CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description James Bryant CLA 2018-01-22 17:54:28 EST
I'm having trouble building a project that uses aspectJ, Java 9, and JAXB.
This project builds with no issues under aspectJ 1.0.9.RC2 and Java 1.8.0_144.  But when I build the same project under Java 9.0.1, I get errors from ajc relating to missing JAXB classes.  There are no other issues reported.  Just missing JAXB classes.
As we all know, Java 9 added modules.  They put JAXB in a module and then disabled it by default.
I have not found a way to make ajc include this module.  Instead, I have added the JAXB jars to the normal dependencies.  The jars do contain the classes that ajc says are missing.
I'm building with Gradle, but I get the same errors if I use the equivalent ajc command.
Comment 1 Andrew Clement CLA 2018-01-23 16:48:14 EST
When supplying the missing ones are you mixing usage of module-path with regular classpath to include jaxb and the dependencies, or are you moving everything to 'normal classpath' when attempting to include those extra classes?  I'll probably need a way to recreate it to dig into it. I guess two issues - the inability to resolve dependencies you are supplying on the classpath and the problem of why they are needed in the first place. I can imagine running on Java9 possibly adjusting the generated code (in a harmless way) or the paths explored during type resolution.
Comment 2 James Bryant CLA 2018-01-26 15:58:30 EST
No, I am only including them on the classpath.  We are not yet using module-path.

To reproduce the issue, you should just be able to create a class that references a JAXB class.  Have some sort of aspect that applies to that class. 
 Then compile with ajc on Java 9 with JAXB on the classpath.  Any references you have to JAXB will cause an error.
Comment 3 James Bryant CLA 2018-01-26 18:58:49 EST
Created attachment 272432 [details]
Simple class that replicates the issue

I've put together a simple class that reproduces the issue.  There's a batch script that will execute it.  But you will have to gather the needed dependencies and adjust the paths accordingly.
This fails every time for me with AspectJ 1.9.0.RC2 and Java 9.0.1.
Comment 4 Andrew Clement CLA 2018-01-26 19:22:43 EST
I was just building an app when that comment came in, let me try it...
Comment 5 Andrew Clement CLA 2018-01-26 20:39:10 EST
I replicated the failure - or at least a failure :) I don't need any aspects (I didn't have the ones you referred to in your batch file anyway) and it compiles with errors. I then took it and put it into a basic eclipse project using eclipse ~4.7.2 which is what AspectJ is based upon.  It didn't compile there either.

I then took it and put it in an Eclipse 4.8 build, this time it did compile (although it seemed to have initially thrown the jars on the modulepath in the UI and I had to move them).

So... that leads me to believe it is an Eclipse compiler issue, and we'll pick up the fix when we pickup a newer copy of JDT.  I was planning to do this at some point, maybe not so soon but looks like I have a good reason to tackle it now.  

Thanks for spending time on the testcase.
Comment 6 James Bryant CLA 2018-01-29 14:40:27 EST
Always happy to help!  Let me know if there is anything else I can do.
Would your fix be able to make it in RC3? If so, is there a current timeline for the release of RC3? Also curious if there's a timeline for the final release of 1.9.0?
I"m being asked to put together a timeline so any information would be a huge help.  
Thanks again!!
Comment 7 Andrew Clement CLA 2018-01-29 14:58:04 EST
I'm working on the upgrade right now. I'd hope by end of week to be at another RC but I won't know what impact all the changes I'm picking up are having on the tests until I finish the upgrade (it takes quite a few hours). When I at least have it compiling, i'll update this bug with progress info.
Comment 8 James Bryant CLA 2018-01-29 15:29:10 EST
Ok, thanks for the info!
Comment 9 Andrew Clement CLA 2018-02-01 14:46:02 EST
JDT upgrade is done, now sorting through testcase failures due to all the updates picked up. Soon I'll add a test for your scenario to check it works.
Comment 10 James Bryant CLA 2018-02-02 09:00:57 EST
Great!  Thanks for the status update!
Comment 11 Andrew Clement CLA 2018-02-02 13:07:07 EST
Just checked and your testcase that was giving me four errors with 1.9.0.rc2:

compiling /Users/aclement/Downloads/playjaxb/src/UsesJAXB.java

/Users/aclement/Downloads/playjaxb/src/UsesJAXB.java:2 [error] The import javax.xml.bind.JAXBContext cannot be resolved
import javax.xml.bind.JAXBContext;
       ^^^^^^^^^^^^^^^^^^^^^^^^^
/Users/aclement/Downloads/playjaxb/src/UsesJAXB.java:7 [error] The import org.w3c.dom.Document cannot be resolved
import org.w3c.dom.Document;
       ^^^^^^^^^^^^^^^^^^^
/Users/aclement/Downloads/playjaxb/src/UsesJAXB.java:17 [error] Document cannot be resolved to a type
Document document;

/Users/aclement/Downloads/playjaxb/src/UsesJAXB.java:18 [error] JAXBContext cannot be resolved to a type
JAXBContext context;
^^

4 errors

is giving me no errors with the proposed 1.9.0.rc3:


compiling /Users/aclement/Downloads/playjaxb/src/UsesJAXB.java

weaver operating in reweavable mode.  Need to verify any required types exist.

woven class UsesJAXB (from /Users/aclement/Downloads/playjaxb/src/UsesJAXB.java)


However, I have 2 failing tests in the full AspectJ suite that I'm still trying to fix before producing a real build.
Comment 12 James Bryant CLA 2018-02-04 12:51:55 EST
Excellent news!  Thanks for the update!
Comment 13 Andrew Clement CLA 2018-02-05 14:37:05 EST
There is a 1.9.0.RC3 available from usual repository. Please give it a try.
Comment 14 James Bryant CLA 2018-02-07 11:05:35 EST
Sorry for the delay.  Ran into problems and wanted to fully investigate before I responded.

The JAXB issue does now appear to be fixed.  But now I'm having alot of issues all of a sudden.  In alot of places I'm now having to add my project dependencies (actual other projects in the product) to aspectpath, otherwise I get classnotfound exceptions for my project classes.  I'm not sure what is going on here.

When I add the projects to aspectpath, I can build a few of them, but then in one of my projects I keep getting this:

[ant:iajc] D:\workspace-git\adminservices\config\api\src\main\java\com\entrust\adminservices\config\primitive\client\ConfigWsClientErrorHandlerAspect.java [error] Internal compiler error: java.lang.RuntimeException: Unknown constant type 18 at org.aspectj.apache.bcel.classfile.ConstantPool.constantToString(ConstantPool.java:212)
[ant:iajc] (no source information available)
[ant:iajc] ABORT
[ant:iajc]
[ant:iajc] Exception thrown from AspectJ 1.9.0.RC3
[ant:iajc]
[ant:iajc] This might be logged as a bug already -- find current bugs at
[ant:iajc]   http://bugs.eclipse.org/bugs/buglist.cgi?product=AspectJ&component=Compiler
[ant:iajc]
[ant:iajc] Bugs for exceptions thrown have titles File:line from the top stack,
[ant:iajc] e.g., "SomeFile.java:243"
[ant:iajc]
[ant:iajc] If you don't find the exception below in a bug, please add a new bug
[ant:iajc] at http://bugs.eclipse.org/bugs/enter_bug.cgi?product=AspectJ
[ant:iajc] To make the bug a priority, please include a test program
[ant:iajc] that can reproduce this exception.
[ant:iajc] Unknown constant type 18
[ant:iajc]
[ant:iajc] when batch building BuildConfig[null] #Files=50 AopXmls=#0
[ant:iajc] Unknown constant type 18
[ant:iajc] java.lang.RuntimeException: Unknown constant type 18
[ant:iajc]      at org.aspectj.apache.bcel.classfile.ConstantPool.constantToString(ConstantPool.java:212)
[ant:iajc]      at org.aspectj.apache.bcel.classfile.ConstantPool.constantToString(ConstantPool.java:254)
[ant:iajc]      at org.aspectj.apache.bcel.classfile.Utility.codeToString(Utility.java:1002)
[ant:iajc]      at org.aspectj.apache.bcel.classfile.Utility.codeToString(Utility.java:160)
[ant:iajc]      at org.aspectj.apache.bcel.classfile.Code.getCodeString(Code.java:372)
[ant:iajc]      at org.aspectj.weaver.bcel.BcelMethod.isEquivalentTo(BcelMethod.java:693)
[ant:iajc]      at org.aspectj.weaver.CrosscuttingMembers.equivalent(CrosscuttingMembers.java:500)
[ant:iajc]      at org.aspectj.weaver.CrosscuttingMembers.replaceWith(CrosscuttingMembers.java:319)
[ant:iajc]      at org.aspectj.weaver.CrosscuttingMembersSet.addOrReplaceAspect(CrosscuttingMembersSet.java:117)
[ant:iajc]      at org.aspectj.weaver.CrosscuttingMembersSet.addOrReplaceAspect(CrosscuttingMembersSet.java:68)
[ant:iajc]      at org.aspectj.weaver.bcel.BcelWeaver.prepareForWeave(BcelWeaver.java:514)
[ant:iajc]      at org.aspectj.ajdt.internal.compiler.AjPipeliningCompilerAdapter.ensureWeaverInitialized(AjPipeliningCompilerAdapter.java:529)
[ant:iajc]      at org.aspectj.ajdt.internal.compiler.AjPipeliningCompilerAdapter.weaveQueuedEntries(AjPipeliningCompilerAdapter.java:509)
[ant:iajc]      at org.aspectj.ajdt.internal.compiler.AjPipeliningCompilerAdapter.queueForWeaving(AjPipeliningCompilerAdapter.java:447)
[ant:iajc]      at org.aspectj.ajdt.internal.compiler.AjPipeliningCompilerAdapter.afterProcessing(AjPipeliningCompilerAdapter.java:432)
[ant:iajc]      at org.aspectj.ajdt.internal.compiler.CompilerAdapter.ajc$after$org_aspectj_ajdt_internal_compiler_CompilerAdapter$5$6b855184(CompilerAdapter.aj:104)
[ant:iajc]      at org.aspectj.org.eclipse.jdt.internal.compiler.Compiler.process(Compiler.java:930)
[ant:iajc]      at org.aspectj.org.eclipse.jdt.internal.compiler.Compiler.processCompiledUnits(Compiler.java:573)
[ant:iajc]      at org.aspectj.org.eclipse.jdt.internal.compiler.Compiler.compile(Compiler.java:473)
[ant:iajc]      at org.aspectj.org.eclipse.jdt.internal.compiler.Compiler.compile(Compiler.java:424)
[ant:iajc]      at org.aspectj.ajdt.internal.core.builder.AjBuildManager.performCompilation(AjBuildManager.java:1069)
[ant:iajc]      at org.aspectj.ajdt.internal.core.builder.AjBuildManager.performBuild(AjBuildManager.java:273)
[ant:iajc]      at org.aspectj.ajdt.internal.core.builder.AjBuildManager.batchBuild(AjBuildManager.java:186)
[ant:iajc]      at org.aspectj.ajdt.ajc.AjdtCommand.doCommand(AjdtCommand.java:114)
[ant:iajc]      at org.aspectj.ajdt.ajc.AjdtCommand.runCommand(AjdtCommand.java:60)
[ant:iajc]      at org.aspectj.tools.ajc.Main.run(Main.java:371)
[ant:iajc]      at org.aspectj.tools.ajc.Main.runMain(Main.java:248)
[ant:iajc]      at org.aspectj.tools.ajc.Main.main(Main.java:84)
[ant:iajc]
[ant:iajc] Feb 07, 2018 9:59:07 AM org.aspectj.weaver.tools.Jdk14Trace info
[ant:iajc] INFO: Dumping to D:\workspace-git\adminservices\config\api\.\ajcore.20180207.095907.860.txt
[ant:iajc]
[ant:iajc] 1 fail|abort, 1 error

FAILURE: Build failed with an exception.


I cannot seem to find a work around for this one and I'm completely stuck.  I also don't understand why I've had to add all my projects to the aspectpath.
Please help! 

Thanks,
James Bryant
Comment 15 James Bryant CLA 2018-02-07 11:10:31 EST
Could this be a possible reversion of https://bugs.eclipse.org/bugs/show_bug.cgi?id=490315 ?

Sounds very similar.
Comment 16 James Bryant CLA 2018-02-07 12:28:00 EST
Ok, some more details:

Adding -XnoInline fixes the "Unknown constant type 18" problem.  So it definitely looks like a regression of https://bugs.eclipse.org/bugs/show_bug.cgi?id=490315
I guess that's the workaround for now, but I'd prefer to not have to turn off inlining.

As I mentioned before, adding all my 'projects' to aspectpath got past the classnotfound issues.

But now I have a new one and nothing I've tried is working so far:

[ant:iajc] D:\workspace-git\adminservices\ens\src\main\java\com\entrust\adminservices\ens\EmailNotificationMessage.java [error] Internal compiler error: java.lang.NullPointerException at org.aspectj.org.eclipse.jdt.internal.compiler.lookup.BinaryModuleBinding.create(BinaryModuleBinding.java:68)
[ant:iajc] (no source information available)
[ant:iajc] ABORT
[ant:iajc]
[ant:iajc] Exception thrown from AspectJ 1.9.0.RC3
[ant:iajc]
[ant:iajc] This might be logged as a bug already -- find current bugs at
[ant:iajc]   http://bugs.eclipse.org/bugs/buglist.cgi?product=AspectJ&component=Compiler
[ant:iajc]
[ant:iajc] Bugs for exceptions thrown have titles File:line from the top stack,
[ant:iajc] e.g., "SomeFile.java:243"
[ant:iajc]
[ant:iajc] If you don't find the exception below in a bug, please add a new bug
[ant:iajc] at http://bugs.eclipse.org/bugs/enter_bug.cgi?product=AspectJ
[ant:iajc] To make the bug a priority, please include a test program
[ant:iajc] that can reproduce this exception.
[ant:iajc]
[ant:iajc] when batch building BuildConfig[null] #Files=8 AopXmls=#0
[ant:iajc] null
[ant:iajc] java.lang.NullPointerException
[ant:iajc]      at org.aspectj.org.eclipse.jdt.internal.compiler.lookup.BinaryModuleBinding.create(BinaryModuleBinding.java:68)
[ant:iajc]      at org.aspectj.org.eclipse.jdt.internal.compiler.lookup.LookupEnvironment.getModuleFromAnswer(LookupEnvironment.java:435)
[ant:iajc]      at org.aspectj.org.eclipse.jdt.internal.compiler.lookup.LookupEnvironment.askForTypeFromModules(LookupEnvironment.java:375)
[ant:iajc]      at org.aspectj.org.eclipse.jdt.internal.compiler.lookup.LookupEnvironment.askForType(LookupEnvironment.java:237)
[ant:iajc]      at org.aspectj.org.eclipse.jdt.internal.compiler.lookup.UnresolvedReferenceBinding.resolve(UnresolvedReferenceBinding.java:111)
[ant:iajc]      at org.aspectj.org.eclipse.jdt.internal.compiler.lookup.BinaryTypeBinding.resolveType(BinaryTypeBinding.java:223)
[ant:iajc]      at org.aspectj.org.eclipse.jdt.internal.compiler.lookup.PackageBinding.getTypeOrPackage(PackageBinding.java:238)
[ant:iajc]      at org.aspectj.org.eclipse.jdt.internal.compiler.lookup.CompilationUnitScope.findImport(CompilationUnitScope.java:550)
[ant:iajc]      at org.aspectj.org.eclipse.jdt.internal.compiler.lookup.CompilationUnitScope.findSingleImport(CompilationUnitScope.java:609)
[ant:iajc]      at org.aspectj.org.eclipse.jdt.internal.compiler.lookup.CompilationUnitScope.faultInImports(CompilationUnitScope.java:458)
[ant:iajc]      at org.aspectj.org.eclipse.jdt.internal.compiler.lookup.CompilationUnitScope.faultInTypes(CompilationUnitScope.java:524)
[ant:iajc]      at org.aspectj.org.eclipse.jdt.internal.compiler.Compiler.process(Compiler.java:897)
[ant:iajc]      at org.aspectj.org.eclipse.jdt.internal.compiler.Compiler.processCompiledUnits(Compiler.java:573)
[ant:iajc]      at org.aspectj.org.eclipse.jdt.internal.compiler.Compiler.compile(Compiler.java:473)
[ant:iajc]      at org.aspectj.org.eclipse.jdt.internal.compiler.Compiler.compile(Compiler.java:424)
[ant:iajc]      at org.aspectj.ajdt.internal.core.builder.AjBuildManager.performCompilation(AjBuildManager.java:1069)
[ant:iajc]      at org.aspectj.ajdt.internal.core.builder.AjBuildManager.performBuild(AjBuildManager.java:273)
[ant:iajc]      at org.aspectj.ajdt.internal.core.builder.AjBuildManager.batchBuild(AjBuildManager.java:186)
[ant:iajc]      at org.aspectj.ajdt.ajc.AjdtCommand.doCommand(AjdtCommand.java:114)
[ant:iajc]      at org.aspectj.ajdt.ajc.AjdtCommand.runCommand(AjdtCommand.java:60)
[ant:iajc]      at org.aspectj.tools.ajc.Main.run(Main.java:371)
[ant:iajc]      at org.aspectj.tools.ajc.Main.runMain(Main.java:248)
[ant:iajc]      at org.aspectj.tools.ajc.Main.main(Main.java:84)
[ant:iajc]
[ant:iajc] Feb 07, 2018 11:22:09 AM org.aspectj.weaver.tools.Jdk14Trace info
[ant:iajc] INFO: Dumping to D:\workspace-git\adminservices\ens\.\ajcore.20180207.112209.375.txt
[ant:iajc]
[ant:iajc] 1 fail|abort, 1 error

FAILURE: Build failed with an exception.

This happens both with or without -XnoInline.
Comment 17 Andrew Clement CLA 2018-02-07 16:17:14 EST
This thing is now all fixed up:

[ant:iajc] Unknown constant type 18
[ant:iajc] java.lang.RuntimeException: Unknown constant type 18
[ant:iajc]      at org.aspectj.apache.bcel.classfile.ConstantPool.constantToString(ConstantPool.java:212)

I thought it was a missing backport from 1.8 > 1.9 of the fix for the bug you referenced but that doesn't seem to quite be it. Anyway, I updated constantToString() with everything it needs to cope with '18' (and all the other ones it was missing).

That leaves us with:

- needing to add things to aspectpath

So it won't work if they are simply on classpath? It has to be aspectpath? 

-[ant:iajc] java.lang.NullPointerException
[ant:iajc]      at org.aspectj.org.eclipse.jdt.internal.compiler.lookup.BinaryModuleBinding.create(BinaryModuleBinding.java:68)

I've added an extra bit of debug (system error line prefixed AJDEBUG when this happens, so we can understand which module is causing trouble).



These changes are all in a 1.9.0.BUILD-SNAPSHOT in the snapshot area of the same spring repository. Perhaps try with that? I'm concerned some of these things I won't be able to address without being able to dig into things.  Are you actively defining modules in your own code or just having to use the JDK modules because you are running on Java 9?
Comment 18 James Bryant CLA 2018-02-07 21:00:45 EST
Ok, I'm no longer seeing: Unknown constant type 18
Even without the -XnoInline flag.  So that seems to be fixed.

I'm not seeing the second exception anymore.  But I am seeing this:

[ant:iajc] AJDEBUG: Unable to determine module for name [C@45394b31

To answer your question, we have not converted anything to modules yet.  All of the projects are just as they were for Java 8.

I am seeing additional issues now, though:

[ant:iajc] D:\workspace-git\adminservices\aes\src\main\java\com\entrust\adminservices\toolkit\internal\xap\XAPObject.java:20 [error] The type org.w3c.dom.Document is ambiguous

That one is happening whether the org.w3c.dom jar is on the classpath or not.

If I take my the dependencies on other projects out of the aspectpath and I still get the class not found exceptions. 

I'm also seeing this on certain projects.  The projects I see it on fail when being built with parallel building on:

[ant:iajc] [warning] incorrect classpath: D:\workspace-git\adminservices\config\api\target\classes\java\main
[ant:iajc] incorrect classpath: D:\workspace-git\adminservices\config\api\target\resources\main
[ant:iajc] incorrect classpath: D:\workspace-git\adminservices\config\api\target\classes\java\main
[ant:iajc] incorrect classpath: D:\workspace-git\adminservices\config\api\target\resources\main
[ant:iajc]

When I build those projects non-parallel, they build, but I still get this:

[ant:iajc] [warning] incorrect classpath: D:\workspace-git\adminservices\config\api\target\resources\main
[ant:iajc] incorrect classpath: D:\workspace-git\adminservices\config\api\target\resources\main
[ant:iajc]

We've always built in parallel without problems.  And in this case, even though parallel was on, only the problematic project was being built at the time.


So, we whacked one mole, but some others seem to be popping up.  I can try and create something to reproduce it, but it might take me some time to put that together.
Comment 19 James Bryant CLA 2018-02-08 10:22:34 EST
Found some more debugs:

[ant:iajc] AJDEBUG: Unable to determine module for name [C@2fd953a6
[ant:iajc] AJDEBUG: Unable to determine module for name [C@49231d79
[ant:iajc] AJDEBUG: Unable to determine module for name [C@100d5018
[ant:iajc] AJDEBUG: Unable to determine module for name [C@45f32b12
[ant:iajc] AJDEBUG: Unable to determine module for name [C@2dc54b50
Comment 20 Andrew Clement CLA 2018-02-08 12:30:13 EST
doh, i missed that the module name was a char array in my haste, so those debug messages are giving me the array address rather than the contents of the thing. Fixing that right now so we get more sensible debug messages.

I am starting to think the problems are *all* related to the classpath handling that had to be rewritten for Java9 support.  The need to understand modules meant we had to replace the simple stuff we had (which worked great...) with much more complicated processing that uses more JDT than we have before (previously path entries were jars or directories, now they might additionally be modules).  The messages like this:

[ant:iajc] [warning] incorrect classpath: D:\workspace-git\adminservices\config\api\target\resources\main

are, I suspect, due to driving more of this JDT infrastructure that more deeply processes classpaths and verifies them.  Do those paths actually exist and have content?

I am thinking it might also be the cause of the 'The type org.w3c.dom.Document is ambiguous' message.  It took a while to get the new code to work reliably for my tests but they are all passing now. If it is the classpath handling it might be enough for you to just give me the classpath (or the whole command line) that is being used for building that module, rather than any sources. If I have the paths I can try driving it through this classpath handling code and looking at the result. Can you turn on debugging to see the ajc command line being used?  Feel free to send it to my email rather post on the public bug (aclement AT pivotal.io) - any observations you have on that path would be interesting too (any elements that are there but don't exist or whatever).

(I've just published a new snapshot that should produce correct module names on those AJDEBUG lines - I expected it to get past that position now as in addition to printing the debug line I told it to default to a particular module and continue rather than fail).
Comment 21 James Bryant CLA 2018-02-08 16:28:38 EST
Just got this out of the latest version:

[ant:iajc] AJDEBUG: Unable to determine module for name java.activation

and also:

[ant:iajc] AJDEBUG: Unable to determine module for name java.xml.bind
[ant:iajc] AJDEBUG: Unable to determine module for name java.xml.bind
[ant:iajc] AJDEBUG: Unable to determine module for name java.xml.bind
[ant:iajc] AJDEBUG: Unable to determine module for name java.xml.bind

I can turn up logging to get more output including the command line arguments.  I'll get back to you on that.
Comment 22 Andrew Clement CLA 2018-02-08 18:08:01 EST
Interesting.  I did find a path that was duplicating classpath entries (which I went looking for because you mentioned "The type org.w3c.dom.Document is ambiguous") - a fix for that is now in the build snapshots.

This one "[ant:iajc] [warning] incorrect classpath: D:\workspace-git\adminservices\config\api\target\resources\main"
is logging coming out from deeper classpath processing, as suspected. It is annoying that it isn't routed through the regular messaging infrastructure but instead sent directly to sysout/err.


I know you are running on Java9 but are you passing the -1.9 flag or the -1.8 flag?  if you aren't using modules I'd have thought we could avoid problems related to module lookups (Unable to determine module for name java.xml.bind) by just passing -1.8 - I'm not saying it shouldn't behave supplying -1.9, just exploring options for making it behave on Java9 whilst we sort through things.
Comment 23 James Bryant CLA 2018-02-08 19:23:20 EST
I'm not seeing any change in the new build.  Getting the exact same errors. 
Are you sure a new build made it to the repo.  My tooling didn't seem to think there was an update.  I forced it to re-download, but, again, no change in behavior.
Comment 24 Andrew Clement CLA 2018-02-08 19:51:31 EST
I just repushed the code to the maven repo to make sure the changes are definitely up there but I can't guarantee it'll fix the ambiguous error as I can't recreate any problems right now (I've not seen the duplication thing cause a problem, but it clearly wasn't right).

At the moment I'm playing around with your test sample code trying to recreate some kind of failure that I can investigate (like the 'Unable to determine module for name java.xml.bind' problem) but no luck yet.
Comment 25 James Bryant CLA 2018-02-09 15:35:31 EST
I'm pretty certain this time I downloaded a newer version.  Unfortunately, the ambiguous Document class issue still persists.  Seems to be no change in behavior in this one.

I sent you the command arguments I'm seeing in the debug output.

I did try with the -1.8 flag, and the issue actually did go away.  I still have the issue of needing various projects on the aspectpath, though.

If we can get past the ambiguous class issue by building with -1.8, that works for us in the meantime.  Due to other issues holding us back, we can't deploy Java 9 to the field yet.  So in reality, we'll be using 1.8 compatibility mode in our build for the time being.  Interestingly, I thought we had been testing with 1.8 mode on, but it wasn't until I looked closely at the command arguments that I saw it was passing 1.9 instead.  We had set the compatibility mode in the wrong place in our build this whole time.

Still haven't had a chance to try and build something to reproduce it.  I promise I'll try if I can just get a chance to.
Comment 26 Andrew Clement CLA 2018-02-09 16:51:01 EST
I've been digging into things more. Actually going back to the original comment.  Your UsesJAXB code will compile with javac using "javac UsesJAXB.java --add-modules java.xml.bind" (after I stripped out the log4j bits). But that doesn't work for AspectJ, I think actually because JDT underneath isn't quite doing things right and we (AJC) aren't surfacing some of that correctly either. We are blindly passing through --add-modules to be dealt with by JDT but it just isn't working.

Getting it all behaving with -1.8 would seem a nice stepping stone to take before throwing additional complexities around -1.9 into the mix. If you get anywhere with recreating problems I'm happy to look at them. Even happy to look at the actual app itself although I imagine it isn't open source which can make things tricky.

thanks for sending over the command line, that's useful.
Comment 27 Andrew Clement CLA 2018-02-09 16:57:29 EST
Also a bit surprised to hear when folks are rolling out Java9 in anger given the short term nature of the release (end of public updates is just next month), but I guess if you need to be on the bleeding edge you'll be on 10 (18.3) in march, I haven't even looked at supporting that yet!  But it is rather smaller in scope I think, no changes the size of this module stuff.
Comment 28 Andrew Clement CLA 2018-02-09 17:04:59 EST
Will compile if I'm explicit about the modulepath:

ajc UsesJAXB.java -1.9 --module-path ~/jvms/j904/Home/jmods --add-modules java.xml.bind

Something javac seems to implicitly realize.
Comment 29 James Bryant CLA 2018-02-09 17:45:57 EST
Having a bit of trouble with that one.  The ant task doesn't appear to want to take the --add-modules parameter:

iajc doesn't support the "add-modules" attribute

Does the ant task need to be updated to accept the new parameter of something?

Yea, Java 9 is a short release, but our thinking is if we can get to Java 9 now, we can easily jump to 10 when it comes out, and then to OpenJDK when they discontinue JDK.

The big jump is the one to 9.  So we're trying to go through that pain sooner rather than later.  

I still curious about the need projects on the aspectpath issue is all about.  Is doing that a good way to be using the tool?  Could there be any consequences to have them on the aspectpath?
Comment 30 Andrew Clement CLA 2018-02-09 19:18:43 EST
> Does the ant task need to be updated to accept the new parameter of something?
Yes, it does, I will do that. I think I was waiting for things to settle down on the JDT side before doing it.


> I still curious about the need projects on the aspectpath issue is all about.  Is doing that a good way to be using the tool?  Could there be any consequences to have them on the aspectpath?

I am very curious about this too.

There would only be consequences if those jars contained aspects themselves, otherwise the aspectpath, like the classpath should just be getting used to resolve types.  I did ask a few comments ago but not sure if you saw it in the noise - does it not work if you instead include those things in the classpath?  It shouldn't matter which one they are on if just used for resolving. If it does matter, that is another clue.  I'm not saying they should be on the classpath either, ideally, just interested if it makes a difference.
Comment 31 James Bryant CLA 2018-02-12 11:49:03 EST
To answer you question, I'm not having to include every project on the aspectpath, but I am having to include some still.  

I cleared my cache and pulled in the latest snapshot, but the ant task still doesn't accept the add-modules option.

If I get a chance, I'll try and run from the command line and see what I get.
Comment 32 James Bryant CLA 2018-02-12 15:40:15 EST
There's a new wrinkle.

While I've been trying to figure this out, I have been putting into place other parts of our build process upgrade.  
Today, I decided to upgrade our builds to Gradle 4.5.1 from 4.4.1.  I fully expected no problems.  But, to my surprise we suddenly encountered the project dependency problems.  There was no change other than to upgrade Gradle.  I'm trying to figure out what difference between the two is causing this.  I'm curious if this apparent Gradle issue is our real problem.  The build in this case is running RC2 and Java 8.
Comment 33 Andrew Clement CLA 2018-02-13 16:21:48 EST
> I cleared my cache and pulled in the latest snapshot, but the ant task still doesn't accept the add-modules option.

Yesterday was a holiday here in Canada, I'm going to get to this bit imminently.

> While I've been trying to figure this out, I have been putting into place other parts
> of our build process upgrade.  
> Today, I decided to upgrade our builds to Gradle 4.5.1 from 4.4.1.  I fully expected
> no problems.  But, to my surprise we suddenly encountered the project dependency
> problems.  There was no change other than to upgrade Gradle.  I'm trying to figure
> out what difference between the two is causing this.  I'm curious if this apparent
> Gradle issue is our real problem.  The build in this case is running RC2 and Java 8.

This is an interesting finding. Is this build running AspectJ 1.9 or AspectJ 1.8 when it hits the dependency issue?
Comment 34 James Bryant CLA 2018-02-14 10:11:19 EST
The build was using 1.9.0.RC2.

I don't have any fresh info at the moment.  I've had to put this aside while I tend to other priorities.  But, I will let you know the latest results once I have a chance to work on it again.
Comment 35 Andrew Clement CLA 2018-02-14 17:35:25 EST
> The build was using 1.9.0.RC2.

Sorry, i see now you said that before, I didn't need to ask.

To support these java9 args and future args I've just published the version that allows compilerArgs to be set on iajc, which will support any options:

<iajc destdir="bin" failonerror="true"
     showWeaveInfo="true" source="1.9" target="1.9"
     debug="true"  fork="true"  maxmem="256m">
     <compilerArg value="--add-modules"/>
     <compilerArg value="java.xml.bind,java.io"/>
    <src path="src" />
    <classpath>
            <pathelement location="${aspectj.home}/lib/aspectjrt.jar"/>
    </classpath>
</iajc>

It'll be more work to manually add them all, this future proofs us a little better and is something we can use to play with any options right now.
Comment 36 James Bryant CLA 2018-02-16 13:31:50 EST
Awesome!  I'm going to try and give this a try today if I have a chance.  I'll report back my results as soon as I can.
Comment 37 James Bryant CLA 2018-02-18 19:05:20 EST
Good news!  I have completed a successful build using a combination of Gradle 4.4.1, the latest AspectJ 1.9.0.BUILD-SNAPSHOT, and Java 9.0.4.  Looks like we have resolved all the blocking issues!  Thank you so much for your help!

Sorry for the delay on this.  I needed an updated version of the Gradle AspectJ plugin to support the new compilerArg feature.  The author appears to have abandoned the project, so I had to fork it and fix it myself.

The only remaining issue is the odd behavior with respect to resolving other projects on the classpath.  As you know, I managed to replicate this behavior just using Java 8 with Gradle 4.5.1.  When using Gradle 4.4.1 with either Java 8 or Java 9, there is no problem. The the variable here is Gradle, not Java.  This leads me to think (obviously) it may be a Gradle issue instead of an AspectJ issue.

As far as this ticket, feel free to close it as the original issue is fully resolved at this point.  If the remaining issue does turn out to be an AspectJ issue, I can open another ticket.

Now that this is resolved, when would you expect to be able to release the next RC.  I've been able to get approvals for the RC builds to be put in our internal repo, but I might not have that luck with SNAPSHOT builds.  I've been having to pull your snapshots directly from the repo in my local build.

And again, thank you very much for your help!
Comment 38 James Bryant CLA 2018-02-19 08:59:09 EST
Update, I figured out the remaining issue.  It turned out to be a task dependency problem where the aspectJ task was being executed before the projects it depended on.  Tweaking the task dependencies in the Gradle plugin fixed the problem.
So, I have no remaining issue.  
As always, thanks again for all the help!
Comment 39 Andrew Clement CLA 2018-02-19 12:28:21 EST
That is great news that you are working again.  This was the one thing holding up the next RC, and I'm trying to get out a final release but I need to get it approved by the eclipse guys for formal release, it isn't something I can declare on my own.  So I think next RC is imminent then formal (non RC) release a little after that.

There was one 'debug' thing left in, if I recall correctly, the one that produced:

AJDEBUG: Unable to determine module for name XXX

are those messages gone now? Since I'd like to remove that if we don't need it anymore.
Comment 40 James Bryant CLA 2018-02-20 12:51:58 EST
I have not seen this statement in my latest testing.  I think that issue got resolved.
I'll be eagerly anticipating the new release!
Comment 41 Andrew Clement CLA 2018-02-21 15:41:25 EST
1.9.0.rc4 is published. Currently progressing through a release review process for formal 1.9.0 in march.
Comment 42 James Bryant CLA 2018-02-25 11:50:07 EST
We verified that RC4 is working for us.  Thanks again!  We'll be eagerly awaiting the final release.
Comment 43 Andrew Clement CLA 2018-10-11 14:58:33 EDT
Believed all fixed up.