Bug 527420 - Java file, which is generated by annotation processor, is not compiled
Summary: Java file, which is generated by annotation processor, is not compiled
Status: NEW
Alias: None
Product: JDT
Classification: Eclipse Project
Component: APT (show other bugs)
Version: 4.6   Edit
Hardware: PC Windows 7
: P3 normal (vote)
Target Milestone: ---   Edit
Assignee: Jay Arthanareeswaran CLA
QA Contact:
URL:
Whiteboard: stalebug
Keywords:
Depends on:
Blocks:
 
Reported: 2017-11-17 12:49 EST by Sergej Isbrecht CLA
Modified: 2022-09-19 11:34 EDT (History)
4 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Sergej Isbrecht CLA 2017-11-17 12:49:41 EST
Environment:
Version: Neon.3 Release (4.6.3)
Build id: 20170314-1500

java version "1.8.0_144"
Java(TM) SE Runtime Environment (build 1.8.0_144-b01)
Java HotSpot(TM) 64-Bit Server VM (build 25.144-b01, mixed mode)

Lets say I have a processor A, which generates a Java-File, which in turn has a Annotation, which will be consumed by another processor B. Processor B will generate another java file.

Lets say processor A generates class X. Class x has a annotation, which will be consumed by processor B. Class Y will be generated by processor B.

All java files are generated as expected. For example class X and class Y. But eclipse has a compiling problem. 

Lets say I have class a1, a2, a3 which will be consumed by processor A. All java files will be generated for a1, a2, a3. The classes a1, a2 have a .class file in the bin folder but not a3. It is missing. The three classes created by processor B are all compiled.

This failure does not happen in IntelliJ Idea / Gradle.

I published a project in GitHub: https://github.com/SergejIsbrecht/EclipseAnnotationBug

Checkout master, go to folder consumer. Execute ./gradlew build eclipse and import main project (EclipseAnnotationBug) into Eclipse. Start a clean build and look at the compiler errors.

generated files in .apt_generated folder:
CalzonePizzaFactory.java           
ImmutableMargheritaPizzaFactory.java  
MargheritaPizzaFactory.java
ImmutableCalzonePizzaFactory.java  
ImmutableSoloMioPizzaFactory.java     
SoloMioPizzaFactory.java

compiled files in bin
CalzonePizzaFactory.class
ImmutableCalzonePizzaFactory$Builder.class
ImmutableCalzonePizzaFactory.class
ImmutableMargheritaPizzaFactory$Builder.class
ImmutableMargheritaPizzaFactory.class
ImmutableSoloMioPizzaFactory$Builder.class
ImmutableSoloMioPizzaFactory.class
SoloMioPizzaFactory.class

Missing:
MargheritaPizzaFactory.class is missing.

Therefore I can not compile the project. If I reopen Eclipse, the missing file .class file will be compiled, but will be missing again, when a clean-build happens.

One interesting thing happens. If I don't use the annotation-processor provided by immutables.org, but instead implement the processor B on my one, all class-files will be generated correctly.

Please have a look at branch example2 on https://github.com/SergejIsbrecht/EclipseAnnotationBug . Use ./gradlew build cleanEclipse eclipse to init project.

Thank you for your time. Best regards,


Sergej Isbrecht
Comment 1 Walter Harley CLA 2017-11-18 15:38:34 EST
"If I don't use the annotation-processor provided
by immutables.org, but instead implement the processor B on my one, all
class-files will be generated correctly."

This seems very relevant, since we know that in general, the Eclipse annotation processor tooling does correctly handle cascaded generation (there are many test cases for it as well as lots of processors out there in the real world that do it).

I wonder if there is something peculiar about the immutables.org processor, or about the way that it is configured in this project?  Can you be more precise about how you implemented the processor B yourself, and what might be different than the immutables.org version?

(Full disclosure: I'm no longer set up to debug APT problems myself; I'm just trying to gather information for the developers who work on it nowadays.)
Comment 2 Sergej Isbrecht CLA 2017-11-19 09:40:34 EST
Hi Walter,

thank you for your time.

Please have a look at branch example2:

https://github.com/SergejIsbrecht/EclipseAnnotationBug/tree/example2/src/main/java

ImmutableGenerator.java is processor A
AnotherProcessor.java is processor B (own impl)

Processor B is a dummy implementation, which just generates an empty interface. If I use the ProxyProcessor (provided by immutables.org) as Processor B it generates a full implementation.

I looked at the implementation of the ProxyProcessor by immutables.org (https://github.com/immutables/immutables) library, but it is very difficult to understand what is happening there. 

As I already wrote the java files will be generated just fine in both versions, but something is wrong with the class compilation, when using immutables.org processor. If I use Gradle or IntelliJ Idea the class files will be generated like expected.

Could it be that Gradle uses two passes. In the first run the java files will be generated and in the second they will be compiled. Contrary to this, eclipse might handle things differently.
Comment 3 Yuriy Kulikov CLA 2017-11-19 11:16:03 EST
Hello Walter,

I am a collegue of Sergej's and I am familiar with this issue.

What we have observed is that Eclipse would delete the .class files which are compiled from the files generated by the first processor. This does not happen if the checkbox "Scrub output folders when cleaning projects" is not checked.

So it seems that for whatever reason Eclipse discards the .class files for .java files which are passed to the immutables.io annotation processor.

I am quite certain that the issue is with the Eclipse APT, because what is missing is the .class file. All .java files are properly generated. Given the fact that annotation processors do not have any impact on the compilation (apart from generating the code), we can assume that there is a problem with Eclipse compiler. This may be, of course, caused by some unexpected Immutables.io behaviour, but still.

Could you please look into it? Thank you.
Comment 4 Walter Harley CLA 2017-11-19 17:09:18 EST
I'm sure that if you are building inside the IDE, as opposed to building with gradle, the order of generation and compilation may be different.  Gradle and IntelliJ will both use javac rather than using the Eclipse IDE's internal incremental compiler.

In the Eclipse IDE, if I remember (it has been many years since I last looked at this code), the Java build happens as an iteration: compile existing Java files; execute annotation processors; if the processors have generated new Java files, compile those; execute annotation processors on the new files; repeat until there are no more generated files.  This is essentially as defined by the JSR269 spec.

However, it's interesting that the "Scrub output folders when cleaning projects" option would affect the output.  Once the Eclipse Java builder has started, there would never be a clean; the clean would happen in the generic Eclipse builder, prior to the Java-specific build beginning.  The fact that "Scrub output folders when cleaning projects" changes the behavior makes me wonder if .class files are somehow getting created prior to the Java build starting.

Alternately, maybe, another guess might be that the immutables.org processor is creating .class files directly, rather than or in addition to creating .java files?  If I remember, that area of the spec was much less well defined and less well tested.

I suspect this is going to be difficult to make progress on unless we can reproduce it with a simpler processor.

Jay, any thoughts on this?
Comment 5 Jay Arthanareeswaran CLA 2017-11-19 23:07:46 EST
(In reply to Walter Harley from comment #4)
> Alternately, maybe, another guess might be that the immutables.org processor
> is creating .class files directly, rather than or in addition to creating
> .java files?  If I remember, that area of the spec was much less well
> defined and less well tested.

> Jay, any thoughts on this?

Thanks for chipping in, Walter. I don't have a clue at this point either. As you said, it would be interesting to see what exactly the processor does.

I will checkout the given steps/testcase later this week (once RC3 tasks are done).
Comment 6 Sergej Isbrecht CLA 2017-11-27 15:43:06 EST
Dear Jay,

did you find any time to check the processor and the JDT? If not, could you please give us a estimate, when you could have a look?

Best regards
Comment 7 Jay Arthanareeswaran CLA 2018-04-24 04:52:06 EDT
(In reply to Sergej Isbrecht from comment #6)
> Dear Jay,
> 
> did you find any time to check the processor and the JDT? If not, could you
> please give us a estimate, when you could have a look?
> 
> Best regards

Unfortunately, I haven't found time for this with Java 9 and Java 10 work. Will have to move out for now. Meanwhile, if somebody can put this in a simple annotation processor, that'll be wonderful.
Comment 8 Eclipse Genie CLA 2020-09-21 00:11:18 EDT
This bug hasn't had any activity in quite some time. Maybe the problem got resolved, was a duplicate of something else, or became less pressing for some reason - or maybe it's still relevant but just hasn't been looked at yet.

If you have further information on the current state of the bug, please add it. The information can be, for example, that the problem still occurs, that you still want the feature, that more information is needed, or that the bug is (for whatever reason) no longer relevant.

--
The automated Eclipse Genie.
Comment 9 Jay Arthanareeswaran CLA 2020-09-22 02:34:32 EDT
I think this is still relevant. Let me see if I can still make sense of the issue and get somewhere this week.
Comment 10 Eclipse Genie CLA 2022-09-19 11:34:30 EDT
This bug hasn't had any activity in quite some time. Maybe the problem got resolved, was a duplicate of something else, or became less pressing for some reason - or maybe it's still relevant but just hasn't been looked at yet.

If you have further information on the current state of the bug, please add it. The information can be, for example, that the problem still occurs, that you still want the feature, that more information is needed, or that the bug is (for whatever reason) no longer relevant.

--
The automated Eclipse Genie.