Community
Participate
Working Groups
Created attachment 276811 [details] sample workspace I'm using mapstruct as annotation-processor to generate mappers for mapping entities to dtos and vice versa. With M3-Milestone-Release I'm getting a FilerException "Source file already created" when editing a mapper and than saving this file. Now the autobuild is going on and I'm getting the error. When I now call clean projects then the error goes away and the source file is generated. I'd setup and attach a small workspace where the error occurs. Comment #18 of bug 367599 (https://bugs.eclipse.org/bugs/show_bug.cgi?id=367599#c18) seems to be the same problem with another anntotation processor.
(In reply to Dirk Ziegenbalg from comment #0) > With M3-Milestone-Release I'm getting a FilerException Does it work with M2? If not, did it work with 4.9? "Source file already created" when editing a mapper and than > saving this file. Now the autobuild is going on and I'm getting the error. > When I now call clean projects then the error goes away and the source file > is generated. So, you have an ugly workaround?
(In reply to Dani Megert from comment #1) > (In reply to Dirk Ziegenbalg from comment #0) > > With M3-Milestone-Release I'm getting a FilerException > Does it work with M2? If not, did it work with 4.9? No problems with 4.9. M2, I don't know. > > "Source file already created" when editing a mapper and than > > saving this file. Now the autobuild is going on and I'm getting the error. > > When I now call clean projects then the error goes away and the source file > > is generated. > So, you have an ugly workaround? Yes, there is a workaround but I cannot use it because I have ~ 500 projects in my workspace and everytime cleaning and building all is not possible. And when depending projects where build they also get red.
M2 not avaliable for download, tested with M1, same problem here.
Exception is raised in org.eclipse.jdt.internal.apt.pluggable.core.filer.IdeFilerImpl.createSourceFile(IdeFilerImpl.java:177)
Thanks, I can reproduce. I think the whole approach of just checking whether the type exists is broken because of incremental compilation. My patch from bug Bug 539774 just improved the situation w.r.t. change from Bug 534979, but I think we should just disable this check for 4.10RC2
@Jay, WDYT?
I had this typed out before running into collision: --- I was going to type that the annotation processor should catch these exceptions and should not attempt to create the same file again. But then realized that it's the reconciler we are dealing with and what we already have was created during the previous compilation and not previous round of annotation processing. The batch compiler (and Javac) will get away with it since they start with a clean slate. The JDK bug [1] does not talk about what to do in case of multiple compilation attempts or incremental build. The solution should be to : 1. Somehow identify that the existing file is the compiled output of what we are trying to create during the previous build/save. 2. And if (1) returns true and this is 0th round of annotation processing, i) Don't report ii) Overwrite the existing file silently. [1] https://bugs.openjdk.java.net/browse/JDK-8193576
New Gerrit change created: https://git.eclipse.org/r/133478
(In reply to Eclipse Genie from comment #8) > New Gerrit change created: https://git.eclipse.org/r/133478 I am sure this patch has its limitations, but this is all I could muster with the limited time I have. This takes care of the issue reported, though. When I am more awake tomorrow, will take another look. Meanwhile, if Till or someone can glance through the patch and provide feedback, that will be nice. In the patch, I am ignoring the .class files that are found in this project's output location (but only if it's the first round of annotation processing).
(In reply to Jay Arthanareeswaran from comment #9) > (In reply to Eclipse Genie from comment #8) > > New Gerrit change created: https://git.eclipse.org/r/133478 > > I am sure this patch has its limitations, but this is all I could muster > with the limited time I have. This takes care of the issue reported, though. Remain the other two bugs fixed?
I have incorporated Till's suggestion of null check and also added new tests. (In reply to Dani Megert from comment #10) > Remain the other two bugs fixed? Yes, those fixes are unaffected.
Gerrit change https://git.eclipse.org/r/133478 was merged to [master]. Commit: http://git.eclipse.org/c/jdt/eclipse.jdt.core.git/commit/?id=656a7f631fb76e187296f81dcd900dfff9ff6a65
Thanks Till and Manoj! The fix is in master now.
@Jay, after thinking about this longer: Wouldn't incremental compilation still fail for classes generated after the first round?
(In reply to Till Brychcy from comment #14) > @Jay, after thinking about this longer: Wouldn't incremental compilation > still fail for classes generated after the first round? Yes, but the annotation processor or whoever is creating the file should ensure that such attempt is not made. From the aforementioned JDK bug: "Any subsequent attempt to create the same file during a run will throw a FilerException" The CU that contains the annotation and as a result triggers annotation processing is likely to be compiled more than once. I have captured this in the testcase as well. Please see FilerTesterProc#testBug542090a() and testBug542090b().
(In reply to Jay Arthanareeswaran from comment #15) > (In reply to Till Brychcy from comment #14) > > @Jay, after thinking about this longer: Wouldn't incremental compilation > > still fail for classes generated after the first round? > > Yes, but the annotation processor or whoever is creating the file should > ensure that such attempt is not made. From the aforementioned JDK bug: I think I was not clear: what about files that are regularly created only in a later round (e.g. because an annotation processor is applied to a generated file from the previous round). Wouldn't the type for these still exist during incremental compilation and an error be thrown?
(In reply to Till Brychcy from comment #16) > I think I was not clear: what about files that are regularly created only in > a later round (e.g. because an annotation processor is applied to a > generated file from the previous round). > Wouldn't the type for these still exist during incremental compilation and > an error be thrown? Yes, agree. Those scenarios still will be affected. I wonder if I should remove that check at all.
Tested with Build id: I20181205-1800 Problem already exists (Exception now raised on another line): Source file already exists : org.eclipse.annotationprocessing.test.mapper.TestMapperImpl at org.eclipse.jdt.internal.apt.pluggable.core.filer.IdeFilerImpl.createSourceFile(IdeFilerImpl.java:182)
(In reply to Jay Arthanareeswaran from comment #17) > (In reply to Till Brychcy from comment #16) > > I think I was not clear: what about files that are regularly created only in > > a later round (e.g. because an annotation processor is applied to a > > generated file from the previous round). > > Wouldn't the type for these still exist during incremental compilation and > > an error be thrown? > > Yes, agree. Those scenarios still will be affected. > > I wonder if I should remove that check at all. Maybe doing it only during full builds might be an option? Still I'd say, let's disable it for 4.10, so we have more time to think about it.
(In reply to Jay Arthanareeswaran from comment #17) > Yes, agree. Those scenarios still will be affected. > > I wonder if I should remove that check at all. Or the other option is to ignore this error if we are in a reconciler. We have the Phase#RECONCILE, but what we are really dealing with here is the Phase#BUILD, both in case of full build and incremental build. Looks like this is going to be bit more involved to differentiate incremental compilation. I would like to defer the rest of the effort to post 4.10 if nobody has any objection.
(In reply to Jay Arthanareeswaran from comment #21) > Looks like this is going to be bit more involved to differentiate > incremental compilation. I would like to defer the rest of the effort to > post 4.10 if nobody has any objection. But with check disabled for 4.10?
New Gerrit change created: https://git.eclipse.org/r/133587
(In reply to Eclipse Genie from comment #23) > New Gerrit change created: https://git.eclipse.org/r/133587 This rolls back some of the changes made via bug 534979. I have also disabled few tests. Till, please take a look.
(In reply to Eclipse Genie from comment #23) > New Gerrit change created: https://git.eclipse.org/r/133587 The tests are green. We just missed the last RC2 build. Once Till reviews, I will request for another build.
Gerrit change https://git.eclipse.org/r/133587 was merged to [master]. Commit: http://git.eclipse.org/c/jdt/eclipse.jdt.core.git/commit/?id=178dc273b0765dcfaf9cca6a214c89ecf4c18d31
Dirk, can you try out the latest I build and report back? TIA! http://download.eclipse.org/eclipse/downloads/drops4/I20181206-0320/
(In reply to Jay Arthanareeswaran from comment #27) > Dirk, can you try out the latest I build and report back? TIA! > > http://download.eclipse.org/eclipse/downloads/drops4/I20181206-0320/ Same error with this I build (raised on IdeFilerImpl.java:182) because it contains an "old" version of apt.pluggable (org.eclipse.jdt.apt.pluggable.core_1.2.300.v20181205-0900.jar). I wait for a new I build and give it a new try tomorrow. Hopefully the exception should'nt be raised anymore.
(In reply to Dirk Ziegenbalg from comment #28) > (In reply to Jay Arthanareeswaran from comment #27) > > Dirk, can you try out the latest I build and report back? TIA! > > > > http://download.eclipse.org/eclipse/downloads/drops4/I20181206-0320/ > > Same error with this I build (raised on IdeFilerImpl.java:182) because it > contains an "old" version of apt.pluggable > (org.eclipse.jdt.apt.pluggable.core_1.2.300.v20181205-0900.jar). I wait for > a new I build and give it a new try tomorrow. Hopefully the exception > should'nt be raised anymore. Tested with Build id: I20181206-0815 it now works fine. I found another problem with annotation processing. For this I will file a fresh bug. Thanks for the fast response.
(In reply to Dirk Ziegenbalg from comment #29) > Tested with Build id: I20181206-0815 it now works fine. I found another > problem with annotation processing. For this I will file a fresh bug. Thanks > for the fast response. Thanks Dirk, for your quick verification too. I will keep this in open but move to 4.11.
So you moved this bug to 4.11 I don't need to open a new because in my oinion it is caused by the same "source". With the tested build I now get the same exception raised on another location when I rename the CU and do saving CU: javax.annotation.processing.FilerException: Source file already created: /Test/.apt_generated/org/eclipse/annotationprocessing/test/mapper/TestMapperImpl.java at org.eclipse.jdt.internal.apt.pluggable.core.filer.IdeFilerImpl.createSourceFile(IdeFilerImpl.java:161) at org.mapstruct.ap.internal.processor.MapperRenderingProcessor.createSourceFile(MapperRenderingProcessor.java:68) I've checked with M1 -> same result. With the latest 4.9 release this error doesn't occur. I think, there is a general problem with incremental compilation detecting source-files generated by apt.
(In reply to Dirk Ziegenbalg from comment #31) > I've checked with M1 -> same result. With the latest 4.9 release this error > doesn't occur. > > I think, there is a general problem with incremental compilation detecting > source-files generated by apt. We need to take a close look at the scenarios you are talking about in light of the below JDK bug and find out what the expected behavior. Simplest way of finding if we have a problem is by comparing the full build with saving just a file or two. https://bugs.openjdk.java.net/browse/JDK-8193576
Jay, you are right. I think you should throw away any generated source when a CU is saved because you don't know what has changed and what was generated before and what will/should now be generated. So a full annotation-processing-round is required, which could slow down the incremental build significant depending on the annotation processor(s).
Comment 18 in bug 367599 referenced by Dirk Ziegenbalg in the opening post of this bug is mine. I want to confirm that indeed it was the same problem as yours just with a different annotation processor (Google's AutoValue) and was fixed with the release of 4.10RC2 (and subsequent release 4.10). As you have already noticed, this affected many annotation processors. Just for reference, apart from AutoValue, I had also noticed a reported issue in Immutables ( https://github.com/immutables/immutables/issues/866 ) probably stemming from this bug and likely fixed too with the fix provided here.
Bulk move out of 4.11
Sorry, I couldn't provide the required attention during 4.12. I will revisit during 4.13 to assess the situation and take this to completion.
Moving out to 4.14.
Bulk move out of 4.14
Sorry, this didn't get the required attention since I was busy with Java 14 work. Hopefully in the next version!
I don't see myself sparing time for this in near future. Moving out. Anyone willing to investigate, please feel free to take over.
The behavior I see with my annotator is that my Processor's global variables are reset at the end of the round where ProcessingEnvironment.processingOver() returns true (I assume the processor class gets unloaded). On subsequent rounds, however, eclipse is not giving me a full list of annotated elements to repopulate my global context. Also, and strangely, it sometimes allows and sometimes forbids updating the source files created on the previous round. It should be consistent, IMHO. If you give me PE.processingOver(), allow me to rewrite the source file. If you unload my processor class, allow me to rescan the annotations. Manually performing a clean build works though.
Hi, I believe this issue is related. https://github.com/google/dagger/issues/2987 Can someone confirm that it is? I have a sample repro there also. The gist of the issue is that when files change on disk (due to fetching code or modifying code), eclipse will fail with the error "Could not generate unknown file: Source file already created:" I am not sure it is related, but I have used the following approach to ensure that annotation processing works when a single file is updated, but an output file takes multiple files as input. https://stackoverflow.com/questions/56133377/how-to-get-all-elements-with-an-annotation-in-an-intellij-incremental-build
I think I'm affected by this as well, also with mapstruct, see: https://github.com/mapstruct/mapstruct/discussions/2920 I tried to compare ECJ and Javac: Javac has this code (Liberica JDK 17.0.3.1): private void checkNameAndExistence(ModuleSymbol mod, String typename, boolean allowUnnamedPackageInfo) throws FilerException { checkName(typename, allowUnnamedPackageInfo); ClassSymbol existing = elementUtils.getTypeElement(typename); boolean alreadySeen = aggregateGeneratedSourceNames.contains(Pair.of(mod, typename)) || aggregateGeneratedClassNames.contains(Pair.of(mod, typename)) || initialClassNames.contains(typename) || containedInInitialInputs(typename); if (alreadySeen) { if (lint) log.warning(Warnings.ProcTypeRecreate(typename)); throw new FilerException("Attempt to recreate a file for type " + typename); } if (lint && existing != null) { log.warning(Warnings.ProcTypeAlreadyExists(typename)); } if (!mod.isUnnamed() && !typename.contains(".")) { throw new FilerException("Attempt to create a type in unnamed package of a named module: " + typename); } } For the source(s) that fail in ECJ, existing is != null but alreadySeen is false. Because lint is not enabled, there is not even a warning (which I find ok). So, is javac too lenient or is ECJ too strict? I'd guess the latter. This is preventing me from using ECJ via Maven in my project(s), which is a pity because ECJ is so much faster as we have many (generated) generics.
(In reply to Falko Modler from comment #43) > I think I'm affected by this as well, also with mapstruct, see: > https://github.com/mapstruct/mapstruct/discussions/2920 > > ... FWIW, my problem turned out to be a parameterization issue in plexus-compiler-eclipse (the component that maven-compiler-plugin uses internally to actually call the compiler): https://github.com/codehaus-plexus/plexus-compiler/issues/232
