Community
Participate
Working Groups
See this thread: https://www.eclipse.org/lists/eclipse-dev/msg11669.html I wrote: >> Jay, it seems that mostly you are maintaining the file which >> currently is named org.eclipse.jdt.compiler.apt/lib/javax15api.jar. >> Before I am going to ask a question herwe is what I know so far: >> >> -- This JAR file exists since Java 9, getting renamed to the >> currently supported Java version ever since, as the Git history >> for the lib folder tells us: >> https://github.com/eclipse/eclipse.jdt.core/commits/master/org.eclipse.jdt.compiler.apt/lib >> >> -- The contained files are binaries from the JDK, more precisely >> from the java.compiler module. When I unpacked them from JDK 15 >> and 16, I saw that they are compiled to the JDK's target >> version, i.e. Java 15 classes have the corresponding class file >> version. >> >> -- Why however the files for Java 15 in javax15api.jar have a Java >> 14 class file version, I do not know. Maybe you actually >> extracted the sources from OpenJDK and compiled them before >> zipping them up. That would suggest, they could also be compiled >> using an older version in order to lower the minimum compiler >> version necessary to build the compiler module for APT. >> >> I know this is not really relevant for JDT Core, because the test >> suite runs all tests on different Java versions up to the latest >> supported one using toolchains. But AspectJ forks the two modules >> org.eclipse.jdt.apt.core and org.eclipse.jdt.compiler.apt, applying >> our changes and then compiling and packaging them into a simple JAR >> (not as OSGi bundles). Out target byte code version is Java 8, but >> because of the compile-time dependency on javax15api.jar we need to >> run the build at least on JDK 14 because of the Java 14 class file >> format inside there. >> >> Questions: >> >> 1. Are you compiling those files manually? >> >> 2. If so, would you consider always compiling them to the lowest >> possible target version? That way we could return to a minimum >> version of Java 8 for our own build, because the rest of AspectJ >> Core does not need more, only the tests for higher Java versions >> do. But they are skipped automatically if we build with an older >> JDK. Jay replied: > I can't think of a reason why the JAR has to be on the latest version. > If at all, I would imagine they are compiled at the same compliance as > the containing project, which is 11. > > Let me try to compile them with 1.8 and see how it goes. > > Meanwhile, can you spare a moment to raise a bug please?
New Gerrit change created: https://git.eclipse.org/r/c/jdt/eclipse.jdt.core/+/180119
(In reply to Eclipse Genie from comment #1) > New Gerrit change created: > https://git.eclipse.org/r/c/jdt/eclipse.jdt.core/+/180119 Alexander, here's a small experiment that shows that the latest source code from java.compiler module can't be compiler at 1.8 compliance level. Yet, at the moment there's only one compiler error that we may be able to fix by altering the code.
Well, it is a little bit more than just one compile error, because there is Javadoc stuff linking to APIs which do not exist in Java 8, usage of the unknown @Deprecated(since=...) parameter, unknown APIs for creating immutable lists and sets and of course the unknown Runtime.version() stuff. In one place even the 'var' syntax for variable declaration was used. I backported all of that quick & dirty from Java 15 to Java 8. Actually, you should upgrade the files to Java 16 and backport from there, because JDT already supports Java 16. I am attaching a patch. It builds with your Ant build script running JDK 8. If you don't like it, just do your own backport. If you like me to hand in a PR instead of a patch, just let me know.
Created attachment 286298 [details] Backport Java 15 to 8 (diff on top of Jay's Gerrit patch)
Created attachment 286301 [details] Backport Java 15 to 8 (minimal diff on top of Jay's Gerrit patch, excluding Javadoc changes)
(In reply to Alexander Kriegisch from comment #3) > Well, it is a little bit more than just one compile error, because there is > Javadoc stuff linking to APIs which do not exist in Java 8, usage of the > unknown @Deprecated(since=...) parameter, unknown APIs for creating > immutable lists and sets and of course the unknown Runtime.version() stuff. > In one place even the 'var' syntax for variable declaration was used. I > backported all of that quick & dirty from Java 15 to Java 8. Actually, you > should upgrade the files to Java 16 and backport from there, because JDT > already supports Java 16. Agree, it could have been Java 16 or even 17 for the beta branch. But having to make so many changes to make it work is not desirable. We don't want to be reconciling our changes with the JDK changes if there are significant changes. Just like the jclMin* jars' content, should we just take the APIs skeleton code? That way, when there are new JDK APIs we can simply incorporate them in our version. No code, just the constants and method signatures! Also, Andrey brought it up that the classes were missing the copyrights but they should include them. So, we should keep the copyright in the next iteration.
Did you look at the second patch? I changed less than 10 actual lines of code, which is quite minimal and should be easy enough to maintain and upgrade. Actually, you handed in the patch with the full files, I just built on top of your work in order to contribute a bit and make your life easier. If now you changed your mind and just want to commit skeletons, up to you. You are one of the maintainers of this product, I am merely a newbie contributor. Actually though, I do not feel like my proposed changes are rocket science or scary in any way. It is not like I ported Jigsaw back to Java 1.4 or so. Concerning licence issues, well I am not a lawyer and not really interested in those kinds of things. IMO we are all honest OSS developers here, donating our time and code to a good purpose. Nobody here wants to violate any licences. Hence, my approach is to just do what I want to do, push the code out into the public, and if some copyright buff explains to me nicely that I have done something wrong and forgot to mention someone's copyright or failed to copy one of those useless licence headers anywhere - IMO licences belong in a central place for the whole product, not into each single file - then I am going to do my best to try and fix it in order to be compliant. I am aware of the fact that this is not the style of a serious organisation like Eclipse, so I am just going to contribute in whichever way I see fit and let other people sort out the bureaucratic stuff. The personal time and code I donate is all I want to give. If that is not good enough for anyone, feel free to reject my contributions. I play fairly and openly, having no malicious intentions whatsoever. I do hope that suffices.
(In reply to Alexander Kriegisch from comment #7) > Did you look at the second patch? I changed less than 10 actual lines of > code, which is quite minimal and should be easy enough to maintain and > upgrade. Actually, you handed in the patch with the full files, I just built > on top of your work in order to contribute a bit and make your life easier. I did go through the changes and quite agree that they are straight forward. But what I am worried about is that one has to manually come up with these fixes, now as well as in future when we would need to bring in new changes from the JDK. What I am hoping for (I don't yet know if we can achieve this), is a process that will allow us to take the code out of JDK and run in through some automated script and be done with it.
I tried taking the source from JDK 16 afresh and tried stripping out implementation details and noticed that what I have been using to generate the javax*.jar is kind of old. Alexandar, to get the full picture, may be you can try with the latest source code once.
Did you upload or commit it somewhere, or are you telling me to extract it from the OpenJDK repository?
(In reply to Alexander Kriegisch from comment #10) > Did you upload or commit it somewhere, or are you telling me to extract it > from the OpenJDK repository? It's usually bundled with JDK itself as src.zip/java.compiler.
I just downloaded it from this tag 'jdk-16+36' here: https://github.com/openjdk/jdk/tree/jdk-16%2B36/src/java.compiler/share/classes I am attaching the original sources and a patch I made in 15 minutes, quick & dirty. It compiles, I only had to take a not so nice shortcut in javax.tools.ToolProvider::matches, because otherwise I would have had to backport a bunch of module-related classes from java.lang.
Created attachment 286312 [details] Java 16+36 compiler module classes
Created attachment 286313 [details] Backport Java 16+36 to 8
Andrey, you reviewed Jay's original patchset. Would you mind inspecting these, please? https://bugs.eclipse.org/bugs/attachment.cgi?id=286312 https://bugs.eclipse.org/bugs/attachment.cgi?id=286313 Simply click https://bugs.eclipse.org/bugs/attachment.cgi?id=286313&action=diff, if you want to see what exactly I backported to Java 8. I would like to know how to proceed. Should I create a new Gerrit patch with the updated original sources from JDK 16 and my backport, so they can be compiled with Java 8? I would push two separate commits, so you can see the originals and the diff on top of them separately. I would greatly appreciate your guidance. In order to improve the chances of doing it right, I would like to know: 1. Is it OK to put the sources in the same directory as Jay did, i.e. javax.api/src? 2. Should we proceed with my backport of the full classes, or have you decided internally to go with Jay's idea to just commit no-op skeletons which make JDT compile, without providing the actual JDK functionality? I cannot judge if the latter would be enough, because I do not know if any existing tests might actually call one of those JDK methods and expect correct results.
Alexander, We can definitely take your patch and bundle the jar with jdt.core projects. We can look at doing that during RC1. In long term, though, I am looking at coming up with an automatic solution that will convert the given a JDK source code (java.compiler alone, I suppose) and convert that into a something that can be compiled at compliance 1.8. The key here is "automatic". The reason being, when something changes in JDK 17 again, we don't want to be repeating what you just did or reconciling our changes with theirs. I have some WIP thing in my workspace, which I will share here shortly.
Once again: I would not add full JDK sources (modified or not) to JDT source tree, because jdt sources are redistributed and that would be redistribution of GPL code inside EPL licensed bundle and a license violation. Same for compiled code produced from original or modified GPL code. I can imagine, we can redistribute stubs of that API only, but even that would be questionable, if derived directly from modified sources. Note, that EPL and GPL are incompatible, unless EPL licensed bundle does dual licensing and the combined work is distributed under GPL, but that is not the case for JDT AFAIK, amd would affect all JDT included packages distributed ny Eclipse.org. I'm pretty sure we already violate GPL in JDT if existing org.eclipse.jdt.compiler.apt/lib/javax15api.jar was created from GPL code. https://www.eclipse.org/legal/epl-2.0/faq.php#h.it3upld1gcpe https://www.gnu.org/licenses/license-list.html.en#EPL2 *Why* do we need that jar at all? Can we avoid redistribution of GPL code or code/binaries derived from that?
(In reply to Andrey Loskutov from comment #17) > I would not add full JDK sources (modified or not) to JDT source tree, > (...) > > Note, that EPL and GPL are incompatible, unless EPL licensed bundle > does dual licensing and the combined work is distributed under GPL, > but that is not the case for JDT AFAIK (...) Thanks for your insights. I did not expect this meta discussion to become the main issue when originally asking the simple question why the JAR contains Java 14 class files and if that is strictly necessary. > I'm pretty sure we already violate GPL in JDT if existing > org.eclipse.jdt.compiler.apt/lib/javax15api.jar was created from GPL > code. I cannot provide many insights there. I just wanted to lend a hand in order to expedite this issue and make the build once more Java 8 compatible. What I can see in Git is that this JAR file exists redundantly in no less than 4 different modules since February 2020. Maybe the issue should have been discussed back then. Are we making anything worse by upgrading it and creating a separate issue investigating the licencing situation and eventually sorting it out with the help of an Eclipse lawywer or some other IP trail expert? The person who introduced these files - Jay, as far as I can see - might be able to shed some light on the following questions: > *Why* do we need that jar at all? Can we avoid redistribution of GPL > code or code/binaries derived from that?
(In reply to Andrey Loskutov from comment #17) > *Why* do we need that jar at all? > Can we avoid redistribution of GPL code or code/binaries derived from that? OK, I see that org.eclipse.jdt.compiler.apt has lot of classes extending javax.lang.model.element (that contains new constructs in Java 9+) and that we hope the code compiled against the "javax15api.jar" library should also work with Java 8. What I don't get is why do we *ship* "javax15api.jar" library at all? I don't see it referenced by the manifest, so if the classes would be needed at runtime, we would have trouble in Java 8-14 anyway (Java 15 classes won't load in old VM's) => we can build against the GPL licensed library/code, but we should not *ship* that. With this: any insights how to change the build to NOT package the jar into org.eclipse.jdt.compiler.apt? If we don't ship this, this bug will be solved as a side effect.
(In reply to Andrey Loskutov from comment #19) > With this: any insights how to change the build to NOT package the jar into > org.eclipse.jdt.compiler.apt? If we don't ship this, this bug will be solved > as a side effect. Correction: not org.eclipse.jdt.compiler.apt but org.eclipse.jdt.compiler.tool contains lib/javax15api.jar. As funny as it could be, the lib/javax15api.jar is used but is not shipped with org.eclipse.jdt.compiler.apt, however it is shipped with org.eclipse.jdt.compiler.tool jar. OMG. I'm not a maven/tycho user, I can't understand how get rid of the lib/javax15api.jar packaged into the org.eclipse.jdt.compiler.tool jar. The patch below tries to remove the jar from manifest, build.properties, and to add it to the pom exclude excludeResources - all without *any* effect. I use following command to run, and it seem to rebuild the jar in the target directory every time, but it still packages that lib/javax15api.jar into org.eclipse.jdt.compiler.tool jar. WHY??? mvn clean verify -Pbuild-individual-bundles -DskipTests=true @Christoph, Alex, Sravan: any idea? I believe you know how the tycho works? diff --git a/org.eclipse.jdt.compiler.tool/META-INF/MANIFEST.MF b/org.eclipse.jdt.compiler.tool/META-INF/MANIFEST.MF index 9f4528a..8a75d7f 100644 --- a/org.eclipse.jdt.compiler.tool/META-INF/MANIFEST.MF +++ b/org.eclipse.jdt.compiler.tool/META-INF/MANIFEST.MF @@ -11,4 +11,3 @@ Fragment-Host: org.eclipse.jdt.core;bundle-version="[3.3.0,4.0.0)" -Bundle-ClassPath: lib/javax15api.jar, - . +Bundle-ClassPath: . Automatic-Module-Name: org.eclipse.jdt.compiler.tool diff --git a/org.eclipse.jdt.compiler.tool/build.properties b/org.eclipse.jdt.compiler.tool/build.properties index d813e13..10f4ec2 100644 --- a/org.eclipse.jdt.compiler.tool/build.properties +++ b/org.eclipse.jdt.compiler.tool/build.properties @@ -19,5 +19,4 @@ about.html,\ - compiler_tool_fragment.properties,\ - lib/javax15api.jar + compiler_tool_fragment.properties +bin.excludes = lib/javax15api.jar src.includes = about.html -jars.extra.classpath = lib/javax15api.jar \ No newline at end of file diff --git a/org.eclipse.jdt.compiler.tool/pom.xml b/org.eclipse.jdt.compiler.tool/pom.xml index 05b863a..e45b380 100644 --- a/org.eclipse.jdt.compiler.tool/pom.xml +++ b/org.eclipse.jdt.compiler.tool/pom.xml @@ -39,3 +39,3 @@ <excludeResources> - <exclude>**/package.html</exclude> + <exclude>**/*api.jar</exclude> </excludeResources> @@ -45,2 +45,10 @@ <plugin> + <groupId>org.eclipse.tycho</groupId> + <artifactId>tycho-packaging-plugin</artifactId> + <version>${tycho.version}</version> + <configuration> + <useDefaultExcludes>false</useDefaultExcludes> + </configuration> +</plugin> + <plugin> <artifactId>maven-antrun-plugin</artifactId>
Andrey, Jay, why is this jar needed at all ? If there is any licensing concern I would ask you to involve IP team to be 100% sure. Supporting old versions comes at a too heavy price IMHO - we can't have better compatibility support than the JVM itself. If such things keep popping up we would not have a choice but to reduce JVM version support. Convoluting releng scripts even more is a topic I'm totally not going to support nor investigate. Only path I'm interested in is actually simplifying releng.
(In reply to Alexander Kurtakov from comment #21) > Andrey, Jay, why is this jar needed at all ? See comment 19, it is only needed at compile time: org.eclipse.jdt.compiler.apt has lot of classes extending javax.lang.model.element (that contains new constructs in Java 9+) and that we hope the code compiled against the "javax15api.jar" library should also work with Java 8. We don't need to package this jar into shipped bundles (but we can package it into test bundles if needed, because we don't re-distribute tests). > If there is any licensing > concern I would ask you to involve IP team to be 100% sure. I have strong licensing concerns, yes, see comment 17. > Supporting old versions comes at a too heavy price IMHO - we can't have > better compatibility support than the JVM itself. If such things keep > popping up we would not have a choice but to reduce JVM version support. > Convoluting releng scripts even more is a topic I'm totally not going to > support nor investigate. Only path I'm interested in is actually simplifying > releng. It is about supporting *future* JDK versions! If I see it right, the javax15api.jar is needed to support all the shiny new toys we get with Java 9+: modules, records, etc. So it would be great if someone could check how one can NOT ship a library with tycho that is used at compile time only.
(In reply to Andrey Loskutov from comment #22) > (In reply to Alexander Kurtakov from comment #21) > > Andrey, Jay, why is this jar needed at all ? > > See comment 19, it is only needed at compile time: > > org.eclipse.jdt.compiler.apt has lot of classes extending > javax.lang.model.element (that contains new constructs in Java 9+) and that > we hope the code compiled against the "javax15api.jar" library should also > work with Java 8. > > We don't need to package this jar into shipped bundles (but we can package > it into test bundles if needed, because we don't re-distribute tests). > > > If there is any licensing > > concern I would ask you to involve IP team to be 100% sure. > > I have strong licensing concerns, yes, see comment 17. Please send the question to the IP team. > > > Supporting old versions comes at a too heavy price IMHO - we can't have > > better compatibility support than the JVM itself. If such things keep > > popping up we would not have a choice but to reduce JVM version support. > > Convoluting releng scripts even more is a topic I'm totally not going to > > support nor investigate. Only path I'm interested in is actually simplifying > > releng. > > It is about supporting *future* JDK versions! If I see it right, the > javax15api.jar is needed to support all the shiny new toys we get with Java > 9+: modules, records, etc. > > So it would be great if someone could check how one can NOT ship a library > with tycho that is used at compile time only. Wouldn't it be an option to compile on latest Java (BREE Java16) with different -target/release ? That should make it use the classes from the JVM and no shipping should involved nor having the jar in git, no?
(In reply to Alexander Kurtakov from comment #23) > Please send the question to the IP team. I've opened bug 573648, I have no other idea how to ask IP team for questions :-) > Wouldn't it be an option to compile on latest Java (BREE Java16) with > different -target/release ? That should make it use the classes from the JVM > and no shipping should involved nor having the jar in git, no? Good idea. How to do that with tycho? :-) Actually we only need to exclude the jar from being packaged into the generated bundle - I assume that should be a very easy task?
New Gerrit change created: https://git.eclipse.org/r/c/jdt/eclipse.jdt.core/+/180788
(In reply to Eclipse Genie from comment #25) > New Gerrit change created: > https://git.eclipse.org/r/c/jdt/eclipse.jdt.core/+/180788 This doesn't work and need some more tycho magic :-(
Created attachment 286424 [details] List of modules containing javax15api.jar
(In reply to Alexander Kriegisch from comment #18) > What I can see in Git is that this JAR file exists redundantly in no > less than 4 different modules since February 2020. Correcting myself: The file exists since Java 9 support in mid 2017 and has changed its name since then several times from "java9api.jar" up to "javax15api.jar". So this is a long-standing issue. And I am repeating: The file is not in one 1 in 4 modules, see screenshot from my IDE: https://bugs.eclipse.org/bugs/attachment.cgi?id=286424 So if it is to be removed, the person(s) refactoring it should check which classes depend on it in each of those modules and make sure that whatever replaces it is a single entity and not something that needs to be committed 4x.
> Wouldn't it be an option to compile on latest Java (BREE Java16) with > different -target/release ? That should make it use the classes from the JVM > and no shipping should involved nor having the jar in git, no? The idea is good, but still you would need my changes (or at least some of them) in order to be able to do that. For example, we have to -- remove a module-info.java from the original code (or not copy it in the first place), -- not use private static interface methods (not allowed in Java 8), -- not use 'var' which became a reserved keyword in Java 10, -- not use annotations like @jdk.internal.PreviewFeature. Just check out my diff again: https://bugs.eclipse.org/bugs/attachment.cgi?id=286313&action=diff
(Sorry for the double-post browser glitch.) Or were you talking about compiling the 4 modules on Java 16 with target/release 8? Then of course, the whole point of this bug, to make the JAR's class files Java-8-compatible in order to be able to compile the whole project with JDK 8, would be moot. I am just throwing in my two cents here, you guys have all the knowledge and experience to decide without me. I am just trying to contribute a few thoughts to the discussion I unintentionally kicked off by creating this Bugzilla issue.
(In reply to Andrey Loskutov from comment #20) > I'm not a maven/tycho user, I can't understand how get rid of the > lib/javax15api.jar packaged into the org.eclipse.jdt.compiler.tool jar. > > The patch below tries to remove the jar from manifest, build.properties, and > to add it to the pom exclude excludeResources - all without *any* effect. > > I use following command to run, and it seem to rebuild the jar in the target > directory every time, but it still packages that lib/javax15api.jar into > org.eclipse.jdt.compiler.tool jar. WHY??? > > mvn clean verify -Pbuild-individual-bundles -DskipTests=true > > > @Christoph, Alex, Sravan: any idea? I believe you know how the tycho works? I'll take a look, just need to take a look how this lib/javax15api.jar building is done.
(In reply to Andrey Loskutov from comment #20) > but it still packages that lib/javax15api.jar into > org.eclipse.jdt.compiler.tool jar. WHY??? I think that the "baseline-replace" is playing tricks on you, I locally get: >> MavenProject: org.eclipse.jdt:org.eclipse.jdt.compiler.tool:1.2.1200-SNAPSHOT: >> baseline and build artifacts have same version but different contents >> ... >> The main artifact has been replaced with the baseline version That means that your locally build version is overwritten with the baseline that of course still contains the javax15api.jar regardless of what you have changed :-) Increasing the version in pom/manifest I'm able to exclude the javax15api.jar from the final artifact.
> I'll take a look, just need to take a look how this lib/javax15api.jar > building is done. If you don't want to use the Ant build file Jay provided with the original Gerrit patch, here is a little POM file I was using for verification in my local copy: <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion> <groupId>org.acme</groupId> <artifactId>javaxapi</artifactId> <version>1.0.0-SNAPSHOT</version> <properties> <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding> </properties> <build> <sourceDirectory>javax.api/src</sourceDirectory> <plugins> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-compiler-plugin</artifactId> <version>3.8.1</version> <configuration> <!--<source>16</source>--> <release>8</release> </configuration> </plugin> </plugins> </build> </project>
(In reply to Alexander Kriegisch from comment #31) > (Sorry for the double-post browser glitch.) > > Or were you talking about compiling the 4 modules on Java 16 with > target/release 8? Then of course, the whole point of this bug, to make the > JAR's class files Java-8-compatible in order to be able to compile the whole > project with JDK 8, would be moot. Actually the idea of compilation against Java 16 with -release 8 to get rid of the jar will not work, because in that case compiler would still need the jar, because it will only see Java 8 level API. Please note again, the whole point of this bug (wrong class version/license violation) has a very simple solution - the jar shouldn't be shipped at all. It is only required for compilation, at runtime we don't need it. Why we don't need it at runtime is easy: the "new" classes we have there from latest JDK are of course available if we run on that JDK, and of course they can't be loaded on any older JDK version because JVM would refuse to load classes with unsupported class version. So we do not need any "downport" patches to compile the jar content with Java 8 class version - it is not necessary, because it can stay at any / latest class version, because we only need it at compile time. And for producing jar - one probably just need a simple Java script that would need to be executed on desired (latest) JVM and that would just copy JDK class bytes from desired packages to the target jar.
(In reply to Andrey Loskutov from comment #35) > (In reply to Alexander Kriegisch from comment #31) > > (Sorry for the double-post browser glitch.) > > > > Or were you talking about compiling the 4 modules on Java 16 with > > target/release 8? Then of course, the whole point of this bug, to make the > > JAR's class files Java-8-compatible in order to be able to compile the whole > > project with JDK 8, would be moot. > > Actually the idea of compilation against Java 16 with -release 8 to get rid > of the jar will not work, because in that case compiler would still need the > jar, because it will only see Java 8 level API. > OK so -release 8 will not work but what about -target 8 ?
(In reply to Alexander Kurtakov from comment #36) > > > OK so -release 8 will not work but what about -target 8 ? There is no point to use this either, because for *compilation* compiler will accept jar with *any* known class file version. So compilation can stay "as is", just the packaging should be changed to not ship that jar. The last patch set on https://git.eclipse.org/r/c/jdt/eclipse.jdt.core/+/180788 seem to fix that - I only needed to touch forceQualifierUpdate.txt, because stupid "replace with baseline" step fooled me again. I can't remember how many hours I've wasted trying to convince maven to do something just to realize later all my efforts were eliminated by this step. Anyway, last patch set should do the job.
(In reply to Alexander Kurtakov from comment #36) > OK so -release 8 will not work but what about -target 8 ? No, with more recent javac versions that does not work anymore: $ javac --source 16 --target 8 --source-path javax.api/src -d out files warning: source release 16 requires target release 16
(In reply to Andrey Loskutov from comment #37) > The last patch set on > https://git.eclipse.org/r/c/jdt/eclipse.jdt.core/+/180788 seem to fix that A few questions: 1. Am I understanding what you said earlier correctly, you want to compile the 4 modules depending on javax15.jar - I think you only changed one - on JDK 16, just with a target of 8? 2. So building on JDK is no longer an option? I am asking, because the original trigger for me to open the ticket at all is - see above - that we actually always used to build the AspectJ fork of the JDT Core subset we use with JDK 8. Backporting the files, like I suggested, would have enabled that. Now I guess, we are losing this option and always need to build using the latest JDK supported by JDT Core. 3. Does that also mean, the JAR can be deleted from Git in those 4 places?
(In reply to Alexander Kriegisch from comment #39) > (In reply to Andrey Loskutov from comment #37) > > The last patch set on > > https://git.eclipse.org/r/c/jdt/eclipse.jdt.core/+/180788 seem to fix that > > A few questions: > > 1. Am I understanding what you said earlier correctly, you want to > compile the 4 modules depending on javax15.jar - I think you only > changed one - on JDK 16, just with a target of 8? No. The patch only makes sure the javax15api.jar is not packaged anymore. *How* the jar itself is generated is not part of the patch. > 2. So building on JDK is no longer an option? Not sure what you mean. The javax15api.jar building is not changed, other code was building on Java 8 anyway. > I am asking, because the > original trigger for me to open the ticket at all is - see above - > that we actually always used to build the AspectJ fork of the JDT > Core subset we use with JDK 8. Backporting the files, like I > suggested, would have enabled that. Now I guess, we are losing this > option and always need to build using the latest JDK supported by > JDT Core. I'm not sure why your build should be affected *now* (the javax15api.jar shouldn't be included in jdt jars that you are re-packaging anymore), and the compiler should be able to use javax15api.jar. > 3. Does that also mean, the JAR can be deleted from Git in those 4 > places? No. javax15api.jar is still needed for *compilation*. But it shouldn't be part of any generated artifact anymore.
First of all, many thanks for your patience, Andrey. Sorry for being inquisitive and meticulous, I just want to understand what everybody is talking about, because we have much digressed from the original question in this ticket/issue. (In reply to Andrey Loskutov from comment #40) > No. The patch only makes sure the javax15api.jar is not packaged anymore. > *How* the jar itself is generated is not part of the patch. OK, so this is only about the distribution issue which came up here as a result of discussing the build and byte code version issues. Actually, it is great that you have addressed that. we in AspectJ did so a long time ago, simply excluding the dependency from Maven Assembly or Maven Shade. It is good that upstream this is taken care of, too. > The javax15api.jar building is not changed, other > code was building on Java 8 anyway. No, it was not. If you read my original inquiry, you will realise that javax15api contains Java14 class files, which no compiler <14 can handle. Hence, AspectJ needed to switch to JDK 14 just in order to be able to use this file. The same should be the case for Eclipse JDT Core in all modules depending on that JAR. This issue has not been addressed yet. Or more precisely, I have addressed it by providing the backport, which compiles to Java 8 byte code. So if this gets checked in, overwriting the 4 existing JAR instances in Git, both JDT Core and AspectJ can be built on JDK 8 again. Presently, they cannot. > I'm not sure why your build should be affected *now* (the javax15api.jar > shouldn't be included in jdt jars that you are re-packaging anymore), and > the compiler should be able to use javax15api.jar. See above. A JDK 8 javac cannot compile Java 14 byte code. > javax15api.jar is still needed for *compilation*. But it shouldn't be > part of any generated artifact anymore. I was under the impression that you strongly discouraged having the sources and even the binaries in the Git repository due to IP trail doubts. So this issue is still unsolved, because nobody has yet presented a way to generate the JAR on the fly without any source or class files committed to Git. At present, nobody really can check how the sources have been built or whether they have just been extracted from a binary JDK module. Jay wanted to address that by at least committing the sources and add a build configuration for them. I understand that due to the licencing problem, we should not do that. The question then is: What is the alternative? For now, I suggest to just commit an new javax16.api (I can attach one to this ticket for convvenience), ideally my Java 8 backport. Then later, if anyone else who comes up with a smoother, more automated version - which for now is just theory and yet to be seen - that someone can hopefully completely refactor the 4 JARs out of the Git repo. But for now, we are just stuck, having to build on Java 14+.
If you are already building some custom version of the source why not also build a custom jar for your distribution also?
Created attachment 286425 [details] Java 16 API classes, backported + compiled to target 8 When I unpack this JAR and verify the class file versions, I get this result: $ find -name '*.class' | xargs javap -v | grep -E 'major version' | sort | uniq major version: 52 Major version 52 means Java 8.
(In reply to Christoph Laeubrich from comment #42) > If you are already building some custom version of the source why not also > build a custom jar for your distribution also? Because this is an issue is JDT Core proper, not just in AspectJ. Besides, before I created this issue, I had no idea where the JAR came from or how it was built, because there are no sources or build instructions in JDT Core. From that point of view, it is just some opaque, rogue library of questionable origin. Now of course, I can backport and build it myself because of what I have learned here. But that does not solve the problem in JDT Core. Sorry to repeat myself, but people here claim that JDT Core can be built with JDK 8. But because of this one file, it cannot! Is it not worth addressing the issue?
(In reply to Alexander Kriegisch from comment #44) > But that does not solve the problem in > JDT Core. Sorry to repeat myself, but people here claim that JDT Core can be > built with JDK 8. But because of this one file, it cannot! JDT can be build with this file and produce Java 8 compatible classes, at least our build runs on Java 11 and it compiles just fine, even if the jar contains classes from Java 14. So you probably only need to update your compiler used during the build.
We already did. Because of this JAR we build with Java 14. If it was not for that file, we could build on 8. I very much doubt that it compiles on Java 11. Maybe the Jenkins job runs on JDK 11, then forks another more recent JVM. I have no idea about all things Tycho, OSGi and so forth. But nothing else explains how something can be built against a dependency containing Java 14 class files.
If it is a non-goal of this project to be buildable with JDK 8, I am fine with that. Just nobody told me so. Also Jay said on the mailing list, before asking me to raise this bug: > I can't think of a reason why the JAR has to be on the latest version. > If at all, I would imagine they are compiled at the same compliance as > the containing project, which is 11. > > Let me try to compile them with 1.8 and see how it goes. If that implies that the minimum build platform is JDK 11 and only the build target is 8, fine. You still need to eliminate the JDK 14 class files then or upgrade to 14 as a minimum build platform.
(In reply to Alexander Kriegisch from comment #46) > We already did. Because of this JAR we build with Java 14. If it was not for > that file, we could build on 8. I very much doubt that it compiles on Java > 11. Maybe the Jenkins job runs on JDK 11, then forks another more recent > JVM. Please check https://ci.eclipse.org/jdt/job/eclipse.jdt.core-Gerrit/4820/consoleFull I don't see there anything suspicious, only place where Java 16 is used is to compile *tests*, for the rest it looks like Java 11 is used.
Sorry for missing out on all the action here. And before I say anything, let me add this to the context: there are lot of people asking us to move to JavaSE-11 and so far we have resisted that since it's not just the Eclipse IDE but also the ECJ we are talking about. Alexandar, all the jdt.core non-test bundles are at BREE 1.8. So, it is fair to say that the goal is to still compile JDT with 1.8. What happened with the JAR in question is an oversight that went unnoticed until you reported it. I will fix this soon, most likely RC1. It would have been lot simpler if it was a one time activity, so I am looking for something that can be repeated. Only point that I don't understand is, why is Andrey's patch not helping you if the Jar itself is not included in the bundles.
Actually, I don't even know what "BREE" means. Sorry for being a newbie here. Anyway, I just learned something important about why we had misunderstandings about the compilability of JDT Core on Java 8/11 vs. 14/16: For AspectJ we use a simplified build with Maven Compiler (using javac), whereas JDT Core plain vanilla uses Tycho Compiler (using ecj). So I tried something on the console: ------------------------------------------------------------------------ $ javac -cp path/to/javax15api.jar -sourcepath src/main/java -d target/classes src/main/java/de/foo/App.java -source 8 -target 8 src\main\java\de\foo\App.java:10: error: cannot access RecordComponentElement import javax.lang.model.element.RecordComponentElement; ^ bad class file: ..\eclipse.jdt.core\org.eclipse.jdt.compiler.apt\lib\javax15api.jar(javax/lang/model/element/RecordComponentElement.class) class file has wrong version 58.0, should be 52.0 Please remove or make sure it appears in the correct subdirectory of the classpath. ------------------------------------------------------------------------ $ java -jar path/to/ecj-3.26.0.v20210407-0512.jar -cp path/to/javax15api.jar -sourcepath src/main/java -d target/classes src/main/java/de/foo/App.java -source 8 -target 8 ---------- 1. WARNING in C:\Users\alexa\Documents\java-src\foobar\src\main\java\de\foo\App.java (at line 18) javax.lang.model.element.RecordComponentElement recordComponentElement = new RecordComponentElement() { ^^^^^^^^^^^^^^^^^^^^^^ The value of the local variable recordComponentElement is not used ---------- 1 problem (1 warning) ------------------------------------------------------------------------ Bingo! So here we go: Javac cannot pull this off, but ECJ can for some reason. Now my world is one step further towards being in order again. So what is the reason ECJ can do this? Well, the latest ECJ version is capable of compiling Java 16 (soon 17) source and target versions, no matter which JDK it is actually running on. Javac however is a native binary (starting a JVM, but anyway) bound to the JDK it is part of, because it uses its internal compiler classes. Therefore, it cannot understand anything more recent than the containing JDK does. That explains why JDT Core builds on Java 11, even though there are class files from a more recent Java version involved in the compilation process. Thanks for bearing with me. Maybe for all of you this information is a triviality, because you live in the Eclipse world and therefore using ECJ is the normal case for you. I however first has to find out the hard way, trying to wrap my head around this important fact which to me was not explicitly clear.
So as far as AspectJ is concerned, we have multiple options now: 1. build with a recent JDK + Maven Compiler (Javac) 2. backport javax15.jar to Java 8 or 11, whatever we need 3. configure Maven Compiler to use ECJ 4. move our build closer to upstream, using Tycho (ECJ) Probably it is not going to be either of 2. or 4. We are using 1. already and can continue to do so or migrate to 2.
+1 for Andrey's patch. But not sure if we have time for another build in M3. If not possible, I am okay for RC1 too.
(In reply to Jay Arthanareeswaran from comment #52) > +1 for Andrey's patch. But not sure if we have time for another build in M3. > If not possible, I am okay for RC1 too. Lets do it in RC1
New Gerrit change created: https://git.eclipse.org/r/c/jdt/eclipse.jdt.core/+/180805
Gerrit change https://git.eclipse.org/r/c/jdt/eclipse.jdt.core/+/180788 was merged to [master]. Commit: http://git.eclipse.org/c/jdt/eclipse.jdt.core.git/commit/?id=fd8920c95ad8e6bc6f7806c42a870f2135224e38
Gerrit change https://git.eclipse.org/r/c/jdt/eclipse.jdt.core/+/180805 was merged to [master]. Commit: http://git.eclipse.org/c/jdt/eclipse.jdt.core.git/commit/?id=ab1ffb380751852391d86fb1a800aba9a8443c16
With build https://download.eclipse.org/eclipse/downloads/drops4/I20210521-0150/ I see javax15api.jar is not shipped anymore with org.eclipse.jdt.compiler.tool_1.2.1200.v20210521-0550.jar, so the JDK classes are not redistributed anymore with JDT, independently which class version they have. That also should fix possible license violation as described in comment 17. For the rest (how to build javax15api.jar in the future) I've created dedicated bug 573689.
No "thumbs up" buttons here, so let me just say thank you for the interesting discussion, your patch and the newly created issue. :-)
Verified for 4.20 RC1 using build I20210525-1800
Sorry to comment on this one again instead of https://bugs.eclipse.org/bugs/show_bug.cgi?id=573689, but actually, the work here is not quite done, having lost focus of the actual title: "javax15api.jar contains Java 14 (58.0) class files" This is still the case, nobody has taken care of that. This means, that JDK 14+ is needed in order to build anything which needs any of those 4 javax15api.jar copies. Not Java 8, not Java 11, but really Java 14. The discussion here was all about licencing issues, but nobody has taken care of replacing those files. So shouldn't this be reopened? Please note that I am not talking about how to automate generating the files in the future, that really ought to be discussed in bug 573689.
(In reply to Alexander Kriegisch from comment #60) > Sorry to comment on this one again instead of > https://bugs.eclipse.org/bugs/show_bug.cgi?id=573689, but actually, the work > here is not quite done, having lost focus of the actual title: > > "javax15api.jar contains Java 14 (58.0) class files" > > This is still the case, nobody has taken care of that. Sorry about that. Let me do that for M1.
(In reply to Alexander Kriegisch from comment #60) > Sorry to comment on this one again instead of > https://bugs.eclipse.org/bugs/show_bug.cgi?id=573689, but actually, the work > here is not quite done, having lost focus of the actual title: > > "javax15api.jar contains Java 14 (58.0) class files" > > This is still the case, nobody has taken care of that. This means, that JDK > 14+ is needed in order to build anything which needs any of those 4 > javax15api.jar copies. Not Java 8, not Java 11, but really Java 14. We build SDK on Java *11* every day. Not 15, not 14, but really Java 11. What's the point to care about the contents if no one needs Java 14+ to build JDT?
(In reply to Andrey Loskutov from comment #62) > We build SDK on Java *11* every day. > Not 15, not 14, but really Java 11. > What's the point to care about the contents if no one needs Java 14+ to > build JDT? Sorry Andrey, if my point was unclear before. I thought I had explained it in almost annoying detail before. Short recap: For AspectJ, we use (and modify) modules org.eclipse.jdt.core and org.eclipse.jdt.compiler.apt. When we compile the latter, we can only do so if we add javax15api.jar to the classpath, because the classes in org.eclipse.jdt.compiler.apt depend on that API. On JDKs 8-13, this simply does not work, because javax15api.jar contains Java 14 class files and the compiler complains about it. I actually still do not have any clue how you can build that module on Java 11, depending on javax15api.jar. If somebody could explain that to me in layman's terms, so I can emulate it for our project, that would be great. But even if we would be able to do so, for me the question still is: Why put a dependency with Java 14 class files into a project being built on Java 11? Java 12-15 API, fine. But not Java 14 or 15 class files. Jay said since the beginning, it was done accidentally and he wants to fix it, if I understand correctly.
New Gerrit change created: https://git.eclipse.org/r/c/jdt/eclipse.jdt.core/+/182373
(In reply to Eclipse Genie from comment #64) > New Gerrit change created: > https://git.eclipse.org/r/c/jdt/eclipse.jdt.core/+/182373 Since all jdt projects are moving to Java 11, I recompiled with Java 11 instead of 8.
Gerrit change https://git.eclipse.org/r/c/jdt/eclipse.jdt.core/+/182373 was merged to [master]. Commit: http://git.eclipse.org/c/jdt/eclipse.jdt.core.git/commit/?id=77dbdd1ceca03500fb473a49e3f16b2a555e45b6
*** Bug 568362 has been marked as a duplicate of this bug. ***
