Community
Participate
Working Groups
Email from Ed Merks, Date: Fri, 8 Oct 2021 at 01:31 The latest p2 complains about the lack of checksum artifact metadata in the EPP repository: !ENTRY org.eclipse.equinox.p2.artifact.repository 2 0 2021-10-08 06:22:17.049 !MESSAGE No digest algorithm is available to verify download of binary,epp.package.java.executable.win32.win32.x86_64,4.22.0.20211007-1500. I think as long as it's signed, this is not an actual problem because the signature will be verified, but it's still annoying.
For whatever reason the EPP does not generate checksums in the artifacts.xml it publishes. AFAICT it never has (I checked Luna R and 2018-09). What I expect to see in the artifacts.xml is something like this: <artifact classifier="osgi.bundle" id="org.eclipse.tm.terminal.connector.local" version="4.7.0.202008310315"> <properties size="8"> <!-- snip --> <property name="download.md5" value="2ce1ebcfd6a0d85e83cbfed7741657a5"/> <property name="download.checksum.md5" value="2ce1ebcfd6a0d85e83cbfed7741657a5"/> <property name="download.checksum.sha-256" value="7349baac86569cf2059c8ae6cbd16565d9cf165519474d18b1a8dad4a3aeaeb9"/> </properties> </artifact> But for EPP it looks like this: <artifact classifier="org.eclipse.update.feature" id="org.eclipse.epp.package.modeling.feature" version="4.21.0.20210910-1200"> <properties size="7"> <property name="artifact.size" value="20301"/> <property name="download.size" value="20301"/> <property name="maven-groupId" value="org.eclipse.epp"/> <property name="maven-artifactId" value="org.eclipse.epp.package.modeling.feature"/> <property name="maven-version" value="4.21.0-SNAPSHOT"/> <property name="download.stats" value="org.eclipse.epp.package.modeling.feature.feature.jar/4.21.0.20210910-1200"/> <property name="download.contentType" value="application/zip"/> </properties> </artifact>
The platform change that introduced this is Bug 576429 - note that the change is only available in I-builds, 2021-12 M1 does not include this change, so I assume Oomph/Installer was built against a more recent I-build than M1.
There is a N&N entry too: https://www.eclipse.org/eclipse/news/4.22/platform_isv.php#logUnsafe
I did some experiments, AFAICT the checksums are never produced in the build, not that they go away. I tried using the fix artifacts mojo[1], but that didn't add the checksums. I asked a question[2] on tycho discussions to see if there are suggestions there [1] https://www.eclipse.org/tycho/sitedocs/tycho-p2/tycho-p2-repository-plugin/fix-artifacts-metadata-mojo.html [2] https://github.com/eclipse/tycho/discussions/326
Created attachment 287288 [details] list of bundles that seem to be missing sha256 Turns out this is not just EPP that has the issue, about 10% of the jars in simrel have the issue too, including a lot of the third-party content and packages from the following projects org.eclipse.zest, org.eclipse.xtend, org.eclipse.xpand, org.eclipse.uml2, org.eclipse.mylyn, org.eclipse.libra, org.eclipse.gef, org.eclipse.emf, org.eclipse.draw2d, org.eclipse.buildship See attachment for full list (which I did a quick bit of grepping on the simrel's artifacts.xml to generate). Most of the bundles that are missing sha256 do have md5, but the N&N entry says that will log going forward.
Created attachment 287290 [details] list of bundles that seem to be missing sha256
> !ENTRY org.eclipse.equinox.p2.artifact.repository 2 0 2021-10-08 > 06:22:17.049 > !MESSAGE No digest algorithm is available to verify download of > binary,epp.package.java.executable.win32.win32.x86_64,4.22.0.20211007-1500. I imagine p2 doesn't add checksums for binary artifacts. That's something to fix in p2; please open a dedicated issue about it > I think as long as it's signed, this is not an actual problem because > the signature will be verified, but it's still annoying. Binaries are not jar-signed, because they're not jars. (In reply to Jonah Graham from comment #5) > Turns out this is not just EPP that has the issue, about 10% of the jars in > simrel have the issue too, including a lot of the third-party content and > packages from the following projects org.eclipse.zest, org.eclipse.xtend, > org.eclipse.xpand, org.eclipse.uml2, org.eclipse.mylyn, org.eclipse.libra, > org.eclipse.gef, org.eclipse.emf, org.eclipse.draw2d, org.eclipse.buildship This is more or less older Orbit bundles and project that have used an ancient build technology. That's something the Planning Council have to deal with; having some guarantee the right artifacts are transferred/received is probably more important than having signed artifacts in the "supply-chain" safety.
See https://git.eclipse.org/r/c/equinox/rt.equinox.p2/+/186418 about possible wording changes for the message.
(In reply to Mickael Istria from comment #7) > > !ENTRY org.eclipse.equinox.p2.artifact.repository 2 0 2021-10-08 > > 06:22:17.049 > > !MESSAGE No digest algorithm is available to verify download of > > binary,epp.package.java.executable.win32.win32.x86_64,4.22.0.20211007-1500. > > I imagine p2 doesn't add checksums for binary artifacts. That's something to > fix in p2; please open a dedicated issue about it I imagined wrong: I verified on Platform, and the download.checksum.sha256 are present for binary content and I couldn't find any particular option or setting to add the signatures. So p2/Tycho seems to create the checksums by default, so it's probably something wrong in EPP build.
(In reply to Mickael Istria from comment #9) > So p2/Tycho seems to create the checksums by > default, so it's probably something wrong in EPP build. Yup - I agree, just haven't been able to track down the problem. FWIW it is not just binary artifacts that are a problem, but *all* bundles/features are also missing checksums in p2 data.
Turns out that this was caused by Bug 518965. Because Bug 518965 "create the p2 mirror from Tycho internal repo (so we can skip the assemble repo)." the checksums were lost in the final output as the Tyhco internal repo does not contain the checksums. So for now I have (essentially) reverted Bug 518965, but that comes with a big performance penalty as the build will probably take a long time again. I am continuing to experiment with what else I can do. There is probably an issue to be filed for Tycho in here, but not sure what it is yet (could be: "Tycho internal repos should contain checksum properties"?)
New Gerrit change created: https://git.eclipse.org/r/c/epp/org.eclipse.epp.packages/+/186982
(In reply to Jonah Graham from comment #1) > For whatever reason the EPP does not generate checksums in the artifacts.xml > it publishes. AFAICT it never has (I checked Luna R and 2018-09). Turns out Luna is missing some checksums - but Oxygen (the last release before Bug 518965) did have all the checksums - https://download.eclipse.org/technology/epp/packages/oxygen/R/artifacts.jar
New Gerrit change created: https://git.eclipse.org/r/c/epp/org.eclipse.epp.packages/+/186986
An alternative of using the fixup mojo looks a more likely fix. I couldn't get this to work at first due to user error - I have raised https://github.com/eclipse/tycho/issues/348 to make the user error easier to spot for the next user.
Gerrit change https://git.eclipse.org/r/c/epp/org.eclipse.epp.packages/+/186986 was merged to [master]. Commit: http://git.eclipse.org/c/epp/org.eclipse.epp.packages.git/commit/?id=03264cc8431287a6906bf07e745ae9028ad7ad08
(In reply to Jonah Graham from comment #5) > Created attachment 287288 [details] > list of bundles that seem to be missing sha256 > > Turns out this is not just EPP that has the issue, about 10% of the jars in > simrel have the issue too, including a lot of the third-party content and > packages from the following projects org.eclipse.zest, org.eclipse.xtend, > org.eclipse.xpand, org.eclipse.uml2, org.eclipse.mylyn, org.eclipse.libra, > org.eclipse.gef, org.eclipse.emf, org.eclipse.draw2d, org.eclipse.buildship > > See attachment for full list (which I did a quick bit of grepping on the > simrel's artifacts.xml to generate). Most of the bundles that are missing > sha256 do have md5, but the N&N entry says that will log going forward. The SimRel part of this is in Bug 576906 The EPP part is fixed using Tycho's fix-artifacts-metadata mojo
(In reply to Jonah Graham from comment #6) > Created attachment 287290 [details] > list of bundles that seem to be missing sha256 A common factor is that many, perhaps all of the Eclipse bundles, have not been built recently. EMF itself is notably missing because of course it is built very regularly. (The listed 'EMF' bundles are from EMF Services.)
(In reply to Ed Willink from comment #18) > (In reply to Jonah Graham from comment #6) > > Created attachment 287290 [details] > > list of bundles that seem to be missing sha256 > > A common factor is that many, perhaps all of the Eclipse bundles, have not > been built recently. EMF itself is notably missing because of course it is > built very regularly. (The listed 'EMF' bundles are from EMF Services.) Thanks Ed - I think we are going to try to solve this without requiring respins from all those projects. Please follow along in Bug 576906 for the SimRel issue.
'old-build-date' doesn't really make sense; why should the date matter? 'old-tycho-build-version' is much more likely. Projects with limited releng capacity may be lagging a long way behind.
(In reply to Ed Willink from comment #20) Please join the conversation on this topic in Bug 576906
