Re: [cross-project-issues-dev] SimRel JIPP migration to new infrastructu

[Aleksandar Kurtakov <akurtako@xxxxxxxxxx>]

On Fri, Jan 15, 2021 at 8:14 AM Ed Merks <ed.merks@xxxxxxxxx> wrote:

Fred,

Yes, people don't look at reports.  That's why no one noticed that bad licenses and unsigned content are present for 2021-03 M1; they've been present in staging for a while:

  https://download.eclipse.org/oomph/archive/reports/download.eclipse.org/staging/2021-03/index.html

In my experience with the same topic but in Platform releng the only way to make issues not be introduced is make them break the build whenever possible, everything else pretty much left keeping the state on the few people that generate and look at the report.

 

Note that I've opened https://bugs.eclipse.org/bugs/show_bug.cgi?id=570380 because https://git.eclipse.org/c is completely unreliable these days, which is why the above report has a bunch of error icons due to of random failures loading the icons during generation.

This "gitc" problem also makes reporting on SimRel broken:

  https://ci.eclipse.org/oomph/job/simrel-analyzer/

And it makes it next to impossible for me to generate the product catalog for M1.

Broken processes are frustrating and unreliable infrastructure is both annoying and time consuming.

Regards,
Ed

On 14.01.2021 15:38, Frederic Gurr wrote:

Hi,

As you might know, the SimRel JIPP has been migrated to our new
cluster-based infrastructure (see
https://bugs.eclipse.org/bugs/show_bug.cgi?id=568118).

This has some implications on the SimRel aggregation build:
On the old infra, the SimRel JIPP could utilize up to 16 CPUs of the
host machine (hipp8) and had file system access (via NFS mounts) to
download.eclipse.org. On the new infra, resource restrictions are in
place and there is no direct access to download.eclipse.org (only via
SCP/SSH).

Therefore we are using a dedicated "promotion" build agent with 4vCPUs
that provides filesystem access. Due to the CPU resource limits of the
build agent, an aggregation build (simrel.runaggregator.pipeline) now
takes approximately 2 hours on the new infra (vs ~1h hour on the old
infra). One hour of this build was spent on "repo reports" alone. AFAIK
nobody reads the repo reports on a regular basis...if ever. Hence I've
split out the repo report generation from the aggregation build.
I'd suggest to run the separate repo reports job
(https://ci-staging.eclipse.org/simrel/job/simrel.reporeports.pipeline/)
at least every day/night, but not after every aggregation build anymore.

Please also note, more changes (e.g. native Tycho build) will come soon,
but the focus is on the migration for now.

If you have any questions or concerns, please let me know.

Regards,

Fred

_______________________________________________
cross-project-issues-dev mailing list
cross-project-issues-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/cross-project-issues-dev

--

Alexander Kurtakov

Red Hat Eclipse Team