Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [cbi-dev] Interrogations on the new Jenkins/Kubernetes instances
  • From: Denis Roy <denis.roy@xxxxxxxxxxxxxxxxxxxxxx>
  • Date: Tue, 11 Jun 2019 11:00:39 -0400
  • Autocrypt: addr=denis.roy@xxxxxxxxxxxxxxxxxxxxxx; prefer-encrypt=mutual; keydata= mQENBFSQVYcBCACphCllQd59H3sF23T1XUD8h6pNPvRwuKKpfcwTAL7qe98PbMBQBLuy3ocj 04PcvlRj8lCR9JmChFc9V5Bi61MTdCrshEbiyomw/ZWdfFbmc4/rlMWg00gTO94SP0zMl96x np0aNi8cGwD258NUVD1+tlvraukr6yixD25qCGQ313lOO71l1R8eJun/Kx7SEFXtMFJ7Er4r /QnfFioFFwWGiUXbGhioCZhggQWnq3U+LOZzGKp0SuWSQuu0RzQB57Fy8EN2bmX61EKGaFnr TR/8yk8EdifupYgDc6XCxndktFJYFse4v4XQSPNIIUwcBT/Ji732xdXiR4YnL0SNAVBDABEB AAG0NkRlbmlzIFJveSAoRWNsaXBzZSkgPGRlbmlzLnJveUBlY2xpcHNlLWZvdW5kYXRpb24u b3JnPokBOAQTAQIAIgUCWWjoCAIbAwYLCQgHAwIGFQgCCQoLBBYCAwECHgECF4AACgkQQI23 cA21ZyMHGgf/THQqUR43XqUhSytf3q0Wmq1KBCDUpLXQYfpe1uh464robj4iUsoCMDhlEzbU uy0p+de87aVv68ZGRPOlz0oIt80NZbHxLErhunjFGhJvnw/V5jf2lYThiWysDcQMUSIGERuy 3wIyVFVF4RDZB4jjvrPvQ/dy+x+mN38t76C8x5sp4oup19sy9u4EUzPpEInBNs8ADoKO9kOl ZbTOkIFSqC4aI0yABRMbur+mks+b9IaVVQytK4zlzTROuBNHnjx6kgfAV5L8ezQpgBkVLrLw HwB+Rmsnx/qbRXrRmESiiD6aVQQntkfj0zXK/XzMpm1+9qxA8TBj6nFXo/ivNqDx7LkBDQRU kFWHAQgAy/xnfKzcxiCyHtP2a3mbRW04xMwDoLWj+3khfudiHgQWNLDiN/oTAJag5cOrh17L ewRDuhEctJ71Nb+bkMNHvJvq5KC0oB1x54ocQGa493Q/9irIYaEbXVcVqyggmTO6lPvLcA9+ mdQZNvfFlnxS05z+c2CvNNqhgrFU4xS6MTuGEA6oduha0mbDodPKNBh8MkL7OjlYO/44H4l0 MsBYJ57QhQqeOVv1C8Sc2hstk8ZcCc3G2W29Beh0Li5WFYf7F8L0JtxuDSLGisWJL5cAZvdE Zrd/BNAOE02+fANl9iYkX+pG1rjYMdtm/UpLo38jovcyWhOc+TE7qab72wO8iQARAQABiQEf BBgBAgAJBQJUkFWHAhsMAAoJEECNt3ANtWcjxEMH/Rer0weIKA1jV72FNQpHaKuGuUmqW6ak qQHSRKZEECjTa9E1M/Mvdr21qE/zYR7tFD9zc2wqhxBDBD8azKQLZo2TSTnqyUPdr2Dy3KFS FoavCvaVzEt7c3sA/TxXWXQhxGJL56yj2VZaF+k7j/DtYl6nG62bS0rQbg/3/RllmXa51QIT V5ZPnTo6NFJ5K3lbn7EoQbVT2LdkPmabPqw6DT+ZjQlJS3Q8iI0wSoS6r9D7QQG/VaeKGjoK TRDRn/5YUnof4vjpao8jIRMJ+5RBlRto96jwt7hfNm842GsqT4ddMyYmTLGnYLUae9rl2GmN ahBMbQ9PB8Ta/DeCpIxpOyk=
  • Delivered-to: cbi-dev@xxxxxxxxxxx
  • List-archive: <https://www.eclipse.org/mailman/private/cbi-dev>
  • List-help: <mailto:cbi-dev-request@eclipse.org?subject=help>
  • List-subscribe: <https://www.eclipse.org/mailman/listinfo/cbi-dev>, <mailto:cbi-dev-request@eclipse.org?subject=subscribe>
  • List-unsubscribe: <https://www.eclipse.org/mailman/options/cbi-dev>, <mailto:cbi-dev-request@eclipse.org?subject=unsubscribe>
  • Openpgp: preference=signencrypt
  • User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.2.1

I'm not a release engineer but I can repeat what I've read elsewhere:


On 2019-06-11 8:28 a.m., LE MENEZ Quentin wrote:

Hi Mikaël,

 

I tried correcting the points mentioned but found other problems. Again, if you could help me with them it’d be a great help:

1-      The first point I made was mainly about keeping the workspace for the downloaded maven dependencies which can be a pain to download every single time.(which, if I understood correctly will be the case if the workspace is deleted every single time)

This is an issue and I believe we have ideas for resolving this, but for now, the world needs to be re-downloaded every single time.


2-      I tried to add a promote step to the job but I cannot figure out how one would access the archive files from the job as the promote step is launched in another agent. I gather you can get them through a (curl ?) request but that can be a big overhead for larger project archives (e.g. RCPs, zipped p2s containing hundreds of plugins, …). There must be a way to do it cleanly but I didn’t find how.

If you're using a pipeline build, you can use stash for this:

https://jenkins.io/doc/pipeline/steps/workflow-basic-steps/



3-      I tried to publish the contents in the same workspace state (after the build) but the ssh connection did not seem to get through (no error shown just a timeout).

Is the destination still /home/data/httpd/download.eclipse.org or should I call another destination ?

I think, so, based on these docs:

https://wiki.eclipse.org/Jenkins#How_do_I_deploy_artifacts_to_download.eclipse.org.3F


I don't have any answers for the rest, but I'm sure Mikael will chime in tomorrow.


Denis




As a bonus question it should be possible to use variables in the request but if I transform “ssh genie.shared@xxxxxxxxxxxxxxxxxxxxxxxxxxxx mkdir –p /home/data/***” into “ssh $sshGenie@$sshRemote mkdir -p $destinationUpdateSite” it times out. Is it because the previously mentioned destination is erroneous ?

 

Thanks again,

Quentin

 

Ps : The zip command seem to only be available in the ‘migration’ profile. Should I open a bug for it to be integrated inside the “ui-test” (as I get the feeling that migration is not supposed to last) along the one for Gerrit problems ?

Pps: to provide further information I included the script I intended to use as a promotion script. If you see any kind of problems with it feel free to tell me J

 

 

De : cbi-dev-bounces@xxxxxxxxxxx <cbi-dev-bounces@xxxxxxxxxxx> De la part de Mikaël Barbero
Envoyé : vendredi 7 juin 2019 09:58
À : Common-build Developers discussion <cbi-dev@xxxxxxxxxxx>
Objet : Re: [cbi-dev] Interrogations on the new Jenkins/Kubernetes instances

 

Hi Quentin,

 

See my answers inline

 



Le 6 juin 2019 à 17:58, LE MENEZ Quentin <Quentin.LEMENEZ@xxxxxx> a écrit :

 

Hi Everyone,

 

I might be asking something that is already documented, and if so please direct me to the relevant documentation, but if not here are my questions:

For context I was configuring the XWT jobs on https://ci.eclipse.org/xwt/ and I am not admin of this instance.

 

1-      The xwt-master workspace just vanished without any actions from me (so I gather the message The agent this project has run on for the last time was removed indicates me that the pod/node was used by another job hence the workspace was deleted ?). How so ?

 

Agents on the new infra are spawn on-demand, a new workspace is created at the same time and everything is teared down once the build is done. If you want some files to outlive the duration of the jobs, you should either use the archiveArtifacts Jenkins feature and/or cp the files to the download area (https://wiki.eclipse.org/Jenkins#How_do_I_deploy_artifacts_to_download.eclipse.org.3F). Note that if you push files that are not releases, you should put them in a folder strcuture that will not be mirrored (see https://wiki.eclipse.org/IT_Infrastructure_Doc#Use_mirror_sites.2Fsee_which_mirrors_are_mirroring_my_files.3F for the list of file patterns that are not send to mirrors).



2-      I cannot seem to be able to run the zip command from a shell script build step. Should I use something else ?

 

"zip" command is not available in the default agent image, "unzip" is though. Advanced images are available in "migration" and "ui-test" agents. See https://wiki.eclipse.org/CBI/Jenkins_Migration_FAQ#How_do_I_run_UI-tests_on_the_new_infra.3F for how to use them.



3-      The gerrit Job can’t seem to add the +1 to the gerrit contribution even after adding the review parameters 1 and +1. I gather that the Gerrit Reporting Values and Verified Commands are not set in this instance and therefore nothing is sent by the job to the gerrit instance & patch ? 

 

It should work out of the box. Please, open a ticket and we will investigate.



4-      This instance does not seem to have any executors pre-locked. Is/Will this be the future default configuration ? i.e. will they be dispatched by the kubernetes manager and assigned whenever a free node is available ?

 

As stated above, agents / executors are dynamically provisioned. You won't see any until you start a build. Up to 2 agents can be provisioned simultaneously for your Jenkins instance, i.e. you can run up to 2 jobs concurrently.

 

Cheers,

 

Mikaël Barbero 

Team Lead - Release Engineering | Eclipse Foundation

📱 (+33) 642 028 039 | 🐦 @mikbarbero

Eclipse Foundation: The Platform for Open Innovation and Collaboration

 


_______________________________________________
cbi-dev mailing list
cbi-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/cbi-dev
--

Denis Roy

Director, IT Services | Eclipse Foundation, Inc.

Eclipse Foundation: The Platform for Open Innovation and Collaboration

Twitter: @droy_eclipse


Back to the top