Bug 507917 - Convert cbi.p2repo.analyzers.build to Gerrit triggered build, plus ....
Summary: Convert cbi.p2repo.analyzers.build to Gerrit triggered build, plus ....
Status: VERIFIED FIXED
Alias: None
Product: CBI
Classification: Technology
Component: p2 Repository Analyzers (show other bugs)
Version: unspecified   Edit
Hardware: PC Linux
: P2 normal (vote)
Target Milestone: ---   Edit
Assignee: David Williams CLA
QA Contact: David Williams CLA
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-11-21 16:47 EST by David Williams CLA
Modified: 2016-12-08 10:22 EST (History)
2 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description David Williams CLA 2016-11-21 16:47:55 EST
I have tried to convert to build at 
https://hudson.eclipse.org/cbi/view/p2RepoRelated/job/cbi.p2repo.analyzers.build/

to be triggered by "Gerrit Events" but it is not working for some reason. 

The repo is not set up quite like other Gerrit repos I have worked with (such the main pom is not at the "root" of the repo. 

But before I start "trizl and error" changing things, thought I'd open this bug, to see if Fred has an "easy fix". Perhaps I missed a field or an asterisk somewhere? Or, is there a "magic switch" somewhere?
Comment 1 David Williams CLA 2016-11-21 16:52:01 EST
I should add, I would ultimately like to set up the "work flow" as I did for "p2repo.aggregtor" where there is an Gerrit build (that is not even cleaned, each time) and then another that does a "clean build and deploy" (to p2 update site, which doesn't exist yet :) and then a third one that adds that specific simple repo to the composite (so it is a separate step, after some initial testing). 

Fred, this may be "routine" for you, but woudl you like to help?
Comment 2 David Williams CLA 2016-11-22 16:14:46 EST
FYI, I had to remove $GERRIT_REFSPEC from repo's "ref spec" since I changed to build on SCM change (given that Gerrit trigger has not been set up yet). 

But once Gerrit is working, feel free to disable "SCM change" and add $GERRIT_REFSPEC back to ref spec.
Comment 3 David Williams CLA 2016-11-23 19:44:19 EST
Adding "plus..." to the title and upping severity and priority to better remember this bug is to cover the "suite" of jobs that ulimately put a repo on "downloads server". This appears more important than I first thought, after seeing bugs such as bug 508082. While that specific bug was easy enough to fix (by restarting the HIPP instnance) it is a good reminder that anythnig on a critical path should not rely on servivces from Hudson running on a HIPP instance but instead rely only on "downloads" server. 

I will likely start to work on the suite of jobs soon, if Fred doesn't. And leave just the "Gerrit" problem for him to investigate.
Comment 4 David Williams CLA 2016-12-01 12:01:00 EST
NOTE: I have changed the name of the ".build" job to "cbi.p2repo.analyzers_Gerrit". 
See, 
https://hudson.eclipse.org/cbi/view/p2RepoRelated/job/cbi.p2repo.analyzers_Gerrit/

EVEN THOUGH is it not a "Gerrit build" yet! Once this bug is fixed, it will be. 

The reason was that I am creating some of the other needed jobs, namely
cbi.p2repo.analyzers_cleanAndDeploy
and soon
cbi.p2repo.analyzers_addComposites

And I simply wanted the "analyzers jobs" to follow the same pattern as the "aggregator jobs" since long term I think that will be easier to keep track of. 

= = = = 

I have changed the two "test jobs" to work off the new "_Gerrit" version (after initial failure). 

Note: I have also disconnected those test jobs from the "cleanAndDeploy" job. 
Eventually, we will have to make the "source" of the executables a parameter on a "parameterized triggered job" for that to work. 

= = = = =

One thing to be aware of is that in the _Gerrit build, the "product names" are 
the simple form, such as 
org.eclipse.cbi.p2repo.analyzers.product-linux.gtk.x86_64.tar.gz
BUT in the "cleanAndDeploy" jobs, the product names
are the simple form *in Hudson*, but when copied to the download site, I add the "buildId" to them, so for example
org.eclipse.cbi.p2repo.analyzers.product-linux.gtk.x86_64.tar.gz
becomes
org.eclipse.cbi.p2repo.analyzers.product_I20161201-1633_linux.gtk.x86_64.tar.gz

Kind of confusing, we can improve on that in the future.
Comment 5 David Williams CLA 2016-12-01 12:05:48 EST
(In reply to David Williams from comment #4)
> [...]
> but when copied to the download site, 
> [...]

And, I should have said, the download site is the same as the update site. 
The update site is 
.../download.eclipse.org/cbi/updates/analyzers/4.6/<buildId>

And, soon, 
.../download.eclipse.org/cbi/updates/analyzers/4.6/
will be a composite. 
I plan to keep "latest 3 I-builds" in the composite.
Comment 6 Eclipse Genie CLA 2016-12-01 13:43:18 EST
New Gerrit change created: https://git.eclipse.org/r/86173
Comment 8 David Williams CLA 2016-12-01 15:21:42 EST
FYI, I have implemented the "cleanAndDeploy" build for analyzers, and have implemented the "addComposities" job. 

This included changing numerous other jobs where a) the builds were using "lastSuccessful" build from Hudson -- they now use the update site or the "product" available from that site. To use the composite to install the analyzers, the repository is 

http://download.eclipse.org/cbi/updates/analyzers/4.6/

To use a "product" from a site, once must browse "down" into the latest I-build and find the "product" they want. For example, at the time of this writing, you can see 

http://download.eclipse.org/cbi/updates/analyzers/4.6/I20161201-1633/
has 
org.eclipse.cbi.p2repo.analyzers.product_I20161201-1633_linux.gtk.x86_64.tar.gz
org.eclipse.cbi.p2repo.analyzers.product_I20161201-1633_macosx.cocoa.x86_64.tar.gz
org.eclipse.cbi.p2repo.analyzers.product_I20161201-1633_win32.win32.x86_64.zip

= = = = = = 

So, all that remains is to fix the "Gerrit job"! 

Any insights, Fred? Anything you'd suggest I try? (still think the "Gerrit site" somehow needs to add "Hudson CI" as a reviewer ... or, does that happen automatically if everything else is ok? 
If it helps, see 
https://git.eclipse.org/r/#/admin/projects/cbi/org.eclipse.cbi.p2repo.analyzers
Comment 9 Frederic Gurr CLA 2016-12-01 17:14:31 EST
Sorry for the late reply. I will take a look at this tomorrow.
Comment 10 David Williams CLA 2016-12-02 17:00:02 EST
This seems to be fixed, now. I hope you did something to fix it. :) 

I think the main difference in settings from now, to when it did not work, is that I do not recall "anonymous" having "build" permission. Was that it? 

There were two other flags I had set, 
Prune remote branches before build
and 
Skip internal tag

No particular reason, other than I say other jobs with those set and seemed like it would help keep the local build clone "cleaned up"

I have now set those two and will test with a "junk change".
Comment 11 Eclipse Genie CLA 2016-12-02 17:05:38 EST
New Gerrit change created: https://git.eclipse.org/r/86290
Comment 12 Frederic Gurr CLA 2016-12-02 17:07:27 EST
(In reply to David Williams from comment #10)
> This seems to be fixed, now. I hope you did something to fix it. :) 

Unfortunately I didn't do something...yet. Sorry. It's at the top of my list though.
Comment 13 David Williams CLA 2016-12-02 17:13:00 EST
Yep, even with those two fields  checked, it still works. That makes me think it was just that "anonymous" needed "build permission" in the security matrix. 

= = = =

Oh, shoot. I am looking the "aggregator_Gerrit" build, not the "analyzers_Gerrit" build. (And, it always worked).  

But, I still bet it is the "anonymous" setting. I'll try that, for *analyzers_Gerrit".
Comment 14 Eclipse Genie CLA 2016-12-02 17:15:41 EST
New Gerrit change created: https://git.eclipse.org/r/86293
Comment 15 David Williams CLA 2016-12-02 17:24:19 EST
You snooze, you lose. :) 

I'll take "credit" for this fix, even though I simply looked at the wrong job and thought you made a change. I would have thought I'd have done that before, but guess I never thought of "Gerrit" as "anonymous". 

But, giving "anonymous" build permission was the "trick". (After fixing one typo).
Comment 16 Frederic Gurr CLA 2016-12-02 17:38:46 EST
Hmm. I'm pretty sure that's not the right solution though.

Compare it to a plaform gerrit job like: https://hudson.eclipse.org/platform/job/eclipse.jdt.ui-Gerrit/configure

It doesn't give anonymous build permissions and still works. I guess it's an easy workaround for now, but should not be a permanent solution.
Comment 17 Eclipse Genie CLA 2016-12-03 13:39:21 EST
New Gerrit change created: https://git.eclipse.org/r/86306
Comment 18 David Williams CLA 2016-12-03 13:59:53 EST
(In reply to Frederic Gurr from comment #16)
> Hmm. I'm pretty sure that's not the right solution though.
> 
> Compare it to a plaform gerrit job like:
> https://hudson.eclipse.org/platform/job/eclipse.jdt.ui-Gerrit/configure
> 
> It doesn't give anonymous build permissions and still works. I guess it's an
> easy workaround for now, but should not be a permanent solution.

It beyond me, but today I disabled the "anonymous build" flag and Gerrit still works. I disabled it on both the 
cbi.p2repo.aggregator_Gerrit job and the 
cbi.p2repo.analyzers_Gerrit job. 

As a wild guess, I wonder if there is a system "cron job" or similar that runs every now and then to "associate" Gerrit with Hudson? Other than that, I have no idea why it "solved the problem" one day, but next day did not seem to be needed. 

Now, I just hope it continues to work tomorrow. :)
Comment 19 Eclipse Genie CLA 2016-12-08 09:46:50 EST
New Gerrit change created: https://git.eclipse.org/r/86718
Comment 21 David Williams CLA 2016-12-08 10:22:21 EST
This does seem to still work. Not sure what was wrong at the begging. Guess I should created a "back up configs" job (bug 508913).