Bug 191571 - Enable automatic bug fixing in promotion of MDT and EMF components
Summary: Enable automatic bug fixing in promotion of MDT and EMF components
Status: VERIFIED FIXED
Alias: None
Product: Modeling
Classification: Modeling
Component: Releng (show other bugs)
Version: unspecified   Edit
Hardware: PC Linux
: P3 enhancement (vote)
Target Milestone: ---   Edit
Assignee: Nick Boldt CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2007-06-07 15:50 EDT by Christian Damus CLA
Modified: 2008-02-11 11:29 EST (History)
5 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Christian Damus CLA 2007-06-07 15:50:15 EDT
It would be nice if the Web UI for promotion of the following builds had the option (on by default) to transition ASSIGNED bugs to FIXED:

  - EMF Query
  - EMF Transaction
  - EMF Validation
  - MDT OCL

And the "Assigned Bugs" query in the right-hand side of the page could filter on the component, too, now that they are better aligned between Bugzilla and CVS/Web.
Comment 1 Nick Boldt CLA 2007-08-14 18:14:35 EDT
Support for doing this commandline has been added to promoteToEclipse.sh; coming up next, web UI in promo.php. 

Users will be responsible for setting up their own Bugzilla login details, which I'll need to document in the wiki.
Comment 2 Nick Boldt CLA 2007-08-29 19:49:09 EDT
OK, we're set [1].

[1]http://wiki.eclipse.org/Modeling_Project_Releng/Releasing#Automatically_Fixing_Assigned_Bugs

Please set it up on emf.toro as described in the wiki, then try it out.

If it works as you expect, please comment here so I know this bug can be closed.

Thanks!
Comment 3 Nick Boldt CLA 2007-09-06 18:52:59 EDT
Testing with Teneo... I'm getting a socket read error when trying to connect from emft.eclipse.org to bugs.eclipse.org.

[updBug] Running /opt/sun-java2-1.4/bin/java
  -Dant.home=/opt/apache-ant-1.6
  -Dant.library.dir=/opt/sun-java2-1.4/lib
  -classpath /opt/sun-java2-1.4/lib/rt.jar
    :/opt/apache-ant-1.6/lib/ant.jar
    :/opt/apache-ant-1.6/lib/ant-launcher.jar
    :./bugTools.jar org.apache.tools.ant.launch.Launcher
  -buildfile updateBugState.xml
  -Dlogin=1234
  -Dlogincookie=1234567890
  -Ddebug=1
  -Dstatus=ASSIGNED
  -Dresolution=FIXED
  -DbugList=202136,202127,200823,197820,197174,177595
  -DbuildID=I200709051239
  -DbuildAlias=

Buildfile: updateBugState.xml

updateBugState:

BUILD FAILED
/home/www-data/modeling-releng-common/tools/scripts/updateBugState.xml:12: Error in reading from stream! Java said: Server redirected too many  times (20)

This comes from this method:

  private String slurpStream(HttpsURLConnection conn)
  {
    String ret = EMPTY;
    try
    {
      BufferedReader in = new BufferedReader(new InputStreamReader(conn.getInputStream()));
      char[] tmp = new char[8192];
      int r;
      while ((r = in.read(tmp, 0, 8192)) != -1)
      {
        ret += new String(tmp, 0, r);
      }

      in.close();
    }
    catch (java.io.IOException e)
    {
      throw new BuildException(Messages.getString("UpdateBugStateTask.streamReadError") + "!" + SP + JS + SP + e.getMessage()); //$NON-NLS-1$ //$NON-NLS-2$
    }

    return ret;
  }

(/org.eclipse.build.tools/src_bugzilla/org/eclipse/releng/services/bugzilla/UpdateBugStateTask.java)

Denis:

Are the vservers allowed to connect to bugzilla?

Neil:

Can you suggest a solution here?

Comment 4 Denis Roy CLA 2007-09-07 09:46:52 EDT
> Denis:
> 
> Are the vservers allowed to connect to bugzilla?

Ehrm, yeah, I don't see why not?
Comment 5 Nick Boldt CLA 2007-09-07 11:36:04 EDT
(In reply to comment #4)
> > Are the vservers allowed to connect to bugzilla?
> Ehrm, yeah, I don't see why not?

Well, omitting the java connection and just using wget, I get:

$ wget -O 202136.xml https://bugs.eclipse.org/bugs/show_bug.cgi?id=202136&ctype=xml

https://bugs.eclipse.org/bugs/show_bug.cgi?id=202136
           => `202136.xml'
Resolving bugs.eclipse.org... 206.191.52.51, 206.191.52.52, 206.191.52.53, ...
Connecting to bugs.eclipse.org|206.191.52.51|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://bugs.eclipse.org/bugs/show_bug.cgi?id=202136 [following]
--11:36:13--  https://bugs.eclipse.org/bugs/show_bug.cgi?id=202136
           => `202136.xml'
Reusing existing connection to bugs.eclipse.org:443.
HTTP request sent, awaiting response... 302 Found
Location: https://bugs.eclipse.org/bugs/show_bug.cgi?id=202136 [following]
--11:36:13--  https://bugs.eclipse.org/bugs/show_bug.cgi?id=202136
           => `202136.xml'
 ...

Reusing existing connection to bugs.eclipse.org:443.
HTTP request sent, awaiting response... 302 Found
Location: https://bugs.eclipse.org/bugs/show_bug.cgi?id=202136 [following]
--11:36:14--  https://bugs.eclipse.org/bugs/show_bug.cgi?id=202136
           => `202136.xml'
Reusing existing connection to bugs.eclipse.org:443.
HTTP request sent, awaiting response... 302 Found
Location: https://bugs.eclipse.org/bugs/show_bug.cgi?id=202136 [following]
20 redirections exceeded.

Comment 6 Denis Roy CLA 2007-09-07 11:39:34 EDT
Ah, we play some port mapping trickery on the load balancer, so I think you'll need to connect to a specific port as you're not hitting the load balancer from the vserver.  Please try this:

https://bugs.eclipse.org:8443/
Comment 7 Nick Boldt CLA 2007-09-07 12:00:08 EDT
(In reply to comment #6)
> Ah, we play some port mapping trickery on the load balancer, so I think you'll
> need to connect to a specific port as you're not hitting the load balancer from
> the vserver.  Please try this:
> https://bugs.eclipse.org:8443/

Yes, that works... but only on the vserver. 

In order for my script to work cross-server and cross-firewall, I'll need either to do some sort of port mapping thingamabob on emft.eclipse (ssh tunnelling?) or have something like

  if (hostname like '*.eclipse.org') then port=8443; else port=443; fi

What do you recommend? Or, could port 8443 be open to the public as well as the vservers?
Comment 8 Denis Roy CLA 2007-09-07 15:05:31 EDT
> In order for my script to work cross-server and cross-firewall, I'll need
> either to do some sort of port mapping thingamabob on emft.eclipse (ssh
> tunnelling?) or have something like

I found the culprit, and I've applied a fix.  This will propagate to the webservers tonight.

Long story short, you'll be able to access https://bugs.eclipse.org/bugs/ as usual once this is applied.
Comment 9 Nick Boldt CLA 2007-09-10 01:36:52 EDT
(In reply to comment #8)
> > In order for my script to work cross-server and cross-firewall, I'll need
> > either to do some sort of port mapping thingamabob on emft.eclipse (ssh
> > tunnelling?) or have something like
> I found the culprit, and I've applied a fix.  This will propagate to the
> webservers tonight.
> Long story short, you'll be able to access https://bugs.eclipse.org/bugs/ as
> usual once this is applied.

I've been trying sporadically over the weekend and I can still only:

connect from non-eclipse.org domain to bugs.eclipse.org on port  443
connect from *.eclipse.org   domain to bugs.eclipse.org on port 8443

If I try the opposite, I get:

connect from non-eclipse.org domain to bugs.eclipse.org on port 8443
  = timeout / can't connect

connect from *.eclipse.org   domain to bugs.eclipse.org on port  443
  = HTTP/1.1 302 Found
  = 20 redirections exceeded

So, as it stands, it's still one port for *.eclipse.org and one port for non-eclipse.org. I can live with this if necessary, but I thought your fix was to avoid needing to use 8443?
Comment 10 Nick Boldt CLA 2007-09-15 01:09:52 EDT
Update:

a) changing my Ant task to allow a variable port number is doable, but painful, as it'll mean chopping up variables and adding new get/set methods to permit passing in the new variables, plus rewriting the XML and some shell to detect the server name, etc. Or, making the task smart enough to try multiple ports in sequence until it finds one that works.

b) I can connect from emf.torolab -> bugs.eclipse on :443 ONLY

c) I can connect from emft.eclipse -> bugs.eclipse on :8443 ONLY

d) I can connect from build.eclipse -> bugs.eclipse on :443 OR :8443, but ONLY if I use `wget --no-check-certificate` to get around this error:

    WARNING: Certificate verification error for 
    bugs.eclipse.org: unable to get local issuer
    certificate

(I'm not even sure how to code --no-check-certificate in java, so that'll be an interesting exercise!)

It's been a full week so I'm guessing whatever you fixed in comment #8 has had time to propagate. On the back of the bottle of 'M. Roy's Elixir De SSL', it clearly says, 'Si les symptômes persistent, consultez un médecin.'

M'aidez, svp, docteur, m'aidez? ;-)
Comment 11 Nick Boldt CLA 2008-02-02 13:45:48 EST
Looks like connections from emft.eclipse.org -> bugs.eclipse.org are working again!

$ cd /tmp; wget https://bugs.eclipse.org:443/bugs/show_bug.cgi?id=202136
--13:42:52--  https://bugs.eclipse.org/bugs/show_bug.cgi?id=202136
           => `show_bug.cgi?id=202136'
Resolving bugs.eclipse.org... 206.191.52.54, 206.191.52.55, 206.191.52.51, ...
Connecting to bugs.eclipse.org|206.191.52.54|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]

    [ <=>                                 ] 20,145        --.--K/s

13:42:53 (1.12 MB/s) - `show_bug.cgi?id=202136' saved [20145]

Moving to FIXED and looking for someone to volunteer to test this on their bugs... ;-)
Comment 12 Martin Taal CLA 2008-02-04 01:26:35 EST
Hi Nick,
I tried this with the latest build but it did not seem to work. 
I checked this option:
Move Resolved Bugs to Verified? (-bugzonly)

It seems that the promo step did not find the issues. The release notes lists them however.

The promo log can be found here:
http://emft.eclipse.org/promo_logs/promo_log_teneo_0.8.0.I200802040041_2008-02-04-01.14.05_bugzonly.txt

Btw, for me it is not clear if the bugs are moved to 'verified' (did not know that that status was there) or fixed.

gr. Martin
Comment 13 Nick Boldt CLA 2008-02-04 10:54:31 EST
Sounds like a timing issue -- if the database isn't current, the bug state task won't find the bugs and will do nothing. Might be a coincidence due to the time you chose to promote your build -- if there's already a database update in progress, your new bits may take a while to be inserted. Strangely, your build was in the releases table, but no bugs were found (yet).

The easy fix here is "wait a few minutes and try again", as when I ran it again just now, it found the three bugs:

http://emft.eclipse.org/promo_logs/promo_log_teneo_0.8.0.I200802040041_2008-02-04-10.45.55_bugzonly.txt

However, it complained about your login credentials being invalid.

To fix that, see this:

http://wiki.eclipse.org/Modeling_Project_Releng/Releasing#Automatically_Fixing_Assigned_Bugs

Then re-run the promote with the [x] -bugzonly flag checked, and you should be golden.

Comment 14 Martin Taal CLA 2008-02-05 06:21:29 EST
I tried the bugzillalogin.sh script and it returned this:
Bugzilla login or email address: mtaal@elver.org
Bugzilla password:
Logging in and fetching cookie values...

Paste the following into UpdateBugStateTask.properties:
---- 8< ---- cut here ---- 8< ----


---- 8< ---- cut here ---- 8< ----

I pasted the part into UpdateBugStateTask.properties (although it did not seem very meaningfull). But I still get the error:
promote] ERROR: logincookie cannot be read from /home/mtaal/.ssh/UpdateBugStateTask.properties.
[promote] Run `./bugzillaLogin.sh 2> /home/mtaal/.ssh/UpdateBugStateTask.properties` to regenerate login credentials.

gr. Martin
Comment 15 Nick Boldt CLA 2008-02-05 13:02:14 EST
(In reply to comment #14)
> ---- 8< ---- cut here ---- 8< ----
> 
> 
> ---- 8< ---- cut here ---- 8< ----

When I run it using my login, I get this:

---- 8< ---- cut here ---- 8< ----

login=8190
logincookie=abc3fgs123

---- 8< ---- cut here ---- 8< ----

If you try again without suppressing errors (just as./bugzillaLogin.sh), what do you see?

Or, what do you see in your ~/.ssh/UpdateBugStateTask.properties file? Should just be the two lines starting with "login..." as above.
Comment 16 Martin Taal CLA 2008-02-11 07:10:46 EST
Now it seems to work. See here:
http://emft.eclipse.org/promo_logs/promo_log_teneo_0.8.0.I200802110503_2008-02-11-07.06.23_bugzonly.txt

although the bugzillaLogin script still showed an empty result my UpdateBugStateTask.properties contains valid values:

login=28237
logincookie=BAkRUeqVXa

So it works for me.

gr. Martin
Comment 17 Nick Boldt CLA 2008-02-11 11:29:33 EST
(In reply to comment #16)
> Now it seems to work. 

Excellent!

FYI, I've updated this section to explain how to make bug state changing automatic during your builds (rather than a manual 2nd step on the promo.php page):

http://wiki.eclipse.org/Modeling_Project_Releng/Releasing#Automatically_Fixing_Assigned_Bugs