Bug 212060 - Migrate GEF build to use common modeling releng
Summary: Migrate GEF build to use common modeling releng
Status: RESOLVED FIXED
Alias: None
Product: GEF
Classification: Tools
Component: RelEng (show other bugs)
Version: 3.4   Edit
Hardware: PC Linux
: P3 normal (vote)
Target Milestone: 3.4.0 (Ganymede) M5   Edit
Assignee: Nick Boldt CLA
QA Contact:
URL:
Whiteboard:
Keywords:
: 214552 (view as bug list)
Depends on:
Blocks:
 
Reported: 2007-12-05 15:00 EST by Nick Boldt CLA
Modified: 2008-02-08 01:44 EST (History)
3 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Nick Boldt CLA 2007-12-05 15:00:59 EST
 
Comment 1 Anthony Hunter CLA 2007-12-05 16:41:26 EST
We are going to do the migration after GEF 3.4 M4
Comment 2 Nick Boldt CLA 2008-01-07 17:32:54 EST
*** Bug 214552 has been marked as a duplicate of this bug. ***
Comment 3 Nick Boldt CLA 2008-01-14 12:21:03 EST
Still stuck trying to run tests:

Failed to invoke suite():org.eclipse.swt.SWTError: No more handles [gtk_init_check() failed]

http://emf.torolab.ibm.com/gef/gef/downloads/drops/3.4.0/N200801141138/buildlog.txt

> Check out https://bugs.eclipse.org/bugs/show_bug.cgi?id=115822  
> 1. The DISPLAY environment variable is set. 
> 2. The host/port information is correct (e.g. added to /etc/hosts).  
> 3. The X server is up.

3. X is up -- kdm is running and a working xorg.conf has been installed. Running `ps ax --forest`, I get in part:

    1 ?        S      0:01 init [5]
 4015 ?        Ss     0:00 /usr/bin/kdm
 4025 tty7     Ss+    0:01  \_ /usr/bin/X -br -nolisten tcp :0 vt7 -auth /var/run/xauth/A:0-5vtRiz
 4043 ?        S      0:00  \_ -:0
 4052 ?        S      0:00      \_ /usr/bin/kdm_greet

Should that "-nolisten" be there?

1. DISPLAY is being set right before invoking the tests:

export DISPLAY=":0.0"

2. What does that mean? Where in /etc/hosts do you define X server ports?

$ cat /etc/hosts

127.0.0.1       localhost
9.26.101.101    emf.torolab.ibm.com     emf

# The following lines are desirable for IPv6 capable hosts
# (added automatically by netbase upgrade)

::1     ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts

I've even added 

-Dosgi.bundlefile.limit=100

and 

export CVS_RSH=ssh
ulimit -c unlimited
export JAVA_HIGH_ZIPFDS=500
export LANG=C

and nothing (yet) works. Failure is happening running both ui-test and core-test ant targets, suggesting it's maybe not an x server issue, but a java / threading issue? I'm stumped. Any suggestions?
Comment 4 Nick Boldt CLA 2008-01-15 03:38:15 EST
Boo-freakin'-yeah!

JUnit tests now pass. X server works. I'm even switched the tests over to be "ui-test" instead of "core-test", and they still work! 

I've updated the wiki with lessons learned here:

http://wiki.eclipse.org/EMF_Build_Server_Setup#Verify_X_server

And I've also unearthed 637 new CHKPII errors. However, as 633 of are javadoc problems, I've added reference/api/*.html to the CHKPII skiplist. Also found invalid XML in topics_Samples.xml, which I've fixed. 

There are three issues remaining, all from the gef.doc plugin:

http://emf.torolab.ibm.com/gef/downloads/drops/3.4.0/N200801150106/testresults/chkpii/org.eclipse.nls.htm.xml
Failing files are:
  /about.html
  /guide/guide.html
  /guide/index.html

However, CHKPII doesn't find these now that I've fixed the fact that the doc plugin wasn't being jarred.

Anthony: do you want to fix these? 

Meanwhile... playing with different window managers and headless X server connections have not been entirely successful -- yet. In one case, changes in relengbuildgtk.sh caused the build's tests to not run at all [1]; in another, we had 4 failures [2].

[1]http://emf.torolab.ibm.com/gef/downloads/?showAll=1&hlbuild=N200801150223&sortBy=date&project=gef#N200801150223
(see build log; search for "Java Result: 13")

[2]http://emf.torolab.ibm.com/gef/gef/downloads/drops/3.4.0/N200801150313/testresults/html/org.eclipse.draw2d.test_linux.gtk.html
Comment 5 Anthony Hunter CLA 2008-01-15 09:54:39 EST
(In reply to comment #4)
> JUnit tests now pass. X server works. I'm even switched the tests over to be
> "ui-test" instead of "core-test", and they still work! 

Good stuff

> Anthony: do you want to fix these? 

Yes, we should raise a separate Bugzilla for these issues.
Comment 6 Nick Boldt CLA 2008-01-16 02:23:02 EST
Guys:

After much debugging, I can't figure that there's anything wrong with the build system in terms of its ability to run the tests. I've done/undone everything I changed since the last good build, and I still get 4 JUnit failures. I've even reverted the mapfile to its 1/15 2:06am state to see if by chance something broke along the way. 

Maybe it's because I've using a newer JDK, a newer kernel, or a newer basebuilder/Eclipse. I'm not sure. What's on your existing build server?

java version "1.4.2_15"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.2_15-b02)
Java HotSpot(TM) Client VM (build 1.4.2_15-b02, mixed mode)

Linux emf 2.6.15-26-server #1 SMP Fri Sep 8 21:00:37 UTC 2006 i686 GNU/Linux

(I've tried three versions: "v20071108", "M4_34", "vI20070503-1400" -- all have been proven to work at one point or another, despite using different versions of the Equinox launcher. I've tried 3.4M4, I20080108, and I20080115. All work, except for the 4 JUnit failures.)

I do see some changes that sound like they're related to the JUnit failures -- perhaps they only manifest on Linux (as you told me you couldn't reproduce them on Windows).

Here's the 4 commits done by Pratik (cvs user pshah) in/around the time I was working on the build:

http://www.eclipse.org/modeling/emf/searchcvs.php?q=file%3A%25gef%25+author%3Apshah+days%3A7&project=0&fullpath=Y

While you figure out if there's in fact a linux gtk bug in there, I'll try to get the build running on my thinkpad and also run a build from within a full UI Eclipse instance. 

If you want to build and promote using your old system this week, you're more than welcome. I've got a number of other things to set up before the web will be ready anyway...

Oh, and don't forget, there's also the 24 compiler warnings... do you get those with your old builder? If not, I wonder why not.
Comment 7 Anthony Hunter CLA 2008-01-16 10:39:17 EST
(In reply to comment #6)
> [...] I still get 4 JUnit failures. [...]

As discussed, I also got errors when I tried to change the build machine from an older RedHat to Ubuntu. There may be a GTK issue at work here.

The latest code and JUnits work for me on Windows and RedHat, so I will test on Ubuntu when I get a chance. This is not a show stopper in my opinion.

> If you want to build and promote using your old system this week, 

Will do for the January 16 weekly integration build

> Oh, and don't forget, there's also the 24 compiler warnings... do you get those
> with your old builder? If not, I wonder why not.

Yes, there are a number of warnings. 

Comment 8 Nick Boldt CLA 2008-02-08 01:44:23 EST
Remaining items here are:

* mapfile generation not working (but GEF doesn't use that so I'm not concerned here; obvious workaround if you want a "generated" mapfile is to set the map to use HEAD as the tag, commit to CVS, and build from that)

* -addSDK flag not working -- the purpose here is to add GEF as a dependency to the build page for others to consume; but as GEF is build on emf.toro and the only projects that consume it are built on emft.eclipse, this wouldn't benefit them anyway

* Bug 216984: 4 JUnit test failures

And as I just sorted out Jar signing and packing (bug 173651), I think we're done here.