Bug 377115 - Build SWT and consume native fragments
Summary: Build SWT and consume native fragments
Status: CLOSED FIXED
Alias: None
Product: CBI
Classification: Technology
Component: build help (show other bugs)
Version: 1.0   Edit
Hardware: PC Linux
: P3 normal (vote)
Target Milestone: ---   Edit
Assignee: Krzysztof Daniel CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on: 384874
Blocks: 405753
  Show dependency tree
 
Reported: 2012-04-18 13:36 EDT by Paul Webster CLA
Modified: 2018-01-04 04:49 EST (History)
9 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Paul Webster CLA 2012-04-18 13:36:10 EDT
Currently a platform build rebuilds org.eclipse.swt for each of the swt fragments that has native binaries in it (slight modifications to the source, int->long for 32-bit->64-bit, some common java source and some separate java source).

The CBI platform build needs to be able to correctly build the swt fragments and include them in the published p2 build repo.

PW
Comment 1 Andrea Ross CLA 2012-07-09 21:53:03 EDT
marking this as CBI 1.0

I'm also interested in knowing were we stand against the functionality completed thus far?
Comment 2 Paul Webster CLA 2012-07-10 11:09:42 EDT
If I read the mailing list correctly, SWT can be built on one platform, but instead of building the fragment it puts all of the code in org.eclipse.swt

PW
Comment 3 Thanh Ha CLA 2012-07-10 16:19:44 EDT
Not sure this is the right bug to comment on but per the mailing list:
  http://dev.eclipse.org/mhonarc/lists/cbi-dev/msg00474.html

It seems Andrey noticed that the SWT build produced by CBI has some issues (pasting his assessment below):


Second, it looks like SWT build is not optimal yet: AFAIK SWT maintains their own very special build and version schema, and so the generated jar has wrong version/content (see attached picture):

"3.8 from eclipse.org":
org.eclipse.swt_3.8.0.v3833.jar 18 KB

"3.8 custom":
org.eclipse.swt_3.8.0.201207061258.jar 7.2 MB

However this seems not break the UI (after quick smoke test).

And last one: even if I specified to re-build swt native binaries on Linux 64 gtk, they still contain parts built on eclipse.org:

libcairo-swt.so
libswt-mozilla-gtk-3833.so
libswt-xpcominit-gtk-3833.so
libswt-xulrunner-fix.so
libswt-xulrunner-gtk-3833.so

I guess there are some preconditions for the build (like xulrunner/mozilla SDK installed) which are not considered by the maven build. I wonder if this can/should be manual step before the build, or can be done by maven downloading those SDK's to the build area.
Comment 4 Paul Webster CLA 2012-09-21 08:33:46 EDT
When I inspect eclipse.platform.swt.binaries/bundles/org.eclipse.swt.gtk.linux.x86_64/target/classes it seems to have a lot of compiled classes in it.

But the generated jar org.eclipse.swt.gtk.linux.x86_64-3.100.1-SNAPSHOT.jar doesn't include them, it seems to only include their package directories.

PW
Comment 5 Krzysztof Daniel CLA 2012-09-21 09:03:43 EDT
My fedora build (before applying the patch from bug 384874) produces something that looks like a valid fragment (http://i.imgur.com/KoaMT.png).

I have not fully understood when you are getting the error - before applying the patch or after that (or in both cases) ?
Comment 6 Thanh Ha CLA 2012-09-21 09:13:34 EDT
(In reply to comment #4)
> When I inspect
> eclipse.platform.swt.binaries/bundles/org.eclipse.swt.gtk.linux.x86_64/
> target/classes it seems to have a lot of compiled classes in it.
> 
> But the generated jar org.eclipse.swt.gtk.linux.x86_64-3.100.1-SNAPSHOT.jar
> doesn't include them, it seems to only include their package directories.
> 
> PW

I checked with my build and my org.eclipse.swt.gtk.linux.x86_64 jar seems to include a bunch of class files while my org.eclipse.swt jar includes a bunch of source code (.java, .c, etc..) which seems like the complete contents of eclipse.platform.swt/bundles/org.eclipse.swt
Comment 7 Paul Webster CLA 2012-09-21 09:20:57 EDT
Right now this calls out to org.eclipse.swt/buildFragment.xml and buildSWT.xml

But when I run with bree-libs, I get:

    [javac] Compiling 609 source files to /opt/pwebster/git/cbi/r4/eclipse.platform.releng.aggregator/eclipse.platform.swt.binaries/bundles/org.eclipse.swt.gtk.linux.x86_64/temp.folder/@dot.bin
    [javac] /opt/pwebster/git/cbi/r4/eclipse.platform.releng.aggregator/eclipse.platform.swt.binaries/bundles/org.eclipse.swt.gtk.linux.x86_64/temp.folder/@dot.src/org/eclipse/swt/SWT.java:46: error: cannot access Object
    [javac] public class SWT {
    [javac]        ^
    [javac]   class file for java.lang.Object not found

So the compile fails for me for some reason.

PW
Comment 8 Paul Webster CLA 2012-09-21 09:21:53 EDT
Instead of running the SWT ant scripts, can each fragment not have pom that does that work intead?

PW
Comment 9 Paul Webster CLA 2012-09-21 10:28:02 EDT
(In reply to comment #7)
> Right now this calls out to org.eclipse.swt/buildFragment.xml and
> buildSWT.xml
> 
> But when I run with bree-libs, I get:

Worse, when it fails it doesn't fail the building of this plugin.

PW
Comment 10 Paul Webster CLA 2012-09-21 14:04:16 EDT
(In reply to comment #7)
> Right now this calls out to org.eclipse.swt/buildFragment.xml and
> buildSWT.xml
> 
> But when I run with bree-libs, I get:
> 
>     [javac] Compiling 609 source files to
> /opt/pwebster/git/cbi/r4/eclipse.platform.releng.aggregator/eclipse.platform.
> swt.binaries/bundles/org.eclipse.swt.gtk.linux.x86_64/temp.folder/@dot.bin
>     [javac]
> /opt/pwebster/git/cbi/r4/eclipse.platform.releng.aggregator/eclipse.platform.
> swt.binaries/bundles/org.eclipse.swt.gtk.linux.x86_64/temp.folder/@dot.src/
> org/eclipse/swt/SWT.java:46: error: cannot access Object
>     [javac] public class SWT {
>     [javac]        ^
>     [javac]   class file for java.lang.Object not found

This appears to be related to using an IBM JRE, it doesn't add the vm.jar.  Switching to Oracle JVM allowed this to go forward.

We're still looking for a maven way to replace these ant calls.

PW
Comment 11 Krzysztof Daniel CLA 2012-09-24 04:12:26 EDT
> We're still looking for a maven way to replace these ant calls.

The way is there but I'm too faint hearted to follow it. Build login in SWT is buried in three places - buildFragment.xml, which contains:
	* definiton of what goes where
	* SWT 32<->64 bit conversion,
build.{sh, bat} files, which (at least for linux) do a lot of conditional checks and platform specific variables settings, 
and make (which could be eventually replaced with http://duns.github.com/maven-nar-plugin/).

This is not something that I can commit to right now :-( - but I'll definitely try to fail the build if the underlying ant call fails.
Comment 12 Paul Webster CLA 2012-09-24 07:57:37 EDT
(In reply to comment #11)
> 
> This is not something that I can commit to right now :-( - but I'll
> definitely try to fail the build if the underlying ant call fails.

Andrew was going to try and get someone with a lot of maven experience to help us out here as well.

PW
Comment 13 David Williams CLA 2014-04-17 13:04:15 EDT
(In reply to Paul Webster from comment #10)

> This appears to be related to using an IBM JRE, it doesn't add the vm.jar. 
> Switching to Oracle JVM allowed this to go forward.
> 
> We're still looking for a maven way to replace these ant calls.
> 

If this is a matter of "what's on the classpath" and Toolchain.xml is being used, I think ToolChains gives a why to configure the "bootclasspath" for "non-standard" layouts ... 

See 

http://eclipse.org/tycho/sitedocs/tycho-compiler-plugin/compile-mojo.html#useJDK 

Especially the sections that says 

= = = =

The default value of the bootclasspath used for compilation is <jdkHome>/lib/*;<jdkHome>/lib/ext/*;<jdkHome>/lib/endorsed/* . For JDKs with different filesystem layouts, the bootclasspath can be specified explicitly in the configuration section. Example:

<configuration>
  <jdkHome>/path/to/jdk/1.5</jdkHome>
  <bootClassPath>
    <includes>
      <include>jre/lib/amd64/default/jclSC160/*.jar</include>
    </includes>
    <excludes>
      <exclude>**/alt-*.jar</exclude>
    </excludes>
  </bootClassPath>
</configuration>
Comment 14 David Williams CLA 2014-04-17 13:26:44 EDT
(In reply to David Williams from comment #13)
> (In reply to Paul Webster from comment #10)
> 
> > This appears to be related to using an IBM JRE, it doesn't add the vm.jar. 
> > Switching to Oracle JVM allowed this to go forward.
> > 
> > We're still looking for a maven way to replace these ant calls.
> > 
> 
> If this is a matter of "what's on the classpath" and Toolchain.xml is being
> used, I think ToolChains gives a why to configure the "bootclasspath" for
> "non-standard" layouts ... 
> 

FYI, I say this since the "PDE equivalent" of toolschain, for WTP, we used to have settings as below, depending on which JRE was being used: 

# some standard ones for Sun
J2SE-1.4=${env.JAVA_4_HOME}/jre/lib/rt.jar:${env.JAVA_4_HOME}/jre/lib/jsse.jar
J2SE-1.5=${env.JAVA_5_HOME}/jre/lib/rt.jar:${env.JAVA_5_HOME}/jre/lib/jsse.jar

# some standard ones for IBM's JRE
#J2SE-1.4=${env.JAVA_4_HOME}/jre/lib/core.jar:${env.JAVA_4_HOME}/jre/lib/xml.jar:${env.JAVA_4_HOME}/jre/lib/graphics.jar:${env.JAVA_4_HOME}/jre/lib/server.jar:${env.JAVA_4_HOME}/jre/lib/ibmorbapi.jar:${env.JAVA_4_HOME}/jre/lib/security.jar
#J2SE-1.5=${env.JAVA_5_HOME}/jre/lib/core.jar:${env.JAVA_5_HOME}/jre/lib/vm.jar:${env.JAVA_5_HOME}/jre/lib/xml.jar:${env.JAVA_5_HOME}/jre/lib/graphics.jar:${env.JAVA_5_HOME}/jre/lib/server.jar:${env.JAVA_5_HOME}/jre/lib/ibmorbapi.jar:${env.JAVA_5_HOME}/jre/lib/security.jar


But, I have not personally tried this with our current builds.
Comment 15 Alexander Kurtakov CLA 2014-04-17 13:42:07 EDT
If we give up on the swt-debug.jar in the standalone swt downloads it should be doable to get rid of the ant calls (or at least reduce significantly).
Comment 16 Alexander Kurtakov CLA 2018-01-04 04:49:02 EST
Marking this one as fixed. All swt fragments have proper poms with even possibility to rebuild natives if -Dnatives=ws.os.arch is passed to it..