Bug 528435 - Document how to build swt.jar with local code changes.
Summary: Document how to build swt.jar with local code changes.
Status: CLOSED WONTFIX
Alias: None
Product: Platform
Classification: Eclipse Project
Component: SWT (show other bugs)
Version: 4.8   Edit
Hardware: PC Linux
: P3 normal (vote)
Target Milestone: ---   Edit
Assignee: Platform-SWT-Inbox CLA
QA Contact:
URL:
Whiteboard: stalebug
Keywords: triaged
Depends on:
Blocks: 528173
  Show dependency tree
 
Reported: 2017-12-11 09:59 EST by Leo Ufimtsev CLA
Modified: 2020-08-15 04:53 EDT (History)
3 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Leo Ufimtsev CLA 2017-12-11 09:59:16 EST
It's currently not very clear on how to build local swt.jar containing the natives.

E.g use case is to develop/verify OSGI functionality. E.g compile swt.jar and inject it into a recent eclipse I-Build.

(e.g https://git.eclipse.org/r/#/c/113014/)
Comment 1 Andrey Loskutov CLA 2017-12-11 10:56:50 EST
Would be great to add some works how this SWT build can be used in the parent platform aggregator maven build.

My use case: I want to build 4.7.2 locally with patched SWT, and SWT patches aren't yet merged upstream (but present in the local SWT git clone). Would be great to document how this could be done, so that at the end I have a custom SDK platform build with patched SWT.
Comment 2 Leo Ufimtsev CLA 2017-12-11 11:01:09 EST
(In reply to Andrey Loskutov from comment #1)
> Would be great to add some works how this SWT build can be used in the
> parent platform aggregator maven build.
> 
> My use case: I want to build 4.7.2 locally with patched SWT, and SWT patches
> aren't yet merged upstream (but present in the local SWT git clone). Would
> be great to document how this could be done, so that at the end I have a
> custom SDK platform build with patched SWT.

Is this for gtk64 bit or for other platforms also?
Comment 3 Andrey Loskutov CLA 2017-12-11 11:03:29 EST
(In reply to Leo Ufimtsev from comment #2)
> (In reply to Andrey Loskutov from comment #1)
> > Would be great to add some works how this SWT build can be used in the
> > parent platform aggregator maven build.
> > 
> > My use case: I want to build 4.7.2 locally with patched SWT, and SWT patches
> > aren't yet merged upstream (but present in the local SWT git clone). Would
> > be great to document how this could be done, so that at the end I have a
> > custom SDK platform build with patched SWT.
> 
> Is this for gtk64 bit or for other platforms also?

I need this for GTK3 64 bit only, I'm on RHEL 7.2.
Comment 4 Leo Ufimtsev CLA 2017-12-11 14:32:47 EST
(In reply to Andrey Loskutov from comment #3)
> (In reply to Leo Ufimtsev from comment #2)
> > (In reply to Andrey Loskutov from comment #1)
> > > Would be great to add some works how this SWT build can be used in the
> > > parent platform aggregator maven build.
> > > 
> > > My use case: I want to build 4.7.2 locally with patched SWT, and SWT patches
> > > aren't yet merged upstream (but present in the local SWT git clone). Would
> > > be great to document how this could be done, so that at the end I have a
> > > custom SDK platform build with patched SWT.
> > 
> > Is this for gtk64 bit or for other platforms also?
> 
> I need this for GTK3 64 bit only, I'm on RHEL 7.2.

Could you test the script I wrote here?
https://git.eclipse.org/r/#/c/113014/

It should generate the *swt*.jar that you're looking for.
If the script is suitable for your needs, I'll merge it in as part of docu update.
Comment 5 Andrey Loskutov CLA 2017-12-11 16:04:53 EST
(In reply to Leo Ufimtsev from comment #4)
> Could you test the script I wrote here?
> https://git.eclipse.org/r/#/c/113014/
> 
> It should generate the *swt*.jar that you're looking for.
> If the script is suitable for your needs, I'll merge it in as part of docu
> update.

I will, but this is probably not enough. Once the SWT bits are generated, I have no idea how to integrate this artifacts into the overall maven SDK build (I'm not the maven expert in any way).
Comment 6 Leo Ufimtsev CLA 2017-12-11 16:26:04 EST
(In reply to Andrey Loskutov from comment #5)
> (In reply to Leo Ufimtsev from comment #4)
> > Could you test the script I wrote here?
> > https://git.eclipse.org/r/#/c/113014/
> > 
> > It should generate the *swt*.jar that you're looking for.
> > If the script is suitable for your needs, I'll merge it in as part of docu
> > update.
> 
> I will, but this is probably not enough. Once the SWT bits are generated, I
> have no idea how to integrate this artifacts into the overall maven SDK
> build (I'm not the maven expert in any way).

hmm.

So I guess you use maven to build the whole eclipse?

Try:
1)  <edit> ~/git/eclipse.platform.swt.binaries/bundles/binaries-parent/pom.xml 
  Replace: <forceContextQualifier>v20171209-1014</forceContextQualifier>
  With   : <forceContextQualifier>v20000000-0000</forceContextQualifier>

2) Make some change in swt. E.g println when display initializes.

3) Build your thing.

4) Inspect if build Eclipse has local changes of SWT source code.

The script does the above except it just builds the swt jar.

Let me know.

Alternative: Checkout a slightly older swt.binaries, the older forceContextQualifier will force a fresh pull of swt sources from local repo instead of remote. (usually?).
Comment 7 Sravan Kumar Lakkimsetti CLA 2017-12-12 00:25:44 EST
(In reply to Andrey Loskutov from comment #5)
> (In reply to Leo Ufimtsev from comment #4)
> > Could you test the script I wrote here?
> > https://git.eclipse.org/r/#/c/113014/
> > 
> > It should generate the *swt*.jar that you're looking for.
> > If the script is suitable for your needs, I'll merge it in as part of docu
> > update.
> 
> I will, but this is probably not enough. Once the SWT bits are generated, I
> have no idea how to integrate this artifacts into the overall maven SDK
> build (I'm not the maven expert in any way).

You need to update qualifier in swt repo and swt binaries repo like https://git.eclipse.org/c/platform/eclipse.platform.swt.git/commit/?id=2b729d07987504e1da5cf98781c8b12475b90c6a
and https://git.eclipse.org/c/platform/eclipse.platform.swt.binaries.git/commit/?id=cd3a1f8987b460853dc10ab2ebfd9418c649bf0d
Comment 8 Leo Ufimtsev CLA 2017-12-12 09:48:17 EST
(In reply to Sravan Kumar Lakkimsetti from comment #7)
> (In reply to Andrey Loskutov from comment #5)
> > (In reply to Leo Ufimtsev from comment #4)
> > > Could you test the script I wrote here?
> > > https://git.eclipse.org/r/#/c/113014/
> > > 
> > > It should generate the *swt*.jar that you're looking for.
> > > If the script is suitable for your needs, I'll merge it in as part of docu
> > > update.
> > 
> > I will, but this is probably not enough. Once the SWT bits are generated, I
> > have no idea how to integrate this artifacts into the overall maven SDK
> > build (I'm not the maven expert in any way).
> 
> You need to update qualifier in swt repo and swt binaries repo like
> https://git.eclipse.org/c/platform/eclipse.platform.swt.git/commit/
> ?id=2b729d07987504e1da5cf98781c8b12475b90c6a
> and
> https://git.eclipse.org/c/platform/eclipse.platform.swt.binaries.git/commit/
> ?id=cd3a1f8987b460853dc10ab2ebfd9418c649bf0d

As a side note, I found that when I run maven/build from inside the binary project, updating (up or down) the forceQualifier from SWT project did not have an impact on weather the binary project used the local swt sources or pulled upstream. (it always pulled from upstream).
But lowering the forceQualifier from binary project forced it to use the local sources instead of pulling from upstream.
Very odd.. ?
Comment 9 Andrey Loskutov CLA 2017-12-13 10:52:16 EST
OK, I've tried to understand all it, and after some fails I believe I found the solution.

If I get it right, for a single platform (e.g. gtk-linux-x86_64) the SWT build should produce 3 artifacts, which all must have same version (e.g. v20171213-1345, which is not existing on eclipse.org):

org.eclipse.swt_3.106.2.v20171213-1345.jar
org.eclipse.swt.gtk.linux.x86_64_3.106.2.v20171213-1345.jar
org.eclipse.swt.gtk.linux.x86_64.source_3.106.2.v20171213-1345.jar

To build custom SWT bundles one must change two files:
eclipse.platform.swt/bundles/org.eclipse.swt/pom.xml
eclipse.platform.swt.binaries/bundles/binaries-parent/pom.xml

and set forceContextQualifier to "v20171213-1345".

After that

cd eclipse.platform.swt.binaries/bundles/org.eclipse.swt.gtk.linux.x86_64
mvn clean verify -Pbuild-individual-bundles

should generate all the bundles above with extra patches.

But what I get in the target directory are only *two* jars with *wrong* names:

org.eclipse.swt.gtk.linux.x86_64-3.106.2-SNAPSHOT-sources.jar
org.eclipse.swt.gtk.linux.x86_64-3.106.2-SNAPSHOT.jar

The code / sources *inside* the jars seem to be right, I see changed classes.

But now with that, how I can trigger the "right" jars generation as they appear in the SDK package?

So here I believe I'm supposed to run the full SDK build. The SDK build is not incremental, because it always fails to generate some bits if they are already there, so before running it one is supposed to run git submodule foreach 'git clean -fdx' and git clean -fdx in eclipse.platform.releng.aggregator.

This would of course delete all the bundles we just generated in SWT, and fail to "download" SWT bundles with the "v20171213-1345" qualifier.

Therefore I've tried to "install" generated SWT bundles locally via:

cd eclipse.platform.swt.binaries/bundles/org.eclipse.swt.gtk.linux.x86_64
mvn clean install verify -Pbuild-individual-bundles

After that I can:

cd eclipse.platform.releng.aggregator
git submodule foreach 'git clean -fdx'
git clean -fdx
mvn -e clean verify

and after half of the internet is downloaded to my hard disk by maven (~1 hour) magically my fresh SDK build contains 3 patched SWT jars with right names and content!
Comment 10 Sravan Kumar Lakkimsetti CLA 2017-12-14 01:34:06 EST
(In reply to Andrey Loskutov from comment #9)
> OK, I've tried to understand all it, and after some fails I believe I found
> the solution.
> 
> If I get it right, for a single platform (e.g. gtk-linux-x86_64) the SWT
> build should produce 3 artifacts, which all must have same version (e.g.
> v20171213-1345, which is not existing on eclipse.org):
> 
> org.eclipse.swt_3.106.2.v20171213-1345.jar
> org.eclipse.swt.gtk.linux.x86_64_3.106.2.v20171213-1345.jar
> org.eclipse.swt.gtk.linux.x86_64.source_3.106.2.v20171213-1345.jar
> 
> To build custom SWT bundles one must change two files:
> eclipse.platform.swt/bundles/org.eclipse.swt/pom.xml
> eclipse.platform.swt.binaries/bundles/binaries-parent/pom.xml
> 
> and set forceContextQualifier to "v20171213-1345".
> 
> After that
> 
> cd eclipse.platform.swt.binaries/bundles/org.eclipse.swt.gtk.linux.x86_64
> mvn clean verify -Pbuild-individual-bundles
> 
> should generate all the bundles above with extra patches.
> 
> But what I get in the target directory are only *two* jars with *wrong*
> names:
> 
> org.eclipse.swt.gtk.linux.x86_64-3.106.2-SNAPSHOT-sources.jar
> org.eclipse.swt.gtk.linux.x86_64-3.106.2-SNAPSHOT.jar
> 
> The code / sources *inside* the jars seem to be right, I see changed classes.
> 
> But now with that, how I can trigger the "right" jars generation as they
> appear in the SDK package?
> 
> So here I believe I'm supposed to run the full SDK build. The SDK build is
> not incremental, because it always fails to generate some bits if they are
> already there, so before running it one is supposed to run git submodule
> foreach 'git clean -fdx' and git clean -fdx in
> eclipse.platform.releng.aggregator.
> 
> This would of course delete all the bundles we just generated in SWT, and
> fail to "download" SWT bundles with the "v20171213-1345" qualifier.
> 
> Therefore I've tried to "install" generated SWT bundles locally via:
> 
> cd eclipse.platform.swt.binaries/bundles/org.eclipse.swt.gtk.linux.x86_64
> mvn clean install verify -Pbuild-individual-bundles
> 
> After that I can:
> 
> cd eclipse.platform.releng.aggregator
> git submodule foreach 'git clean -fdx'
> git clean -fdx
> mvn -e clean verify
> 
> and after half of the internet is downloaded to my hard disk by maven (~1
> hour) magically my fresh SDK build contains 3 patched SWT jars with right
> names and content!

To get new bundles in to SDK build you need to 
1. update forceContextQualifier with timestamp in UTC. 
2. Run mvn -e clean verify -DskipTests=true from eclipse.platform.releng aggregator

Maven build will do the necessary rebuilds and package with new content.
Comment 11 Eclipse Genie CLA 2020-08-15 04:53:27 EDT
This bug hasn't had any activity in quite some time. Maybe the problem got resolved, was a duplicate of something else, or became less pressing for some reason - or maybe it's still relevant but just hasn't been looked at yet. As such, we're closing this bug.

If you have further information on the current state of the bug, please add it and reopen this bug. The information can be, for example, that the problem still occurs, that you still want the feature, that more information is needed, or that the bug is (for whatever reason) no longer relevant.

--
The automated Eclipse Genie.