Bug 330240 - new PDE features and product
Summary: new PDE features and product
Status: RESOLVED WONTFIX
Alias: None
Product: Platform
Classification: Eclipse Project
Component: Releng (show other bugs)
Version: 3.7   Edit
Hardware: PC Mac OS X - Carbon (unsup.)
: P3 normal (vote)
Target Milestone: ---   Edit
Assignee: Kim Moir CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on: 327899
Blocks:
  Show dependency tree
 
Reported: 2010-11-15 08:29 EST by Jeff McAffer CLA
Modified: 2011-08-26 13:45 EDT (History)
4 users (show)

See Also:


Attachments
patch (28.72 KB, patch)
2010-12-01 16:11 EST, Kim Moir CLA
no flags Details | Diff
mylyn/context/zip (43.03 KB, application/octet-stream)
2010-12-01 16:11 EST, Kim Moir CLA
no flags Details
patch (28.72 KB, patch)
2010-12-01 16:37 EST, Kim Moir CLA
no flags Details | Diff
tweaked compiler adapter bundle (13.54 KB, application/java-archive)
2010-12-02 22:12 EST, Jeff McAffer CLA
no flags Details
patch (30.93 KB, patch)
2010-12-14 09:51 EST, Kim Moir CLA
no flags Details | Diff
patch (52.09 KB, patch)
2010-12-14 11:08 EST, Kim Moir CLA
no flags Details | Diff
patch (5.46 KB, patch)
2010-12-15 16:37 EST, Kim Moir CLA
no flags Details | Diff
patch (2.20 KB, patch)
2010-12-17 09:51 EST, Kim Moir CLA
no flags Details | Diff
patch (9.82 KB, patch)
2010-12-19 21:35 EST, Kim Moir CLA
no flags Details | Diff
patch (806 bytes, patch)
2010-12-20 16:12 EST, Kim Moir CLA
no flags Details | Diff
patch (1.34 KB, patch)
2010-12-20 16:12 EST, Kim Moir CLA
no flags Details | Diff
patch (8.31 KB, patch)
2010-12-20 16:12 EST, Kim Moir CLA
no flags Details | Diff
patch (7.89 KB, patch)
2010-12-20 21:59 EST, Kim Moir CLA
no flags Details | Diff
patch to fix build page presentation (5.93 KB, patch)
2010-12-21 16:43 EST, Kim Moir CLA
no flags Details | Diff
patch to revert (11.86 KB, patch)
2011-01-14 08:28 EST, Kim Moir CLA
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Jeff McAffer CLA 2010-11-15 08:29:26 EST
As a result of Bug 327966, there are a couple of new features, a plugin and a product in PDE build land. I have added them to the core.map file and tagged.

    plugin@org.eclipse.pde.build.compiler.adapter
    feature@org.eclipse.pde.build.feature
    feature@org.eclipse.pde.build.product.feature

The features should show up in the p2 repo and the product should show up on the Eclipse download page. As with the OSGi starter kit that we just setup, only the major platforms need to be provided.  Win32/64, Linux 32/64 and Mac 32/64.  These should go in a new section just before the PDE Runtimes on the download page with the following heading/description

PDE Builders

The PDE Builders are self-contained, executable PDE Build configurations that can be used to build OSGi and Eclipse-based systems. They can also be used as the basis for more sophisticated build systems that run tests, do API scans, publish builds etc.

This product has the same requirements as the OSGi starter kit in that the executables should be branded (pdebuild.exe in this case) and really only the console executable is relevant.  I did not include the windows executables however in this setup as that was not a sustainable approach.  Ideally we would ahve a pde build solution for creating products that just have the console launcher.
Comment 1 Kim Moir CLA 2010-12-01 15:35:02 EST
Hi Jeff

Two questions
1) What do you want the name of these zips to be?
2) There aren't any branding icons in the pde.build.product feature, I guess we'll just use the default?
Comment 2 Kim Moir CLA 2010-12-01 16:11:31 EST
Created attachment 184305 [details]
patch
Comment 3 Kim Moir CLA 2010-12-01 16:11:38 EST
Created attachment 184306 [details]
mylyn/context/zip
Comment 4 Jeff McAffer CLA 2010-12-01 16:22:39 EST
(In reply to comment #1)
> 1) What do you want the name of these zips to be?

PDE-builder-<version>-<platform>.zip sounds good.

> 2) There aren't any branding icons in the pde.build.product feature, I guess
> we'll just use the default?

The default (eclipse) icon is fine.  We don't have an image to use for PDE build AFAIK.  For the most part these are command line tools anyway.
Comment 5 Kim Moir CLA 2010-12-01 16:37:03 EST
Created attachment 184308 [details]
patch
Comment 6 Kim Moir CLA 2010-12-02 10:58:37 EST
org.eclipse.jdt.core.compiler.batch isn't a bundle that is included in the map files but it is included in the pde.build feature.  It's constructed from the ecj jars and thus not available in the build until after the build has been signed and the repo created.   The fact that it is includes the compiler adapter means that the product has be assembled after the repo has been signed and created which take a lot of  build script calisthenics.  Would it be possible to just include jdt.core bundle in the org.eclipse.pde.build.feature?
Comment 7 Jeff McAffer CLA 2010-12-02 11:13:57 EST
unfortunately not.  jdt.core drags in a lot of other stuff and considerable work has gone into making this all go just on ECJ.

Note that related to this the JDT folks have made the jdtCompilerAdapter.jar (included in jdt.core) into a bundle that we can consume in this product/feature setup.  That jar/bundle will come out of the normal build embedded in jdt.core but will need to be extracted and published as an individual bundle.

So, would it make sense to do think of / implement the build in two steps.  Effectively do the normal build that produces and publishes all the bundles that will be needed as well as many/most of the features.  Then a second "build" that really just assembles these into deliverables.  This second step may generate some additional features but generally would just be building/publishing products and then assembling them.

Adding Andrew to the fun since we have talked about some of this kind of thing in the past and he may have some insight
Comment 8 Kim Moir CLA 2010-12-02 11:23:47 EST
Yes, I know how to implement it :-)  I just wanted to confirm with you because this will take more work than I anticipated.
Comment 9 Jeff McAffer CLA 2010-12-02 11:56:18 EST
> Yes, I know how to implement it :-) 

Of course you do!  I did not mean to imply otherwise.

> I just wanted to confirm with you because
> this will take more work than I anticipated.

Let me take a pass on the current feature/product before you spend too much more time.  The changes around jdtCompilerAdapter.jar make the pde.build.compiler.adapter bundle obsolete.  So there will likely be two things that need special treatment.  the ecj bundle needs to be available and the ..jdt.core.compiler.adapter bundle (jdtCompilerAdapter.jar) needs to be extracted from the jdt.core bundle and made available.

I took a look at the latest nightly build and it seems that the jdtCompilerAdapter.jar in jdt.core has the right plugin.xml but the MANIFEST.MF is wrong.  It may have been overwritten by the zip/jar process as it is not an OSGi manifest as set out in bug 327899 comment 15.  I'll reopen that bug as it needs to be addressed for this to all work.

In the mean time, I'll update the features etc in anticipation of that problem being resolved.
Comment 10 Jeff McAffer CLA 2010-12-02 22:12:15 EST
Created attachment 184436 [details]
tweaked compiler adapter bundle

I updated the pde.build.feature to use the JDT compiler adapter rather than our own and purged the PDE supplied adapter from the repo.  Attached is a jdtCompilerAdapter.jar extracted from a recent jdt.core bundle and fixed up to have the right manifest info (to work around the JDT build problem).

We should be good to go now.
Comment 11 Kim Moir CLA 2010-12-13 13:48:37 EST
Regarding, comment #10, do you want me to include this version of the adapter in the repo as a bundle that doesn't change?
Comment 12 Jeff McAffer CLA 2010-12-13 14:03:28 EST
No.  That attachment was provided for folks wanting to mess with this prior to the JDT build structure being fixed.  The jdtCompilerAdapter.jar included in the jdt.core bundle of the current builds is correct and should be used.  you can now ignore the attached compiler adapter.
Comment 13 Kim Moir CLA 2010-12-14 09:51:54 EST
Created attachment 185135 [details]
patch
Comment 14 Kim Moir CLA 2010-12-14 11:08:36 EST
Created attachment 185147 [details]
patch
Comment 15 Kim Moir CLA 2010-12-15 16:37:56 EST
Created attachment 185271 [details]
patch
Comment 16 Kim Moir CLA 2010-12-17 09:51:13 EST
Created attachment 185426 [details]
patch
Comment 17 Kim Moir CLA 2010-12-19 21:35:23 EST
Created attachment 185511 [details]
patch
Comment 18 Kim Moir CLA 2010-12-20 16:12:10 EST
Created attachment 185595 [details]
patch
Comment 19 Kim Moir CLA 2010-12-20 16:12:35 EST
Created attachment 185596 [details]
patch
Comment 20 Kim Moir CLA 2010-12-20 16:12:59 EST
Created attachment 185597 [details]
patch
Comment 21 Kim Moir CLA 2010-12-20 21:59:10 EST
Created attachment 185608 [details]
patch
Comment 22 Kim Moir CLA 2010-12-21 10:15:49 EST
Jeff, noticed this in the org.eclipse.equinox.p2.extras.feature's feature.xml


<includes
         id="org.eclipse.equinox.p2.core.feature"
         version="0.0.0"/>

   <requires>
      <import feature="org.eclipse.equinox.core.feature" version="1.0.0" match="compatible"/>
      <import feature="org.eclipse.equinox.p2.core.feature" version="1.0.0" match="compatible"/>
   </requires>

Why is the org.eclipse.equinox.p2.core.feature both included and required?
Comment 23 John Arthorne CLA 2010-12-21 11:41:07 EST
(In reply to comment #22)
> Why is the org.eclipse.equinox.p2.core.feature both included and required?

That must be a bug - there is no value in requiring a feature that is already included.
Comment 24 Kim Moir CLA 2010-12-21 11:50:56 EST
I'm going to remove the require statement and leave the include statement.
Comment 25 Kim Moir CLA 2010-12-21 16:43:10 EST
Created attachment 185673 [details]
patch to fix build page presentation
Comment 26 Kim Moir CLA 2010-12-21 16:45:05 EST
Fixed.  

This wasn't really fun to implement.  Adding nested jars from other bundles to the repo and then assembling products. Ugly Ant code.

Released for N20101221-2000.
Comment 27 Jeff McAffer CLA 2010-12-22 10:11:39 EST
Thanks Kim.  Not sure about the require/include thing.  That must have been a leftover from previous structures.  Including is the way to go here I think.
Comment 28 Kim Moir CLA 2011-01-14 08:23:25 EST
Reopening.

I'm going to revert this fix because it's causing compile errors in the build. I tried to fix this yesterday but due to a variety of issues, it didn't work.  So I'm reverting it so we can have a clean integration build.
Comment 29 Kim Moir CLA 2011-01-14 08:28:57 EST
Created attachment 186823 [details]
patch to revert
Comment 30 Kim Moir CLA 2011-08-26 13:45:35 EDT
The pde builder zips were removed because it was causing compile errors.  I spent a huge amount of time on trying to build these zips properly and it didn't work. Since Jeff is no longer involved with Eclipse, I assume I can close the request for the pde build zips.