Community
Participate
Working Groups
When an pde.exportFeatures task is executed within an ant script, then the process is forked and the ant script continued in parallel. This makes it very difficult to write a script where the input of the next task is dependent on the export task.
*** Bug 62242 has been marked as a duplicate of this bug. ***
*** Bug 65118 has been marked as a duplicate of this bug. ***
*** Bug 65189 has been marked as a duplicate of this bug. ***
*** Bug 65146 has been marked as a duplicate of this bug. ***
Just to clarify... As the export wizard indicates, the pde.export* tasks only save the settings of the export operation so that it could be later be executed without having to go through the wizard again. No more promises were made. This includes the fact that it will run as a background job. As many of you noticed, the fact that the task runs in the background and the fact that it requires UI classes makes it difficult to incorporate the task into a bigger ant script. But these tasks were not originally intended to be run in headless mode, nor were they intended to be run as part of a bigger Ant script. I believe the PDE/Build component provides tasks that allow you to do all the things you want to do. In fact, the pde-build tasks are the very same tasks that the Eclipse Releng team uses to build Eclipse itself. The pde.export* tasks were therefore not designed to duplicate these functions. Having said that and having seen how popular these export tasks have become, we will work to make them headless and execute synchronously. However, this actually requires quite a bit of work and would have to be addressed post-3.0.
The bugzilla report that I opened only asks that it be synchronous, which sounds a lot easier to implement than also making it headless. I'm not saying don't make it headless, as that sounds useful as well, but I am asking that the synchronous support not be delayed because of this other requirement.
I agree that headless and synchronous are two separate issues. I don't mind having to build my RCP app from Eclipse; however, it's impossible to automate the build with the current state of the pde.export* tasks. Just getting the synchronous ability will help a lot.
The making it synchronous part is actually the more difficult part, given the current implementation. The implementation of the export job generates build/assembly/cleanup scripts, programmatically creates an Ant runner and executes these scripts. Making it a synchronous task so that it can be called from an arbitrary script requires a solution in which the FeatureExport task does not programmatically create an Ant runner, since no two Ant runners can run in the same VM at the same time in Eclipse. Therefore, I invite you to take a look at FeatureExportJob in the org.eclipse.pde.ui plug-in, and see how much work it would be to do all the build/assembly/cleanup without a single call to AntRunner.
*** Bug 84065 has been marked as a duplicate of this bug. ***
I have indicated in comment 5 what the original purpose of these tasks was and that PDE/Build already provides synchronous headless tasks for all your building needs. It would be redundant (and not that easy) to turn the pde.export* tasks headless and synchronous, since the pde.export* tasks call the pde/build tasks under the covers anyway. Since the list of people interested in this effort is evergrowing, I would appreciate a response to the following question(s): Are the pde/build tasks insufficient in any way? I mean, they are used by the releng team to build Eclipse so they certainly have all the pieces you need. Why do we you rely on pde.export* tasks?
I'll echo Wassim's comments and up them with a call for patches. All available people are flat out on RCP product tooling and related items. Time spent driving this will take away from that effort. Patch review and collaboration are definitely possible but someone else has to drive/solve the problems.
I think that the result of this work should be the removal of the pde.export* tasks as we know them today. Indeed if we want to solve all the problems we will have to recraft something that is open-ended and we'll end up duplicating what's in pde.build and the infrastructure already in place. Therefore, I'd rather see a consolidation effort around pde build infrastructure allowing the "files that people save" to be used as a base for "in place build" (in the workspace) as well as in a releng style build. Let me describe why I think that by describing today's world: PDE Build: It provides infrastructure to do releng style build, it does not run in the background. This infrastructure is driven by 2 global files: build.properties and customTargets.xml (for an example see: org.eclipse.releng.eclipsebuilder/rcp folder in eclipse repository ). The values from those two files are used to configure various scripts generators (BuildScriptGenerator, AssembleScriptGenerator, etc.) The first file decides which platform we are building for, the name of the archive, the baseOS, the compiler options, the feature to build, the list of plugins in the target, etc... The customTargets.xml is mainly used to decide the name of the archive for each configuration (it also provides extra call backs at different stage of the build process allowing extra tools to run). PDE-UI: PDE-UI FeatureExportJob class drives pde.build by letting it know the feature to build, the configuration for which to build, the plugins in the target, the baseOS, the compiler options, etc... Once the scripts are generated (by calling BuildScriptGenerator or AssembleBuildScript), pde ui selectively invokes tasks on them in order to produce the various results. In fact, PDE UI plays the role of build.properties and the customTargets.xml. The pde.export tasks remembers most of those setting for further export and also provides hooks in the WorkspaceModelManager to give flexibility in the list of plugins. I hope this explanation made clear how both pde.build and pde.ui works together, and how redundant it would be to enhance pde.ui which would introduce new files, new concepts, and more confusion in our already complex build story. If pde.ui were to fully rely on the build.properties and customTargets.xml, then "saving the export operation" would result is giving those two files to the end user. Before going and hacking pde.ui, the first cut should be done on pde.build to give more flexibility (most of the time it consists in exposing some of the properties on ant tasks) and recreate the same kind of drop that pde.ui creates today. Some PDE build limitations - make fetch step optional - the list of plugins can't be passed in. - no ways to set the output format - the generic scripts might not be flexible enough
I'm using the pde.exportFeatures task because it was easy to find by telling the export wizard to save itself as an ant task. Where is the documentation for how to use PDE/Build tasks to integrate synchronous feature builds into a larger ant build?
Here: http://dev.eclipse.org/viewcvs/index.cgi/~checkout~/pde-build-home/articles/Automated%20Builds/article.html?rev=HEAD&content-type=text/html
Comment 12 is really helpful information, that is not found in the documentation that clearly. As a concequence, people were using the simplest accessible tool, which is the pde.export tasks. However, they would only be useful, if they could run sychronously. Since this won't be the case, I agree with Pascal to remove them. As I experienced today, the learning curve for pde.build is high, if you want to use them in your custom builds. But for the time being it is too early for me to come up with a good suggestion on how to improve them.
I've lurked and learned from this conversation but have also been active recently in exploring the pde-build tasks as used by the rel-eng team to create the eclipse downloads. Much more there than I was aware of, partially because I did not need to create a repeatable/automated build environment and partially because I had what I needed from the pde-ui processing. The document referenced in comment 14 is very helpful, but I still struggled a bit getting it working (not being an ant wiz only part of the problem). It took a msg or two from Pascal to point out some of my obvious blunders but I also got a bit of help just by searching the newsgroup so there are others that have found/discovered this function and are using it today. I agree with the limitations identified in comment 12. The current structure forces features, and even a root feature over other features to get a multi- feature build. That itself might be a good thing as such a structure is not wrong. It is probably just an issue of getting feature buy-in (not everybody likes them :-). I've made the fetch optional, but so far only by having the feature/plugin set in a build dir. I've even run the scripts locally in the current JRE/JVM as well as via antRunner. I drive it all from content in a workspace as I find it easier to manipulate the ant files using Eclipse itself. This has been fun enough, and I think important enough given the need for formalized repeatable builds, for us to actually re-scope the EclipseCon tutorial two of us are sponsoring to include this as one of the three major learning activities. Tutorial 24: "Creating, Packaging, Testing and Deploying Features in Eclipse 3.0" now has a module that is based on the rel-eng scripts which use the pde- build tasks. This structured activity gets you going with a small system that emulates fetch (we don't want a CVS dependency during the tutorial) and prepares a small system for RCP deployment. Fetch guidance is provided as a take home activity. The whole thing is based on the document from Sonia and Pascal. (Sorry Jeff, we've not been able to contain the incorporation of the RCP deploy package but will talk to it a bit:-) I like this so much now that I may replace our current formal education offering exercise with this process. Today we teach feature/build using only the function provided by the pde-ui. This new approach covers both quick (pde- ui) and repeatable (pde-build). One question that hits me as I've scanned these posts - is it the desire of some to be able to run the build using the projects in the workspace as the source vs having a stand-alone build process that fetches them? I've got a no- fetch process for testing (those tasks are just commented out in the build.xml main target) but the input is in sub-dirs and not the actual workspace plugin project. If I've heard the desire correctly I may have to try and see if I can get the pde-build scripts working such that the active workspace is used as input instead. Is that what people are after? FYI this is the structure of the process we will use in one of the learning activities that is part of our EclipseCon structured tutorial: Exercise 2: Build and Package a Feature Using AntRunner 2-1 Introduction 2-1 Exercise Concepts 2-2 Exercise Setup 2-2 Exercise Instructions 2-3 Part 1: Setup Build Automation Project 2-3 Step 1: Create Workspace Project with Alternative Location 2-3 Step 2: Add the Build Scripts to the New Project 2-3 Part 2: Customize the Ant Build Scripts 2-4 Step 1: Modify build.properties to Control the Ant Processing 2-4 Step 2: Customize the build.xml File to Reflect Your Processing Requirements 2-5 Step 3: Customize the customTargets.xml File to Reflect the Root Feature to be Built 2-5 Part 3: Setup the Feature and Plug-in Directory Tree (CVS Fetch Emulation) 2-6 Step 1: Unzip the Feature Set to the Build Directory 2-6 Part 4: Run the Ant Build Script 2-7 Step 1: Run as an Ant Task 2-7 Step 2: Run using the Eclipse antRunner Application 2-7 Step 3: Review the Files Created by the Ant Script 2-8 Optional Activities 2-8 Part 1: Enable CVS Fetch Processing 2-8 Step 1: Create and Store Map Files 2-9 Step 2: Customize the Ant Scripts to Find and Process the Map Files 2-10
Pat, I think you are on the right track here. Basically we are stuck between the full glory that is PDE build and the simple, just do it, that is PDE UI. Anything that makes the former easier is a good thing. PDE UI is not intended to be in this game at all. My cut on this is that features are the way to structure builds. PDE UI is gracious enough to help hide this by generating a root level feature under the covers but there is no magic there. Nor is that at odds with going it on your own and creating features which can be fed into PDE Build. So our approach is to say that people with all but trivial usecases should define features to collect up/structure their plugins and fragments. Do this regardless of whether or not you are shipping features. If you are not shipping then simply delete them at the end or just ship them, they are small. Once you have features that capture the world, you can right click and export and, if you did it right, you have a running system. You can then take that top-most feature and stick it in a releng build structure complete with fetch etc and build it every night. Twice if you like. ok, so the challenge is when you are in both worlds. You have some stuff in your workspace and some in your target and you still want to use some sort of scripting releng-ish build. The PDE Build stuff is not happy about disjoint plugins scattered all over the place. It has structures that allow you to tell it what it needs but that is non-trivial work. In fact, it seems that that is what the PDE UI build tasks are doing -- using the PDE model of the world to generate context for PDE Build. I didn't add much here but it was fun to write...
Agreed the automation of a build can be tricky to setup the first time, but with the draft article and a little help I've got it working in multiple ways... Running pure Ant in the same JRE without a CVS Fetch Running antRunner application with/without CVS Fetch Both of those options are based on using the expected structure, which is a dir with /features and /plugins dirs where the projects can be found (and the fetch target too). I think I heard somewhere in this bug report, or elsewhere, that some would like to do all the packaging and such as part of the active development workspace. My first try at this failed, but I have it working now, and with the right tickles to the definition of an external builder, I have it working such that when I modify the contents of any project I define as relevant, the Ant script kicks off, running under the control of the antRunner application, and my distribution zip (what I want for this scenario) is created. Takes a bit of strange workspace setup (the dir structure required by the PDE build is still the same, but making a workspace do real work and support this structure from the same source is a bit tricky. The only rule so far is that you must NOT have feature and plug-in ids that match. That goes against the common pattern where they do (partially for the free branding association), but if you really want this to work you have to use id patterns such as: F:com.mine.toolcore.feature F:com.mine.toolui.feature P:com.mine.toolcore P:com.mine.toolui Anything works, as long as the IDs are not the same. That allows you to create a strange workspace setup with projects in other than the default location, a project with the build control content, and a project where the primary build outputs are visible. The workspace directory itself ends up holding the temporary files and assemble scripts, but you know where they are and don't really have to look at them very often anyway. This will be covered a bit in the EclipseCon tutorial as well (added charts and created a demo yesterday).
I gave this issue an honest try, but got stuck. The tasks are now headless. But to make them synchronous, I tried a lot of things including using SubAnt to launch nested tasks from within the export task, but I kept getting classloading errors. I will adjust the summary and add the helpwanted keyword. if there is anybody out there who can solve the synchronous issue, I will be more than happy to accept the contribution.
*** Bug 139108 has been marked as a duplicate of this bug. ***
*** Bug 238383 has been marked as a duplicate of this bug. ***
I'd just like to know whether this problem will ever get solved - or whether there is a useful workaround. The documentation of the PDE/build tasks is not trivial and obviously unfinished as it starts with "??? introduction blah blah???". See it from a users point of view: -) He implements several plugins, which relate to each other. -) The plugin export wizard exists, and works very nice. However using the wizard cannot be automated. -) The wizard provides the option to save an ANT (obviously in order to automate exporting - what else should it be used for). This ANT task "basically" works, but has several drawbacks which leave it useless. 1) it's not possible to export multiple plugins. (I assume this is because config files generated for exporting get overwritten if multiple exports are scheduled). 2) it's not possible to do anything with the exported plugins, as they simply don't exist once the task finishes. I'm relatively new to plugin development so I might miss something. To be honest - if such basic stuff doesn't work it scares off developers. What's scaring me most is that this bug is known since 4 years now and there still seems to be no solution (although you know the request is popular). If it is that hard to make this task synchronous - maybe it is an option to rework the export wizard so it generates an ANT script using the proper PDE/build tasks?
(In reply to comment #22) > I'd just like to know whether this problem will ever get solved - or whether > there is a useful workaround. The documentation of the PDE/build tasks is not > trivial and obviously unfinished as it starts with "??? introduction blah > blah???". The workaround is to use the PDE Build tasks rather than the export tasks provided PDE that run in the IDE. See bug 238383#c1 for a description/pointer to the tasks.
Wouldn't it be a good idea to drop the export ANT task completely? Instead the export wizard could create an ANT script using the PDE build tasks. (They are not that simple, so this is a typical job for a wizard). I think it's somehow misleading if a wizard generates a script, which is basically useless. Beside of that - also the PDE build tasks only work within eclipse (I know - when running on the command line eclipse can be executed headless, but still .... it's tricky). In our concrete case we simply want to build / package / deploy an ODA driver (which by its nature is an eclipse plugin). We have a centralized build process (based on cruise control) which regularly builds our EAR. Right now we found no useful solution to automatically build the plugin (which is part of the EAR) on this build server. The workaround is, that the ODA developer builds the plugin locally, and checks it in. (Needless to say that this is ugly, as all sources should be build by a central build - we don't want to rely on developers who tend to forget to build/checkin stuff). The perfect solution would be, if the wizard creates an ANT script using only native ANT tasks, but likely this isn't as simple as it sounds. Anyways - thanks for the information. Chris
For those willing to automate the build of plugins or products, there are several tools that you could use. A while ago, I wrote a headless pde export and made available via http://eclipseassembly.sourceforge.net/ There is also ant4eclipse http://ant4eclipse.sourceforge.net Finally, you can also try http://help.eclipse.org/help32/index.jsp?topic=/org.eclipse.pde.doc.user/guide/tasks/pde_product_build.htm
As of now 'LATER' and 'REMIND' resolutions are no longer supported. Please reopen this bug if it is still valid for you.
