[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
[
List Home]
|
[pde-build-dev] Re: [osgi-dev] Continuous Integration of OSGi Bundles
|
I'm including pde-build-dev in the To: list in hopes of moving this to a more proper place. Please send replies to that list. :)
Jeff,
I think for optimal performance, the build server would keep a local copy of the repository and simply perform an update on it, instead of checking out projects each time. Our current system keeps local checkouts of each project, but it does so in a way that requires per-project updating. The optimal way would be to tell the builder to checkout a particular subversion path (
e.g. ,
http://svn.company.com/osgi-project/bundles) which contains some or all of the bundles within subdirectories (e.g.,
http://svn.company.com/osgi-project/bundles/com.company.somebundle), and then determine which bundles are contained within, if any. If the bundles path contained 150 bundle subdirectories, it would be much faster to update the root bundles path than to update each of the 150 bundles paths individually.
Perhaps our SVN environment is slow for some reason, or JavaSVN is slow, I haven't investigated very closely. At any rate, I would imagine an extended PDE Build would allow stuff like this to be configurable.
-Jeremy
On 4/30/07, Jeff McAffer <Jeff_McAffer@xxxxxxxxxx> wrote:
There are a number of changes in 3.3
but fundamentally the structure is the same. As for your performance
issue... Interesting. per-project check out with SVN is slow?
that does not bode well for incremental building of only those parts
that need to be built. The incremental approach seems to imply that
you would only check out the projects that need building and those which
are already built can just be used directly.
In any event, with the new provisioning
work that we are doing in Equinox we are looking to see if there are opportunities
to improve the overall build story. Input from other people with
realworld build problems (and solutions) would be great. That should
likely go on the equinox-dev or pde-build-dev lists as appropriate.
Jeff
Jeff,
Perhaps I'll look towards starting with the current PDE build, then. I'll
try to catch up on its lists. Are you aware of any large changes
since 3.2 was released?
The PDE part of our build really isn't the problem. The problem is
that I'm storing each bundle as a project checked out into an Eclipse workspace
(all headlessly). Updating the projects from subversion is much like
highlighting all projects in a workspace and selecting Team -> Update:
the update happens project-per-project. The much more efficient way
to do this would be to simply update the root directory of all the bundle
projects, which Subversion does very well. However, this change would
require some rearchitecting, and since we have other larger needs anyway
we want to just redo the whole thing.
One issue with the PDE export is that it's a mysterious black box of long
running processes. I'm sure that PDE Build has better build output,
since it's not intended as a UI operation. Also, the PDE export job
fails completely if any of the bundles have unresolved manifests (with
a CoreException). It will complete successfully if a bundle has compilation
problems, though. Optimally the export job would treat manifest resolution
failures like compilation failures.
Thanks,
Jeremy
On 4/30/07, Jeff McAffer <Jeff_McAffer@xxxxxxxxxx
>
wrote:
Jeremy,
There has been some discussion in the area of updating PDE build to do
much of what you have described. Currently however there are not
enough people in that community to do the required work. Do you think
you/your organization would have time to work in this area?
Jeff
p.s., In case you didn't know, the Export Deployable Plugins and Fragments
option in the Eclipse UI actually runs PDE build under the covers. Not
sure why performance is degrading. You might check that you are not
doing a build in the workspace (Look at the Project > Build Automatically
menu setting). That would be a waste since PDE build does its own
build.
Greetings,
We're looking to re-implement/replace our current continuous
integration system, which has been building our OSGi project for the
past 10 months or so. The current builder was quickly hacked together
over a period of a week, and we're starting to run into its
limitations now that our project has surpassed 150 bundles.
Some of the big requirements that we set for our current system were:
1. Automatic discovery of new bundle projects in the source
repository. We didn't want to configure the builder each time we
added a new bundle to the project.
2. Build as much as needed, but no more. Based on what was changed
since the previous build, we wanted to use OSGi dependency analysis to
figure out what needed to be built (i.e., changed bundles and bundles
depending upon them). Because of this need, the builder must be able
to fetch dependencies required for build from our bundle repository.
3. Resolve build dependencies based upon OSGi metadata (not, for
instance, Maven dependencies).
4. Build Eclipse PDE bundle projects, since that's what we use as our
development environment.
To cover these requirements, we wrote a custom continuous integration
system consisting of:
1. A repository scanner (written using JavaSVN) able to detect new
projects in our repository; and
2. A bundle builder based around Eclipse PDE export operations.
We did not use the PDE build system but rather do the equivalent of
launching an Eclipse instance, selecting bundles and invoking "Export
Deployable plug-ins and fragments".
This system has worked fairly well for us. However, it's becoming
slower and slower, and it is not easily extensible. We'd like to
be
able to make use of some existing builder for integrated
testing/reporting/etc.
I've noticed that a lot of people are using Maven to build OSGi
bundles. However, as far as I can tell, Maven requires its own
dependency format, and cannot be extended resolve based upon OSGi
dependency metadata. Please correct me if this is incorrect.
Does anyone have any systems similar to this, or any pointers to build
systems that can be easily extended to support these types of
features?
Thanks,
Jeremy Volkman
_______________________________________________
OSGi Developer Mail List
osgi-dev@xxxxxxxxxxxxx
http://www2.osgi.org/mailman/listinfo/osgi-dev
_______________________________________________
OSGi Developer Mail List
osgi-dev@xxxxxxxxxxxxx
http://www2.osgi.org/mailman/listinfo/osgi-dev
_______________________________________________
OSGi Developer Mail List
osgi-dev@xxxxxxxxxxxxx
http://www2.osgi.org/mailman/listinfo/osgi-dev
_______________________________________________
OSGi Developer Mail List
osgi-dev@xxxxxxxxxxxxx
http://www2.osgi.org/mailman/listinfo/osgi-dev
