[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
[emf-dev] Re: Galileo build failed 
(Extending distribution to -dev lists...)

----------

The system works like this:

* Every new build produces a "category.xml" - it's not a valid XML document, just a fragment for the category contributed to the overall site*.xml

view-source:http://modeling.eclipse.org/modeling/emf/updates/milestones/emf-emf-2.5-S_r0/category.xml

* Each new build (say, for EMF (Core) 2.5 on milestones site) will push out the oldest of the 3 stable builds and keep only the new one + two older ones. For I/M builds, we keep the three most recent ones; for R, we keep two.

http://emft.eclipse.org/modeling/emf/updates/milestones/
http://modeling.eclipse.org/modeling/emf/updates/milestones/

* Every new build also collects a list of jars that are still required for the 1-3 builds kept on the site per project. This is done by storing a copy of each build's features/ and plugins/ folder ON THE BUILD SERVER. The list of jars to be kept is held on download.eclipse.org, and each build server produces its own list.

http://download.eclipse.org/modeling/emf/updates/milestones/jarlist.clean.modeling.eclipse.org.txt
http://download.eclipse.org/modeling/emf/updates/milestones/jarlist.clean.emft.eclipse.org.txt
http://download.eclipse.org/modeling/emf/updates/milestones/jarlist.clean.emf.torolab.ibm.com.txt

* This is done so that we don't end up with an ever-larger update site with irrelevant / unreferenced / obsolete jars. We need multiple files because not all builds are produced on the same server, and thus we'd either need to store all the update site jars + category.xml for each build on download.eclipse.org, or cheat by use of the listfile. I went the listfile route to save disc usage on download.eclipse.

* Every time a build is published, all jars NOT on the aggregate jarlist.clean*.txt files for a given site are deleted. All category.xml files for that site are merged to create the new site.xml files.

* The system, in theory, does its own maintenance and cleanup every time a new build is added to the pile. Also in theory, we should never end up with site.xmls that reference jars that are missing because the full list of required jars comes from Master zip (signed + packed jars) -> unpacked site on build server -> directory listing of included jars -> jarlist.clean*.txt files.

----------

So, the reason that EMF (Core) 2.4.0RC3 is still on the update site is that it's the second newest milestone/stable build for EMF (Core) 2.4. There's also 2.4.2RC1 on there. This is intentional.

What may not be intentional is that there are three site*.xml files in the site, and I'm not sure from which one the content.jar and artifacts.jar metadata are produced. I'd expect it to be the site.xml (73K) file which should include EVERYTHING, not the smaller site-galileo.xml (16K) or site-ganymede.xml (40K), but I'll have to verify that.

We may need to split these sites into separate streams for eclipse 3.4 and 3.5, if p2 or the amalgam/galileomatic is getting confused. This would also prevent end users from installing, say, CDO 2.0 onto Eclipse 3.4. The only complication here is that unlike, say, PDT, who spawned a /2.0/updates/ site, we'd have to do /e35/updates/ or /galileo/updates/ because everyone's at a different version number within a given project.

Of course renaming/moving update sites will also mean updating all your feature.xml files to point at the new site.

Nick


Dave Steinberg wrote: 
Hi guys,

I have no idea how Nick's scripts remove old builds and manifest entries. It seems odd that an old entry for a missing feature matters at all when you're trying to update something newer, but then, I have no idea how that works either.

Should I investigate, or can I hope Nick will?

Cheers,
Dave

--
Dave Steinberg
Rational Software - IBM Toronto Lab
905-413-3705 (t/l 313-3705)
mailto:davidms@xxxxxxxxxx


Inactive hide details for Richard Gronback ---02/05/2009 01:14:35 PM---I suspect the problem here is the cleanup script. The reRichard Gronback ---02/05/2009 01:14:35 PM---I suspect the problem here is the cleanup script. The referenced jar seems to be from 2.4.0RC3, which would make sense to no l


From:	
Richard Gronback <richard.gronback@xxxxxxxxxxx>

To:	
Ed Merks <ed.merks@xxxxxxxxx>, Dave Steinberg/Toronto/IBM@IBMCA

Cc:	
<nickboldt@xxxxxxxxx>, Anthony Hunter/Ottawa/IBM@IBMCA

Date:	
02/05/2009 01:14 PM

Subject:	
Re: Galileo build failed

------------------------------------------------------------------------



I suspect the problem here is the cleanup script. The referenced jar seems to be from 2.4.0RC3, which would make sense to no longer have on the site, although the entry in the site.xml persists.

- Rich


On 2/5/09 12:46 PM, "Ed Merks" <_ed.merks@xxxxxxxxxx> wrote:

            Dave,

            I'm not sure if some magical promotion step was missing or
            went astray...


            Richard Gronback wrote:

                        Sorry, was testing XSD (GEF error was expected).
                        XSD seems to install fine,
                        though exceptions are thrown due to the update
                        site problem mentioned
                        yesterday.

                        The file
                        features/org.eclipse.emf.all_2.4.0.v200806021643.jar
                        is referenced
                        in the site manifest, but doesn't exist.

                        Thanks,
                        Rich


                        [exec] install.feature:
                        [exec] [exec] !SESSION 2009-02-05 12:32:11.159
                        -----------------------------------------------
                        [exec] [exec] eclipse.buildId=I20090202-1535
                        [exec] [exec] java.fullversion=J2RE 1.5.0 IBM J9
                        2.3 Linux ppc-32
                        j9vmxp3223-20071007 (JIT enabled)
                        [exec] [exec] J9VM - 20071004_14218_bHdSMR
                        [exec] [exec] JIT - 20070820_1846ifx1_r8
                        [exec] [exec] GC - 200708_10
                        [exec] [exec] BootLoader constants: OS=linux,
                        ARCH=ppc, WS=gtk,
                        NL=en_US
                        [exec] [exec] Framework arguments: -application
                        org.eclipse.equinox.p2.director.app.application
                        -installIU
                        org.eclipse.xsd.sdk.feature.group -version
                        2.5.0.v200902031500
                        -profileProperties
                        org.eclipse.update.install.features=true
                        -metadataRepository_
                        __http://download.eclipse.org/modeling/emf/updates/milestones/_
                        -artifactRepository_
                        __http://download.eclipse.org/modeling/emf/updates/milestones/_
                        [exec] [exec] Command-line arguments: -os linux
                        -ws gtk -arch ppc
                        -application
                        org.eclipse.equinox.p2.director.app.application
                        -consoleLog
                        -installIU org.eclipse.xsd.sdk.feature.group
                        -version 2.5.0.v200902031500
                        -profileProperties
                        org.eclipse.update.install.features=true
                        -metadataRepository_
                        __http://download.eclipse.org/modeling/emf/updates/milestones/_
                        -artifactRepository_
                        __http://download.eclipse.org/modeling/emf/updates/milestones/_
                        [exec] [exec]
                        [exec] [exec] !ENTRY org.eclipse.equinox.p2.core
                        4 0 2009-02-05
                        12:32:25.186
                        [exec] [exec] !MESSAGE Provisioning exception
                        [exec] [exec] !STACK 1
                        [exec] [exec]
                        org.eclipse.equinox.internal.provisional.p2.core.ProvisionException:
                        Could
                        not connect to _
                        __http://download.eclipse.org/modeling/emf/updates/milestones/features/org.ecl_
                        ipse.emf.all_2.4.0.v200806021643.jar


                        On 2/5/09 12:33 PM, "Richard Gronback"
                        <_richard.gronback@xxxxxxxxxxxx>
                        <_mailto:richard.gronback@xxxxxxxxxxxx> wrote:



                                    Contribution org.eclipse.gef.sdk version
                                    3.5.0.v20080806-1610-7B7H-97OO5c7YCeEHSDd_OSWDJJV
                                    failed to install from_
                                    __http://download.eclipse.org/tools/gef/updates/milestones/_.


                                    Check the log file for more
                                    information:_
                                    __http://build.eclipse.org/galileo/build/galileo-I20090205-1228.log.txt_.









--
Nick Boldt :: http://wiki.eclipse.org/User:Nickb
Release Engineer :: Eclipse Modeling & Dash CBI