Bug 233744 - [Provisioning] Improve PDT Releng & Website Resources
Summary: [Provisioning] Improve PDT Releng & Website Resources
Status: CLOSED FIXED
Alias: None
Product: z_Archived
Classification: Eclipse Foundation
Component: PDT (show other bugs)
Version: unspecified   Edit
Hardware: PC Linux
: P1 normal with 1 vote (vote)
Target Milestone: ---   Edit
Assignee: Alon Peled CLA
QA Contact:
URL:
Whiteboard:
Keywords: contributed
: 241341 (view as bug list)
Depends on: 228145
Blocks: 231849 238960 248995 251304
  Show dependency tree
 
Reported: 2008-05-23 14:55 EDT by Nick Boldt CLA
Modified: 2020-05-14 10:15 EDT (History)
9 users (show)

See Also:
nboldt: iplog+


Attachments
delete, create, and move, but no backup (2.09 KB, application/x-sh)
2008-05-28 00:40 EDT, Nick Boldt CLA
no flags Details
updated script (2.18 KB, application/x-shellscript)
2008-05-28 09:42 EDT, Denis Roy CLA
no flags Details
updated script with one more plugin (2.25 KB, application/octet-stream)
2008-05-28 10:32 EDT, Roy Ganor CLA
no flags Details
updated script, including releng changes (2.18 KB, application/octet-stream)
2008-05-28 10:42 EDT, Roy Ganor CLA
no flags Details
patch to look for library.xml in one of two places (1.30 KB, patch)
2008-07-15 17:06 EDT, Nick Boldt CLA
no flags Details | Diff
website and releng projects as of 2008-09-10 16:19:07 (37.72 KB, application/zip)
2008-09-11 21:39 EDT, Nick Boldt CLA
nboldt: iplog+
Details
patch to change signing to run as apeled in tools/pdt instead of nickb in tools/gef (1.22 KB, patch)
2008-09-15 23:07 EDT, Nick Boldt CLA
no flags Details | Diff
mylyn/context/zip (5.13 KB, application/octet-stream)
2008-09-15 23:07 EDT, Nick Boldt CLA
no flags Details
patch to update build/promote defaults: run as apeled and default contact email to alon@zend.com (1.07 KB, patch)
2008-09-15 23:20 EDT, Nick Boldt CLA
no flags Details | Diff
trivial change to readme.html (1.42 KB, patch)
2008-09-15 23:30 EDT, Nick Boldt CLA
no flags Details | Diff
all the missing variables and includes in _projectCommon.php (2.83 KB, patch)
2008-09-22 16:49 EDT, Nick Boldt CLA
no flags Details | Diff
do not use Eclipse 3.5 milestone builds for building PDT (912 bytes, patch)
2008-09-29 14:48 EDT, Nick Boldt CLA
no flags Details | Diff
patch to display nightlies at the bottom of the new downloads page (931 bytes, patch)
2008-10-02 10:47 EDT, Nick Boldt CLA
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Nick Boldt CLA 2008-05-23 14:55:34 EDT
(In reply to bug 166178 comment 18)
> Hi Nick, I tried to release an integration yesterday. Our releng scripts are
> broken for some reason (probably the dependencies cleanup we have done lately).
> I will look into it this week.

That or changes in basebuilder [1] you need to adopt. If you had publicly available build logs I could have a look...

[1]http://wiki.eclipse.org/Platform-releng-basebuilder
 
> Can you tell me what's the risk in refactoring our packages to this one?
> I am trying to think about all the scripts we have in it sounds like a big
> change, that can cause the releng to be messed up.

Unless you have hardcoded path dependencies (which is BAD!), then all you'd need to do is update your map file(s) and whatever scripts you might use to extract stuff from CVS... eg:

cvs -d :pserver:anonymous@dev.eclipse.org -Q ex -r someTag -d org.eclipse.php.foo org.eclipse.php.foo

becomes:

cvs -d :pserver:anonymous@dev.eclipse.org -Q ex -r someTag -d org.eclipse.php.foo org.eclipse.pdt/plugins/org.eclipse.php.foo

> BTW - I don't quite understand the needed change. by your convention:
> > /cvsroot/tools/org.eclipse.php.core ==>
> > /cvsroot/tools/org.eclipse.pdt/org.eclipse.core

No, I'm not suggesting you refactor everything -- there's another bug for that, to avoid the apache mirror problems w/ the .php namespace (bug 203794). :) 

I'm just suggesting you NEST your cvs modules under a parent folder:

/cvsroot/tools/org.eclipse.php.core ==>
/cvsroot/tools/org.eclipse.pdt/plugins/org.eclipse.php.core

The benefit here is that checking everything out from CVS can be done with a single statement, instead of umpteen, and the /cvsroot/tools/ directory is cleaner. (Admittedly you can also check everything out using Team Project Set files [2], but when you need a single overarching project in your workspace for everything, this is the best -- and it works w/ .psf files too.)

[2] http://wiki.eclipse.org/WTP/Team_Project_Set_(.psf)

Now if I had my druthers (as that annoying busybody who keeps harassing you guys about process, planning, usability and website content! :) ), you'd reorg your CVS (again, no code refactoring, just moving things around) according to this system [3]:

[3]http://wiki.eclipse.org/Modeling_Project_Releng/Plugin_And_Feature_Files#Directory_Structure

BEFORE:

/cvsroot/tools/
  org.eclipse.php
  org.eclipse.php.ui
  org.eclipse.php.core
  org.eclipse.php.delivery  -- not sure what this plugin's for?
  org.eclipse.php.help      -- is this documentation, or pdt's integration w/ Eclipse help? Or both?
  org.eclipse.php.debug.core
  org.eclipse.php.debug.daemon
  org.eclipse.php.debug.ui
  org.eclipse.php.server.core
  org.eclipse.php.server.ui
  org.eclipse.php.server.apache.core
  org.eclipse.php.server.apache.ui
  org.eclipse.php-feature
  org.eclipse.php.sdk-feature
  org.eclipse.php.test
  org.eclipse.php.test-feature
  org.eclipse.php-releng

AFTER:

/cvsroot/tools/org.eclipse.pdt/
  examples/ [empty, for now]
  doc/ [empty, for now]
  plugins/org.eclipse.php
  plugins/org.eclipse.php.ui
  plugins/org.eclipse.php.core
  plugins/org.eclipse.php.delivery  -- not sure what this plugin's for?
  plugins/org.eclipse.php.help  -- assuming this belongs in plugins/, not in doc/
  plugins/org.eclipse.php.debug.core
  plugins/org.eclipse.php.debug.daemon
  plugins/org.eclipse.php.debug.ui
  plugins/org.eclipse.php.server.core
  plugins/org.eclipse.php.server.ui
  plugins/org.eclipse.php.server.apache.core
  plugins/org.eclipse.php.server.apache.ui
  features/org.eclipse.php-feature
  features/org.eclipse.php.sdk-feature
  tests/org.eclipse.php.test
  tests/org.eclipse.php.test-feature
  plugins/org.eclipse.php-releng -- replaced by a new module so that both the old and new build systems can run in parallel to ease migration (did this for GEF and GMF and it worked great)
/cvsroot/tools/
  org.eclipse.pdt.releng -- new module based on http://wiki.eclipse.org/Modeling_Project_Releng/Releng_Module

But that's not crucial, just good practice for CVS cleanliness.
Comment 1 Roy Ganor CLA 2008-05-24 15:18:27 EDT
Let's focus on the structure again:

DELETE UNUSED Fragments:

1. /cvsroot/tools/org.eclipse.php.delivery
2. /cvsroot/tools/org.eclipse.php.server.apache.core
3. /cvsroot/tools/org.eclipse.php.server.apache.ui


CREATE NEW Folders:

1. /cvsroot/tools/org.eclipse.pdt/examples/ 
2. /cvsroot/tools/org.eclipse.pdt/doc/
3. /cvsroot/tools/org.eclipse.pdt/plugins/
4. /cvsroot/tools/org.eclipse.pdt/features/
5. /cvsroot/tools/org.eclipse.pdt/tests/
6. /cvsroot/tools/org.eclipse.pdt/org.eclipse.pdt.releng/


MOVE Fragments:
1. /cvsroot/tools/org.eclipse.php
==> 
/cvsroot/tools/org.eclipse.pdt/plugins/org.eclipse.php

2. /cvsroot/tools/org.eclipse.php.core
==> 
/cvsroot/tools/org.eclipse.pdt/plugins/org.eclipse.php.core

3. /cvsroot/tools/org.eclipse.php.ui
==> 
/cvsroot/tools/org.eclipse.pdt/plugins/org.eclipse.php.ui

4. /cvsroot/tools/org.eclipse.php.help
==> 
/cvsroot/tools/org.eclipse.pdt/plugins/org.eclipse.php.help

5. /cvsroot/tools/org.eclipse.php.debug.core
==> 
/cvsroot/tools/org.eclipse.pdt/plugins/org.eclipse.php.debug.core

6. /cvsroot/tools/org.eclipse.php.debug.ui
==> 
/cvsroot/tools/org.eclipse.pdt/plugins/org.eclipse.php.debug.ui

7. /cvsroot/tools/org.eclipse.php.debug.daemon
==> 
/cvsroot/tools/org.eclipse.pdt/plugins/org.eclipse.php.debug.daemon

8. /cvsroot/tools/org.eclipse.php.server.core
==> 
/cvsroot/tools/org.eclipse.pdt/plugins/org.eclipse.php.server.core

9. /cvsroot/tools/org.eclipse.php.server.ui
==> 
/cvsroot/tools/org.eclipse.pdt/plugins/org.eclipse.php.server.ui

10. /cvsroot/tools/org.eclipse.php.test-releng
==> 
/cvsroot/tools/org.eclipse.pdt/plugins/org.eclipse.php.test-releng

11. /cvsroot/tools/org.eclipse.php-feature
==> 
/cvsroot/tools/org.eclipse.pdt/features/org.eclipse.php-feature

12. /cvsroot/tools/org.eclipse.php.sdk-feature
==> 
/cvsroot/tools/org.eclipse.pdt/features/org.eclipse.php.sdk-feature

13. /cvsroot/tools/org.eclipse.php.test
==> 
/cvsroot/tools/org.eclipse.pdt/tests/org.eclipse.php.test

14. /cvsroot/tools/org.eclipse.php.test-feature
==> 
/cvsroot/tools/org.eclipse.pdt/tests/org.eclipse.php.test-feature

1. I don't see any thumb rule here so I detailed everything.
2. We should have an instruction page before the move - Can you clone the EMF one to the PDT project?
3. I prefer that the webmaster will orchestrate it instead of me ;) how do we actually execute it?


Comment 2 Denis Roy CLA 2008-05-26 09:10:27 EDT
(In reply to comment #1)
> 3. I prefer that the webmaster will orchestrate it instead of me ;) how do we
> actually execute it?

Yes, we'll help here.  Comment 1 is what needs to be done, we'll write a small script that takes a backup of your CVS tree then execute all the mkdir and mv commands.   That way we can easily back out of this if anything explodes.

Just confirm comment 1 is what needs to be done, we'll script it.  Then tell us 'when' you want us to run it, so you can test.
Comment 3 Nick Boldt CLA 2008-05-28 00:40:28 EDT
Created attachment 102296 [details]
delete, create, and move, but no backup

(In reply to comment #2)
> Yes, we'll help here.  Comment 1 is what needs to be done, we'll write a small
> script that takes a backup of your CVS tree then execute all the mkdir and mv
> commands.   That way we can easily back out of this if anything explodes.

Minus the backing up part (because I don't know where you'd put that), the attached script will do exactly that.

> Just confirm comment 1 is what needs to be done, we'll script it.  Then tell us
> 'when' you want us to run it, so you can test.

See comments in the script about last move; other than that it looks fine, but I'll leave it to Roy to give the final +1.

As to WHEN, I'd suggest a few possible routes:

a) after Ganymede drops
b) this week, so that Friday's Bug Day can be used for me to produce a new .releng project for PDT
c) after your 1.1M1 is complete, so that people will survive if there's a bit of a pause between builds, and have something to use/test that's more stable/complete than an N or I build

I think we should also discuss WHERE we'll be running these builds as I'm out of space on emft.eclipse.org. Denis, is there a free vserver slot I can have for non-Modeling builds like PDT? Perhaps a tools.eclipse.org, which could also house VE, GEF, and others one day?
Comment 4 Nick Boldt CLA 2008-05-28 00:48:35 EDT
> 2. We should have an instruction page before the move - Can you clone the EMF
one to the PDT project?

Only reorg doc I can find for EMF is this one:

http://wiki.eclipse.org/EMF/EMF_2.3/New_Features_Migration_Guide#EMF_2.3M6_vs._2.3M7 

But that's only relevant if you split your features up, as in bug 197573 (which I'd be happy to contribute a patch for one day).

A CVS reorg really only affects a document like this:

http://wiki.eclipse.org/EMF/Setting_up_a_development_environment

which, for PDT, can probably be simplified down to just providing a .psf file, like this one:

http://wiki.eclipse.org/WTP/Team_Project_Set_%28.psf%29

Or... were you thinking something else?
Comment 5 Denis Roy CLA 2008-05-28 09:42:03 EDT
Created attachment 102360 [details]
updated script

Here's the script, with the backup line.

As for a vserver, I'm not opposed, but wouldn't it be better to do this on build.eclipse.org?
Comment 6 Roy Ganor CLA 2008-05-28 10:32:21 EDT
Created attachment 102381 [details]
updated script with one more plugin
Comment 7 Roy Ganor CLA 2008-05-28 10:42:36 EDT
Created attachment 102385 [details]
updated script, including releng changes

OK some more changes (I hope this will be a final release):

1. regarding Nick's question - org.eclipse.php-releng should be moved as well
2. test-feature should be located in the features folder
Comment 8 Nick Boldt CLA 2008-05-31 02:53:16 EDT
> > I think we should also discuss WHERE we'll be running these builds as I'm out
> > of space on emft.eclipse.org. Denis, is there a free vserver slot I can have
> > for non-Modeling builds like PDT? Perhaps a tools.eclipse.org, which could 
> > also house VE, GEF, and others one day?
> As for a vserver, I'm not opposed, but wouldn't it be better to do this on
> build.eclipse.org?

Perhaps. I've had a long-standing TODO to migrate my build system to work on build.eclipse.org -- had some problems early on involving paths, JDKs (versions & missing .so files), write permissions, and possibly virtual framebuffers (for headless eclipse GUI testing)... but I think most of those are sorted by now. 

I suppose I could look at running Tools.GEF on build.eclipse, and if that works, Tools.PDT should be no major problem -- in fact, it would save time and bandwidth during builds.

I take it your preference would be to run on build.eclipse.org, in order to take advantage of more ram and more disk, at the expense of more sharing (eg., running multiple parallel builds) ?
Comment 9 Denis Roy CLA 2008-06-04 13:27:41 EDT
(In reply to comment #8)
> I take it your preference would be to run on build.eclipse.org

I think it doesn't really matter.  I simply wanted to avoid 'wasting' project vservers for the sole purpose of producing builds, as the number of slots for vservers is limited.

In the end, use whichever tool works best for you.
Comment 10 Nick Boldt CLA 2008-06-04 16:49:34 EDT
(In reply to comment #9)
> I simply wanted to avoid 'wasting' project vservers for
> the sole purpose of producing builds.

Uh, what else are these vservers used for, if not for running builds and testing code?

That might sound like a dumb question, but I really thought that's what they were created for. ecf.eclipse.org runs builds (and Remy's IRC bot, KOS-MOS); dsdp.eclipse.org runs builds. emft.eclipse.org runs builds. build.eclipse.org runs builds (and signing).

In all seriousness, I'm curious -- for what other purposes are the vservers used?
Comment 11 Nick Boldt CLA 2008-06-05 15:01:39 EDT
> Hi Nick, Denis, 
> I would like to schedule a date to execute this resource refactoring.  
> What is your preferred date between June 15th and June 19th? 
> Roy Ganor 

My preference in ascending order:
June 19: after all the bits are locked and ready for Ganymede
June 26: after Ganymede GAs 
July 03: after the party dies down :)
Comment 12 Denis Roy CLA 2008-06-06 10:54:28 EDT
Yeah, after Ganymede sounds good (and lower risk).

> In all seriousness, I'm curious -- for what other purposes are the vservers
> used?

To run apps you can't run on build -- for instance, Mylyn runs a few Bugzillas to test their connectors, Babel runs the translation website, RAP runs some RAP apps, ECF runs some communications software stuff, Dash runs the commits stuff, etc. etc.
Comment 13 Nick Boldt CLA 2008-06-06 11:01:53 EDT
(In reply to comment #12)
> > In all seriousness, I'm curious -- for what other purposes are the vservers
> To run apps you can't run on build -- for instance, Mylyn runs a few Bugzillas
> to test their connectors, Babel runs the translation website, RAP runs some RAP
> apps, ECF runs some communications software stuff, Dash runs the commits stuff,

So, EMF projects like Teneo which need to connect to a database for testing would need to continue to run on emft.eclipse (or else have access to creating/destroying mysql tables on build.eclipse). Beyond that... emft.eclipse is just a build server these days, so I should be able to port them over to build.eclipse one day. 

Of course with build.eclipse crashing every couple days, do we really want the extra load of 20 more builds? :) Will it handle the extra abuse?

Comment 14 Roy Ganor CLA 2008-06-08 14:27:25 EDT
ok, save the day!

June 26: after Ganymede GAs 
Comment 15 Nick Boldt CLA 2008-06-23 10:35:52 EDT
(In reply to comment #14)
> ok, save the day!
> June 26: after Ganymede GAs 

June 26 is now Build Harder day:

http://wiki.eclipse.org/Build_Workshop_2:_Build_Harder

So, if we do this early in the day, we can actually create a vserver for PDT and test it to verify the image is complete. PDT only needs platform + WTP to build, correct? And 3rd party dependencies (non-Eclipse, non-EPL) when compiling?

It's also been suggested that there be an update site for the 1.1 stream, and once you're running my system on a vserver at eclipse.org, that will be incorporated into the publishing process, so that you can have Releases, Intergrations, and Milestones as three separate and automatically maintained sites.

Comment 16 Roy Ganor CLA 2008-06-23 10:57:36 EDT
Indeed PDT requires Eclipse + WTP (and its dependencies) + DLTK. no third party is required.

We can do it early that day. sounds great to me (thank you for being on top of this issue)

Integration site is more than welcome! (although I will not recommend using it by anyone who works on a production app)

Good luck in the "Build Harder" day, I am very excited to see that Eclipse will share the releng knowledge between its projects. We will pay attention to this day's results to see how PDT is going to catch up!

We haven't thought about who's going to actually do it... Denis / Nick?

Comment 17 Nick Boldt CLA 2008-06-24 13:15:09 EDT
(In reply to comment #16)
> We haven't thought about who's going to actually do it... Denis / Nick?

I'm not a PDT committer, so it'll have to be Denis that moves stuff around in CVS. 

If you want me to have access to your releng stuff (as I do for other projects) you might want to start the committer election process now. Or wait until I've submitted enough patches... 

Comment 18 Roy Ganor CLA 2008-06-25 10:47:32 EDT
You're right, I can't do it either since the new folders we should create.

Desnis, I know that you're busy these days but it is for a good cause :) can you please commit the changes?



(In reply to comment #17)
> (In reply to comment #16)
> > We haven't thought about who's going to actually do it... Denis / Nick?
> I'm not a PDT committer, so it'll have to be Denis that moves stuff around in
> CVS. 
> If you want me to have access to your releng stuff (as I do for other projects)
> you might want to start the committer election process now. Or wait until I've
> submitted enough patches... 

Comment 19 Roy Ganor CLA 2008-06-26 11:18:17 EDT
thank you guys,

this page should help when starting to develop Eclipse PDT 

http://wiki.eclipse.org/PDT_Development_Environment

(actually it is a clone from the EMF project)
Comment 20 ????? ???????? CLA 2008-06-26 12:36:09 EDT
Roy, please do not mark this bug RESOLVED, FIXED at least until there is a live Integration builds update site.
Comment 21 Nick Boldt CLA 2008-06-26 23:25:20 EDT
Reopening. 

We still need:

+ a working build
+ a usable psf file, linked from the wiki in comment 19
+ release notes w/ cvs/bugzilla integration
+ ...
Comment 22 Roy Ganor CLA 2008-06-28 08:37:44 EDT
(In reply to comment #21)
> Reopening. 
> We still need:
> + a working build
> + a usable psf file, linked from the wiki in comment 19
> + release notes w/ cvs/bugzilla integration
> + ...

oppps (actually I wanted to drop this issue from Denis/Matt tasks list)


Comment 23 Nick Boldt CLA 2008-06-28 13:06:28 EDT
(In reply to comment #22)
> oppps (actually I wanted to drop this issue from Denis/Matt tasks list)

Then you should just remove 'em from cc: ...

Comment 24 Roy Ganor CLA 2008-07-01 03:13:06 EDT
(In reply to comment #21)
> Reopening. 
> We still need:
> + a working build
> + a usable psf file, linked from the wiki in comment 19
> + release notes w/ cvs/bugzilla integration
> + ...

ok, status is:
1. a working build is ready under our downloads page
2. a usable psf file is ready under the Wiki's developer zone 
3. release notes automation is NOT ready
4. update site generation is NOT ready
5. Unit tests result are NOT supplied
6. All-in-one package is NOT supplied yet

anothing else I have forgot? 
Comment 25 ????? ???????? CLA 2008-07-01 03:57:58 EDT
(In reply to comment #16)
> Integration site is more than welcome! (although I will not recommend using it
> by anyone who works on a production app)
Comment 26 Andreas Goetz CLA 2008-07-07 15:53:28 EDT
+1 for update site as to allow me stay updated as the current version is unusable for me (bug 215099)
Comment 27 Nick Boldt CLA 2008-07-10 01:54:41 EDT
First attempt -- build isn't perfect yet, but the harness is in place.

http://emft.eclipse.org/pdt/downloads/?sortBy=date -- I've copied over two recent 2.0 builds, and attempted to run a third using my new o.e.pdt.releng project.
http://emft.eclipse.org/pdt/build/ -- UI for running builds

Because I'm not a PDT committer, and because I like to work from CVS, the source for the website and .releng project are here:

/cvsroot/modeling/releng-common/templates/org.eclipse.pdt.website
/cvsroot/modeling/releng-common/templates/org.eclipse.pdt.releng

Tomorrow I'll see if I can get the build moving along further. Eventually I'll bundle all the code up and attach it here.

Has anyone in PDT contacted the EPP folks about starting an EPP PDT bundle [0], to replace your current "all-in-one" [1] zips?

[0] http://wiki.eclipse.org/EPP/How_to_create_a_package
[1] http://download.eclipse.org/tools/pdt/downloads/release.php?release=R20080603
Comment 28 Roy Ganor CLA 2008-07-10 02:26:37 EDT
Thanks for asking about EPP, we already created an issue: Bug #238960. you are more than welcome to join the thread :)

About the web interface - it looks promising! although my visit was short (5 sec.), Alon probably join this thread and have real feedback!

Thanks a lot for a great job! looking forward see your code running on the nightly!






Comment 29 Alon Peled CLA 2008-07-12 12:36:06 EDT
I must say this looks like a very promising start!

At which stage can I ask/comment about the web interface? I wouldn't like to comment on work in progress, that is about to change
Comment 30 Nick Boldt CLA 2008-07-12 19:01:03 EDT
(In reply to comment #29)
> I must say this looks like a very promising start!
> 
> At which stage can I ask/comment about the web interface? I wouldn't like to
> comment on work in progress, that is about to change

Comments welcome, but bear in mind that this early prototype is based on the Modeling/GEF build, so I won't be changing much unless it's really a problem. 

I am open to requests for the Common Build Infra (the Dash project, running on build.eclipse) as that one needs a facelift and will need to change anyway. Please file such requests in bug 238626.

(You can think of the two build systems as v1.x and v2.0. 1.x is in maintenance mode. 2.0 is still in the planning stages, intended to be rolled out later this year (eg., 4Q) if possible. I'm attempting to get you building with the current 1.x version so you can get used to it and use if for PDT 2.0's release in September.)
Comment 31 Nick Boldt CLA 2008-07-15 00:41:33 EDT
Currently, my version of the PDT build is failing because your test feature does not include org.eclipse.test or org.eclipse.ant.optional.junit. I've tried a few workarounds, but the simplest fix is to add this to you test feature:

	<plugin id="org.eclipse.test"
	        download-size="0"
	        install-size="0"
	        version="0.0.0" />

	<plugin id="org.eclipse.ant.optional.junit"
	        download-size="0"
	        install-size="0"
	        version="0.0.0"
	        fragment="true" />

Can you add this to your test feature? If you're worried about breaking your current build, you can copy that feature and rename its folder in CVS (eg., org.eclipse.php.test_feature) and I can build from that instead.

If you'd like a full feature-in-a-zip (with this modification), I can attach one. 

In lighter news, the SDK & Runtime are building OK:

http://emft.eclipse.org/pdt/downloads/drops/2.0.0/N200807142309/eclipse/N200807142309/

Comment 32 Michael Spector CLA 2008-07-15 01:18:36 EDT
(In reply to comment #31)

Added.
Comment 33 Nick Boldt CLA 2008-07-15 17:06:39 EDT
Created attachment 107530 [details]
patch to look for library.xml in one of two places

the new pdt build is breaking due to this error:

     [exec] BUILD FAILED
     [exec] /home/www-data/build/pdt/downloads/drops/2.0.0/N200807151623/testing/N200807151623/testing/test.xml:109: The following error occurred while executing this line:
     [exec] /home/www-data/build/pdt/downloads/drops/2.0.0/N200807151623/testing/N200807151623/testing/test.xml:85: The following error occurred while executing this line:
     [exec] /home/www-data/build/pdt/downloads/drops/2.0.0/N200807151623/testing/N200807151623/testing/test.xml:70: The following error occurred while executing this line:
     [exec] /home/www-data/build/pdt/downloads/drops/2.0.0/N200807151623/testing/N200807151623/testing/target/eclipse/dropins/eclipse/plugins/org.eclipse.php.test_2.0.0.v20080709/test.xml:23: The following error occurred while executing this line:
     [exec] java.io.FileNotFoundException: /home/www-data/build/pdt/downloads/drops/2.0.0/N200807151623/testing/N200807151623/testing/target/eclipse/plugins/org.eclipse.php.test_2.0.0.v20080709/library.xml (No such file or directory)

The problem is that the library.xml file is in the new Eclipse 3.4-standard eclipse/dropins/eclipse/plugins/org.eclipse.php.test_2.0.0.v20080709, not in eclipse/plugins/org.eclipse.php.test_2.0.0.v20080709.

This patch should let Ant decide which file/path is valid and assign the property based on the correct path. If the new path doesn't exist, it will fall back to the old path (so that your current build should still work after this change).
Comment 34 Nick Boldt CLA 2008-07-15 22:50:08 EDT
Meanwhile, the build looks good. I've got signing, packing, digesting, and p2 metadata generation working. Here's a build you can try:

http://emft.eclipse.org/pdt/downloads/?project=pdt&showAll=1&hlbuild=I200807151712#I200807151712
http://emft.eclipse.org/tools/pdt/downloads/drops/2.0.0/I200807151712/

And here's an update site you can test:

http://emft.eclipse.org/tools/pdt/updates/interim/

If someone can either create a fork of the test plugin (/cvsroot/tools/org.eclipse.pdt/tests/org.eclipse.php.test/) or apply the patch in comment 33, I can try the testing-while-building part of the system. 

--

After that we only need to set up access permissions so you guys can use it without me... and we're done here. :)
Comment 35 Roy Ganor CLA 2008-07-16 03:57:31 EDT
From now on Alon should handle and support Nick's requests,
Good luck Alon!
Comment 36 Nick Boldt CLA 2008-07-16 12:07:30 EDT
Alon: please find me online via one of these many methods [1], so that we can IM -- I find that much more efficient than email/bug report conversation, especially for iterative development (fix a file, spin a build, investigate results, fix a file, spin a build...). 

--

After the last patch was applied I re-ran the tests and appear to be getting a lot of undefined variables [2].

[1] http://wiki.eclipse.org/User:nickb
[2] http://emft.eclipse.org/pdt/downloads/drops/2.0.0/N200807160214/buildlog.txt

I think the next step would either be to look at how you currently run your tests and where these variables are set ... or better yet, replace your test.xml and library.xml with more up-to-date versions from basebuilder & my system.

I can submit a patch for my proposed change, but I need to know if all your tests as currently assembled can be run through a UI thread (headless Eclipse + virtual framebuffer to synthesize the UI). 

As this patch would be a breaking change from your current tests' operation, I'm thinking rather than replace your test.xml with my changes we should just commit an alternate version, eg., test-new.xml.
Comment 37 Nick Boldt CLA 2008-07-16 22:18:41 EDT
Trying a new approach (using a whole new test.xml [1] & basebuilder's library.xml to run all three suites -- base, headless, and ui), but now I'm stuck because your tests won't compile [2]. Is there a tag other than HEAD I should use for the o.e.php.test plugin?

[1]http://dev.eclipse.org/viewcvs/index.cgi/releng-common/templates/org.eclipse.pdt.releng/builder/tests/test-new.xml?revision=1.1&root=Modeling_Project&view=markup
[2]http://emft.eclipse.org/pdt/downloads/drops/2.0.0/N200807161728/buildlog.txt
Comment 38 Alon Peled CLA 2008-07-17 02:56:36 EDT
I am afraid the code was broken due to changes based on DLTK. All of our code is tightly coupled with that of DLTK, and we need to use their latest integration build for the nightly builds.
I can create a tag so you can stabilize the nightly builds, but sooner or later we will probably need an even newer DLTK build for the other plugins.
Comment 39 Nick Boldt CLA 2008-07-23 12:00:08 EDT
(In reply to comment #38)
> I am afraid the code was broken due to changes based on DLTK. All of our code
> is tightly coupled with that of DLTK, and we need to use their latest
> integration build for the nightly builds.
> I can create a tag so you can stabilize the nightly builds, but sooner or later
> we will probably need an even newer DLTK build for the other plugins.

Running with the latest DLTK 1.0 I build [0], I get this logged [1] error. Do I need to use newer tags than are in the current map [2]? 

[javac] 164. ERROR in /home/www-data/build/pdt/downloads/drops/2.0.0/N200807231041/eclipse/plugins/org.eclipse.php.core/src/org/eclipse/php/internal/core/compiler/ast/parser/AbstractASTParser.java (at line 73)
[javac] 	} catch (CoreException e) {
[javac] 	         ^^^^^^^^^^^^^
[javac] Unreachable catch block for CoreException. This exception is never thrown from the try statement body

[0]http://emft.eclipse.org/pdt/downloads/drops/2.0.0/N200807231041/build.cfg
[1]http://emft.eclipse.org/pdt/downloads/drops/2.0.0/N200807231041/buildlog.txt
[2]http://emft.eclipse.org/pdt/downloads/drops/2.0.0/N200807231041/directory.txt
Comment 40 Alon Peled CLA 2008-07-24 02:19:28 EDT
It looks like the tags in the map file you are using are not up to date.
The current tags should compile.
Comment 41 Nick Boldt CLA 2008-07-26 23:13:31 EDT
(In reply to comment #40)
> It looks like the tags in the map file you are using are not up to date.
> The current tags should compile.

OK, using the latest tags, everything compiles but the tests won't run:

http://emft.eclipse.org/pdt//downloads/drops//2.0.0/N200807261335/testresults/html/org.eclipse.php.test_linux.gtk.html

Any ideas?

Comment 42 Nick Boldt CLA 2008-07-27 19:07:14 EDT
(In reply to comment #41)
> OK, using the latest tags, everything compiles but the tests won't run:
> http://emft.eclipse.org/pdt//downloads/drops//2.0.0/N200807261335/testresults/html/org.eclipse.php.test_linux.gtk.html
> Any ideas?

I notice today's posted N build also has no successful tests:

http://download.eclipse.org/tools/pdt/downloads/drops/N20080727/results/TEST-php.html

(I'm glad it's not just me!)

Is it possible that these tests are missing some runtime jars, eg., xmlunit [1]?

[1]http://wiki.eclipse.org/Modeling_Project_Releng/Building#java.lang.Exception:_Could_not_find_plugin_.22org.eclipse.emf.mwe.tests.22

Comment 43 Nick Boldt CLA 2008-07-30 03:45:54 EDT
Spoke w/ Alon tonight / this morning, and here's a short list of possible issues to explore:

* newer releng.basebuilder tag (http://wiki.eclipse.org/Platform-releng-basebuilder) - find out what's changed between 3.3 and 3.4
* xmlunit missing from classpath (bug 240989)
* junit version change (3.8 -> 4.0)
* text.xml in test plugin (diff with latest in basebuilder/eclipsebuilder/o.e.test)
* buildAll.xml/customTargets.xml/customTest.xml/etc. in pdt.releng
Comment 44 Andreas Goetz CLA 2008-08-30 09:46:40 EDT
Any news on this one? Still no 2.0 update site available- this is really painful and honestly doesn't encourage external contributions...
Comment 45 Nick Boldt CLA 2008-08-30 11:59:21 EDT
(In reply to comment #44)
> Any news on this one? Still no 2.0 update site available- this is really
> painful and honestly doesn't encourage external contributions...

Other than the JUnits failing (prolly due to the removal of XMLUnit?) my build system works great. If someone in PDT can either confirm for me that *their* build now works for the JUnits or help me fix mine (same tests, same method to run them) then we can move forward here.

http://emft.eclipse.org/pdt/downloads/

As to an update site, here's the interim one I threw up last month:

http://emft.eclipse.org/tools/pdt/updates/interim/

Because I'm not a PDT committer, I can't publish this update site on download.eclipe.org (unless I put it under modeling/ not tools/pdt/). I can keep publishing a PDT update site on emft.eclipse.org, but really, the JUnit problem needs to be sorted out first,

Bottom line -- I've done all I can without direct intervention from the PDT folks. Now we need to work together to move this forward.

Comment 46 Roy Ganor CLA 2008-09-01 05:06:50 EDT
*** Bug 241341 has been marked as a duplicate of this bug. ***
Comment 47 Nick Boldt CLA 2008-09-02 19:31:37 EDT
Roy, Alon:

Seems like most of the posts in the PDT newsgroup and mailing list are "What build contains the fix for such-and-such bug?" If you adopt my build system, you can tell people to go look in the release notes for such information, as it'll be updated w/ every published I, M, S, or R build. And bugzilla will be updated automatically w/ the build ID of the first build to contain the fix, too. 

If you don't consider the JUnit problem to be blocking, we can move forward w/ having you guys run and publish builds on an eclipse.org server. Once all the permissions are sorted out and the first build is promoted, it pretty much just runs itself (ie., we can set up untended scheduled weekly I builds).

Find me online (wiki.eclipse.org/User:nickb), reply here, or send me a note with how you'd like to proceed -- and when -- so I can plan to be available for you.
Comment 48 Alon Peled CLA 2008-09-03 03:10:36 EDT
I totally agree that the JUnit problem is not a blocker, and sorry that we didn't discuss this sooner. I would like to continue and adopt your build system, as soon as possible. I believe I will be available to do this at the end of next week, or early in the week after. If you can specify what we need to do on our side in order to continue, that will be very useful and help us plan our schedule better.
Comment 49 Andreas Goetz CLA 2008-09-03 12:33:30 EDT
Nick,

thanks a lot for providing this site and the outstanding level of support!

I cannot however install PDT 2.0 from your site- not sure this is a local problem or one with the build:

An error occurred while collecting items to be installed
  Error closing the output stream for org.eclipse.php.core/osgi.bundle/2.0.0.v20080722 on repository file:/C:/eclipse/. 
  Error unzipping D:\Temp\org.eclipse.php.core_2.0.0.v2008072260168.jar: Invalid zip file format

Consistently getting the same error over a number of days.

Any idea?
Comment 50 Nick Boldt CLA 2008-09-03 12:51:33 EDT
(In reply to comment #49)
> Nick,
> thanks a lot for providing this site and the outstanding level of support!

> I cannot however install PDT 2.0 from your site- not sure this is a local
> problem or one with the build:
> An error occurred while collecting items to be installed
>   Error closing the output stream for
> org.eclipse.php.core/osgi.bundle/2.0.0.v20080722 on repository
> file:/C:/eclipse/. 
>   Error unzipping D:\Temp\org.eclipse.php.core_2.0.0.v2008072260168.jar:
> Invalid zip file format
> Consistently getting the same error over a number of days.

Hmm. Not sure what's wrong there. The next time PDT releases changes to its map file (it's been a month and a half since their last I build!), I'll respin a build and an update site. Meanwhile, try installing from this archived update site:

http://emft.eclipse.org/pdt/downloads/drops/2.0.0/I200808301152/pdt-Update-I200808301152.zip
Comment 51 Nick Boldt CLA 2008-09-10 16:19:07 EDT
Re: comment 49, with an updated map file comes a new build.

New Update site (with p2 metadata & digest):

http://emft.eclipse.org/pdt/updates/interim/

New Update zip:

http://emft.eclipse.org/pdt/downloads/drops/2.0.0/I200809101408/pdt-Update-I200809101408.zip

I think the problem was that I'd forgotten that to run this I currently need to pass in hacked-up CVS paths:

./buildUpdateSite.sh -sub pdt -Q -branch 2.0.0 -buildID I200809101408 \
   -wwwCVSRep nickb@dev.eclipse.org:/cvsroot/modeling \
   -updatesCVSPath releng-common/templates/org.eclipse.pdt.website/updates 

Eventually, the web content would be in /cvsroot/org.eclipse/www/pdt/updates/, and those last two hack-flags would not be required.

---------

Re: comment 48, here's what I need:

* content in @dev.eclipse.org:/cvsroot/modeling/releng-common/templates/org.eclipse.pdt.website/ needs to be copied into @dev.eclipse.org:/cvsroot/org.eclipse/www/pdt/

* content in @dev.eclipse.org:/cvsroot/modeling/releng-common/templates/org.eclipse.pdt.releng/ needs to be copied into @dev.eclipse.org:/cvsroot/tools/org.eclipse.pdt/org.eclipse.pdt.releng/

* whoever the user is from PDT who will be running the build on emft.eclipse.org, that person needs to do this:

http://wiki.eclipse.org/Modeling_Project_Releng/Releasing#The_First_Time

See also:

http://wiki.eclipse.org/Modeling_Project_Releng/Component_Creation

(Most of that you've already done, but some of it will be new.)

* finally, once the new builds are published (which includes release notes, search cvs data, update site w/ p2 metadata, and zips), we need to migrate & verify the website:

a) move old builds into new location (if applicable) or add build.cfg files to them so that they appear correctly on the new downloads page
b) link old downloads page from new downloads page (if applicable)
c) test release notes & search CVS pages

* after that, all you need to do is get used to using the new build/promote pages on emft.eclipse.org. When that's done, we can look at scheduled (automated) builds.

http://wiki.eclipse.org/Modeling_Project_Releng/Building/Scheduled_Builds

So, find me on IM when you're ready, and I'll assist as I can. 24hrs advanced notice would be great, if possible. :)
Comment 52 Andreas Goetz CLA 2008-09-11 05:26:57 EDT
Dunno if this is a local/ proxy or build problem, but I'm still unable with latest install site at http://emft.eclipse.org/pdt/updates/interim/
Getting new type of error message though (installing on 3.4):

An error occurred while collecting items to be installed
  Error closing the output stream for org.eclipse.php.core/osgi.bundle/2.0.0.v20080910 on repository file:/C:/eclipse/. 
  Error unzipping D:\Temp\org.eclipse.php.core_2.0.0.v2008091033920.jar: only DEFLATED entries can have EXT descriptor
Comment 53 Alon Peled CLA 2008-09-11 09:55:58 EDT
> Re: comment 48, here's what I need:
> 
> * content in
> @dev.eclipse.org:/cvsroot/modeling/releng-common/templates/org.eclipse.pdt.website/
> needs to be copied into @dev.eclipse.org:/cvsroot/org.eclipse/www/pdt/

Is it OK to put all these files under a folder, or do you expect them on the root of the site?
I have created a 'releng' folder under the pdt site and put all the files there, but haven't committed to CVS yet.

> * content in
> @dev.eclipse.org:/cvsroot/modeling/releng-common/templates/org.eclipse.pdt.releng/
> needs to be copied into
> @dev.eclipse.org:/cvsroot/tools/org.eclipse.pdt/org.eclipse.pdt.releng/

Done
Comment 54 Nick Boldt CLA 2008-09-11 21:39:15 EDT
Created attachment 112374 [details]
website and releng projects as of 2008-09-10 16:19:07

Forgot to attach this so that this bug can be properly tagged as a contribution with the "iplog+" flag.
Comment 55 Nick Boldt CLA 2008-09-11 21:50:45 EDT
Since at least one of my patches are in CVS now (though in the wrong path for consistency w/ other projects -- see below), I'm tagging this bug with the 'contributed' keyword for followup IP log stuff.

FYI, for my system to work w/o extra hacks in place, the expected path for the .releng project is:

/cvsroot/$parent/org.eclipse.$project/org.eclipse.$project.releng, ie.,
/cvsroot/tools/org.eclipse.pdt/org.eclipse.pdt.releng.

You've accidentally committed my stuff into /cvsroot/tools/org.eclipse.pdt/releng/org.eclipse.pdt.releng. Can you ssh to the server and move the folder up a level?

Thanks!

(In reply to comment #52)
>   Error unzipping D:\Temp\org.eclipse.php.core_2.0.0.v2008091033920.jar: only
> DEFLATED entries can have EXT descriptor

Never seen that one before, sorry. Might have to post this to the equinox.p2 newsgroup for help. 

Comment 56 Nick Boldt CLA 2008-09-11 22:22:32 EDT
(In reply to comment #52)
> >   Error unzipping D:\Temp\org.eclipse.php.core_2.0.0.v2008091033920.jar: only
> > DEFLATED entries can have EXT descriptor

Actually, that file doesn't exist in the update site, but there's a org.eclipse.php.core_2.0.0.v20080910.jar. Something is adding a 5-digit numerical suffix to the end of the jar when referencing it. 

This sounds like an old bug from pre-Eclipse 3.4.0 -- bug 228145, which has perhaps come back? I was able to reproduce this. I'll attach details on that bug.

However, I was able to successfully install like this:

* start clean Eclipse 3.4
* download latest DLTK (dltk-core-sdk-I-I200809081043-200809081043-incubation.zip) and PDT (pdt-SDK-I200809101408.zip)
* unpack zips into eclipse/dropins/dltk and eclipse/dropins/pdt folders
* install WTP, DTP, and GEF from Ganymede update site
* restart when prompted.
Comment 57 Alon Peled CLA 2008-09-15 11:19:40 EDT
> * whoever the user is from PDT who will be running the build on
> emft.eclipse.org, that person needs to do this:
> 
> http://wiki.eclipse.org/Modeling_Project_Releng/Releasing#The_First_Time

Did it all. Don't have access to emft.eclipse.org, but I guess I am not supposed to have it.

> 
> See also:
> 
> http://wiki.eclipse.org/Modeling_Project_Releng/Component_Creation
> 

I need some explanation about what you mean by setting up a build server.
Comment 58 Nick Boldt CLA 2008-09-15 23:07:46 EDT
Created attachment 112618 [details]
patch to change signing to run as apeled in tools/pdt instead of nickb in tools/gef
Comment 59 Nick Boldt CLA 2008-09-15 23:07:51 EDT
Created attachment 112619 [details]
mylyn/context/zip
Comment 60 Nick Boldt CLA 2008-09-15 23:20:51 EDT
Created attachment 112622 [details]
patch to update build/promote defaults: run as apeled and default contact email to alon@zend.com
Comment 61 Nick Boldt CLA 2008-09-15 23:24:35 EDT
(In reply to comment #60)
> Created an attachment (id=112622) [details]
> patch to update build/promote defaults: run as apeled and default contact email
> to alon@zend.com

This patch should actually be applied to /cvsroot/org.eclipse/www/pdt/build/_common.php, not the temporary mocked up version I created elsewhere in CVS. 

After committing the whole website project (https://bugs.eclipse.org/bugs/attachment.cgi?id=112374), you can apply this patch to fix up the default settings for the build & promote pages, but you might have to twiddle the patch file a little so that Eclipse will apply it to the right file in your workspace. Of course it's trivial to make the changes by hand, too. 
Comment 62 Nick Boldt CLA 2008-09-15 23:30:08 EDT
Created attachment 112623 [details]
trivial change to readme.html

fixes a label and URL; cosmetic changes only
Comment 63 Nick Boldt CLA 2008-09-22 16:49:07 EDT
Created attachment 113189 [details]
all the missing variables and includes in _projectCommon.php

here's all the stuff that wasn't committed yet, keeping your existing nav items
Comment 64 Nick Boldt CLA 2008-09-22 23:31:09 EDT
We're making progress on the promote process. Last attempt appears successful (with one minor error I can't understand yet). I can't confirm this until the patches above are applied so we can see the webpages.

---

Unfortunately, I kicked a new build using the latest deps [1], and it's failing to compile:

    [javac] 139. ERROR in /home/www-data/build/pdt/downloads/drops/2.0.0/I200809221822/eclipse/plugins/org.eclipse.php.core/src/org/eclipse/php/internal/core/compiler/ast/parser/PHPSourceElementParser.java (at line 27)
    [javac] 	super.parseSourceModule(contents, astCache, filename);
    [javac] 	      ^^^^^^^^^^^^^^^^^
    [javac] The method parseSourceModule(ISourceModule, ISourceModuleInfoCache.ISourceModuleInfo) in the type AbstractSourceElementParser is not applicable for the arguments (char[], ISourceModuleInfoCache.ISourceModuleInfo, char[])


[1]
    * http://download.eclipse.org/eclipse/downloads/drops/M20080911-1700/eclipse-SDK-M20080911-1700-linux-gtk.tar.gz
    * http://download.eclipse.org/modeling/emf/emf/downloads/drops/2.4.1/R200808251517/emf-sdo-xsd-SDK-2.4.1.zip
    * http://download.eclipse.org/tools/gef/downloads/drops/3.4.1/M200809101400/GEF-SDK-M200809101400.zip
    * http://download.eclipse.org/webtools/downloads/drops/R3.0/M-3.0.2-20080918035538/wtp-sdk-M-3.0.2-20080918035538.zip
    * http://download.eclipse.org/technology/dltk/downloads/drops/R1.0/I-I200809180958-200809180958/dltk-core-sdk-I-I200809180958-200809180958-incubation.zip
    * http://www.eclipse.org/downloads/download.php?file=/datatools/downloads/drops/N_DTP_1.6.1/dtp-sdk-1.6.1RC3-200809191336.zip
Comment 65 Nick Boldt CLA 2008-09-23 00:39:21 EDT
BTW, here's how to prevent the build page's UI from selecting Eclipse 3.5 milestones as input to the build.

It's so easy I won't bother w/ a patch. Open /cvsroot/org.eclipse/www/pdt/build/_common.php, and change the regex:

	"regex" => "M200.*/eclipse-SDK-|[SR]-.*200.*/eclipse-SDK-|[MSR]200.*/emf-sdo-xsd-SDK-|[MSR]200.*/GEF-ALL-|wtp-|dltk-|dtp-",


to

	"regex" => "M200.*/eclipse-SDK-|R-.*200.*/eclipse-SDK-|[MR]200.*/emf-sdo-xsd-SDK-|[MR]200.*/GEF-|wtp-|dltk-|dtp-",

With the above, only Eclipse M and R builds will match; same for EMF. For GEF, the pattern will match both GEF-SDK and GEF-ALL zips, again only M and R. If you find the patterns for wtp, dltk, or dtp are too loose, you can add more restriction to them.

Once you update this file in CVS, you need to also update it in /var/www/html/pdt/build/ on emft.eclipse.org (using `cvs update` in that folder to grab the latest).
Comment 66 Nick Boldt CLA 2008-09-23 16:28:53 EDT
(In reply to comment #64)
>     [javac] 139. ERROR in
> /home/www-data/build/pdt/downloads/drops/2.0.0/I200809221822/eclipse/plugins/org.eclipse.php.core/src/org/eclipse/php/internal/core/compiler/ast/parser/PHPSourceElementParser.java
> (at line 27)
>     [javac]     super.parseSourceModule(contents, astCache, filename);
>     [javac]           ^^^^^^^^^^^^^^^^^
>     [javac] The method parseSourceModule(ISourceModule,
> ISourceModuleInfoCache.ISourceModuleInfo) in the type
> AbstractSourceElementParser is not applicable for the arguments (char[],
> ISourceModuleInfoCache.ISourceModuleInfo, char[])

This is fixed by moving up to today's mapfile (tag v20080923-1839)... however, the tests are now broken because of this change by rganor on Tue Sep 23 @ 00:58:24 2008 UTC:

http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.pdt/features/org.eclipse.php.test-feature/feature.xml?root=Tools_Project&r1=1.16&r2=1.17

The log shows this error:

/home/www-data/build/pdt/downloads/drops/2.0.0/N200809231613/org.eclipse.pdt.releng/builder/tests/customTargets.xml:185: /home/www-data/build/pdt/downloads/drops/2.0.0/N200809231613/eclipse/test.assembly/eclipse/dropins/eclipse/plugins/${org.eclipse.test} not found.

And the fix is documented here:

http://wiki.eclipse.org/Modeling_Project_Releng/Building#eclipse.2Ftest.assembly.2Feclipse.2F_plugins.2F.24.7Borg.eclipse.test.7D_not_found

Comment 67 Alon Peled CLA 2008-09-23 19:21:48 EDT
(In reply to comment #66)
> (In reply to comment #64)
> >     [javac] 139. ERROR in
> > /home/www-data/build/pdt/downloads/drops/2.0.0/I200809221822/eclipse/plugins/org.eclipse.php.core/src/org/eclipse/php/internal/core/compiler/ast/parser/PHPSourceElementParser.java
> > (at line 27)
> >     [javac]     super.parseSourceModule(contents, astCache, filename);
> >     [javac]           ^^^^^^^^^^^^^^^^^
> >     [javac] The method parseSourceModule(ISourceModule,
> > ISourceModuleInfoCache.ISourceModuleInfo) in the type
> > AbstractSourceElementParser is not applicable for the arguments (char[],
> > ISourceModuleInfoCache.ISourceModuleInfo, char[])
> 
> This is fixed by moving up to today's mapfile (tag v20080923-1839)... however,
> the tests are now broken because of this change by rganor on Tue Sep 23 @
> 00:58:24 2008 UTC:
> 
> http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.pdt/features/org.eclipse.php.test-feature/feature.xml?root=Tools_Project&r1=1.16&r2=1.17
> 
> The log shows this error:
> 
> /home/www-data/build/pdt/downloads/drops/2.0.0/N200809231613/org.eclipse.pdt.releng/builder/tests/customTargets.xml:185:
> /home/www-data/build/pdt/downloads/drops/2.0.0/N200809231613/eclipse/test.assembly/eclipse/dropins/eclipse/plugins/${org.eclipse.test}
> not found.
> 
> And the fix is documented here:
> 
> http://wiki.eclipse.org/Modeling_Project_Releng/Building#eclipse.2Ftest.assembly.2Feclipse.2F_plugins.2F.24.7Borg.eclipse.test.7D_not_found
> 

fixed the feature. Integration build from latest tag succeeded.
Comment 68 Andreas Goetz CLA 2008-09-25 11:40:49 EDT
Hope this is not OT, but:

After upgrading to PDT Feature v2.0.0.20080923-1839 using P2 update, the PHP and PHP Debug perspectives are gone. "Installed software" shows the feature is there (along with DLTK 0.95.1), logfile does not contain any error messages.
Comment 69 Nick Boldt CLA 2008-09-29 14:48:08 EDT
Created attachment 113786 [details]
do not use Eclipse 3.5 milestone builds for building PDT

simple change to regex pattern to exclude S builds
Comment 70 Nick Boldt CLA 2008-09-30 11:49:01 EDT
re: comment 68 -- that's OT for this bug. Please post this to the newsgroup or open a new bug / comment on another existing bug.

Last promoted build worked 100% -- no permissions problems or missing files. Success!

So, here's what's left to do:

* migrate some old builds to the new format, if you want them listed on the new Downloads page -- this will require renaming zips, creating a build.cfg file, and moving them into the appropriate version folders. You could skip this and just do the next step if you prefer.
* link old, archived downloads (eg., 1.0.x R, 0.7.x R) from the new Downloads page (see /pdt/downloads/extras-pdt.php, line 4)
* link the old Downloads page from the new Downloads page (see /pdt/downloads/extras-pdt.php, line 14)
* update the left nav in _projectCommon.php to link to the Update site page and new Downloads page [1, 2]
* provide a link in the left nav for Search CVS - PDT project, last 7 days [3]

[1] http://www.eclipse.org/pdt/updates/
[2] http://www.eclipse.org/pdt/downloads/
[3] http://www.eclipse.org/pdt/searchcvs.php?q=project%3A+org.eclipse.pdt+days%3A+7

For the last two items above, here's what I'd suggest to replace your left nav:

$Nav->addCustomNav("About This Project", "/projects/project_summary.php?projectid=tools.pdt", "", 1);
$Nav->addNavSeparator($projectName, "$rooturl/");

$Nav->addCustomNav("Installation", "http://wiki.eclipse.org/PDT/Installation", "_self", 2);
$Nav->addCustomNav("Downloads", "$downurl/$PR/downloads/", "_self", 2);
$Nav->addCustomNav("Update Manager", "$rooturl/updates/", "_self", 2);

$Nav->addNavSeparator("Documentation", $rooturl . "/docs.php");
$Nav->addCustomNav("Tutorials", "http://wiki.eclipse.org/PDT/TUTORIALS", "_self", 2);
$Nav->addCustomNav("FAQ", "http://wiki.eclipse.org/PDT/FAQ", "_self", 2);
$Nav->addCustomNav("Development", 		"/pdt/development.php", 			"_self", 2);	
$Nav->addCustomNav("Plan", $rooturl . "/plans/php_plan_2_0.php", "_self", 2);

$Nav->addCustomNav("Release Notes", $rooturl . "/news/relnotes.php?project=$projct&amp;version=HEAD", "_self", 2);
$Nav->addCustomNav("Search CVS", $rooturl . "/searchcvs.php?q=file%3A+org.eclipse." . strtolower($projectName) . "%2F" . "+days%3A+7", "_self", 2);

$Nav->addNavSeparator("Community", "http://wiki.eclipse.org/PDT");
$Nav->addCustomNav("Wiki", "http://wiki.eclipse.org/PDT", "_self", 2);
$Nav->addCustomNav("Newsgroup Search", "http://www.eclipse.org/newsportal/thread.php?group=eclipse.tools.pdt", "_self", 2);
$collist = "%26query_format%3Dadvanced&amp;column_changeddate=on&amp;column_bug_severity=on&amp;column_priority=on&amp;column_rep_platform=on&amp;column_bug_status=on&amp;column_product=on&amp;column_component=on&amp;column_version=on&amp;column_target_milestone=on&amp;column_short_short_desc=on&amp;splitheader=0";
$Nav->addCustomNav("Open Bugs", "$bugurl/bugs/colchange.cgi?rememberedquery=product%3D" . $projectName . "%26bug_status%3DNEW%26bug_status%3DASSIGNED%26bug_status%3DREOPENED%26order%3Dbugs.bug_status%2Cbugs.target_milestone%2Cbugs.bug_id" . $collist, "_self", 2);
$Nav->addCustomNav("Submit A Bug", "$bugurl/bugs/enter_bug.cgi?product=" . $projectName, "_self", 2);

#$Nav->addCustomNav("Contributors", "http://www.eclipse.org/$PR/project-info/team.php", "_self", 2);
$Nav->addCustomNav("IP Log", "http://www.eclipse.org/projects/ip_log.php?projectid=tools.pdt", "_self", 2);

Let me know if you'd like a hand w/ anything above.
Comment 71 Nick Boldt CLA 2008-09-30 11:53:03 EDT
* after renaming the existing http://download.eclipse.org/tools/pdt/downloads/index.php to indexOld.php, you'll want to put a redirect page in there so that people will bounce to the new page automatically. Here's some code for that new index.php:

<?php header("Location: http://www.eclipse.org/pdt/downloads/"); ?>
Comment 72 Alon Peled CLA 2008-10-02 07:14:40 EDT
(In reply to comment #69)
> Created an attachment (id=113786) [details]
> do not use Eclipse 3.5 milestone builds for building PDT
> 
> simple change to regex pattern to exclude S builds
> 

done
Comment 73 Nick Boldt CLA 2008-10-02 10:47:46 EDT
Created attachment 114105 [details]
patch to display nightlies at the bottom of the new downloads page

untested, hope this works
Comment 74 Nick Boldt CLA 2008-10-12 02:38:13 EDT
As discussed in pdt newsgroup [0], people are confused around PDT install instructions. Perhaps the left nav link, http://www.eclipse.org/pdt/install.php, ought to either bounce directly to the wiki [1] so that it can be maintained by the community, or at least link from there to the more complete wiki page [1].

[0]http://www.eclipse.org/newsportal/article.php?id=2619&group=eclipse.tools.pdt#2619
[1]http://wiki.eclipse.org/PDT/Installation

Thoughts?
Comment 75 Andreas Goetz CLA 2008-10-23 04:36:52 EDT
The wiki is not updated either. http://wiki.eclipse.org/PDT/Installation#From_Update_Site still instructs to install via zip files which is not recommended.

Eclipse has reached 3.4.1 and we're still not able to properly install PDT from an update site...
Comment 76 Nick Boldt CLA 2008-11-17 01:00:32 EST
* Release notes updated for 1.0.5 and 2.0.0M1 builds. Alon, Roy, etc. let me know if anything's missing.

http://www.eclipse.org/pdt/news/
http://www.eclipse.org/pdt/news/relnotes.php?version=R1_0_5_20080723

* Ability to publish N builds via UI and crontab has been added. I've tested promoting an N via UI, and will turn on the crontab for tomorrow. 

* I've set up weekly I builds based on CVS changes for Thursdays at 9:30am EST (GMT-5). Does that work? Note that the build will automatically tag & release any changes since the last build so you no longer have to do so yourself. (I can do the same for daily N builds if you'd like.)

* Install instructions updated; wiki linked from homepage so there's ONE set of instructions instead of two.

Comment 77 Roy Ganor CLA 2008-11-19 07:17:15 EST
(In reply to comment #76)
> * Release notes updated for 1.0.5 and 2.0.0M1 builds. Alon, Roy, etc. let me
> know if anything's missing.
> 
> http://www.eclipse.org/pdt/news/
> http://www.eclipse.org/pdt/news/relnotes.php?version=R1_0_5_20080723
> 
> * Ability to publish N builds via UI and crontab has been added. I've tested
> promoting an N via UI, and will turn on the crontab for tomorrow. 
> 
> * I've set up weekly I builds based on CVS changes for Thursdays at 9:30am EST
> (GMT-5). Does that work? Note that the build will automatically tag & release
> any changes since the last build so you no longer have to do so yourself. (I
> can do the same for daily N builds if you'd like.)
> 
> * Install instructions updated; wiki linked from homepage so there's ONE set of
> instructions instead of two.
> 

Very Impressive!

Can you tell me on what dependencies list this nightly / integration build works?

Comment 78 Nick Boldt CLA 2008-11-19 14:30:38 EST
> Can you tell me on what dependencies list this nightly / integration build
> works?

There are two ways to feed dependencies to scheduled builds:

a) regular expression (defined in www/pdt/build/_common.php) that matches the latest available dependencies in the list on the build page -- this list is fed by http://emft.eclipse.org/dependencies.urls.txt, which is updated every time someone adds a new dep via a build page.

b) static list of URLs set either in the regex in (a) or in apache's crontab script.

Here are some examples:

# use a regex and build an N build only if something has changed in CVS in the last 24hrs; do not tag & release
/home/www-data/build/modeling/scripts/start_cron.sh -sub pdt -antTarget runWithoutTest -version 2.0.0 -regex "M200.*/eclipse-SDK-|R-.*200.*/eclipse-SDK-|[MSR]200.*/emf-sdo-xsd-SDK-|[MSR]200.*/GEF-|wtp-|dltk-|dtp-" -basebuilderBranch RC2_34 -searchcvs pdt,pdt.releng > $HOME/cron_logs/start_cron.sh.pdt.20N.txt 2>&1

# use a regex and build an N build on schedule (regardless of CVS changes)
/home/www-data/build/modeling/scripts/start_cron.sh -sub pdt -noSearchCVS     -antTarget runWithoutTest -version 2.0.0 -regex "M200.*/eclipse-SDK-|R-.*200.*/eclipse-SDK-|[MSR]200.*/emf-sdo-xsd-SDK-|[MSR]200.*/GEF-|wtp-|dltk-|dtp-" -basebuilderBranch RC2_34 -searchcvs pdt,pdt.releng > $HOME/cron_logs/start_cron.sh.pdt.20N.txt 2>&1

# build 2.0I only if something changes in CVS and tag/release those changes
/home/www-data/modeling-releng-common/tools/org.eclipse.releng.tools.tagandrelease/update_tags.sh /home/www-data/modeling-releng-common/tools/org.eclipse.releng.tools.tagandrelease/pdt.HEAD.options.txt -debug 2 > $HOME/cron_logs/update_tags.sh.pdt.20I.txt 2>&1

# build 1.0.5M only if something changes in CVS and tag/release those changes
/home/www-data/modeling-releng-common/tools/org.eclipse.releng.tools.tagandrelease/update_tags.sh /home/www-data/modeling-releng-common/tools/org.eclipse.releng.tools.tagandrelease/pdt.R10m.options.txt -debug 2 > $HOME/cron_logs/update_tags.sh.pdt.10M.txt 2>&1

# static list of URLs for the M builds; ignore CVS, just build on schedule
/home/www-data/build/modeling/scripts/start_cron.sh -sub pdt -buildType M -antTarget runWithoutTest -basebuilderBranch RC2_34 -cvsbranch R1_0_5_20080723 -searchcvs pdt.pdt.releng -noSearchCVS -version 1.0.5 -email "alon@zend.com,nickboldt+build@gmail.com" -regex "http://download.eclipse.org/eclipse/downloads/drops/R-3.4.1-200809111700/eclipse-SDK-3.4.1-linux-gtk.tar.gz|http://download.eclipse.org/modeling/emf/emf/downloads/drops/2.4.1/R200808251517/emf-sdo-xsd-SDK-2.4.1.zip|http://download.eclipse.org/tools/gef/downloads/drops/3.4.1/R200809101400/GEF-SDK-3.4.1.zip|http://download.eclipse.org/webtools/downloads/drops/R3.0/R-3.0.2-20080921203356/wtp-wst-sdk-R-3.0.2-20080921203356.zip|http://download.eclipse.org/datatools/downloads/1.6/dtp-sdk_1.6.0_200806181028.zip" > $HOME/cron_logs/start_cron.sh.pdt.10M.txt 2>&1

 

Comment 79 Nick Boldt CLA 2008-11-29 01:33:47 EST
(In reply to comment #75)
http://wiki.eclipse.org/PDT/Installation#From_Update_Site updated.

I think we're done here. I'm going to close this bug -- if anything within is still not yet done we can open a new one to contain that work.
Comment 80 Gadi Goldbarg CLA 2008-12-01 04:20:07 EST
Closed :)