[cross-project-issues-dev] Callisto Update Site -- to copy, or not to co

[David M Williams <david_williams@xxxxxxxxxx>]

Just wanted to keep all informed, I
had a long heart-to-heart talk (or .. mind-to-mind?) with some of the Update
and Platform team members yesterday, 
and we concluded by agreeing that my
current strategy of using "nearest mirror" for each file was
too risky for this years release. That is, Update 
Manager could (probably) not be made
robust enough to deal with a long, long list semi-random mirrors, each
with a small probability of failure.  
So, Users will still (get to) pick one,
"nearest mirror" 
at the beginning of the update session
-- that is, the mirrors will be sorted in order of "nearest"
to "furtherest" (as best as can be determined), but 
users can still pick which ever one
they want. (And, from some bug posts and notes, some users want/need that
ability, since for those 
that use mirrors a lot, they are well
aware of which ones are good and which ones are not so great). 

The limitations of this approach are
1: we can not do statistics on a file by file basis (same as status quo),
and 2. We will not be providing 
a solution that others outside of Callisto
could mimic. That is, other projects or products that want to "pre-req"
any Callisto/Eclipse projects 
will still have to make copies of the
files, or instruct user to install them first, or (not recommended) point
directly to the eclipse.org version, 
or .. what ever they currently do. (again,
status quo). 

There is an important implication for
"providers" to Callisto ... we will need to *copy* all your update
jars to one or more callisto directories. I've sent note to 
our friendly eclipse.org webmaster,
to see what "warning" we should give mirror partners, but since
he originally suggested it 
should not be any fundamental problem.
(and, some projects have stated they would prefer this anyway, to more
independence in their 
own sites .. they could delete something
there, for example, even if still being used by Callisto). 

And, we do need to keep these copies
to a relatively minimum-required, to not overload mirrors. I'm still mulling
the mechanics, but at worst, 
each project will have to provide me
with a pointer to a directory to copy their "minimum set" of
features and plugin jars from. 
The info I've currently been given is
minimum for some projects, for now, since first one, but some, such as
EMF already have a (nice) 
strategy of having a single update site
directory ... so, I wouldn't want to copy all that. The best case is some
volunteer will write 
a small program to "get the most
recent" features and plugins from the directories you've given me.


Any project preference? Do projects
want "write access" so you can copy over you own files to appropriate
directories? 

Lastly, we discussed the advantages
of having "milestones" and "release candidates" in
the actual "callisto/releases"  directory, so 
1. it matches a discovery site that
comes "shipped" in the platform feature and 2. just to get a
bit better testing (but, being clear, was 
still "only for testing, not production,
etc"). Any objections?  This directory would (still) be "fresh"
each milestone, release candidate, etc., 
for a bit more reliability if "installing
from scratch".  Note we would still have the "staging"
directory to "test out" the update site, and then 
just copy it to the release directory.


And, then, we concluded, we would have
another "/interim" directory that would start off with the first
"milestone" release .. but then 
could be updated weekly, etc., by projects,
as desired, to specifically test the "version interval" features
of OSGi and "stress" Update 
manager by it having to, possibly, peek
at hundreds of features. 

Since I am aware of my fault of long
notes ... I'll even summarize the questions above I'm asking for feed back
about, from the 10 
projects that have deliverables for
Callisto ... 

Any problems providing a "copy
this minimum set" directory? 
Would you prefer ability (and responsibility)
to write to some central directory yourselves? 
         (feel
free to offer an *easy* alternative solution, or provide a tool that automatically
copies most recent only)
Any objections to having milestones
in release directory? 
Any clients of an interim directory?
(we in WTP might use some ... but, I doubt weekly). 

We can discuss at Friday's status meeting,
if discussion is needed.