[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
[
List Home]
|
Re: AW: [equinox-dev] Re: [prov] Deltas for delivering updates
|
Agreed!
So, e.g. a post processing step would be something like
(processor-id="apply-xdelta",data="from-version=1.0",required=true).
Is the intended semantic of ´required==false´ that a
ProcessorManager/Executer may decide on some context whether or execute
or not, e.g. md5 check required or not may be a decision of the ´user´?
Currently ProcessingStep is ´only´ part of the ArtifactDescriptor and so
neither available to requests and repositories because it is not part of
their interface. They only know IArtifactKey. It is only used inside
LocalWrittableRepo and meta data generation.
Given that a request can not execute post processors. As a consequence
ArtifactDescriptor should be made visible/accessible/query-able from the
repository such as
ArtifactDescriptors[] getArtifactDescriptors( IArtifactKey key )
With this a request can than decide for which of the returned
descriptors it is capable of processing the required post processing steps.
Another question that comes into my mind is: Should the post processing
be capable of piping the output of one post processor to the next post
processor?
I could start working on this post processing stuff, if that is ok?
I am not sure whether a repository should be responsible for download at
all. It´s responsibility should be storing/updating/deleting and
retrieving/querying, it should delegate this to requests. Even the
initial feed of the artifact descriptors should be handled by a request,
otherwise dealing with stuff like (internet) proxies, etc. has to be
done in multiple places.
Stefan
Pascal Rapicault wrote:
Answers embedded <pascal></pascal>
PaScaL
"Liebig, Stefan "
<Stefan.Liebig@co
mpeople.de> To
Sent by: "Equinox development mailing list"
equinox-dev-bounc <equinox-dev@xxxxxxxxxxx>
es@xxxxxxxxxxx cc
Subject
07/18/2007 08:39 AW: [equinox-dev] Re: [prov] Deltas
AM for delivering updates
Please respond to
Equinox
development
mailing list
<equinox-dev@ecli
pse.org>
First to provide some clarification for me:
The content (repositoryDescriptor) of an artifact repo is described within
the artifacts.xml.
This maps to a set of artifact descriptors where each artifact descriptor
contains an artifact
key, the artifacts download size and a list of post processing steps. The
post processing steps
seam to be currently unused currently.
<pascal> This is correct. The idea behind the post processing step was to
describe operations that had to be performed on the artifact once it had
been downloaded. The canonical example being unpack (from pack200). It
could be used for MD5 checks and things like that but I'm less sure about
this.
Also these processing steps should probably be used as negotiation between
repositories, since for example I should not download a packed200 jar if I
don't have the support to unpack it.</pascal>
Additionaly the repo descriptor contains a mapping definition to create a
url for accessing an
artifact. This url is than used to fetch the artifact.
<pascal> The mapper is an implementation detail of the simple artifact
repository. Other repository implementers may do the mapping another
way.</pascal>
How could deltas (the plural is for different delta algorithms) be
described here?
Is a delta an artifact in itself?
<pascal> Yes, however it is never explicitly requested for mirror by the
creator of a request.</pascal>
Maybe an artifact key (of a delta) contains an optional version property to
define the ´from version´?
The ´from version´ and the type of alg. are necessary to apply the delta.
The Mapper has than to consider the ´from version´ and the processing step
id for creating the url.
Would the ´from version´ better be part (data field) of a processing step?
The Mapper has than to consider also the processing steps for creating the
url.
<pascal>I like the approach where the type of delta is indicated as a post
processing step. It fits well in the model where first you check that the
target artifact repo can receive the artifact being downloaded, then
download, then apply the post processing steps, because downloading a delta
is just like downloading another artifact that could have its own
processing step, etc. Then the only thing that needs special code for is
how to find if a delta can be downloaded instead of the complete artifact
being requested.
Maybe a first thing would be to implement the support for post processing.
This will likely question how the download is currently handled (I'm not
really in love with what we have now) and may help move some
responsibilities around. For example, maybe the download should be handled
in the target repo itself.
</pascal>
Anyway, in a repository there may be stored a complete artifact, a zipped
artifact, a xdelta artifact
or a jbdiff artifact,.. of one and the same target artifact.
With that the MirrorRequest could be enhanced such that it can query the
source repo for different
variants (complete,delta,..) of the target artifact, choose the appropriate
one, download it, apply
the post processing steps and store it in the target repo.
Stefan
-----Ursprüngliche Nachricht-----
Von: equinox-dev-bounces@xxxxxxxxxxx
[mailto:equinox-dev-bounces@xxxxxxxxxxx] Im Auftrag von
Pascal Rapicault
Gesendet: Mittwoch, 11. Juli 2007 15:50
An: Equinox development mailing list
Cc: equinox-dev@xxxxxxxxxxx; equinox-dev-bounces@xxxxxxxxxxx
Betreff: [equinox-dev] Re: [prov] Deltas for delivering updates
As you correctly identified the separation of responsibility
between ECF and the provisioning code is done as follow:
- ECF is used to do the transports (e.g http, ftp,
bittorrent). We leverage, its API, its transport pluggability
(and eventually its ability to restart downloads).
- The provisioning code (DownloadManager) is responsible to
cause the download of all the requested artifacts using the
artifact repos. It will be responsible for picking the best
mirror (hopefully we can reuse what is in Maya) and also
doing the parallelization.
Can/should delta handling be part of ECF - as a kind of smart
transport/download handling? Or is delta handling so
important to provisioning that it should be part of provisioning?
Given our usage of ECF, I don't think that ECF has enough
context to be able to handle delta (it would not have the
identification information that you are talking about).
Therefore I think the ability to do delta depends on the
repository in which the artifact is being mirrored to and
from and it should be transparent for the download manager.
Therefore with the current code I think the decision or not
to use delta can be made in the MirrorRequest by checking the
characteristics of the artifact repositories at hand, and the
actual logic of getting the right delta could be left to the
implementation of the repositories.
It is also important to note that this code did not receive a
lot of attention and anything could be changed.
PaScaL
"Liebig, Stefan "
<Stefan.Liebig@co
mpeople.de>
To
Sent by:
<equinox-dev@xxxxxxxxxxx>
equinox-dev-bounc
cc
es@xxxxxxxxxxx
Subject
[equinox-dev] Deltas
for delivering
07/11/2007 04:49 updates
AM
Please respond to
Equinox
development
mailing list
<equinox-dev@ecli
pse.org>
I am currently investigating how the delta stuff (binary
deltas) could fit into the ongoing provisioning work.
When working with deltas there are certain constraints such as:
- when requesting a delta for an artifact it is essential
that the request contains an identification (version, md5,
..) of the artifact that is already available on the client,
so that is possible to regenerate the new artifact from the
old artifact and the delta.
- the client should not rely on the fact that the server(s)
can always respond with a delta. In some way the client needs
to have some flexibility on handling the data (deltas,
complete ´files´) which the server can respond.
Currently provisioning uses ECF to handle the transport and download.
Can/should delta handling be part of ECF - as a kind of smart
transport/download handling? Or is delta handling so
important to provisioning that it should be part of provisioning?
Any thoughts?
_______________________________________________
equinox-dev mailing list
equinox-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/equinox-dev
_______________________________________________
equinox-dev mailing list
equinox-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/equinox-dev
_______________________________________________
equinox-dev mailing list
equinox-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/equinox-dev
_______________________________________________
equinox-dev mailing list
equinox-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/equinox-dev
