| 19 |
is not feasible for eclipse.org to create, manage and distribute pre-packaged |
is not feasible for eclipse.org to create, manage and distribute pre-packaged |
| 20 |
configurations to meet all the demands. This proposal details a structure |
configurations to meet all the demands. This proposal details a structure |
| 21 |
which enables Eclipse component providers to contribute their features and |
which enables Eclipse component providers to contribute their features and |
| 22 |
plugins and for Eclipse consumers to efficiently select, download and install |
plug-ins and for Eclipse consumers to efficiently select, download and install |
| 23 |
their components of interest.<br> |
their components of interest.<br> |
| 24 |
Last Modified: 1100 February 17, 2004</p> |
Last Modified: 1100 February 17, 2004</p> |
| 25 |
</blockquote> |
</blockquote> |
| 39 |
the single zip and go. If they need to run on two different platforms they download |
the single zip and go. If they need to run on two different platforms they download |
| 40 |
the zip for that platform and ignore the fact that 95% of the second download |
the zip for that platform and ignore the fact that 95% of the second download |
| 41 |
is the same as the first.</p> |
is the same as the first.</p> |
| 42 |
<p>Users looking to combine the base Eclipse with additional plugins/features |
<p>Users looking to combine the base Eclipse with additional plug-ins/features |
| 43 |
are somewhat less happy. They have to search over several web sites, identifying |
are somewhat less happy. They have to search over several web sites, identifying |
| 44 |
downloads (zips) of interest and correlating versions manually. In general, |
downloads (zips) of interest and correlating versions manually. In general, |
| 45 |
the suppliers of these additional zips do not make it easy for users to understand |
the suppliers of these additional zips do not make it easy for users to understand |
| 46 |
their offerings.</p> |
their offerings.</p> |
| 47 |
<p>The main issues with the current setup are:</p> |
<p>The main issues with the current setup are:</p> |
| 48 |
<ul> |
<ul> |
| 49 |
<li>Does not promote the easy provision, acquisition and use of plugins from |
<li>Does not promote the easy provision, acquisition and use of plug-ins from |
| 50 |
a variety of souces. The basic premise of Eclipse is that plugin consumers |
a variety of souces. The basic premise of Eclipse is that plug-in consumers |
| 51 |
can gather a set of interesting plugins, put them together in an Eclipse platform |
can gather a set of interesting plug-ins, put them together in an Eclipse platform |
| 52 |
configuration and enjoy an integrated seemless environment. Currently plugin |
configuration and enjoy an integrated seemless environment. Currently plug-in |
| 53 |
producers provide their plugins in various forms and the lack of any managed |
producers provide their plug-ins in various forms and the lack of any managed |
| 54 |
acquisition mechanism forces potential users to grovel around for plugins |
acquisition mechanism forces potential users to grovel around for plug-ins |
| 55 |
and then hope that they got the right thing.</li> |
and then hope that they got the right thing.</li> |
| 56 |
<li>Does not scale as the scope of Eclipse increases. The current approach addresses |
<li>Does not scale as the scope of Eclipse increases. The current approach addresses |
| 57 |
the provision/acquisition problem by packaging collections of interesting |
the provision/acquisition problem by packaging collections of interesting |
| 58 |
plugins together in a single zip. This works for Eclipse since it is at the |
plug-ins together in a single zip. This works for Eclipse since it is at the |
| 59 |
bottom of the plugin stack. However, others providers are not so lucky and |
bottom of the plug-in stack. However, others providers are not so lucky and |
| 60 |
are challenged with questions like "how many of my prerequisites should |
are challenged with questions like "how many of my prerequisites should |
| 61 |
I include in my download?". Further, as more and more plugins are added |
I include in my download?". Further, as more and more plug-ins are added |
| 62 |
to the Eclipse world, the size of the complete zip increases. A zip of a reasonable |
to the Eclipse world, the size of the complete zip increases. A zip of a reasonable |
| 63 |
set of SDK drops from projects on eclipse.org could easily reach 150MB (i.e., |
set of SDK drops from projects on eclipse.org could easily reach 150MB (i.e., |
| 64 |
double the Eclipse SDK size). Following the current model, there would be |
double the Eclipse SDK size). Following the current model, there would be |
| 65 |
one of these uber-zips per target OS/WS.</li> |
one of these uber-zips per target OS/WS.</li> |
| 66 |
<li>Does not scale as the Eclipse use scenarios increase. As the range of people |
<li>Does not scale as the Eclipse use scenarios increase. As the range of people |
| 67 |
using Eclipse broadens, there will be increasing demand for alternative collections |
using Eclipse broadens, there will be increasing demand for alternative collections |
| 68 |
of plugins. These will contain either more or less than the Eclipse SDK. While |
of plug-ins. These will contain either more or less than the Eclipse SDK. While |
| 69 |
we can take a stab and produce some common packagings, we will not</li> |
we can take a stab and produce some common packagings, we will not</li> |
| 70 |
<li>Makes poor use of existing disk and bandwidth resources. As mentioned earlier, |
<li>Makes poor use of existing disk and bandwidth resources. As mentioned earlier, |
| 71 |
something like 95% of each OS/WS specific SDK zip is identical content. Further, |
something like 95% of each OS/WS specific SDK zip is identical content. Further, |
| 72 |
from build to build, not all plugins change. The user doc for example tends |
from build to build, not all plug-ins change. The user doc for example tends |
| 73 |
to change slowly. Near the end of a milestone or release cycle we frequently |
to change slowly. Near the end of a milestone or release cycle we frequently |
| 74 |
rebuild to fix isolated problems. Rather than rebuilding just the plugins |
rebuild to fix isolated problems. Rather than rebuilding just the plug-ins |
| 75 |
in question and providing these for download, we rebuild the whole stack and |
in question and providing these for download, we rebuild the whole stack and |
| 76 |
repackage an entire set of full zips. Subsequently, interested developers |
repackage an entire set of full zips. Subsequently, interested developers |
| 77 |
are forced to redownload the bulk of what they already have. This wastes diskspace |
are forced to redownload the bulk of what they already have. This wastes diskspace |
| 105 |
<p>After following the above scenario, the user can manage their configuration |
<p>After following the above scenario, the user can manage their configuration |
| 106 |
either using the steps outlined in Scenario 2 or by restarting their configuration |
either using the steps outlined in Scenario 2 or by restarting their configuration |
| 107 |
with an indication that the update manager UI should be started. This puts them |
with an indication that the update manager UI should be started. This puts them |
| 108 |
back at step 3 but with all their previously installed plugins still installed.</p> |
back at step 3 but with all their previously installed plug-ins still installed.</p> |
| 109 |
<h3>Scenario 2: building on a product install</h3> |
<h3>Scenario 2: building on a product install</h3> |
| 110 |
<p>In this scenario, the user has an Eclipse product installed (possibly just |
<p>In this scenario, the user has an Eclipse product installed (possibly just |
| 111 |
the Eclipse SDK from a downloaded zip file or the configuration from the steps |
the Eclipse SDK from a downloaded zip file or the configuration from the steps |
| 143 |
creating the familiar SDK etc zips. |
creating the familiar SDK etc zips. |
| 144 |
<h3>Update site</h3> |
<h3>Update site</h3> |
| 145 |
<p>eclipse.org (and others) will provide update sites sporting all the relevant |
<p>eclipse.org (and others) will provide update sites sporting all the relevant |
| 146 |
features and plugins. Collections of projects can cooperate and share an update |
features and plug-ins. Collections of projects can cooperate and share an update |
| 147 |
site or each supply/support their own update site. Managing your own eliminates |
site or each supply/support their own update site. Managing your own eliminates |
| 148 |
the need for some sort of shared site update mechanism. However, the user must |
the need for some sort of shared site update mechanism. However, the user must |
| 149 |
be able to navigate these individual sites seamlessly (see the section on Update |
be able to navigate these individual sites seamlessly (see the section on Update |
| 150 |
manager). </p> |
manager). </p> |
| 151 |
<h3>Versioning</h3> |
<h3>Versioning</h3> |
| 152 |
<p>Update sites are not possible unless we use the Eclipse version numbering scheme |
<p>Update sites are not possible unless we use the Eclipse version numbering scheme |
| 153 |
effectively. The main challenge is for plugins to have version numbers that |
effectively. The main challenge is for plug-ins to have version numbers that |
| 154 |
indicate their content. Updating versions so they correctly reflect semantic |
indicate their content. Updating versions so they correctly reflect semantic |
| 155 |
change would be good but not essential. What is required however is that if |
change would be good but not essential. What is required however is that if |
| 156 |
two plugins have the same id and version, their content is identical. The easiest |
two plug-ins have the same id and version, their content is identical. The easiest |
| 157 |
way to do this is to use the version qualifier (i.e., the forth version element). |
way to do this is to use the version qualifier (i.e., the forth version element). |
| 158 |
The qualifier should be the version tag from the map file used for the build |
The qualifier should be the version tag from the map file used for the build |
| 159 |
which generated them (e.g., 3.0.0.v20040212). The map file defines the content |
which generated them (e.g., 3.0.0.v20040212). The map file defines the content |
| 160 |
going into the build. If that does not change, then we can assume that the content |
going into the build. If that does not change, then we can assume that the content |
| 161 |
of the output plugin will not change. In this way, identical plugins will not |
of the output plug-in will not change. In this way, identical plug-ins will not |
| 162 |
be duplicated on the update site and consumers will not have to download them |
be duplicated on the update site and consumers will not have to download them |
| 163 |
if they already have them.</p> |
if they already have them.</p> |
| 164 |
<p>Features on the other hand are packagings of plugins. They are also relatively |
<p>Features on the other hand are packagings of plug-ins. They are also relatively |
| 165 |
small. Since the content of a feature is indirectly defined by the content of |
small. Since the content of a feature is indirectly defined by the content of |
| 166 |
the plugins included in the feature, it is hard to detect change. The safe bet |
the plug-ins included in the feature, it is hard to detect change. The safe bet |
| 167 |
here is to use the build stamp as the qualifier for feature version (e.g., 3.0.0.I200402131200). |
here is to use the build stamp as the qualifier for feature version (e.g., 3.0.0.I200402131200). |
| 168 |
This has the benificial effect of clearly identifying the features for a particular |
This has the benificial effect of clearly identifying the features for a particular |
| 169 |
build. Further, since the entire feature set is regenerated, the collection |
build. Further, since the entire feature set is regenerated, the collection |
| 171 |
output.</p> |
output.</p> |
| 172 |
<p>Nightly builds would not go into the update site. There is no easy way to correlate |
<p>Nightly builds would not go into the update site. There is no easy way to correlate |
| 173 |
the content in a nightly build (out of HEAD) with that of an integration build |
the content in a nightly build (out of HEAD) with that of an integration build |
| 174 |
(using versioned resources). As a result, all plugins would be new and the economies |
(using versioned resources). As a result, all plug-ins would be new and the economies |
| 175 |
of the proposed solution would not be realized. Further, nightly builds are |
of the proposed solution would not be realized. Further, nightly builds are |
| 176 |
primarily there for the individual teams to ensure they have not adversely affected |
primarily there for the individual teams to ensure they have not adversely affected |
| 177 |
other components etc. That is, they do not figure prominently in the scenarios |
other components etc. That is, they do not figure prominently in the scenarios |
| 178 |
driving this proposal.</p> |
driving this proposal.</p> |
| 179 |
<p>Plugin directories currently reflect the id and version of the plugin. With |
<p>Plug-in directories currently reflect the id and version of the plug-in. With |
| 180 |
the addition of the map stamp, the directory names will grow by typically ~10 |
the addition of the map stamp, the directory names will grow by typically ~10 |
| 181 |
chars). It is unlikely but this may push some path length limits. The use of |
chars). It is unlikely but this may push some path length limits. The use of |
| 182 |
the map stamp in a file/dir name may also place additional constraints the set |
the map stamp in a file/dir name may also place additional constraints the set |
| 205 |
list. They select their favourite root, click "all required" and |
list. They select their favourite root, click "all required" and |
| 206 |
get everything they need.</li> |
get everything they need.</li> |
| 207 |
<li>the update manager mechanism may cache downloaded elements |
<li>the update manager mechanism may cache downloaded elements |
| 208 |
(features and plugins) for use in future install operations. That is, if |
(features and plug-ins) for use in future install operations. That is, if |
| 209 |
you already have the plugin/feature, you don't need to download it again. |
you already have the plug-in/feature, you don't need to download it again. |
| 210 |
The |
The |
| 211 |
cache location should be (optionally) set by the user.</li> |
cache location should be (optionally) set by the user.</li> |
| 212 |
<li>ideally the update/install scenarios described will be possible when running |
<li>ideally the update/install scenarios described will be possible when running |
| 228 |
</ul> |
</ul> |
| 229 |
<p>Interesting side note: using the runtime's new ability to run directly out |
<p>Interesting side note: using the runtime's new ability to run directly out |
| 230 |
of jars, it is possible to have many different eclipse configurations sharing |
of jars, it is possible to have many different eclipse configurations sharing |
| 231 |
the same actual plugin files from the update manager's cache/store. This will |
the same actual plug-in files from the update manager's cache/store. This will |
| 232 |
break some assumptions about plugins being together in a plugins directory etc |
break some assumptions about plug-ins being together in a plug-ins directory etc |
| 233 |
(the update reconciler may get upset) and it introduces some challenges for |
(the update reconciler may get upset) and it introduces some challenges for |
| 234 |
cache clean up but it is an interesting possibility.<br> |
cache clean up but it is an interesting possibility.<br> |
| 235 |
</p> |
</p> |
| 236 |
<h3>PDE Build</h3> |
<h3>PDE Build</h3> |
| 237 |
<p>PDE Build already updates feature.xml files with the version numbers of the |
<p>PDE Build already updates feature.xml files with the version numbers of the |
| 238 |
plugins involved in the build. To support the version numbering scheme outlined |
plug-ins involved in the build. To support the version numbering scheme outlined |
| 239 |
above, PDE Build will be updated to </p> |
above, PDE Build will be updated to </p> |
| 240 |
<ul> |
<ul> |
| 241 |
<li>modify the plugin version number in each plugin.xml (i.e., add the map stamp). |
<li>modify the plug-in version number in each plugin.xml (i.e., add the map stamp). |
| 242 |
The version number of a plugin.xml is updated only if it contains <_map |
The version number of a plugin.xml is updated only if it contains <_map |
| 243 |
stamp_> as the version qualifier. This allows plugins to opt out of this |
stamp_> as the version qualifier. This allows plug-ins to opt out of this |
| 244 |
update mechanism.</li> |
update mechanism.</li> |
| 245 |
<li>modify the version number in the feature.xmls (i.e., add the build stamp). |
<li>modify the version number in the feature.xmls (i.e., add the build stamp). |
| 246 |
The version number of a feature.xml is updated only if it contains <_map |
The version number of a feature.xml is updated only if it contains <_map |
| 247 |
stamp_> as the version qualifier. This allows features to opt out of this |
stamp_> as the version qualifier. This allows features to opt out of this |
| 248 |
update mechanism. </li> |
update mechanism. </li> |
| 249 |
<li>(advanced) modify the version number on <import> statements. Some |
<li>(advanced) modify the version number on <import> statements. Some |
| 250 |
plugins and features may specify a perfect match rule in when listing their |
plug-ins and features may specify a perfect match rule in when listing their |
| 251 |
dependencies. While this is not recommended, it is certainly possible. Unfortunately, |
dependencies. While this is not recommended, it is certainly possible. Unfortunately, |
| 252 |
there is a conflict if the intention was that plugin A depend perfectly on |
there is a conflict if the intention was that plug-in A depend perfectly on |
| 253 |
the version of plugin B with which it was built. During the build, B's version |
the version of plug-in B with which it was built. During the build, B's version |
| 254 |
number will be updated according to its map stamp. Plugin A's <import> |
number will be updated according to its map stamp. Plug-in A's <import> |
| 255 |
statement for B needs to be updated to reflect this new version. While this |
statement for B needs to be updated to reflect this new version. While this |
| 256 |
modification is technically possible, it requires more complex analysis of |
modification is technically possible, it requires more complex analysis of |
| 257 |
the plugin.xml file. Updating the version of the plugin itself can be done |
the plugin.xml file. Updating the version of the plug-in itself can be done |
| 258 |
using simple string replacement. In this case the builder has to |
using simple string replacement. In this case the builder has to |
| 259 |
<ul> |
<ul> |
| 260 |
<li>look at each <import> using perfect match and wanting build-time |
<li>look at each <import> using perfect match and wanting build-time |
| 272 |
<p></p> |
<p></p> |
| 273 |
<p>PDE Build will also be updated to output directly to update site format. This |
<p>PDE Build will also be updated to output directly to update site format. This |
| 274 |
is already largely supported but there maybe some modest work required.</p> |
is already largely supported but there maybe some modest work required.</p> |
| 275 |
<p>A possible longer term advance is to change PDE build to only build the plugins |
<p>A possible longer term advance is to change PDE build to only build the plug-ins |
| 276 |
which actually changed. Since we know the map stamp of the plugins we want in |
which actually changed. Since we know the map stamp of the plug-ins we want in |
| 277 |
the output, we can first look for those in the update site (or some cache). |
the output, we can first look for those in the update site (or some cache). |
| 278 |
If present, there is no need to rebuild them. This will have a huge (positive) |
If present, there is no need to rebuild them. This will have a huge (positive) |
| 279 |
impact on rebuilds during release cycles.</p> |
impact on rebuilds during release cycles.</p> |
| 291 |
<p>The key steps that should be taken for 3.0 are </p> |
<p>The key steps that should be taken for 3.0 are </p> |
| 292 |
<ul> |
<ul> |
| 293 |
<li>create/populate an update site</li> |
<li>create/populate an update site</li> |
| 294 |
<li>update PDE build to automate the stamping of plugins and features</li> |
<li>update PDE build to automate the stamping of plug-ins and features</li> |
| 295 |
<li>a modest amount of work on update manager to enable reasonable workflows |
<li>a modest amount of work on update manager to enable reasonable workflows |
| 296 |
(i.e., set a stake in the ground)</li> |
(i.e., set a stake in the ground)</li> |
| 297 |
</ul> |
</ul> |