Bug 89428 - [doc] Underscore replaced with dash in version qualifiers
Summary: [doc] Underscore replaced with dash in version qualifiers
Status: RESOLVED FIXED
Alias: None
Product: Platform
Classification: Eclipse Project
Component: Runtime (show other bugs)
Version: 3.1   Edit
Hardware: PC Windows XP
: P3 major (vote)
Target Milestone: 3.2 RC7   Edit
Assignee: Pascal Rapicault CLA
QA Contact:
URL:
Whiteboard:
Keywords:
: 121424 122984 123394 123940 124032 125975 126688 127257 144029 (view as bug list)
Depends on:
Blocks: 138825
  Show dependency tree
 
Reported: 2005-03-29 17:21 EST by Philippe Ombredanne CLA
Modified: 2006-06-01 18:24 EDT (History)
15 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Philippe Ombredanne CLA 2005-03-29 17:21:03 EST
If I enter a version with a qualifier in the Overview pane of the feature
manifest editor, my edits are transformed in some cases.
For instance, if I enter 3.1.0.31_v20050329_test as a version, my edits gets
transformed as 3.1.0.31-v20050329-test.

Underscores are transfored in dashes. Editing the raw XML of the manifest is a
workaround.
Comment 1 Philippe Ombredanne CLA 2005-06-05 12:52:15 EDT
Note that I understand the rationale behind this behavior.
The version is extracted/parsed from the plugin jar or directory file name and
everything right of the underscore is the verion identifier.
But that does not mean that further underscore should be restricted
Especially since if you want to create a version number that has a CVS tag name
as a qualifier.
Dashes are not allowed in CVS tag names, only underscores ;-)

Comment 2 Konrad Kolosowski CLA 2005-06-05 14:30:20 EDT
org.eclipse.core.runtime.PluginVersionIdentfier implementation mangles 
qualifier.
Comment 3 Pascal Rapicault CLA 2005-06-06 09:13:08 EDT
Konrad, where and why is PDE using PluginVersionIdentifier?
Even though it is not marked as deprecated (probably because we forgot it in
3.0) it is, since all the references runtime have on it are from deprecated classes.

Instead PDE should use the class org.osgi.framework.Version.

Marking for RC2 to mark PluginVersionIdentifier as deprecated.
Comment 4 Konrad Kolosowski CLA 2005-06-06 11:20:43 EDT
There are 63 references in various PDE plug-ins and places, to 
PluginVersionIdentifier, and 19 references to Version
Other plug-ins I checked (Help, Update) also use PluginVersionIdentifier.

PluginVersionIdentifier was used for years, and there was never any known 
reason that PluginVersionIdentfier should not be used.
There are even reasons not to switch:
Version and PluginVersionIdentfier are completely different with respect to 
characters allowed in qualifier.  plugin.xml documentation refers to 
PluginVersionIdentifier which states that qualifier is uninterpreted string.
PDE, Update need to support legacy plug-ins so use of PluginVersionIdentifier 
seems preferred.  Switching to Version now may break old plug-in as it makes 
more restrictions on the qualifier (although allowing "_").
PluginVersionIdentfier has a useful method
public static IStatus validateVersion(String version) that is supposed to 
return helpful message in the IStatus ("object indicating what is wrong with 
the string").  Version is only documented as throwing IllegalArgumentException
It is late, and I don't think it is safe to require everybody to switch to 
suddenly switch to Version, and not nice to ship referencing deprecated 
classes.  You may reference PluginVersionIdentfier from compatibility only, 
but PluginVersionIdentfier is self contained, does not need compatibility.

I think for 3.1, you should only fix PluginVersionIdentfier to allow "_".
Comment 5 Konrad Kolosowski CLA 2005-06-06 15:06:52 EDT
It will be easier to fix this bug in PDE-UI for 3.1.  In most places in PDE 
using PluginVersionIdentfier does not cause visible problems.  Feature editor 
(and wizard) used PluginVersionIdentifier for validating which returned OK, 
then used PluginVersionIdentifier.toString() which returned different String 
than passed in the constructor.
I will just change PDE to use the original string.
Comment 6 Konrad Kolosowski CLA 2005-06-06 15:08:04 EDT
Fixed.
Comment 7 Philippe Ombredanne CLA 2005-06-06 15:40:46 EDT
Thanks Konrad!
You da man:-)
Comment 8 Andrew Niefer CLA 2006-01-13 16:56:28 EST
This is still a general problem for features.  IFeature and IFeatureReference use PluginVersionIdentifier via org.eclipse.update.core.VersionedIdentifier.

The result being that exporting a feature that contains a '_' in its qualifier will fail with "Unable to find feature".  And a releng style build will produce features with the '_' changed to '-'.
Comment 9 Pascal Rapicault CLA 2006-01-13 17:08:36 EST
I can only agree with Andrew.
Today this block us to update from an I build to another, and in a near future could cause grief to deliver Calisto (note calisto is supposed to be shipped with udpate manager).
Raising the priority as I should have done before.
Comment 10 Wassim Melhem CLA 2006-01-13 17:10:38 EST
does this "blocker" belong in Update?  I don't see how PDE/UI is blocking you.
Comment 11 Wassim Melhem CLA 2006-01-13 17:13:23 EST
I will re-close this bug report as the problem specified has been addressed successfully.

Please open new bug reports against the correct components.
Comment 12 Wassim Melhem CLA 2006-01-13 20:17:14 EST
Better yet, Runtime should either deprecate PluginVersionIdentifier or fix it.
Comment 13 Jeff McAffer CLA 2006-01-14 09:18:29 EST
We should consider just tossing the old and in with the new.  What would be the consequences of changing to use Version and thus allowing _?
Comment 14 Pascal Rapicault CLA 2006-01-14 12:10:30 EST
Re-opening and moving in the runtime bicket
Comment 15 Pascal Rapicault CLA 2006-01-16 09:59:33 EST
*** Bug 123940 has been marked as a duplicate of this bug. ***
Comment 16 Pascal Rapicault CLA 2006-01-16 09:59:47 EST
*** Bug 123394 has been marked as a duplicate of this bug. ***
Comment 17 Pascal Rapicault CLA 2006-01-16 10:46:49 EST
Fixed in HEAD.
Comment 18 Pascal Rapicault CLA 2006-01-16 17:08:02 EST
*** Bug 124032 has been marked as a duplicate of this bug. ***
Comment 19 Jeff McAffer CLA 2006-01-16 22:18:20 EST
(In reply to comment #17)
> Fixed in HEAD.

What was the fix?  Please summarize so that in future we can find the bug and know what actually happened.
Comment 20 Pascal Rapicault CLA 2006-01-17 09:10:29 EST
I changed the PluginVersionIdentifier class. It now delegates to Version. This slightly changes / fixes the behavior of the PluginVersionIdentifier since non alphabetical character are no longer replaced by a dash. 
Comment 21 Andrew Irvine CLA 2006-01-19 13:32:24 EST
*** Bug 124032 has been marked as a duplicate of this bug. ***
Comment 22 Jeff McAffer CLA 2006-01-21 20:26:07 EST
This needs a migration guide entry
Comment 23 Pascal Rapicault CLA 2006-02-01 09:00:50 EST
*** Bug 125975 has been marked as a duplicate of this bug. ***
Comment 24 Pascal Rapicault CLA 2006-02-01 09:01:07 EST
*** Bug 121424 has been marked as a duplicate of this bug. ***
Comment 25 David Williams CLA 2006-02-01 11:41:14 EST
may I request clarification on the status of this bug? Sorry, I get a little lost reading all the entries. Here's what I hope I am reading: 

1. PDE is/will be fixed so that '_' is no longer converted to '-' (even for features). 

2. The doc/migration guide referred to is just to warn those that might have somehow been counting on this conversion taking place? 

And ... if I am reading correctly, any estimate on when this would (or does) appear in the basebuilder code packages?
Comment 26 Kim Moir CLA 2006-02-01 13:49:11 EST
If this bug is fixed, basebuilder would be upgraded with the plugins from the latest milestone after it was released, as usual.
Comment 27 David Williams CLA 2006-02-01 23:43:40 EST
ok, so, guess the question is "if/when" this will be fixed, and looks like the target is 3.1 M5! :) 

Thought I'd write down my understanding of "effects" of fixing the remaining feature part bug, in case I am mis-understanding, others can correct me. 

Since "_" is greater than "-" (in the osgi sense of comparing versions) the bug can be fixed in an evolutionary way, without the worst king of update manager errors. 

that is, if a feature is currently incorrectly being labeled with, say, '05-01' and then later, the feature was not manually changed, but just as a result of changing PDE, it would start to be labeled with '05_01', then '05_01' would be "seen" as being "newer" (greater than) '05-01' so the "newer" one would be used. (in other words, the error would be "over inclusion", not "ignoring" the newer one. 

[I do, by the way, think this should be fixed, if at all possible ... very frustrating to waste hours trying to figure out why I kept getting 'hyphen', and some entry in the docs, or readme, would not have prevented that waste of my time]
Comment 28 David Williams CLA 2006-02-03 20:49:53 EST
This is, I think, a worse problem than seems to be being acknowledged by the remarks and observations in this bug. 

Basically, I do not believe '_' can be used in the version qualifier, contrary to OSGI specifications. 

in 3.2, using '_' in even plugin version qualifiers causes erroneous warnings in all the "configuration" details/proproperties. (That's in M4 based builds, not sure if/when the "fixed in head" was applied to a build.)



in 3.1.x, besides the above problem, "source" plugins are not correctly associated with their "code" plugins, if '_' is in the plugin's qualifier. 

To reproduce, you can install recent builds of WTP, such as 
http://download.eclipse.org/webtools/downloads/drops/I-I200602022035-200602022035/
for M4 based builds. 

Or, 
http://download.eclipse.org/webtools/downloads/drops/M-M200602010238-200602010238/
for a 3.1.2 based build. 


Comment 29 David Williams CLA 2006-02-03 21:20:03 EST
To give more details, the erroneous error messages are always of the form, 

Plug-in: "org.eclipse.wst.validation" version: "1.0.1.v20060130-2215" referenced by this feature is not included at runtime. Runtime includes plug-in version "1.0.1.v20060130_2215" supplied by feature "org.eclipse.wst.common_core.feature" version "1.0.0".

Even though there really is no hyphen in the qualifier, in the feature.xml, nor plugin manifest ... they both use the underscore. 

I suspect this same "misreading" is what causes the source plugin not to be found. 

BTW, everything seems to actually run fine (no error messages then) but, the source not being associated is a "blocker" for our SDK users. 

Comment 30 Gunnar Wagenknecht CLA 2006-02-07 02:32:34 EST
*** Bug 126688 has been marked as a duplicate of this bug. ***
Comment 31 Gunnar Wagenknecht CLA 2006-02-07 02:43:23 EST
(In reply to comment #29)
> BTW, everything seems to actually run fine (no error messages then) but, the
> source not being associated is a "blocker" for our SDK users. 

No. I installed WTP SDK I200602022035 in a clean 3.2 M4 with all its dependencies and it doesn't work. I've no WTP new project wizards, no WTP perspectives, no WTP views. Thus, this is a real blocker.

Comment 32 Pascal Rapicault CLA 2006-02-08 12:08:23 EST
I've just tested WTP (wtp-sdk-M200602010238.zip) with all the appropriate dependencies and here what I see:
- out of the box I see the following errors in configuration details:
    - Plug-in "org.eclipse.jst.doc.isv" version "1.0.0.v20060128-0223" referenced by this feature is missing.
    - Plug-in "org.eclipse.wst.doc.isv" version "1.0.0.v20060128-0211" referenced by this feature is missing.
    I did notice that the "_" is changed into a "_".
    But more importantly I also noticed that the plug-ins are *not* present in the zips!

- I then reapplied the same fix than in the 3.2 stream and I then noticed that the error messages are now saying (notice the "_" in place of the "-" ):
    - Plug-in "org.eclipse.jst.doc.isv" version "1.0.0.v20060128_0223" referenced by this feature is missing.
    - Plug-in "org.eclipse.wst.doc.isv" version "1.0.0.v20060128_0211" referenced by this feature is missing.

I have also been able to reproduce the source lookup bug and you should enter a bug against PDE UI (plz cc me).

Given that the workaround to fix the problem consists in making sure "_" is not used in the last segment of the version number, I suggest that we do not change the runtime code nor the update.configurator code.
Comment 33 David Williams CLA 2006-02-09 01:50:16 EST
FYI, don't let the the missing doc.isv's through you off :) they may be missing, and they may produce "red x's" in the configuration manager. But they are not the source of the issues we have. Even out of the box, the other, especially "source" features show as warnings with the '-' and '_' mix up. 

But, if I understand you right ... you are saying that doesn't matter, and the source look-up problem is probably PDE's UI bug?  I've opened bug 127030 to request their investigation. 

Thanks. 

Comment 34 Pascal Rapicault CLA 2006-02-10 12:38:08 EST
*** Bug 127257 has been marked as a duplicate of this bug. ***
Comment 35 Pascal Rapicault CLA 2006-02-17 09:00:01 EST
*** Bug 122984 has been marked as a duplicate of this bug. ***
Comment 36 David Williams CLA 2006-04-04 17:43:53 EDT
This target was changed to "RC3"? I thought it was "already fixed"? 
So, I'm wondering that the fix is? Is it "just doc"?! 

Comment 37 Pascal Rapicault CLA 2006-04-04 18:39:05 EDT
Yes it was moved to rc3 because now this bug is used to track the doc / readme problem. The initial problem has been solved a while ago.
Comment 38 David Williams CLA 2006-04-26 22:51:28 EDT
Sorry to ask again, but I'm wondering if I should (re) open some bug related to 
this issue. Maybe not "runtime" exactly, but 3.1.2 update manager? Maybe PDE build? 

for WTP 1.0.2 I used M6_32 base builder, specifically to create "proper" version qualifiers in features (where the qualifier is based on the features own cvs tag, and also the tags of all the plugins it contains). But, now I see that the pseudo-random-monotonic function allows underscores to surface ... so, our WTP 1.0.2 builds don't work with 3.1.2 update manager. 

To give examples, the following feature were generated with undercores in the qualifier: 

org.eclipse.wst.common_core.feature.source_1.0.2.v200604160100--_OmSTOStcSZvm-

The 3.1.2 update manager complains, for examample, "can't find feature 
org.eclipse.wst.common_core.feature.source_1.0.2.v200604160100---OmSTOStcSZvm-

So ... what to do? Is it fair to ask this qualifer calculator to never use underscores? Should 3.1.2 be fixed? Or is the planned option for developers to just documnet that they should never ever use generateFeatureVersionSuffix 
if they are planning to deploy to 3.1.x? (with the limitations that entails). 

Seems it will be a persistent problem for anyone who wants to use 3.2 to develop fixes/additions to 3.1.x based installations. 

Any suggestions? 


Comment 39 Pascal Rapicault CLA 2006-04-27 09:57:12 EDT
I don't see this bug being backported on 3.1.x, it seems to be a big change for a point release. Moreover there is no guarantee that fixing this will make the complete story smooth, even from a runtime point of view.

For people targeting 3.1.x with a 3.2 build, we should just recommend them to not use the generated suffix, or use pde build from 3.1.
Comment 40 David Williams CLA 2006-04-27 11:49:48 EDT
Thanks Pascal. And, I guess a "surpress underscores" option is too expensive to implement? 


The PDE 3.2 build is better than the 3.1 (it does allow the .qualifier tag to work off of CVS tags, at least), so, I hope there is some level of support for using 3.2 to develop 3.1 code ... seems that's always been the case before. Even if the "generateFeatureVersionSuffixWithNoUnderscores" could not be implemented. 

Thanks again. 
Comment 41 Andrew Niefer CLA 2006-04-27 14:55:48 EDT
There is quite conveniently 64 allowed characters (including the underscore) for OSGi qualifiers.  The "pseudo-random-monotonic" suffixes are created by adding up the qualifiers of the contained plugins/features base 64.  Switching to base 63 is certainly not as pretty and its more expensive.  While excluding the underscore is certainly posssible, its just too late in the 3.2 cycle to do now.
Comment 42 David Williams CLA 2006-04-27 23:19:05 EDT
Thanks for explaining the (base 64) logic behind the "pseudo-random-monotonic" suffixes. 

Let me suggest something more radical. Why, exactly, is this "summation" method used? Why not just use, as a suffix, the highest version qualifier in the contained plugins (actually, I assume this, as would be the summation? that its based on all four feilds of contained plugins). 

I'm sure I'm missing all the mathematical nicities of what you are doing "for free" ... but, if people were serious about it, there are other schemes that would work, right? 

I'm partially persisting in my questions because this has been an expense error for us in WTP ... so, I'm just trying to plan ahead. 

Thanks again, 
Comment 43 Pascal Rapicault CLA 2006-04-28 09:50:26 EDT
A sum needs to be done otherwise modifications will be missed.
For example 
 Build #1
 - F includes P v 4.0.0.v20060427
     includes A v 3.0.0.v20060328
 Build #2
 - F includes P v 4.0.0.v20060427 (same version as before)
     includes A v 3.0.0.v20060404 (new version)

With your proposal the qualifier of F on build#2 will be the same than in build #1.
   
Comment 44 David Williams CLA 2006-05-01 01:21:14 EDT
(In reply to comment #43)
> 
> With your proposal the qualifier of F on build#2 will be the same than in build
> #1.
> 

Thanks Pascal. Keeping in mind my goal is to come up with a verbal rule to implement this "by hand" on the 3.1.2 stream (since we can not use the automatic calculation since it has underscores) ... your remark helps a lot. I was thinking of "most recent date" of the pluign, but in fact, what I had in mind was "the current date" as suffix, when any plugin was changed (which would also have the current date). 

I can not imagine we could ever compose a feature with some past version of a plugin already built. So, "current date" would apply to both the plugin, and the feature suffix. 

Your automatic rule sounds like a good algorithm to handle all possible contengencies, but -- given is has underscores, and in general looks ransom to users and developers ... seems like the "current date" would work for us. 

I write all this down, of course, just to increase the probablility of someone telling me how wrong I am and what a bad plan this is to think of "current date" in the above "workflow" as the suffix :) 

Comment 45 Pascal Rapicault CLA 2006-05-01 06:16:01 EDT
The date based approach may work if you are careful to have enough precision in your date so that you can run multiple builds the same day. Still the difficulties with this approach are:
- figuring out that a plug-in has changed (you need to have a reference to verify against)
- having a mechanism that computes the appropriate number for the feature (you need to have a reference to verify against)
- you can not easily track back the plug-in to the source unless you tag with the source with the date
Comment 46 David Williams CLA 2006-05-02 00:55:54 EDT
Thanks for the further suggetstions and clarifications. 

We in WTP do, BTW, use year, month, date, hours, and minutes, so .. good to go there.
 
I might also add (in case this ever makes it into an algorithm), we also insist on UTC times in CVS time stamps, since we have a world-wide team. Those time stamps get "released" to a map file, and those map files tagged with each build .. so, we should have all the info we need ... 
now if I can  j u s t   f i n d   s o m e o n e  to code it! :)
Comment 47 John Arthorne CLA 2006-05-08 16:12:21 EDT
Updating milestone for unfinished doc work
Comment 48 Jeffrey Liu CLA 2006-06-01 13:33:23 EDT
*** Bug 144029 has been marked as a duplicate of this bug. ***
Comment 49 DJ Houghton CLA 2006-06-01 18:17:16 EDT
Added to README: 

In Eclipse 3.0, the use of an underscore ('_') character in the qualifier segment of a version identifier was not supported, but also was not enforced. If a plug-in version identifier contained an underscore in the qualifier, then this character was transformed to a hyphen ('-') when exporting the plug-in to the file-system and also when installing the plug-in from an update site.

In Eclipse 3.1 the rules for characters allowed in qualifiers was relaxed to include the underscore character, so when an offending plug-in was exported or installed the qualifier was not modified from its original state. This subtle change was accidently left out from the migration guide for 3.0 to 3.1.

The story going forward (and for Eclipse 3.2) is that we are maintaining compatibility with Eclipse 3.1 and plug-ins who use underscore characters in their version qualifiers, should be aware of the aforementioned changes when dealing with old plug-in versions (both when exporting and when they exist on update sites). This means, for instance, that plug-in providers who have old versions of their plug-in on an update site should ensure that the name in the file-system matches the name of the plug-in.

Comment 50 Pascal Rapicault CLA 2006-06-01 18:24:41 EDT
In fact it has been added to the porting guide.