Bug 208967 - Manifest editor can't handle a large number of friends
Summary: Manifest editor can't handle a large number of friends
Status: VERIFIED FIXED
Alias: None
Product: PDE
Classification: Eclipse Project
Component: UI (show other bugs)
Version: 3.4   Edit
Hardware: PC Windows XP
: P3 normal (vote)
Target Milestone: 3.4 M4   Edit
Assignee: Les Jones CLA
QA Contact:
URL:
Whiteboard:
Keywords: bugday, contributed
Depends on:
Blocks:
 
Reported: 2007-11-06 20:33 EST by John Arthorne CLA
Modified: 2007-12-11 16:03 EST (History)
5 users (show)

See Also:


Attachments
Patch for ExportPackageObject to support formatting of x-friends (6.61 KB, patch)
2007-11-09 04:26 EST, Les Jones CLA
no flags Details | Diff
Alternative - minimal patch (3.58 KB, patch)
2007-11-09 04:28 EST, Les Jones CLA
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description John Arthorne CLA 2007-11-06 20:33:04 EST
Build id: I20071101-0010

 - Add a large number of friend bundles to a package in the manifest editor.
 - Save the editor

-> There is now an error marker in MANIFEST.MF complaining that the manifest contains a line that is too long.
Comment 1 Chris Aniszczyk CLA 2007-11-06 22:05:03 EST
moving to PDE UI... this is probably due to the 72 byte limit in MANIFEST.MF
Comment 2 Chris Aniszczyk CLA 2007-11-06 22:11:34 EST
sigh, moving to PDE UI again...

I thought of renaming this bug "Manifest editor has no friends"
Comment 3 Jeff McAffer CLA 2007-11-06 22:24:23 EST
I don't think this is the 72 byte problem.  Lines longer than that seem ok.  I vaguely recall some other limit of around 255.  That seems more in line with the issue we are seeing.
Comment 4 Chris Aniszczyk CLA 2007-11-06 22:27:42 EST
Tom, what's the official word on limit in MANIFEST.MF?
Comment 5 Les Jones CLA 2007-11-07 07:36:44 EST
FWIW: the limit at which the marker appears in the PDE editor is 512 (i.e. at 511 there's no error).

Also, seems strange to me that the editor doesn't appear to be doing any formatting of the manifest itself. For example, if I add a very long header as part of creating a new plug-in project from a template, it gets wrapped correctly. If I use the same template to add a new extension to an existing project, it doesn't. (To do this you'll need to apply the patch in bug 185477).

From a very quick skim of the code it appears that during the new project creation, the persisting of the manifest information to disk goes via the PluginConverter, which ensures that the fields have carriage returns in them (actually, not at the 72 char boundary either, just at commas). Persistence when adding the template to an existing plug-in project uses the 'normal' editor saving mechanism (i.e. via TextFileDocumentProvider::saveDocument() ).

On the subject of whether there is a limit or not, I'm certainly no expert, but as I see it - "OSGi bundles are always valid JAR files." (taken from section 2.3.2 of the OSGi v4.0 core spec) and "No line may be longer than 72 bytes..." taken from the list of additional restrictions applying to manifests in JAr files (See http://java.sun.com/j2se/1.4.2/docs/guide/jar/jar.html). Seems to me there is a limit. ;-)

Either way, if there is a character limit for OSGi manifests then IMHO the save should really be consistently applying it.
Comment 6 Thomas Watson CLA 2007-11-07 09:19:35 EST
"No line may be longer than 72 bytes..." from http://java.sun.com/j2se/1.4.2/docs/guide/jar/jar.html is correct.

There are two competing goals here:
1) PDE-UI should produce human readable manifest.mf "source" files.
2) PDE-Build or who ever builds the real bundle JAR should produce a valid manifest.mf "binary" file for the JAR.

The widely used jar utilities will take a manifest "source" as input and do the proper formatting to ensure proper line lengths for the "binary" manifest in the output JAR.  Is it really important that the "source" manifest confirm to the strict and often unreadable restrictions of the "binary" manifest?  PDE-Build currently does reformat the "source" manifest into a properly formatted "binary" manifest.  Crack open any of the Eclipse jar'ed bundles and you will find an unreadable Export-Package header in its binary manifest.

All of this is moot in the Equinox runtime, we could care less about silly line length limits :)  Where is the error coming from about the line that is to long?  I thought PDE-UI used the same manifest parser from the framework.

One option for PDE is to have the manifest text editor display the raw text in a human readable format (i.e wrapping at ',' ';' etc) but saving the real file in properly 72 bytes line lengths.  Of coarse that would mean the PDE editor would not be displaying the true "raw" text anymore.  Maybe a checkbox to switch between the a "raw" mode and a "readable" mode?  Sounds a bit too fancy with only a little gain.


Comment 7 Thomas Watson CLA 2007-11-07 09:25:50 EST
(In reply to comment #5)
> From a very quick skim of the code it appears that during the new project
> creation, the persisting of the manifest information to disk goes via the
> PluginConverter, which ensures that the fields have carriage returns in them
> (actually, not at the 72 char boundary either, just at commas). Persistence
> when adding the template to an existing plug-in project uses the 'normal'
> editor saving mechanism (i.e. via TextFileDocumentProvider::saveDocument() ).
> 

PlugincConverter!!! what?!?!  I thought PDE stopped using that from the Framework.  That is some internal class in PDE, right?
Comment 8 Les Jones CLA 2007-11-07 09:36:51 EST
Thomas, I like your argument and it's quite convincing. I guess the question would be, is it valid to not care about the 72 char limit in all typical cases of Eclipse usage today? For example, what about running a project within Tomcat, where the project is both a Bundle and a WebApp. How well does Tomcat handle >72 char lines? What about all the other possible vendors?

I agree that having a 'raw vs presentable' flag on the source tab is perhaps a little OTT, as well as causing work for potentially little gain (?)

The message itself would be easy enough to stop if that's deemed to be the appropriate solution (it's coming from the JarManifestErrorReporter which has a hard coded length check of 512).

>PlugincConverter!!! what?!?!  I thought PDE stopped using that from the
>Framework.  That is some internal class in PDE, right?

It is internal, yes (org.eclipse.pde.internal.core.converter.PluginConverter). Can't comment on whether it 'should' be being used I'm afraid.
Comment 9 Les Jones CLA 2007-11-07 09:38:07 EST
Just to be clear, when I say "... what about running a project within
Tomcat..." I don't mean, export and install. I'm thinking of a developer debugging.
Comment 10 Brian Bauman CLA 2007-11-08 13:41:48 EST
I think the easy answer for this one is to do formatting for x-friends just like we do for the uses directive on Export-Package.  To see how we do this check out org.eclipse.pde.internal.core.text.bundle.ExportPackageObject.formatUsesDirective().  This should hopefully also improve the readability of long lists of x-friends.
Comment 11 Les Jones CLA 2007-11-09 04:26:44 EST
Created attachment 82523 [details]
Patch for ExportPackageObject to support formatting of x-friends

Thanks for the pointer Brian; attached is a patch that formats the x-friends directive in a similar way to the uses directive.

Although the new method formatDirective(...) is semantially similar to your formatUsesDirective(...), it looks very different because it's written in my style. I can appreciate that often consistency is a very important aspect of a solution, so in case you'd rather a solution that utilised a style similar to the existing formatUsesDirective(...), I'll also attach another patch that is the same (semantically) but more minimal change.
Comment 12 Les Jones CLA 2007-11-09 04:28:13 EST
Created attachment 82524 [details]
Alternative - minimal patch

As promised, an alternative patch, with more minimal changes. Personally, I prefer the first one, but appreciate that consistency can be important.
Comment 13 Les Jones CLA 2007-11-09 04:35:26 EST
The two patches attached merely address the 'tactical' issue - that of the x-friends directive becoming too long; it doesn't address the wider issue of whether the manifest should be being formatted to a maximum 72 char width within the development environment.

I'm personally not too fussed about the 72 character limit, on the assumption that many platform implementations probably don't care either. However, if it's shown that implementations (such as EE vendors or alternative OSGi implementations) do have an issue with this, then it could become an inhibitor to using Eclipse as the development tooling. 

I guess at that point, we could raise a bug?
Comment 14 Brian Bauman CLA 2007-11-09 11:56:39 EST
Les, thanks for working on this and providing two patches.  In the future, don't worry about doing twice the work (though I appreciate the choices of patches).  Since you have worked so hard and been very active, I figured I would just go with the formatting you prefer :)

There was one small change.  On the loop with the List, I changed it to:

for (ListIterator iterator = usesList.listIterator(); true;) {
	sb.append(iterator.next());
	if (iterator.hasNext()) {
		sb.append(',');
		if (newLine) {
			sb.append(EOL).append(INDENT3);
		}
	} else {
		break;
	}
}

I have been told in some cases ListIterator has optimizations not found in Iterator (but who knows).  This is a little cleaner than using a boolean to mark the first time through.

Thanks for working on this and turning around a patch so quickly.  Yet another very good quality patch!
Comment 15 Brian Bauman CLA 2007-11-09 12:01:32 EST
(In reply to comment #13)
> it doesn't address the wider issue of
> whether the manifest should be being formatted to a maximum 72 char width
> within the development environment.

Just to address this before I forget, this has come up in the past.  It was decided since 72 is really just a number someone made up and that we are interested in providing more readable Manifests, that we do not support this in the workspace.  The runtime has no problems.  Upon export and jarring of bundles, I believe the 72 characters are enforced.  

Unless we find a situation that causes problems, we will choose readability over an arbitrary number.
Comment 16 Brian Bauman CLA 2007-12-11 16:03:18 EST
Verified on I20071211-0010.