Bug 217930 - Provide ee property for binary source mapping
Summary: Provide ee property for binary source mapping
Status: VERIFIED FIXED
Alias: None
Product: JDT
Classification: Eclipse Project
Component: Debug (show other bugs)
Version: 3.4   Edit
Hardware: PC Windows XP
: P3 enhancement (vote)
Target Milestone: 3.4 M6   Edit
Assignee: Darin Wright CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2008-02-05 16:42 EST by Berthold Lebert CLA
Modified: 2008-02-29 13:40 EST (History)
5 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Berthold Lebert CLA 2008-02-05 16:42:14 EST
Build ID: I20080205-0010

Steps To Reproduce:
Following the instructions in bug 210978 for binary to source mapping, the ee file for DesktopEE would need a -Dee.bootclasspath property value longer than 1024 characters. All the files in the \lib\jclDEE\ext directory would need to be listed in -Dee.bootclasspath and their corresponding source in -Dee.src.

Trying to start Eclipse pointing to default.ee with a property value longer than 1024 bytes causes a crash.  I don't know if this is a Java limitation, a J9 problem, or something else.

We could avoid this problem, and make the ee file much easier to maintain, if we add a new ee property which maps binary jars to source files. For example:

ee.src.map=${ee.home}\lib\charconv.zip=${ee.home}\lib\charconv-src.zip;\
${ee.home}\lib\jclDEE\classes.zip=${ee.home}\lib\jclDEE\source\source.zip;\
${ee.home}\lib\jclDEE\*.zip=${ee.home}\lib\jclDEE\source\*-src.zip;\
${ee.home}\lib\jclDEE\ext\*.jar=${ee.home}\lib\jclDEE\source\*-src.zip

The substitution characters * and ? could be used.
Comment 1 Curtis Windatt CLA 2008-02-22 12:39:33 EST
Will consider for 3.4.
Comment 2 Curtis Windatt CLA 2008-02-22 14:55:06 EST
Andrew, can you please comment on this?  Could the launcher can handle ee properties being broken up onto separate lines (ending with '\').  Debug can handle this by simply concatenating the lines as we read them in.
Comment 3 Curtis Windatt CLA 2008-02-22 15:02:20 EST
Berthold, is this map intended to attach source locations for jars outside of the bootclasspath (such as the jars in ee.additional.dirs)?  How important are the wildcard characters, as they may add a fair amount of complexity to the solution?  
Comment 4 Berthold Lebert CLA 2008-02-22 15:31:13 EST
Yes, the mapping would apply to all jars found on any of the paths (boot, endorsed, additional and ext dirs).

The wildcards would provide significant flexibility.  Right now we have to specify all jars in the ee.additional.dirs in the ee.bootclasspath property to be able to map the source file. We have 23 optional files in the addtional.dirs, 17 with source, all follow the same naming convention:
${ee.home}\lib\jclDEE\ext\*.jar=${ee.home}\lib\jclDEE\source\*-src.zip

Without wildcards we would also need to know beforehand which jars are in the ext.dirs or additional.dirs.  We deliver our JRE to Lotus Expeditor, who in turn can remove and/or add any jar file to the ext/additional.dirs as part of their build and deliver it to Lotus Sametime, and so on.
Comment 5 Andrew Niefer CLA 2008-02-22 15:33:51 EST
The launcher can not currently handle lines broken up with \

If the purpose of providing broken lines is the 1024 restriction, then providing such support in the launcher doesn't help anything.  After launcher joins such lines, it looks exactly like a single unbroken line and there is no difference as far as java is concerned and the problem still exists.

The launcher can not really pass the lines broken as they are since they will be considered invalid arguments by java and will be lost.

A solution would be to break it up as:

-Dee.src.map=${ee.home}\lib\charconv.zip=${ee.home}\lib\charconv-src.zip;\
-Dee.src.map.1=${ee.home}\lib\jclDEE\classes.zip=${ee.home}\lib\jclDEE\source\source.zip;\
-Dee.src.map.2=${ee.home}\lib\jclDEE\*.zip=${ee.home}\lib\jclDEE\source\*-src.zip;\
-Dee.src.map.3=${ee.home}\lib\jclDEE\ext\*.jar=${ee.home}\lib\jclDEE\source\*-src.zip

When debug reads it, if ee.src.map ends with \, then it looks for a ee.src.map.1 and so on.


If the purpose of breaking up the lines is simply readability, then yes the launcher could do that.

Note that the launcher's own 1024 problem was fixed in M5 with bug 214635
Comment 6 Berthold Lebert CLA 2008-02-22 15:40:29 EST
This requirement is not about continuation lines \  even if they would be nice to have the ability :)

I used them in my initial bug comment to shorten the lines so they can be easily read in bugzilla.
Comment 7 Curtis Windatt CLA 2008-02-26 16:06:09 EST
What about only applying the wildcards to the file name?  It would simplify the replacement and reduce the chance of map entries clobbering each other.

ex:
${ee.home}\lib\*.zip=${ee.home}\lib\source\*.zip;\
${ee.home}\lib\jclDEE\*.zip=${ee.home}\lib\jclDEE\source\*-src.zip;\

If wildcards apply to the whole path, both of the above patterns would match any zips in the jclDEE folder.

ex2:
${ee.home}\lib\*.zip=${ee.home}\lib\source\src-*.zip;\

If wildcards apply to the whole path, the above pattern would match any zip file in a sub folder and would use a very odd looking source zip (\lib\source\src-subfoldername\zipname.zip).
Comment 8 Berthold Lebert CLA 2008-02-26 16:19:33 EST
>>What about only applying the wildcards to the file name?

That works for me.
Comment 9 Curtis Windatt CLA 2008-02-28 10:55:46 EST
I take back my ealier comment, it was easier to apply the wildcards to the whole path rather than just the filenames.  If people are having problems with wildcards and subfolders it can always be changed later.

The code to parse, match and replace the paths is slow, so a complex map and/or a lot of possibly libraries to match against, will add time to the ee file loading (with 6 map entries and a dozen libraries it added almost a second to the time on my machine).  However, part of the problem is that the code is running more than once, so I filed bug 220654 to improve it.

The map can use ${ee.home}, .., ?, *.  If a path is relative, it is made absolute by appending it to the location of the eeFile.  Both forward and backslashes should be supported as we convert the paths to their OS specific form.  The wildcards in the map value must match the order of the wildcards in the key (ex foo?blah*=src*boo? is invalid).

The -Dee.src property will now have to be a single path (not a list anymore).  This source location will be applied to all of the bootclasspath libraries.  The -Dee.src.map property will apply to all libraries (ext, additional, bootclasspath) and will overwrite the path set by -Dee.src.

I will update the wiki page.

Fixed in HEAD.  See EEVMType, LibraryLocation, LaunchingMessages and the JUnit tests
Comment 10 Curtis Windatt CLA 2008-02-28 10:56:27 EST
Darin, please verify.
Berthold, please try it out and let me know how it goes.
Comment 11 Berthold Lebert CLA 2008-02-29 13:40:05 EST
Verified in N20080229-0010.

Works very well. Thanks!

The WEME/WECE configurations can now use a common mapping scheme like this:

-Dee.src.map=\
${ee.home}\lib\jcl*\classes.zip=${ee.home}\lib\jcl*\source\source.zip;\
${ee.home}\lib\*.zip=${ee.home}\lib\*-src.zip;\
${ee.home}\lib\jcl*\*.zip=${ee.home}\lib\jcl*\source\*-src.zip;\
${ee.home}\lib\jcl*\ext\*.jar=${ee.home}\lib\jcl*\source\*-src.zip

(Note that the continuation character \ is not supported by the Eclipse launcher and is used in above only for readability)