Bug 5399 - Improve file encoding support
Summary: Improve file encoding support
Status: RESOLVED FIXED
Alias: None
Product: Platform
Classification: Eclipse Project
Component: Text (show other bugs)
Version: 1.0   Edit
Hardware: PC Windows 2000
: P1 major (vote)
Target Milestone: 3.0   Edit
Assignee: Platform-Text-Inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords:
: 13949 14435 18751 21161 22332 (view as bug list)
Depends on:
Blocks:
 
Reported: 2001-11-01 04:50 EST by Dani Megert CLA
Modified: 2004-09-22 04:46 EDT (History)
14 users (show)

See Also:


Attachments
test.xsl (2.12 KB, application/octet-stream)
2001-11-05 03:13 EST, Dani Megert CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Dani Megert CLA 2001-11-01 04:50:08 EST
See: news://eclipse.org/9rp7ib%249m4%241%40rogue.oti.com
Comment 1 Erich Gamma CLA 2001-11-01 04:55:15 EST
pls investigate this could be a 1.0 fix candidate. Andre can help you with the 
set-up.
Comment 2 Dani Megert CLA 2001-11-05 03:13:31 EST
Created attachment 67 [details]
test.xsl
Comment 3 Dani Megert CLA 2001-11-05 03:14:53 EST
e-mail sent to me directly by the newsgroup problem reporter:
Hi Dani,

I tested on Websphere Studio Site Developer's 10/15 build,
so whichever Eclipse is used for that is the one i used.

Here's the test file that I used.

(See attached file: test.xsl)

I just searched for one of the Chinese character elements, and the XSL file
wasn't found.
Comment 4 Dani Megert CLA 2001-11-05 06:03:10 EST
Sent e-mail back to orignal problem reporter:

We have looked at the file and have some additional questions. We assume that 
the wrong encoding is caused by the fact that this 
type of file (xsl) is interpreted by IE.

1) What is the default encoding inside Eclipse? Evaluate the following code 
snippet in a scrapbook page:
    System.getProperty("file.encoding")
2) Please compare the chinese characters in IE with the ones displayed in the 
text or Java editor inside Eclipse (open the file with 
the default text editor). Are they rendered the same way?
3) Open the xsl file with Notepad or Word. Are the chinese characters rendered 
the same way as in IE?
4) Change the type of the test file to text (i.e. txt suffix), open it in IE, 
compare the characters. We assume that your search 
scenario will work in this case.
5) Please check the Help->About menu to see the Eclipse build and version info. 
We do not know on which Eclipse build is used for 
which WSSD build.
Comment 5 Dani Megert CLA 2001-12-03 09:09:29 EST
Sent another reminder to the problem reporter.
Comment 6 Phillip Avery CLA 2001-12-03 11:17:43 EST
Here is some more information:

1) What is the default encoding inside Eclipse?

   MS950

2) Please compare the chinese characters in IE with the ones displayed in the 
text or Java editor inside Eclipse (open the file with the default text 
editor). Are they rendered the same way? 

   No, they render as ???

3) Open the xsl file with Notepad or Word. Are the chinese characters rendered 
the same way as in IE? 

   Yes, they are the same.

4) Change the type of the test file to text (i.e. txt suffix), open it in IE, 
compare the characters. We assume that your search scenario will work in this 
case. 

   The characters are still correct in IE.

5) Please check the Help->About menu to see the Eclipse build and version info. 
We do not know on which Eclipse build is used for which WSSD build. 

    Build: 20011026
Comment 7 Dani Megert CLA 2001-12-06 11:46:01 EST
This is a general Eclipse "problem": the platform does not look into the file to 
detect its type and treat it accordingly. In this case it should recognize the 
file as of type XML and use the specified encoder to read the file (i.e. provide 
the contents) - IFile.getContents could do this.

If we only enhance the editor to handle such cases then (text) search would give 
wrong results.

Moving to platform ui for comment.
Comment 8 Kevin Haaland CLA 2001-12-18 20:35:52 EST
Similiar requests have come up before and there were some discussions about 
adding mime-type support. This will require involement from multiple teams:

   SWT & Core to expose platform capabilities and to honour the encodings. 
   UI - To expose this base capability to the user. 

Taking it up one more level, this enhancement would permit true multi-lingual 
applications. 

It's a great idea but I do not believe there is time to address this request 
now. 
Comment 9 Kevin Haaland CLA 2002-01-21 22:41:13 EST
Defer
Comment 10 Randy Giffen CLA 2002-08-08 16:47:52 EDT
Reopen for investigate
Comment 11 Kevin Haaland CLA 2002-09-03 14:01:08 EDT
Encoding support needs to improve in 2.1
Comment 12 Kevin Haaland CLA 2002-09-03 14:02:45 EDT
See duplicate defect report for other cases where encoding support needs work
Comment 13 Kevin Haaland CLA 2002-09-03 14:04:34 EDT
*** Bug 13949 has been marked as a duplicate of this bug. ***
Comment 14 Kevin Haaland CLA 2002-09-03 14:10:02 EDT
*** Bug 14435 has been marked as a duplicate of this bug. ***
Comment 15 Kevin Haaland CLA 2002-09-03 14:11:53 EDT
*** Bug 18751 has been marked as a duplicate of this bug. ***
Comment 16 Kevin Haaland CLA 2002-09-03 14:14:16 EDT
*** Bug 21161 has been marked as a duplicate of this bug. ***
Comment 17 Kevin Haaland CLA 2002-09-03 14:17:56 EDT
*** Bug 22332 has been marked as a duplicate of this bug. ***
Comment 18 Bob Foster CLA 2002-09-05 12:14:59 EDT
Is there somebody named Platform-UI-Inbox?
Comment 19 Kai-Uwe Maetzel CLA 2002-09-27 14:08:52 EDT
This one of the top 10 issues for 2.1 Anyway, whatever bin we choose, it's 
still a cross component issue.
Comment 20 Kai-Uwe Maetzel CLA 2003-02-04 05:21:32 EST
Still open in 2.1. Has been shifted to 2.2. The minor improvements in Platform 
Text don't solve this problem. Moving to Later.
Comment 21 Bob Foster CLA 2004-02-22 13:19:29 EST
Last I looked improve encoding support was on the to-be-done list for 3.0. Is
someone working on it? What progress has been made? Is there any design note?
Reason I ask, I'd like a little notice before discovering that some milestone
has broken the encoding support I have added to my editor plug-in.
Comment 22 DJ Houghton CLA 2004-02-23 10:53:18 EST
Bug report covering the plan item is bug 37933. 

Initial APIs changes have been released. 
Doc to be updated and posted to the web soon. (this week I believe)
Comment 23 Dani Megert CLA 2004-09-22 04:44:54 EDT
Encoding support has been improved for 3.0 and in the 3.1 stream (3.1 M2 or
newer) the UI to change the encoding has been unified. Please open new bug
reports if the current solution does not fit your needs.
Comment 24 Dani Megert CLA 2004-09-22 04:45:27 EDT
Need to reopen to get rid of LATER.
Comment 25 Dani Megert CLA 2004-09-22 04:46:07 EDT
.