Bug 491297 - Hangul Filler character problem in linux
Summary: Hangul Filler character problem in linux
Status: CLOSED WONTFIX
Alias: None
Product: Platform
Classification: Eclipse Project
Component: SWT (show other bugs)
Version: 4.4.2   Edit
Hardware: PC Linux
: P3 normal (vote)
Target Milestone: ---   Edit
Assignee: Platform-SWT-Inbox CLA
QA Contact:
URL:
Whiteboard: stalebug
Keywords: needinfo
Depends on:
Blocks:
 
Reported: 2016-04-08 01:52 EDT by Hayarobi Park CLA
Modified: 2020-08-27 03:03 EDT (History)
2 users (show)

See Also:


Attachments
example eclipse java project (1.17 KB, application/x-bzip)
2018-09-05 09:49 EDT, Hayarobi Park CLA
no flags Details
java editor of eclipse photon in macos 10.13 (18.36 KB, image/png)
2018-09-06 05:39 EDT, Hayarobi Park CLA
no flags Details
package explorer of eclipse photon in macos 10.13 (39.52 KB, image/png)
2018-09-06 05:39 EDT, Hayarobi Park CLA
no flags Details
package explorer of luna in ubuntu 18.04 (5.53 KB, image/png)
2018-09-06 08:29 EDT, Hayarobi Park CLA
no flags Details
java editor of luna in ubuntu 18.04 (3.51 KB, image/png)
2018-09-06 08:30 EDT, Hayarobi Park CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Hayarobi Park CLA 2016-04-08 01:52:41 EDT
The two filler characters, 'HANGUL CHOSEONG FILLER' (U+115F) and 'HANGUL JUNGSEONG FILLER' (U+1160), are similar to unicode combining characters, such as diacritical marks or accents. Combining character is added after the main character, but the filler character is added before main character. There is some legal combinations in unicode standard to combine Hangul filler(s) and Hangul Jamo character(s).

When cursor is after the combination, pressing backspace key will remove a rightmost combining character leaving main character and other combining character before it. (for example, ḗ  is combination of e and two combining characters, it will be ē and then e by pressing backspace two times).
When cursor is "before" the combination, pressing delete key will remove whole characters of combination. It seems reasonable when combining characters are added after main character.

These behaviors of backspace and delete is same in filler combination, but it cause some confusions.

Pressing backspace after filler combination will leave filler character only, but it's not legal notation in unicode standard. This situation sometimes happen while mistyping and backspacing for correction.

The Java editor in linux gnome3 does not show these filler characters as if there's nothing, but compiler treats them as normal characters and allowed in class name or method name. It make confusion, such as content assist does not works as people expected. (It's not so problem in windows, since the editor show the filler characters as while space.)

The classname CᅟlassA, for example, looks like just six alphabets word but there is hangul filler character after C, so it is equvalent to C\u115Flass. So when type C, the content assistent will list this class, but not Cl. It made me hard to find error or bugs lots of times. 

I think it's better to patch like below, at least one of them.
* A single press of backspace removes Hangul jamos and fillers combination at once. It also prevents breaking unicode standard.
* Java editor shows sole filler character as white space in Linux, like in MS Windows.
* Make warning marker when invisible characters are contained in name of class, method or other. It will be also useful to zero width space (U+200b).
Comment 1 Hayarobi Park CLA 2016-04-08 02:17:28 EDT
example word.
a각aᅟᅡᆨa
Comment 2 Eric Williams CLA 2018-07-06 14:22:20 EDT
Can you provide an SWT snippet that reproduces this bug?
Comment 3 Hayarobi Park CLA 2018-09-05 09:49:35 EDT
Created attachment 275700 [details]
example eclipse java project

It's quite long time. Almost all softwares I doubted - eclipse, gnome3, ibus, the font, or something else - were updated since then, and I can't exactly reproduce that bug anymore.

This project has 3 java classes. Two of them contains not-well-formed unicode character in their name. These classes was seen identical on Java editor in two years ago, but My newer eclipse in my PC - neon and photon in ubuntu 18.04 - hold place with blank space, so I can easily notice that I mistyped something.
Comment 4 Eric Williams CLA 2018-09-05 10:37:53 EDT
(In reply to Hayarobi Park from comment #3)
> Created attachment 275700 [details]
> example eclipse java project
> 
> It's quite long time. Almost all softwares I doubted - eclipse, gnome3,
> ibus, the font, or something else - were updated since then, and I can't
> exactly reproduce that bug anymore.
> 
> This project has 3 java classes. Two of them contains not-well-formed
> unicode character in their name. These classes was seen identical on Java
> editor in two years ago, but My newer eclipse in my PC - neon and photon in
> ubuntu 18.04 - hold place with blank space, so I can easily notice that I
> mistyped something.

Are there any configuration steps required (outside of Eclipse) to see the characters? Right now looking in my Java editor, there is no space between the "A" and "Class" for any of the files.
Comment 5 Hayarobi Park CLA 2018-09-06 05:39:06 EDT
Created attachment 275713 [details]
java editor of eclipse photon in macos 10.13
Comment 6 Hayarobi Park CLA 2018-09-06 05:39:48 EDT
Created attachment 275715 [details]
package explorer of eclipse photon in macos 10.13
Comment 7 Hayarobi Park CLA 2018-09-06 05:59:25 EDT
(In reply to Eric Williams from comment #4)
> (In reply to Hayarobi Park from comment #3)
> > Created attachment 275700 [details]
> > example eclipse java project
> > 
> > It's quite long time. Almost all softwares I doubted - eclipse, gnome3,
> > ibus, the font, or something else - were updated since then, and I can't
> > exactly reproduce that bug anymore.
> > 
> > This project has 3 java classes. Two of them contains not-well-formed
> > unicode character in their name. These classes was seen identical on Java
> > editor in two years ago, but My newer eclipse in my PC - neon and photon in
> > ubuntu 18.04 - hold place with blank space, so I can easily notice that I
> > mistyped something.
> 
> Are there any configuration steps required (outside of Eclipse) to see the
> characters? Right now looking in my Java editor, there is no space between
> the "A" and "Class" for any of the files.

I guess you don't have any font that support Hangul. There are some free fonts which support Hangul, such as https://github.com/naver/d2codingfont/releases/tag/VER1.3.2 - I'm using this font in code editor and terminal.

FYI, as the screenshot I uploaded just before, There is overriden squere characters on "la" in macos, but There were spacing in linux. I will upload the screenshot of same area in linux later at home.
Comment 8 Hayarobi Park CLA 2018-09-06 08:29:27 EDT
Created attachment 275718 [details]
package explorer of luna in ubuntu 18.04
Comment 9 Hayarobi Park CLA 2018-09-06 08:30:07 EDT
Created attachment 275719 [details]
java editor of luna in ubuntu 18.04
Comment 10 Eclipse Genie CLA 2020-08-27 03:03:48 EDT
This bug hasn't had any activity in quite some time. Maybe the problem got resolved, was a duplicate of something else, or became less pressing for some reason - or maybe it's still relevant but just hasn't been looked at yet. As such, we're closing this bug.

If you have further information on the current state of the bug, please add it and reopen this bug. The information can be, for example, that the problem still occurs, that you still want the feature, that more information is needed, or that the bug is (for whatever reason) no longer relevant.

--
The automated Eclipse Genie.