Bug 206740 - Source editor widens line spacing if multibyte character in source text
Summary: Source editor widens line spacing if multibyte character in source text
Status: REOPENED
Alias: None
Product: Platform
Classification: Eclipse Project
Component: SWT (show other bugs)
Version: 3.3.1   Edit
Hardware: All All
: P3 normal with 6 votes (vote)
Target Milestone: ---   Edit
Assignee: Platform-SWT-Inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2007-10-18 07:32 EDT by Mike Whittaker CLA
Modified: 2021-12-16 14:19 EST (History)
16 users (show)

See Also:


Attachments
Image from editor window, normal line spacing (7.38 KB, image/png)
2007-10-19 03:23 EDT, Mike Whittaker CLA
no flags Details
Image from editor window, affected line spacing (11.54 KB, image/png)
2007-10-19 03:24 EDT, Mike Whittaker CLA
no flags Details
before and after entering multibyte characters (37.18 KB, image/x-png)
2009-03-31 22:57 EDT, YDM CLA
no flags Details
Line spacing before and after inserting ⇒ (58.74 KB, image/jpeg)
2014-10-01 05:41 EDT, Iulian Dragos CLA
no flags Details
line space widens on c++ editor (49.23 KB, image/png)
2016-04-11 21:46 EDT, Gumui Yul CLA
no flags Details
SWT StyledText testcase (1.46 KB, text/x-java)
2016-10-26 11:13 EDT, Leon Blakey CLA
no flags Details
Line heights with or without special characters (489.81 KB, image/png)
2021-12-16 14:16 EST, Pierre-Yves Bigourdan CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Mike Whittaker CLA 2007-10-18 07:32:12 EDT
Build ID: M20070921-1145

Steps To Reproduce:
1. Create source file containing several source lines
2. Font used for editor is Lucida Typewriter Sans 9
3. Save file, note interline spacing is normal
4. now add (e.g.) text taken from <linux-kernel>/drivers/i2c/algos/i2c-algo-bit.c
- comment including text /* With some changes from ... */
(includes byte sequence:
 hex: 4B 79 EF BF BD 73 74 69 - 'Ky?sti'
5. Save, observe line spacing increases


More information:
Bug is directly related to inclusion/deletion of multibyte character(s).

May be more general Eclipse problem ?
Comment 1 Anton Leherbauer CLA 2007-10-18 10:54:21 EDT
I don't see the increase in the line spacing, but anyway, this is certainly not a CDT specific issue.
Comment 2 Mike Whittaker CLA 2007-10-19 03:23:50 EDT
Created attachment 80742 [details]
Image from editor window, normal line spacing

Shows part of the editor window when line spacing is unaffected by the reported bug.
Comment 3 Mike Whittaker CLA 2007-10-19 03:24:58 EDT
Created attachment 80743 [details]
Image from editor window, affected line spacing

Image from editor window which is affected by the reported bug.
Comment 4 Anton Leherbauer CLA 2007-10-19 04:05:07 EDT
Which encoding do you use?
With UTF-8, I see o-umlaut and a-umlaut:

/* With some changes from Kyösti Mälkki <kmalkki@cc.hut.fi> */

Did you try different fonts?
Comment 5 Mike Whittaker CLA 2007-10-19 05:00:38 EDT
I have not changed the default encodings:
General/Editors/Text Editors/Spelling/Encoding: UTF-8
General/Workspace/Text file encoding: UTF-8

Any others I have missed ?

The rendering is as shown, the mutibyte chars do not render correctly,
perhaps triggering the line spacing issue.

I have tried it with different font sizes and different font typefaces.

Interestingly, if I scroll down to an area of text that does not
include the text in question within the window, and change editor font,
the spacing is correct, but as soon as I scroll back to show,
"With some changes ...."
the window redraws with the wider spacing.

I must admit this is not a new problem to me, I have seen it for
over a year on other installations (also Centos or RedHat, hence Gnome v2.8.0)
but only now decided it was annoying enough to investigate and report.
Comment 6 Felipe Heidrich CLA 2007-11-13 15:08:29 EST
Something is wrong here.

comment 4 indicates the multi-byte char is o-umlaut (UTF-16 0x00F6)
this char in UTF-8 is  0xC3 0xB6

comment 1 the hex dump for the same char is EF BF BD 

comment 5, says the file encoding is UTF-8 and the char is not displayed correctly. It all indicates that the file in reporter's machine is not encoded in UTF-8

Going back to main problem, the line spacing is increasing (the lines are getting taller) because the system had to load a fallback font to draw the text. This is not a bug. It is normal for a fallback font to be taller than the normal font.
Comment 7 Mike Whittaker CLA 2007-11-14 06:29:17 EST
Comparing behaviour with that of 'gedit':

gedit changes line spacing of _only_ the line(s) containing character,
leaves rest of text unaffected.
Eclipse changes spacing of _all_ lines in the file once the offending
character has been rendered.

Can I suggest that Eclipse goes with the less intrusive handling of
this condition rather than spoiling the entire display of the file ?

This is still a display bug somewhere, even if it's not a CDT issue,
hence should (IMHO) be left open until a workaround or solution
is available.

I have tried display using the alternative character modes
(UTF-16, ISO-8859-1, UTF-8) and behaviour is still the same.
Comment 8 Mike Whittaker CLA 2007-11-14 06:38:47 EST
Tried on Eclipse 3.2 on Windows - the offending characters are displayed
without any problems, so this must be some issue to do with the Linux platform.
Comment 9 Mike Whittaker CLA 2007-11-14 06:59:28 EST
Why not change the target milestone to 'Later' ? (rather than 'won't fix')

That way it will still get fixed should the root cause be understood
at some future date.
Comment 10 Felipe Heidrich CLA 2007-11-14 10:58:32 EST
(In reply to comment #7)
> Eclipse changes spacing of _all_ lines in the file once the offending
> character has been rendered.
The reason is simple, StyledText can work in fixed line height mode (all lines have same height) or variable line height mode.
Performance with fixed line height mode is much better.

> This is still a display bug somewhere, even if it's not a CDT issue,
> hence should (IMHO) be left open until a workaround or solution
> is available.

Correct, but that is a different bug (it should go to new problem report intead of reusing this one).

Can you please attach the exactly file that shows the problem?
The one file that has the umlaut encoded as EF BF BD.
Comment 11 YDM CLA 2008-10-07 04:24:25 EDT
I second to this bug.

With my Korean Windows XP and Eclipse Ganimede, I can reproduce the same problem as with Mike. But with prior versions of Eclipse, there wasn't this problem.

I wonder if there's any activities regarding this bug since last comment from Felippe, November 2007.
Comment 12 YDM CLA 2009-03-31 22:57:03 EDT
Created attachment 130492 [details]
before and after entering multibyte characters

This problem has been here for so long without any answers.

Find the png of before and after entering Hangul(Korean) multibyte characters in my Eclipse 3.4.1. Again, this didn't happen before up until version 3.3 as i know. I guess multibyte support has been changed during version ups as there's another problem.

Before 3.4, I was able to block-undo my typings, ie. when I press ctrl-z, all my successive typings were undone at once. But now, each of Hangul consonants and vowels is undone by pressing ctrl-z each time. This make me very tired when I want to undo just a little bit of typings.

I hope there's any progress regarding this. Thanks for your great work!
Comment 13 Iulian Dragos CLA 2014-10-01 05:40:12 EDT
I just found this ticket while preparing to report a bug. This is indeed very annoying, especially when editing Scala code, where the unicode right arrow ⇒ is quite common.

The source code is visible in the screenshot.

Eclipse Luna (4.4.1)
Mac OS X
Comment 14 Iulian Dragos CLA 2014-10-01 05:41:11 EDT
Created attachment 247521 [details]
Line spacing before and after inserting ⇒
Comment 15 Gumui Yul CLA 2016-04-11 21:13:47 EDT
I have got similar problem.
On the C/C++ Editor if I scroll to the line contains '℃' character,then the Editor's line space widens.
It has tested on Eclipse 4.4(Luna 2) ~ 4.6(Neon M6) and it appears on those versions.
I tried to change font but not work.
I am using monospace like 'Ubuntu Mono' in eclipse under Ubuntu Gnome 16.04.
Comment 16 Gumui Yul CLA 2016-04-11 21:46:20 EDT
Created attachment 260877 [details]
line space  widens on c++ editor
Comment 17 Gumui Yul CLA 2016-04-13 01:18:11 EDT
(In reply to Gumui Yul from comment #15)
> I have got similar problem.
> On the C/C++ Editor if I scroll to the line contains '℃' character,then the
> Editor's line space widens.
> It has tested on Eclipse 4.4(Luna 2) ~ 4.6(Neon M6) and it appears on those
> versions.
> I tried to change font but not work.
> I am using monospace like 'Ubuntu Mono' in eclipse under Ubuntu Gnome 16.04.

This problem seems to be related with Fontconfig on Ubuntu.
For me, monospace font is specified as default on Eclipse and 
the monospace font family prefer to 'Ubuntu Mono'.(for this, I setup /etc/fonts/local.conf)
Also I'm using 'NanumBarun' font for korean characters.

Now, Eclipse render multibyte character like '℃' with 'NanumBarun' font instead of 'Ubuntu Mono'.
However I think that the editor should not change line-spacing for whole lines.  Just do it about the line contained multibyte character.

Finally, I solved appending following codes into /usr/share/fontconfig/config.avail/90-fonts-nanum.conf.

  <match target="pattern">
    <test qual="any" name="family">
	<string>monospace</string>
    </test>
    <edit name="family" mode="append" binding="strong">
	<string>Ubuntu Mono</string>
	<string>NanumBarunGothic Light</string>
    </edit>
  </match>
Comment 18 Rob Bruckheimer CLA 2016-09-20 15:04:57 EDT
Happens to me as well.  As soon as I scroll in the editor to where the Japanese characters are, the line spacing throughout the entire file widens and remains until I close the file in the editor.  In the default Text Editor (gedit) only the line which contains the Japanese characters is widened.
Eclipse Neon.  Ubuntu 64-bit 16.04LTS.
Comment 19 Leon Blakey CLA 2016-10-26 11:13:41 EDT
Created attachment 265062 [details]
SWT StyledText testcase

I am able to reproduce this with unicode "Right Angle Bracket" 0x3009 on Ubuntu and Windows 10. This happens on Neon with the dark theme when pasting it into a text file with a few paragraphs.

I'm not sure if 100% related, but I can also reproduce this using SWT StyledText. I've attached a small class where clicking the button will add that character to the StyledText which adjusts all line spacing. 

Windows Font: Segoe UI
Ubuntu Font: Sans
Comment 20 Léonard Potherat CLA 2017-03-17 08:32:57 EDT
I could reproduce this bug with this char in the source of a JS file : ⏎ - http://www.fileformat.info/info/unicode/char/23ce/index.htm.

My version of Eclipse : 
Eclipse for PHP Developers
Version: Neon.2 Release (4.6.2)
Build id: 20161208-0600

My OS : OSX El Capitan 10.11.6 (15G1212)


When I open the file, interlines are correct, but when I scroll to display the line where the char is, all interlines of the file get wider. I think it could be nice to make only the line concerned wider than all the file.
Comment 21 Olof Andersson CLA 2017-11-11 03:11:05 EST
Reproduced it when character 0x0C ("form feed" in ascii) appeared in a file. Everything looks good until you scroll to the line which contains that byte.

Eclipse Mars Release (4.5.0)
Build id: 20150621-1200
Ubuntu 16.04.2 LTS
Comment 22 Roman Knoell CLA 2018-07-31 06:14:32 EDT
I have that bug since years. It makes Eclipse look awful.

In that it is related to this design flaw: https://bugs.eclipse.org/bugs/show_bug.cgi?id=26765

This is not state of the art 2018. Unicode is the default text standard for more than a decade now. Scala, for example, uses ⇒ naturally (instead of ugly “=>”). It‘s little but easier to read, more concise. By defining own operators, many programmers take advantage of (almost full) Unicode support in Scala to creed ate precise programs. The IDE support that and not stand in the way of its users.
Comment 23 Roman Knoell CLA 2018-07-31 06:51:06 EDT
I like to add that I am using macOS. I would not exclude that it is a Mac-only issue. The bug was reported on Linux though.
Comment 24 Eric Williams CLA 2019-02-01 14:03:34 EST
Might be related to bug 527917.
Comment 25 Yurx Cherio CLA 2019-12-10 17:04:39 EST
12 years later this issue is still present.

BTW this affects not only the editor window but also the console view. All of a sudden the log window display changes as soon as a unicode character shows up in the log.


Eclipse IDE for Enterprise Java Developers
Version: 2019-03 (4.11.0)
Build id: 20190314-1200

18.04.3 LTS (Bionic Beaver)
Linux 5.0.0-37-generic #40~18.04.1-Ubuntu SMP Thu Nov 14 12:06:39 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Comment 26 Andrey Filippov CLA 2020-08-28 19:25:04 EDT
Same now with Version: 2020-06 (4.16.0) Build id: 20200615-1200.

I'm sure it worked with some previous versions (I created the following line in Eclipse), but now widens spaces as soon as the UTF-8 characters appear in the editor window.

Andrey
								    		System.out.println(String.format("corr4dirsLMA -> ↔: %7.4f (%7.4f) ↕:%7.4f (%7.4f) ⤡:%7.4f (%7.4f) ⤢:%7.4f (%7.4f)", ...
Comment 27 Andrey Filippov CLA 2020-08-28 19:38:17 EDT
(In reply to Andrey Filippov from comment #26)
> Same now with Version: 2020-06 (4.16.0) Build id: 20200615-1200.
> 
> I'm sure it worked with some previous versions (I created the following line
> in Eclipse), but now widens spaces as soon as the UTF-8 characters appear in
> the editor window.
> 
> Andrey
> 								    		System.out.println(String.format("corr4dirsLMA -> ↔: %7.4f
> (%7.4f) ↕:%7.4f (%7.4f) ⤡:%7.4f (%7.4f) ⤢:%7.4f (%7.4f)", ...

Seems it applies only to some UTF-8 characters:

all of these arrows: ↖↗↘↙ break (cause double-spacing),
while these are OK:  ← → ↑ ↓ ⇖ ⇗ ⇘ ⇙

So for now I'll just replace diagonal ones.

Andrey
Comment 28 Pierre-Yves Bigourdan CLA 2021-12-16 14:16:47 EST
Created attachment 287700 [details]
Line heights with or without special characters

I can reproduce as well. Attached is a screenshot representing three almost identical pieces of code:
* on the left, no special characters have been inserted.
* in the middle, one special character has been inserted. Note that the height of all lines, even those where the character is not present, has doubled.
* on the right, the special character has been deleted. Unfortunately, the line height bug persists until the editor is closed and reopened.

In large files, this bug only kicks in when you scroll to a line containing a special character: the line heights of the entire editor will dynamically be resized then, which is even more annoying.