Bug 64413 - [Bidi] Cannot mix RTOL and LTOR rendering on a control
Summary: [Bidi] Cannot mix RTOL and LTOR rendering on a control
Status: CLOSED WONTFIX
Alias: None
Product: Platform
Classification: Eclipse Project
Component: SWT (show other bugs)
Version: 3.0   Edit
Hardware: PC Windows XP
: P3 normal (vote)
Target Milestone: ---   Edit
Assignee: Platform-SWT-Inbox CLA
QA Contact: Felipe Heidrich CLA
URL:
Whiteboard: stalebug
Keywords: triaged
: 205204 (view as bug list)
Depends on:
Blocks:
 
Reported: 2004-05-27 14:41 EDT by Randy Hudson CLA
Modified: 2020-04-24 20:33 EDT (History)
5 users (show)

See Also:


Attachments
firefox rendering of BiDi (17.04 KB, image/gif)
2004-05-27 15:48 EDT, Randy Hudson CLA
no flags Details
Incorrect treatment of RLE by TextLayout (2.69 KB, image/gif)
2007-11-13 23:50 EST, Randy Hudson CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Randy Hudson CLA 2004-05-27 14:41:25 EDT
I do not understand how TextLayout determines "words".  If I want a character 
to be blue instead of black, that doesn't mean that I have word-break at that 
location. But TextLayout is treating a change of style to mean that the words 
are broken at that point.  for example, I want to display "IBM" using a 
different color for each character.  But in RtoL mode, TextLayout will 
incorrectly paint this as "MBI".

public static void main(String[] args) {
	final Display display = new Display();
	final Shell shell = new Shell();
	
	shell.addPaintListener(new PaintListener() {
		public void paintControl(PaintEvent e) {
			TextLayout tl = new TextLayout(display);
			tl.setOrientation(SWT.RIGHT_TO_LEFT);
			tl.setText("IBM IBM");
			tl.setStyle(new TextStyle(null, new Color
(null,100,200,150),null),1, 1);
			tl.draw(e.gc, 10,10);
			tl.dispose();
		}
	});
	
	shell.setSize(400,300);
	shell.open();
	
	while (!shell.isDisposed())
		if (!display.readAndDispatch())
			display.sleep();

}
Comment 1 Veronika Irvine CLA 2004-05-27 15:03:39 EDT
Felipe, please comment.
Comment 2 Randy Hudson CLA 2004-05-27 15:48:04 EDT
Created attachment 11238 [details]
firefox rendering of BiDi

This attachment shows FireFox rendering a similar string in RTOL mode.	Even
though the string contains italic and bold styled regions, it is still rendered
as if it were a single word, which it is.
Comment 3 Felipe Heidrich CLA 2004-05-27 15:55:22 EDT
You are drawing with a mirrored textlayout on a non-mirrored shell, thus using 
a non-mirrored gc.
Change:
final Shell shell = new Shell();
By:
final Shell shell = new Shell(display, SWT.SHELL_TRIM | SWT.RIGHT_TO_LEFT);

Then it works fine. It is unpredictable the result of the operations you are 
attempting.
Comment 4 Felipe Heidrich CLA 2004-05-27 15:55:48 EDT
I'm sorry, I should have marked  this as invalid.
Comment 5 Felipe Heidrich CLA 2004-05-27 15:55:57 EDT
closing
Comment 6 Randy Hudson CLA 2004-10-27 17:17:31 EDT
I'm reopening this bug because I find it confusing that TextLayout has 
setOrientation(int) if the calling that method has unpredictable results.

I need to be able to mix RTOL and LTOR TextLayouts on a single Control.
Comment 7 Randy Hudson CLA 2004-11-03 11:51:02 EST
It should be possible to render any text on a Canvas, despite the orientation 
used when constructing the canvas. It looks like the use of StyleItem is not 
needed to see inconsistencies.  The following code shows problems in both the 
ordering and overall placement of text.

public static void main(String[] args) {
	final Display display = new Display();
	final Shell shell = new Shell(SWT.SHELL_TRIM | SWT.RIGHT_TO_LEFT);
	shell.setLayout(new GridLayout());

	final String message = "123 ABC 456 def 999";
	
	StyledText text = new StyledText(shell, SWT.MULTI | SWT.BORDER | 
SWT.RIGHT_TO_LEFT);
	text.setLayoutData(new GridData(GridData.FILL_BOTH));
	text.setText(message);
	
	final Canvas canvas = new Canvas(shell, SWT.BORDER | SWT.LEFT_TO_RIGHT);
	canvas.setLayoutData(new GridData(GridData.FILL_BOTH));
	canvas.addPaintListener(new PaintListener() {
		public void paintControl(PaintEvent e) {
			TextLayout layout = new TextLayout(shell.getDisplay());
			layout.setWidth(shell.getClientArea().width);
			layout.setOrientation(SWT.RIGHT_TO_LEFT);
			layout.setAlignment(SWT.RIGHT);
			layout.setText(message);
			layout.draw(e.gc, 0, 0);
			layout.dispose();
		}
	});
	
	shell.setSize(400, 300);
	shell.open();

	while (!shell.isDisposed())
		if (!display.readAndDispatch())
			display.sleep();
}

Comment #3 talks about the mirrored state of a Shell/GC.  But it is possible to 
query locations of each offset, without the existence of Shells or GCs.
Comment 8 Randy Hudson CLA 2005-03-16 16:37:08 EST
Another test case for the message:
final String message = "\u202eThis should be backwards.";
Comment 9 Randy Hudson CLA 2005-05-03 14:38:04 EDT
The following works correctly on some win32 platforms and incorrectly on others.
 Correctly is defined as display "Period.".

public static void main(String[] args) {
	Display display = new Display();
	final Shell shell = new Shell(display, SWT.RIGHT_TO_LEFT | SWT.SHELL_TRIM);
	shell.setText("Shell");
	
	shell.addPaintListener(new PaintListener() {
		public void paintControl(PaintEvent e) {
			e.gc.drawText("Period.", 0, 0);
			e.gc.drawString("Period.", 0, 20);
			TextLayout layout = new TextLayout(null);
			layout.setOrientation(SWT.LEFT_TO_RIGHT);
			layout.setText("Period.");
			layout.draw(e.gc, 0, 40);
			//No, really, I mean it
			char LRO = '\u202d';
			layout.setText(LRO +"Period.");
			layout.draw(e.gc, 0, 60);
		}
	});
	
	shell.setSize(300, 180);
	shell.open();
	while (!shell.isDisposed()) {
		if (!display.readAndDispatch())
			display.sleep();
	}
	display.dispose();
}
Comment 10 Randy Hudson CLA 2005-05-04 17:18:49 EDT
Another case where drawString works but TextLayout fails.

public static void main(String[] args) {
	Display display = new Display();
	final Shell shell = new Shell(display, SWT.LEFT_TO_RIGHT | 
SWT.SHELL_TRIM);
	shell.setText("Shell");
	
	shell.addPaintListener(new PaintListener() {
		public void paintControl(PaintEvent e) {
			//char LRO = '\u202d';
			char RLE = '\u202b';
			String s = "a<";
			e.gc.drawText(RLE + s, 0, 0);
			e.gc.drawString(RLE + s, 0, 20);
			TextLayout layout = new TextLayout(null);
			layout.setOrientation(SWT.RIGHT_TO_LEFT);
			layout.setText(s);
			for (int i = 0; i < s.length(); i++) {
				System.out.print(layout.getLevel(i) +", ");
			}
			layout.draw(e.gc, 0, 40);
		}
	});
	shell.setSize(300, 200);
	shell.open();
	while (!shell.isDisposed()) {
		if (!display.readAndDispatch())
			display.sleep();
	}
	display.dispose();
}
Comment 11 Randy Hudson CLA 2005-05-04 17:24:31 EDT
When I run the snippet from comment 10, I am seeing:
>a
>a
a>

The 3rd line is the output from TextLayout which is bogus.
Comment 12 Felipe Heidrich CLA 2005-05-06 16:20:14 EDT
I'll need to add new API to fix this problem.

There is two things:
1) base direction, is an input for the bidi algorithm, affects levels and the 
reordering.
2) mirroring.

When you call TextLayout#setOrientation(RTL) the TextLayout sets the base 
direction to RTL and computes all the data expecting it to be drawn by a 
mirrored GC.

In some of these cases you draw using a GC that is not mirrored. I can change 
draw() to verify if the GC is mirrored or not, recompute all the data if 
necessary, and then draw correctly.
The problem is: all the measuring APIs don't take a GC. All numbers returned 
by the measuring APIs would only be valid if used in a mirrored window.

The only solution I can think of is adding a new API.
setOrientation would only set the base directon.
add a setMirrored getMirrored to set if the textlayout is to be used by a 
mirrored window or not.

Other option is to change setOrientation to take a input in the format:
setOrientation ((RTL|LTR) | (MIRRORED|NONE)).
Comment 13 Randy Hudson CLA 2005-05-06 17:06:06 EDT
Ignoring new API and flags, the snippet in comment 10 should either fail 
consistently or work consistently on all platforms.
Comment 14 Felipe Heidrich CLA 2005-05-09 14:36:36 EDT
One platform has mirroring and base dir (and mix both), the other doesn't have 
mirroring, if we don't provide a way to isolate base dir and mirroring I don't 
know how to fix this problem. 
Comment 15 Randy Hudson CLA 2005-05-27 13:58:53 EDT
(In reply to comment #14)
> One platform has mirroring and base dir (and mix both)

Mirroring is not enabled in the snippet in comment 10, which fails on win32 and 
works elsewhere. A normal control fails to work with a right-to-left 
textlayout. So, why does TL let the client change the orientation if it is not 
supported?

We are now working around this problem by setting the TextLayout to match the 
mirrored state of the control, and using RLO and LRO control characters.
Comment 16 Felipe Heidrich CLA 2006-03-07 18:17:19 EST
Randy, I think you no longer need to use RLO and LRO control characters.
the fix for Bug 129908 probably resolved this.

Unfortunately we still don't support widget orientation != text_layout orientation.
Comment 17 Randy Hudson CLA 2006-03-07 22:39:25 EST
> Randy, I think you no longer need to use RLO and LRO control characters.

We need to use these characters regardless. Pseudo-HTML example:
<P>$RLO$This should be <span style="border:5px">backwards</span></p>
The word "backwards" will be level 1, so when we paint it separately, we need to force level 1.

> the fix for Bug 129908 probably resolved this.

Isn't that fix for Linux?

> Unfortunately we still don't support widget orientation != text_layout
> orientation.

So that's what this bug is about. In the very least the limitation could be documented in the API. Also, my guess is it works by chance on GTK.
Comment 18 Felipe Heidrich CLA 2006-03-08 11:28:30 EST
basically TextLayout#setOrientation() wasn't working for pango >= 1.4, I thought you were using RLO and LRO to workaround this problem (to have consistency between gtk&win).
Comment 19 Randy Hudson CLA 2006-03-08 12:43:23 EST
In the example in my previous comment, if you render the string "backwards" separately, and expect it to be "sdrawkcab", because in the original document it was preceeded by RLO, we have to use an RLO. So, we just use it all the time, regardless of whether the string is "naturally" RTL. The same is true for LTR. We always use LRO. The exception is when the entire document is level 0, and the control is not mirrored. Then we just paint the strings in pieces.

Similarly, when painting "XY" as two separate strings of one character, we use ZWJs as needed.

To summarize, this bug is not affecting us currently, but it is still confusing API.
Comment 20 Randy Hudson CLA 2007-11-08 10:26:16 EST
> To summarize, this bug is not affecting us currently, but it is still confusing
> API.

As discussed in bug 205204, is seems that number presentation is affected. Does anyone know if an RLE character is inserted in the string painted in a LTOR TextLayout/GC, would number substitution occur? I would try a snippet myself but I don't have the locale configured.
Comment 21 Felipe Heidrich CLA 2007-11-12 13:03:00 EST
bug 205204 is about number substitution, fixing this bug would not matter to them.


What is asked in this PR can't be implemented in the native widgets (text, label, link, combo, etc). I'm not sure if it can be done for graphics (textlayout+canvas) either.
Comment 22 Randy Hudson CLA 2007-11-13 23:48:26 EST
(In reply to comment #21)
> bug 205204 is about number substitution, fixing this bug would not matter to
> them.

I think these two bugs are related. A RTOL TextLayout *would* perform the desired number substitution, but can't be used because of this bug. There are cases where you can solve this by mirroring the entire canvas, and there are cases where only portions of the canvas should perform substitution.

> What is asked in this PR can't be implemented in the native widgets (text,
> label, link, combo, etc). I'm not sure if it can be done for graphics
> (textlayout+canvas) either.

I don't understand the comparison between a widget and a graphics call.

But, when compared to drawString or drawText, TextLayout still behaves incorrectly

public class Snippet3 {
	private static Display d;
	static final char RLE = '\u202b';
	public static void main(String[] args) {
		d = new Display();
		Shell shell = new Shell(d, SWT.DOUBLE_BUFFERED | SWT.SHELL_TRIM | SWT.LEFT_TO_RIGHT);
		shell.setSize(400, 300);
		final TextLayout textLayout = new TextLayout(null);
		textLayout.setOrientation(SWT.LEFT_TO_RIGHT);
		textLayout.setText(RLE + "101 Dalmations.");
		shell.addPaintListener(new PaintListener() {
			public void paintControl(PaintEvent e) {
				e.gc.drawText(textLayout.getText(), 0, 30);
				textLayout.draw(e.gc, 0, 0);
			}
		});
		Label label = new Label(shell, 0);
		label.setText(textLayout.getText());
		label.setBounds(0, 60, 400, 20);
		shell.open();
		
		while (!shell.isDisposed())
			if (!d.readAndDispatch())
				d.sleep();
	}
}
Comment 23 Randy Hudson CLA 2007-11-13 23:50:08 EST
Created attachment 82832 [details]
Incorrect treatment of RLE by TextLayout
Comment 24 Felipe Heidrich CLA 2007-11-14 11:37:07 EST
Randy, digit substitution is a different bug!
Go to TextLayout#itemize() method
localize the code:
SCRIPT_DIGITSUBSTITUTE psds = new SCRIPT_DIGITSUBSTITUTE();
OS.ScriptRecordDigitSubstitution(OS.LOCALE_USER_DEFAULT, psds);
OS.ScriptApplyDigitSubstitution(psds, scriptControl, scriptState);
move these 3 lines outside of the "if ((orientation & SWT.RIGHT_TO_LEFT) != 0)"
so that they always run.

Do you want this behaviour ? Open a separate bug.

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
let's forget about digit substitution now
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=

As far as I understand this problem report is about drawing with a RTL TextLayout in a canvas that is not Mirrored.
In other words, you want to set the paragraph level of the TextLayout to RTL and draw in a canvas that is not mirrored.
Currently this is not possible. If you set TextLayout#setOrientation(SWT.RIGHT_TO_LEFT) it sets the paragraph level to RTL but is ALSO expect you to draw to canvas that IS mirrored.

We have the same situtation with widgets:
windows: SWT.RIGHT_TO_LEFT sets the paragraph level to RTL AND mirrors the widget.
gtk: SWT.RIGHT_TO_LEFT only mirrors the widget. The paragraph level is determine by the first strong character in the string.

The widgets behaviour is determine by the platform. In graphics (TextLayout) we decide to follow the win32 design.

Comment 25 Felipe Heidrich CLA 2007-11-14 16:32:24 EST
Randy, I read my last comment over again. It sounded a bit rude, that was not the intention. Sorry.


The problem is that this report is a bit confusing. So I tried to summarize it on comment #24. The question for you is, based on comment 24: 
did I understand the request in this problem report correctly ? Is that what you want ?
Comment 26 Randy Hudson CLA 2007-11-15 12:44:06 EST
(In reply to comment #24)
> Randy, digit substitution is a different bug!
> Go to TextLayout#itemize() method
> localize the code:
> SCRIPT_DIGITSUBSTITUTE psds = new SCRIPT_DIGITSUBSTITUTE();
> OS.ScriptRecordDigitSubstitution(OS.LOCALE_USER_DEFAULT, psds);
> OS.ScriptApplyDigitSubstitution(psds, scriptControl, scriptState);
> move these 3 lines outside of the "if ((orientation & SWT.RIGHT_TO_LEFT) != 0)"
> so that they always run.
> 
> Do you want this behaviour ? Open a separate bug.

No, I don't think this is the right behavior. It should be possible to turn digit substitution on and off using RLE/LRE control characters within a single TextLayout.  The snippet I attached already shows how TextLayout is not consistent with drawString, which allows mixed digit substitution in a single string.

> =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
> let's forget about digit substitution now
> =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=

Not so fast. Let's look at why setOrientation might be called. There's no real need for the API, other than crossing your fingers that it turns on digit substitution.  I'll elaborate below...

> As far as I understand this problem report is about drawing with a RTL
> TextLayout in a canvas that is not Mirrored.
> In other words, you want to set the paragraph level of the TextLayout to RTL
> and draw in a canvas that is not mirrored.
> Currently this is not possible.

Actually, as my previous posts about workaround suggested, you can basically approximate this (except for the digit substitution, currently). If I want to "set the paragraph level to RTL", I can just prefix my text with a control character and change the alignment to SWT.RIGHT. (I know, the text might have a PDF which pops off the LRE's effect)

Ideally, setOrientation would be a better alternative to inserting the control character and changing the alignment. But in reality (on win32), it's a meaningless API which clients are forced to call to ensure the orientation is in sync with the GC.

Here is a snippet that shows RTL text can be displayed just fine on a LTR text layout (except for the digits).

public class Snippet2 {
	private static Display d;
	static final char RLE = '\u202b';
	static final char LRE = '\u202a';

	public static void main(String[] args) {
		d = new Display();
		Shell shell = new Shell(d, SWT.DOUBLE_BUFFERED | SWT.SHELL_TRIM | SWT.LEFT_TO_RIGHT);
		shell.setSize(400, 300);
		
		shell.setLayout(new RowLayout());
		
		Canvas c1 = new Canvas(shell, SWT.BORDER | SWT.LEFT_TO_RIGHT);
		c1.setLayoutData(new RowData(200,-1));
		Canvas c2 = new Canvas(shell, SWT.BORDER | SWT.RIGHT_TO_LEFT);
		c2.setLayoutData(new RowData(200,-1));
		
		final TextLayout tl1 = new TextLayout(null);
		final TextLayout tl2 = new TextLayout(null);
		tl1.setWidth(200);
		tl1.setOrientation(SWT.LEFT_TO_RIGHT);
		tl1.setAlignment(SWT.RIGHT);
		tl2.setOrientation(SWT.RIGHT_TO_LEFT);
		tl2.setWidth(200);
		
		tl1.setText(RLE+"(101) Dalmations.");
		tl2.setText("(101) Dalmations.");
		c1.addPaintListener(new PaintListener() {
			public void paintControl(PaintEvent e) {
				tl1.draw(e.gc, 0, 0);
			}
		});
		c2.addPaintListener(new PaintListener() {
			public void paintControl(PaintEvent e) {
				tl2.draw(e.gc, 0, 0);
			}
		});

		shell.open();
		
		while (!shell.isDisposed())
			if (!d.readAndDispatch())
				d.sleep();
	}
}

> We have the same situtation with widgets:
> windows: SWT.RIGHT_TO_LEFT sets the paragraph level to RTL AND mirrors the
> widget.

On native LTR widgets, I can set the alignment to RIGHT, and insert RLE character, and things work properly (as if in a "mixed" mode).
Comment 27 Randy Hudson CLA 2008-01-15 10:24:57 EST
*** Bug 205204 has been marked as a duplicate of this bug. ***
Comment 28 Felipe Heidrich CLA 2009-08-14 09:17:48 EDT
Your bug has been moved to triage, visit http://www.eclipse.org/swt/triage.php for more info.
Comment 29 Lina Kemmel CLA 2009-11-12 08:33:33 EST
What are the remaining problem(s) addressed by this bug?
Is it for incorrect treatment of RLE/LRE characters by TextLayout?
Comment 30 Felipe Heidrich CLA 2009-11-12 09:59:06 EST
Randy, note that Bug 133133 is fixed (you should no longer see the problem with the digits).
Snippet in comment 26 works for me. What handling of RLE/LRE is wrong ?
Keep in mind the bidi algorithm is implemented by win32 (uniscribe). If there is a problem in it, it is windows (not SWT).

I think this probem has become a duplicate of Bug 273198.

Agreed ?
Comment 31 Leo Ufimtsev CLA 2017-08-03 12:30:25 EDT
This is a one-off bulk update. (The last one in the triage migration).

Moving bugs from swt-triaged@eclipse to platform-swt-inbox@eclipse.org and adding "triaged" keyword as per new triage process:
https://wiki.eclipse.org/SWT/Devel/Triage

See Bug 518478 for details.

Tag for notification/mail filters:
@TriageBulkUpdate
Comment 32 Eclipse Genie CLA 2020-04-24 20:33:51 EDT
This bug hasn't had any activity in quite some time. Maybe the problem got resolved, was a duplicate of something else, or became less pressing for some reason - or maybe it's still relevant but just hasn't been looked at yet. As such, we're closing this bug.

If you have further information on the current state of the bug, please add it and reopen this bug. The information can be, for example, that the problem still occurs, that you still want the feature, that more information is needed, or that the bug is (for whatever reason) no longer relevant.

--
The automated Eclipse Genie.