Bug 86503 - [typing] Copy/paste text into an external editor produces blue background
Summary: [typing] Copy/paste text into an external editor produces blue background
Status: CLOSED DUPLICATE of bug 529241
Alias: None
Product: Platform
Classification: Eclipse Project
Component: Text (show other bugs)
Version: 3.1   Edit
Hardware: PC Windows XP
: P3 minor with 13 votes (vote)
Target Milestone: ---   Edit
Assignee: Platform-Text-Inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords:
: 160915 227560 254951 282727 286166 398855 471215 (view as bug list)
Depends on: 255013
Blocks:
  Show dependency tree
 
Reported: 2005-02-24 12:43 EST by Konstantin Kolinko CLA
Modified: 2018-08-29 10:11 EDT (History)
19 users (show)

See Also:


Attachments
Screenshot of Word document showing the effect (152.86 KB, image/png)
2017-12-04 10:26 EST, Thomas Wolf CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Konstantin Kolinko CLA 2005-02-24 12:43:10 EST
Ñopying a piece of text from a file edited in Eclipse to an external editor, 
adds blue background to the copied text.

Steps to reproduce:
--------------------
1. Start Eclipse
2. Open an external file for editing. It can be a plain text file, or an XML 
file. Not tested with other types.
3. Make sure that "Highlight current line" is turned on in the editor's 
properties.

4. Select a piece of text, like a single word. All the selected text must be 
on the same line.
5. Copy the text to the clipboard.

6. Start Windows Wordpad Editor.
7. Paste the copied text.
8. You will see that the pasted text will have light-blue background.

The background color in the same as used to "Highlight current line" in the 
editor.

9. Start Microsoft Word.
10. Paste the copied text, or choose "Paste Special...", format: "Formatted 
text (RTF)".
11. You will see that the pasted text will have dark-blue background. I do not 
know, why.

Evaluation
-----------

The problems seems to be related to the option "Highlight current line".

The bug does not reproduce if this option is turned off, or if you select 
several lines of text.

Environment
------------
Eclipse versions: 3.1M4, 3.1M5a
Windows XP SP2
Microsoft Office Word 2003 SP1 (11.6359.6400)

Workaround
------------
Turn off "Highlight current line" in the options of the editor.
Comment 1 Dani Megert CLA 2006-08-22 04:44:49 EDT
Can be reproduced using R3.2 and 3.3 M1.
Comment 2 Dani Megert CLA 2006-10-18 03:37:46 EDT
*** Bug 160915 has been marked as a duplicate of this bug. ***
Comment 3 Dani Megert CLA 2008-04-17 12:35:18 EDT
*** Bug 227560 has been marked as a duplicate of this bug. ***
Comment 4 Dani Megert CLA 2008-11-12 02:48:12 EST
*** Bug 254951 has been marked as a duplicate of this bug. ***
Comment 5 Dani Megert CLA 2008-11-12 06:16:50 EST
In order to fix this we would need control over the copy operation, e.g. telling to ignore LineBackgroundListener. I filed bug 255013 to request that.
Comment 6 Dani Megert CLA 2009-07-08 02:22:58 EDT
*** Bug 282727 has been marked as a duplicate of this bug. ***
Comment 7 Dani Megert CLA 2009-08-11 03:03:41 EDT
*** Bug 286166 has been marked as a duplicate of this bug. ***
Comment 8 Dani Megert CLA 2011-08-10 03:21:09 EDT
*** Bug 45969 has been marked as a duplicate of this bug. ***
Comment 9 Anton Leherbauer CLA 2013-01-23 02:29:36 EST
*** Bug 398855 has been marked as a duplicate of this bug. ***
Comment 10 Dani Megert CLA 2015-07-21 08:49:58 EDT
*** Bug 471215 has been marked as a duplicate of this bug. ***
Comment 11 Eclipse Genie CLA 2016-05-19 04:12:57 EDT
New Gerrit change created: https://git.eclipse.org/r/73129
Comment 12 Tobias Melcher CLA 2016-05-19 04:23:08 EDT
Problem occurs when you copy exactly one line without a line break. StyledText.RTFWriter#writeLine calls

event = getLineBackgroundData(lineOffset, line);
if (event != null && event.lineBackground != null) 
   lineBackground = event.lineBackground;

org.eclipse.jface.text.CursorLinePainter#lineGetBackground is called as listener of ST.LineGetBackground event which then sets event.lineBackground via following code:

if (event.lineOffset <= caret && caret <= event.lineOffset + length && !hasMultiLineSelection(textWidget))
	event.lineBackground= fHighlightColor;

With https://git.eclipse.org/r/#/c/73129/ , I propose to delete the getLineBackgroundData call in RTFWriter#writeLine. The problem reported in this bug then disappears. 
Is this getLineBackgroundData needed in RTFWriter? If yes, could you please give an example for which scenario this is needed?

Thanks and best regards,
Tobias
Comment 13 Dani Megert CLA 2016-05-19 05:20:55 EDT
(In reply to Tobias Melcher from comment #12)
> Problem occurs when you copy exactly one line without a line break.
> StyledText.RTFWriter#writeLine calls
> 
> event = getLineBackgroundData(lineOffset, line);
> if (event != null && event.lineBackground != null) 
>    lineBackground = event.lineBackground;
> 
> org.eclipse.jface.text.CursorLinePainter#lineGetBackground is called as
> listener of ST.LineGetBackground event which then sets event.lineBackground
> via following code:
> 
> if (event.lineOffset <= caret && caret <= event.lineOffset + length &&
> !hasMultiLineSelection(textWidget))
> 	event.lineBackground= fHighlightColor;
> 
> With https://git.eclipse.org/r/#/c/73129/ , I propose to delete the
> getLineBackgroundData call in RTFWriter#writeLine. The problem reported in
> this bug then disappears. 
> Is this getLineBackgroundData needed in RTFWriter? If yes, could you please
> give an example for which scenario this is needed?
> 
> Thanks and best regards,
> Tobias

This would be a change request for SWT. They have to review it and make sure RTF copying is not affected. I suggest to file a separate bug against SWT.
Comment 14 Gunnar Wagenknecht CLA 2016-10-29 16:07:33 EDT
I think it's arguable whether copying the highlight background color is wanted or not. There might be users liking that feature, especially when pasting multiple lines with that one specifically highlighted. Personally, I would copy as screenshot, though.

I also think that users not wanting any formatting while pasting are used to "paste as text" functionality offered these days in all modern editors.

However, I am concerned that the proposed changed removes background color copying altogether, i.e. disabling a feature users might use. I'm not sure this is something we should do, given that there is are workaround (such as "paste as text").
Comment 15 Tobias Melcher CLA 2016-10-30 10:17:29 EDT
(In reply to Gunnar Wagenknecht from comment #14)
> I think it's arguable whether copying the highlight background color is
> wanted or not. There might be users liking that feature, especially when
> pasting multiple lines with that one specifically highlighted. Personally, I
> would copy as screenshot, though.
> 
> I also think that users not wanting any formatting while pasting are used to
> "paste as text" functionality offered these days in all modern editors.
> 
> However, I am concerned that the proposed changed removes background color
> copying altogether, i.e. disabling a feature users might use. I'm not sure
> this is something we should do, given that there is are workaround (such as
> "paste as text").

I agree, removing the background color completely is not the right way.
From my point of view, the current behavior is not acceptable. Copy and paste a single line without a line break is inserting the blue background selection color in RTF. The blue background is not inserted when multiple lines including a line break are copied. This behavior is inconsistent.
So, therefore I vote, that the blue background selection color is not added when pasting a single line.
Comment 16 Thomas Wolf CLA 2017-12-01 09:20:47 EST
(In reply to Tobias Melcher from comment #15)
> (In reply to Gunnar Wagenknecht from comment #14)
> > I think it's arguable whether copying the highlight background color is
> > wanted or not. There might be users liking that feature, especially when
> > pasting multiple lines with that one specifically highlighted. Personally, I
> > would copy as screenshot, though.
> > 
> > I also think that users not wanting any formatting while pasting are used to
> > "paste as text" functionality offered these days in all modern editors.
> > 
> > However, I am concerned that the proposed changed removes background color
> > copying altogether, i.e. disabling a feature users might use. I'm not sure
> > this is something we should do, given that there is are workaround (such as
> > "paste as text").
> 
> I agree, removing the background color completely is not the right way.
> From my point of view, the current behavior is not acceptable. Copy and
> paste a single line without a line break is inserting the blue background
> selection color in RTF. The blue background is not inserted when multiple
> lines including a line break are copied. This behavior is inconsistent.
> So, therefore I vote, that the blue background selection color is not added
> when pasting a single line.

Wait a sec. Select some text (multiple lines) in a diff shown by Egit, including some removed and added lines. For instance, in the history view, bottom-left panel, or in the commit viewer, diff page. EGit shows removed lines with a red-ish background, and add lines with a green-ish background. Now paste into Word. On my Mac (with an old Word-for-Mac-2011) I get the red and green backgrounds copied nicely (though Word displays a different shade of red and green: fully saturated). It seems to me that with your change, I wouldn't get these backgrounds anymore. (They are added through the same mechanism as the "current line background".)

The blue color is not added when you select multiple lines because that blue you see in Eclipse is the selection highlight, not the "current line background". When you select multiple lines, there is no "current line" and consequently no current line background anywhere.

You do get the blue background when you select only part of the line because as long as start and end of the selection are on the same line, there is a "current" line, and it thus has the "current line background".
Comment 17 Tobias Melcher CLA 2017-12-01 12:17:32 EST
Agreed. I propose to disable the logic of CursorLinePainter when copy to RTF logic is executed. We then still have the EGit background colors, but not the annoying blue background color.
Unfortunately, this now gets quite ugly because StyledText (as part of SWT plugin) doesn't know anything about CursorLinePainter because it is part of jface.text.
Comment 18 Tobias Melcher CLA 2017-12-01 12:22:57 EST
(In reply to Gunnar Wagenknecht from comment #14)
> I think it's arguable whether copying the highlight background color is
> wanted or not. There might be users liking that feature, especially when
> pasting multiple lines with that one specifically highlighted. Personally, I
> would copy as screenshot, though.

If you paste multiple selected lines today, now background color is pasted. There then no one specifically highlighted line :-)
Comment 19 Thomas Wolf CLA 2017-12-01 12:48:19 EST
(In reply to Tobias Melcher from comment #18)
> (In reply to Gunnar Wagenknecht from comment #14)
> > I think it's arguable whether copying the highlight background color is
> > wanted or not. There might be users liking that feature, especially when
> > pasting multiple lines with that one specifically highlighted. Personally, I
> > would copy as screenshot, though.
> 
> If you paste multiple selected lines today, now background color is pasted.
> There then no one specifically highlighted line :-)

That's not true. See the EGit example in comment 16. It's just that the CurrentLinePainter only adds the blue "current line background" if the selection is wholly on one line. If the viewer adds other backgrounds, they _are_copied, also when you select multiple lines.

One could avoid having the current line background being pasted by changing the behavior of the CurrentLinePainter: it could highlight the current line only if selection_start == selection_end. When there is a selection, the selection is marked anyway, and the user knows where it is. By contrast, when there's only a caret position, the caret may be hard to see and a "current line background" helps.

Since one does have to select something to copy-paste, this would avoid the blue background on RTF paste, as for multi-line selections, but still give all other backgrounds a viewer might use.

As for Word using saturated colors: RTF has only RGB color support, so if an Eclipse background uses RGBA, it should be converted to RGB (blend with viewer background).
Comment 20 Eclipse Genie CLA 2017-12-01 13:16:32 EST
New Gerrit change created: https://git.eclipse.org/r/112728
Comment 21 Tobias Melcher CLA 2017-12-01 13:20:28 EST
Could you please review change 112728? CursorLinePainter is now disabled during copy to clipboard run. Will this be fine?
Comment 22 Tobias Melcher CLA 2017-12-02 17:42:54 EST
StyledText#RTFWriter is using \\highlight to specify background color.
Based on
http://www.biblioscape.com/rtf15_spec.htm#Heading45
the highlight background colors are predefined and cannot be configured. 

So, using \\highlight is not a good idea. \cb should be better used. 
But based on https://stackoverflow.com/questions/3513758/how-can-i-change-the-background-color-of-specific-characters-in-a-rtf-document, \cb is not implemented by Microsoft, one should use \chshdng0\chcbpatN instead.

I locally modified RTFWriter#writeStyledLine (see code below) and replaced \\highlight with \chshdng0\chcbpatN and problem is now really solved without the need to disable CursorLinePainter.

Could someone please take over the code and push it to gerrit? Setting up a locally buildable and testable SWT plugin project is too complex for me. Thanks.

	void writeStyledLine(String line, int lineOffset, int ranges[], StyleRange[] styles, Color lineBackground, int indent, int alignment, boolean justify) {
		int lineLength = line.length();
		int startOffset = getStart();
		int writeOffset = startOffset - lineOffset;
		if (writeOffset >= lineLength) return;
		int lineIndex = Math.max(0, writeOffset);

		write("\\fi");
		write(indent);
		switch (alignment) {
			case SWT.LEFT: write("\\ql"); break;
			case SWT.CENTER: write("\\qc"); break;
			case SWT.RIGHT: write("\\qr"); break;
		}
		if (justify) write("\\qj");
		write(" ");

		int endOffset = startOffset + super.getCharCount();
		int lineEndOffset = Math.min(lineLength, endOffset - lineOffset);
		for (int i = 0; i < styles.length; i++) {
			StyleRange style = styles[i];
			int start, end;
			if (ranges != null) {
				start = ranges[i << 1] - lineOffset;
				end = start + ranges[(i << 1) + 1];
			} else {
				start = style.start - lineOffset;
				end = start + style.length;
			}
			// skip over partial first line
			if (end < writeOffset) {
				continue;
			}
			// style starts beyond line end or RTF write end
			if (start >= lineEndOffset) {
				break;
			}
			// write any unstyled text
			if (lineIndex < start) {
				// copy to start of style
				// style starting beyond end of write range or end of line
				// is guarded against above.
				write(line, lineIndex, start);
				lineIndex = start;
			}
			// write styled text
			write("{");
			int colorIndex = getColorIndex(style.background, DEFAULT_BACKGROUND);
			if (colorIndex != DEFAULT_BACKGROUND) {
				write("\\chshdng0\\chcbpat");
				write(colorIndex);
			}else {
				if (lineBackground != null) {
					write("\\cb");
					write("\\chshdng0\\chcbpat");
					write(getColorIndex(lineBackground, DEFAULT_BACKGROUND));
					//write(" ");
				}
			}
			write("\\cf");
			write(getColorIndex(style.foreground, DEFAULT_FOREGROUND));
			
			int fontStyle = style.fontStyle;
			Font font = style.font;
			if (font != null) {
				int fontIndex = getFontIndex(font);
				write("\\f");
				write(fontIndex);
				FontData fontData = font.getFontData()[0];
				write("\\fs");
				write(fontData.getHeight() * 2);
				fontStyle = fontData.getStyle();
			}
			if ((fontStyle & SWT.BOLD) != 0) {
				write("\\b");
			}
			if ((fontStyle & SWT.ITALIC) != 0) {
				write("\\i");
			}
			if (style.underline) {
				write("\\ul");
			}
			if (style.strikeout) {
				write("\\strike");
			}
			write(" ");
			// copy to end of style or end of write range or end of line
			int copyEnd = Math.min(end, lineEndOffset);
			// guard against invalid styles and let style processing continue
			copyEnd = Math.max(copyEnd, lineIndex);
			write(line, lineIndex, copyEnd);
			if ((fontStyle & SWT.BOLD) != 0) {
				write("\\b0");
			}
			if ((style.fontStyle & SWT.ITALIC) != 0) {
				write("\\i0");
			}
			if (style.underline) {
				write("\\ul0");
			}
			if (style.strikeout) {
				write("\\strike0");
			}
			write("}");
			lineIndex = copyEnd;
		}
		// write unstyled text at the end of the line
		if (lineIndex < lineEndOffset) {
			write(line, lineIndex, lineEndOffset);
		}
	}
Comment 23 Thomas Wolf CLA 2017-12-03 05:00:25 EST
You may be onto something here.

The latest RTF specification I was able to find is 1.9.1 (Word 2007).[1] I have not found any change history for the RTF specification.

The 1.9.1 spec indicates in Appendix B when which control words were introduced:

\highlight   Word 95
\chcbpat     Word 97

\chshdng is missing from the list completely, although it's mentioned elsewhere in the document. And indeed they say to use \cshdng0\chcbpatN instead of \cbN.

And while the RTF 1.5 spec says that \highlightN accepts only pre-defined colors (1 <= N <= 16); the 1.9.1 spec doesn't say so anymore.

However, that explains why I even sometimes got a *yellow* background in windows instead of the "current line background" when pasting into Word-for-Mac-2011. So that seems still to be the case. Would also explain why the highlight colors are  fully saturated. (I was wrong about the RGBA guess above.)

But with your code I don't understand two things:

1. Why move the default line background into the loop?
2. Why write a \cb (even without color index!) for the default line background?

[1] https://www.microsoft.com/en-us/download/details.aspx?id=10725
Comment 24 Thomas Wolf CLA 2017-12-03 09:15:54 EST
(In reply to Thomas Wolf from comment #23)
> \chshdng is missing from the list completely

Found it; looked in the wrong place. Also introduced in Word 97.

> However, that explains why I even sometimes got a *yellow* background in
> windows instead of the "current line background"

s/windows/Word/
Comment 25 Eclipse Genie CLA 2017-12-04 04:59:57 EST
New Gerrit change created: https://git.eclipse.org/r/112799
Comment 26 Tobias Melcher CLA 2017-12-04 05:07:43 EST
(In reply to Thomas Wolf from comment #23)
> But with your code I don't understand two things:
> 
> 1. Why move the default line background into the loop?
> 2. Why write a \cb (even without color index!) for the default line
> background?
I first tried to replace \highlight with \cb and it was not working. There I moved it into the loop; but with no effect. Take a look at https://git.eclipse.org/r/#/c/112799/ which replaces \highlight with \chshdng0\chcbpat. This should be now clean.
Comment 27 Thomas Wolf CLA 2017-12-04 06:23:04 EST
Looks good to me. Fixes the wrong colors being shown in Word on copy-paste from Eclipse.

IMO this is good enough to consider this bug as fixed (once the change gets accepted). If a user really doesn't want the backgrounds, he can use "paste plain text only" like Gunnar had suggested. But at least he now gets the correct background colors.
Comment 28 Steffen Brüntjen CLA 2017-12-04 06:47:00 EST
(In reply to Thomas Wolf from comment #27)

> IMO this is good enough to consider this bug as fixed (once the change gets
> accepted).

Seriously? This bug is about blue background colors being added to the pasted text, and so are 6 out of 7 bug duplicates! I guess noone here really cares about color fine tuning. The only other duplicate which addresses a different problem (Bug 254951) is about colors from "Mark occurences".
Comment 29 Thomas Wolf CLA 2017-12-04 06:56:42 EST
(In reply to Steffen Brüntjen from comment #28)
> 
> Seriously?

In my opinion: yes, seriously. If you set Eclipse to highlight the current line, the current line has a (normally light blue) background. Why in heaven's name should it not be copied, but other backgrounds should? (Note that many of the duplicates are about the selection color, but it's actually the current line highlighting, not the selection highlight. The selection highlight is not included in the RTF paste.)

If one really does want to exclude that current line highlight, maybe the approach from comment 19 could be done. That would also remove the current highlight in Eclipse when there is a non-empty selection on a single line, and what is shown in Eclipse would still be consistent with what gets pasted into Word.

But asking for a paste result that is inconsistent with what Eclipse actually has IMO makes no sense at all.
Comment 30 Steffen Brüntjen CLA 2017-12-04 09:19:27 EST
Sorry, I didn't grasp that the color does not come from in-line text selection. I guess as soon as text is readable when pasted to Outlook, everyone will be very happy!
Comment 31 Thomas Wolf CLA 2017-12-04 10:26:18 EST
Created attachment 271760 [details]
Screenshot of Word document showing the effect

Just to clarify what this is about: screenshot of a Word (Word-for-Mac-2011) document showing a copy-paste from an EGit diff viewer in Eclipse before and after Tobias' fix.
Comment 32 Niraj Modi CLA 2018-02-16 04:51:56 EST

*** This bug has been marked as a duplicate of bug 529241 ***
Comment 33 Steffen Brüntjen CLA 2018-08-29 10:11:09 EDT
Have there been some recent changes that now add a gray background color to parts of the pasted text? It looks like it's coming from 'mark occurrences' now :-(

It's even worse now since I can not remove the color in Outlook. I am able to set background to red or yellow, but when I choose "no color" (there's no entry "white"), it becomes gray again.

Anyone else with this problem?