Bug 394277 - RichStringProcessor produces events in wrong order
Summary: RichStringProcessor produces events in wrong order
Status: NEW
Alias: None
Product: Xtend
Classification: Tools
Component: Core (show other bugs)
Version: 2.4.0   Edit
Hardware: PC Mac OS X
: P3 normal (vote)
Target Milestone: ---   Edit
Assignee: Project Inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords:
: 472332 (view as bug list)
Depends on:
Blocks: 393995
  Show dependency tree
 
Reported: 2012-11-14 08:40 EST by Moritz Eysholdt CLA
Modified: 2015-07-10 06:28 EDT (History)
3 users (show)

See Also:


Attachments
Impact of this bug on semantic highlighting (19.33 KB, image/png)
2012-11-20 08:36 EST, Moritz Eysholdt CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Moritz Eysholdt CLA 2012-11-14 08:40:57 EST
For the following rich string, the events are in the expected order (aka ordered by offset)

val x = ```
	<<true>>
```

announceNextLiteral()
acceptTemplateText("")
acceptTemplateLineBreak()
acceptTemplateText("			")
acceptSemanticText("")
acceptSemanticText("")
acceptExpression()
announceNextLiteral()
acceptSemanticText("")
acceptSemanticLineBreak()
acceptTemplateText


for the following template expression, however, the announceNextLiteral() for the second literal is called before all text from the previous literal has been accepted. 

val x = ```  
	<<>>     
```          

announceNextLiteral()
acceptTemplateText("")
acceptTemplateLineBreak()
announceNextLiteral()          <<<<<--------
acceptTemplateText("")
acceptSemanticText("			")
acceptSemanticText("")
acceptSemanticLineBreak()
acceptTemplateText

The expected order is:

announceNextLiteral()
acceptTemplateText("")
acceptTemplateLineBreak()
acceptTemplateText("")
acceptSemanticText("			")
announceNextLiteral()          <<<<<--------
acceptSemanticText("")
acceptSemanticLineBreak()
acceptTemplateText


This currently causes the the formatter smoke tests fail. Please note that all involved template expressions are syntactically correct.
Comment 1 Moritz Eysholdt CLA 2012-11-14 08:57:57 EST
I've set the related tests to 'ignore' for now.

see http://git.eclipse.org/c/xtend/org.eclipse.xtend.git/commit/?id=e68cb9b9b783e3ee11d38938c95d35c78ff18c0a
Comment 2 Moritz Eysholdt CLA 2012-11-19 11:28:26 EST
I've implemented a test for this bug and set it to ignore for now:

http://git.eclipse.org/c/xtend/org.eclipse.xtend.git/commit/?id=5f76bf0fd80b1d53cda14dcd6d51b2d29dca943e

Fixing this isn't trivial since 
org.eclipse.xtend.core.richstring.RichStringProcessor.Implementation.caseLiteral(Literal) has the notion of "skipping" literals and announcing their whitespace at a later point. This conflicts with the literals being announced on first sight.
Comment 3 Moritz Eysholdt CLA 2012-11-20 08:36:22 EST
Created attachment 223750 [details]
Impact of this bug on semantic highlighting

This bug actually has an impact on semantic highlighting, too: semantic whitespace  preceding an empty «» isn't visualised as grey-space. See attached screenshot.
Comment 4 Moritz Eysholdt CLA 2012-11-20 08:41:07 EST
the test in the screenshot passes.
Comment 5 Sebastian Zarnekow CLA 2012-11-26 08:37:07 EST
The contract of the IRichStringPartAcceptor did not strictly require the events to be in-order. The semantic was weakend with the introduction of empty expressions in templates:

''' << /* some comment */ >> '''

The presence of << /* comments */ >> should not change the output of a template in any way thus ''' <<>> ''' is strictly equal to '''  '''. Therefore the implementation got somehow crude. Are you still working on a fix for the processor / acceptor ping-pong, Moritz? Otherwise I could look into thus one in the next days.
Comment 6 Moritz Eysholdt CLA 2013-01-07 05:57:51 EST
Sorry for the late response. No, I'm not working on this atm. 

The order of the events would not be so important if the acceptor would also pass the offset information for whitespace...
Comment 7 Sebastian Zarnekow CLA 2013-01-07 06:01:17 EST
Unfortunately the processor *should* be fully independent from an existing node model thus it does not really assume that there is any offset information available. It could probably pass the offset relative to the semantic content of the rich string but this would involve quite a few changes in the architecture. Nevertheless I think that we need the events in the right order to fix some surprising highlighting and validation issues around template expressions.
Comment 8 Sebastian Zarnekow CLA 2014-08-12 05:11:13 EDT
see org.eclipse.xtend.core.tests.smoke.SmokeTestCases.test_04()
Comment 9 Jan Koehnlein CLA 2015-07-10 06:28:04 EDT
*** Bug 472332 has been marked as a duplicate of this bug. ***