Description Tomer Mahlin 2007-09-09 11:45:05 EDT

This defect is a placeholder for discussion related to implementation of API
resolving issue with display of XPath type complex expression.

Problem overview
-----------------

 XPath string should be rendered left-to-right and also has the Unicode
Bidirectional algorithm applied to it in order to render the mixed text of the
segment in the proper orientation. 
  As an example, a XPath path such as 

  /BOOK/CHAPTER/PARAGRAPH
  /YEAR/MONTH [@NAME = "/BOOK/CHAPTER/PARAGRAPH"] 

(where capital letters indicate RTL-oriented text) should render as 

  /KOOB/RETPAHC/HPARGARAP
  /RAEY/HTNOM [@EMAN = "/HPARGARAP/RETPAHC/KOOB"]

when using the Unicode Bidi algorithm and segmenting the string into tokens. Notice that literal is not supposed to be treated as structured text but rather as a plane text.

  However, since the OS does not understand that the string should be segmented
according to each segment of the XPath, it renders the strings as 

  /HPARGARAP/RETPAHC/KOOB
  /HPARGARAP/RETPAHC/KOOB/" = EMAN@] HTNOM/RAEY"]

Approach overview
-----------------
 The design for XPath complex expression is available in bug 202751 
dedicated to this specific type of complex expression design discussion.

 Following the terminology defined in bug 183164 XPath type of complex
expression belong to group 2 languages. This group includes languages which can not be adequately parsed into tokens using oversimplified technique of parsing -
tokenization based on list of separators. For XPath there is at least one additional level which should be treated separately - level of Stings (aka constants or literals). Each string should be treated as a single token. In other words each string should be treated as unstructured or plane text. After isolating strings, the following characters are separators: white space, slash (/), square brackets ( [ and ] ), less-than (<), greater-than (>), equal sign (=), exclamation mark (!), colon (:), at sign (@), period (.), vertical bar (|), parentheses ( ( and ) ), plus (+), minus (-), asterisk (*).

History of handling and related instances addressed in various components
-------------------------------------------------------------------------
  Currently I am aware of 2 instances of XPath type of complex expression issues handled in bugzilla.
   1. bug 140390 (brackets with possibly no BiDi data at all)
   2. bug 179490 (/CommonBaseEvent[(StrTools:strcmp(@globalInstanceId, 'abcCBA123' < 0)])

  Please notice that two of above issues were fixed not for general case of XPath language pattern but for very specific patterns. Consequently, the provided solutions should be replaced by a more general one.

  I know about several additional instances of XPath type of complex expression issues in Eclipse based products. However this information is confidential and should not be shared via bugzilla.


Layer(s)/components in which support for complex expressions should be
implemented
---------------------------------------------------------------------------
  XPath language patterns appear as both labels or text in a single input text fields and in a complex Styled Text based multiline XPath editors. As was correctly pointed out in bug 14039, comment #9 depending on the context the level at which the solution is provided might differ. 
  For single input field / label context enhancement of TextProcessor might be a good idea. However, for more complex case such as XPath editor based on Styled Text editor the right place might be XPath editor itself. This is since this editor already has tokenizers to support colorization, and this same type of code is needed to insert directional characters appropriately.