### Eclipse Workspace Patch 1.0 #P org.eclipse.equinox.bidi Index: META-INF/MANIFEST.MF =================================================================== RCS file: /cvsroot/rt/org.eclipse.equinox/components/bundles/org.eclipse.equinox.bidi/META-INF/MANIFEST.MF,v retrieving revision 1.4 diff -u -r1.4 MANIFEST.MF --- META-INF/MANIFEST.MF 9 Feb 2011 21:28:55 -0000 1.4 +++ META-INF/MANIFEST.MF 25 May 2011 16:24:07 -0000 @@ -18,4 +18,4 @@ org.eclipse.osgi.util;version="1.1.0", org.osgi.framework;version="1.5.0", org.osgi.util.tracker;version="1.4.0" -Bundle-Activator: org.eclipse.equinox.bidi.internal.BidiComplexActivator +Bundle-Activator: org.eclipse.equinox.bidi.internal.STextActivator Index: plugin.properties =================================================================== RCS file: /cvsroot/rt/org.eclipse.equinox/components/bundles/org.eclipse.equinox.bidi/plugin.properties,v retrieving revision 1.2 diff -u -r1.2 plugin.properties --- plugin.properties 3 Feb 2010 20:01:00 -0000 1.2 +++ plugin.properties 25 May 2011 16:24:07 -0000 @@ -10,9 +10,9 @@ ############################################################################### pluginName = Bidirectional Text Support providerName = Eclipse.org - Equinox -bidiExpressionPointName = Add new complex expression type -bidiExtensionName = Default bi-directional processors -sqlProcessorName = Processor for SQL expressions +bidiExpressionPointName = Add new structured text type +bidiExtensionName = Default bidirectional processors +sqlProcessorName = Processor for SQL statements regexProcessorName = Processor for regular expressions javaProcessorName = Processor for Java statements commaProcessorName = Processor for comma-separated lists Index: plugin.xml =================================================================== RCS file: /cvsroot/rt/org.eclipse.equinox/components/bundles/org.eclipse.equinox.bidi/plugin.xml,v retrieving revision 1.3 diff -u -r1.3 plugin.xml --- plugin.xml 9 Feb 2011 21:28:55 -0000 1.3 +++ plugin.xml 25 May 2011 16:24:07 -0000 @@ -1,68 +1,68 @@ - + Index: schema/BiDiExpressionType.exsd =================================================================== RCS file: schema/BiDiExpressionType.exsd diff -N schema/BiDiExpressionType.exsd --- schema/BiDiExpressionType.exsd 9 Feb 2011 21:28:55 -0000 1.3 +++ /dev/null 1 Jan 1970 00:00:00 -0000 @@ -1,146 +0,0 @@ - - - - - - - - - Use this extension point to describe processing for strings that need special Bi-directional handling, such as file paths, SQL expressions, or Java code. - - - - - - - - - - - - - - - - - - - - - - - - - Unique identifier of this extension - - - - - - - - - - - - - - - - - - - - - - Type of the expression that this processor will work on. - - - - - - - The processor that provides specialized bi-directional handling for this type of strings. - - - - - - - - - - The description of the string type. - - - - - - - - - - - - - - - 1.0 - - - - - - - - - A processor for the complex expressions that support "minus" delimiters can be described as: - -<pre> -public class ComplExpSepMinus extends ComplExpBasic -{ - public ComplExpSepMinus() - { - super("-"); - } -} -</pre> - - - - - - - - - - TBD - - - - - - - - - All the classes mentioned as Examples - - - - - - - - - Copyright (c) 2010 IBM Corporation and others.<br> -All rights reserved. This program and the accompanying materials are made -available under the terms of the Eclipse Public License v1.0 which -accompanies -this distribution, and is available at -<a -href="http://www.eclipse.org/legal/epl-v10.html">http://www.eclipse.org/legal/epl-v10.html</a> - - - - - Index: schema/STextType.exsd =================================================================== RCS file: schema/STextType.exsd diff -N schema/STextType.exsd --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ schema/STextType.exsd 1 Jan 1970 00:00:00 -0000 @@ -0,0 +1,142 @@ + + + + + + + + + Use this extension point to describe processing for strings that need special bidirectional handling, such as file paths, SQL statements, or Java code. + + + + + + + + + + + + + + + + + + + + + + + + Unique identifier of this extension + + + + + + + + + + + + + + + + + + + + + + Type of the structured text that this processor will work on. + + + + + + + The processor that provides specialized bidirectional handling for this type of strings. + + + + + + + + + + The description of the string type. + + + + + + + + + + + + + + + 1.0 + + + + + + + + + A processor for the structured text strings that support "minus" delimiters can be described as: + +<pre> +public class STextSepMinus extends STextProcessor +{ + public STextSepMinus() + { + super("-"); + } +} +</pre> + + + + + + + + + TBD + + + + + + + + + All the classes mentioned as Examples + + + + + + + + + Copyright (c) 2011 IBM Corporation and others.<br> +All rights reserved. This program and the accompanying materials are made +available under the terms of the Eclipse Public License v1.0 which +accompanies this distribution, and is available at +<a +href="http://www.eclipse.org/legal/epl-v10.html">http://www.eclipse.org/legal/epl-v10.html</a> + + + + Index: src/org/eclipse/equinox/bidi/BidiComplexEngine.java =================================================================== RCS file: src/org/eclipse/equinox/bidi/BidiComplexEngine.java diff -N src/org/eclipse/equinox/bidi/BidiComplexEngine.java --- src/org/eclipse/equinox/bidi/BidiComplexEngine.java 27 Apr 2011 14:53:08 -0000 1.1 +++ /dev/null 1 Jan 1970 00:00:00 -0000 @@ -1,496 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2010, 2011 IBM Corporation and others. - * All rights reserved. This program and the accompanying materials - * are made available under the terms of the Eclipse Public License v1.0 - * which accompanies this distribution, and is available at - * http://www.eclipse.org/legal/epl-v10.html - * - * Contributors: - * IBM Corporation - initial API and implementation - ******************************************************************************/ -package org.eclipse.equinox.bidi; - -import org.eclipse.equinox.bidi.custom.BidiComplexFeatures; -import org.eclipse.equinox.bidi.internal.BidiComplexImpl; - -/** - * This class acts as a mediator between applications and complex - * expression processors. - * The purpose of complex expression processors is to add directional - * formatting characters to ensure correct display. - * This class shields applications from the - * intricacies of complex expression processors. - *

- * For a general introduction to complex expressions, see - * {@link - * the package documentation}. - * - *

How to Specify a Processor

- * - *

All the methods in this class have a first argument which - * designates a type of processor. - * - *

It can be specified as a string (usually one of the - * literals to be found in {@link IBidiComplexExpressionTypes} - * or as an instance of {@link custom.IBidiComplexProcessor}. - * - *

Such an instance can be obtained using the - * {@link custom.BidiComplexStringProcessor#getProcessor getProcessor} - * method for the registered processors, or by instantiating a private processor. - * - *

When the same processor is used in multiple calls, it may be - * beneficial to obtain a reference to the processor and to use it - * in following calls, rather than to specify the processor by its - * type expressed as a string, which necessitates a registry search - * for each call. - * - *

A processor reference is also the only way to examine the - * features of a processor by calling its - * {@link custom.IBidiComplexProcessor#getFeatures getFeatures} method. - * - *

Specifying null for the processor as first argument - * of a method causes this method to behave as a no-op. - * - *

State

- * - *

Most of the methods in this class have a text - * argument which may be just a part of a larger body of text. - * When it is the case that the text is submitted in parts with - * repeated calls, there may be a need to pass information from - * one invocation to the next one. For instance, one invocation - * may detect that a comment or a literal has been started but - * has not been completed. In such cases, a state - * argument must be used. - * - *

The state argument must be an array of integers - * with at least one element. Only the first element is used by - * the methods of this class. - * - *

When submitting the initial part of the text, the first element - * of state must contain the value {@link #STATE_INITIAL} - * or any value <= 0. - * - *

After calling a method with a non-null state argument, - * a value is returned in the first element of state. This - * value should be passed unmodified to the method when calling it again - * with the text which is the continuation of the text submitted in the - * last call. - * - *

When the text submitted to a method is not a continuation and is not - * expected to have a continuation , e.g. it is processed all by itself, - * the state argument should be specified as null. - * - *

Code Samples

- * - *

The following code shows how to transform a certain type of complex - * expression (directory and file paths) in order to obtain the full - * text corresponding to the lean text of such an expression. - * - * 

- *
- *    String leanText = "D:\\\u05d0\u05d1\\\u05d2\\\u05d3.ext";
- *    String fullText = BidiComplexEngine.leanToFullText(IBidiComplexExpressionTypes.FILE, null, null, leanText, null);
- *    System.out.println("full text = " + fullText);
- *
- *
- * - *

The following code shows how to transform successive lines of Java - * code in order to obtain the full - * text corresponding to the lean text of each line. - * - * 

- *
- *    int[] state = new int[1];
- *    state[0] = BidiComplexEngine.STATE_INITIAL;
- *    String leanText = "int i = 3; // first Java statement";
- *    String fullText = BidiComplexEngine.leanToFullText(IBidiComplexExpressionTypes.JAVA, null, null, leanText, state);
- *    System.out.println("full text = " + fullText);
- *    leanText = "i += 4; // next Java statement";
- *    fullText = BidiComplexEngine.leanToFullText(IBidiComplexExpressionTypes.JAVA, null, null, leanText, state);
- *    System.out.println("full text = " + fullText);
- *
- *
- * - *

This class provides a user-oriented API but does not provide - * an actual implementation. The real work is done by the class - * {@link BidiComplexImpl}. Users of the API need not be concerned by, and - * should not depend upon, details of the implementation in - * BidiComplexImpl. - * - * @author Matitiahu Allouche - * - */ -public class BidiComplexEngine { - /** - * Constant to use in the first element of the state - * argument when calling most methods of this class - * to indicate that there is no context of previous lines which - * should be initialized before performing the operation. - */ - public static final int STATE_INITIAL = 0; - - private static final int[] EMPTY_INT_ARRAY = new int[0]; - - /** - * Prevent creation of a BidiComplexEngine instance - */ - private BidiComplexEngine() { - // nothing to do - } - - /** Add directional formatting characters to a complex expression - * to ensure correct presentation. - * - * @param processor designates one of the registered processors. - * It can be a string containing a keyword according to - * the processor type, or a processor reference. - * For more details, see above - * How to Specify a Processor. - *

If this argument is null, this method - * returns the text string. - * - * @param features specifies features that affect the processor's - * behavior. - *

This argument may be specified as null, - * in which case the processor will use its standard features - * (as returned by the processor - * {@link custom.IBidiComplexProcessor#getFeatures getFeatures} - * method). - * - * @param environment specifies an environment whose characteristics - * may affect the processor's behavior. - *

This argument may be specified as null, - * in which case the processor will assume a standard - * environment as specified in - * {@link BidiComplexEnvironment#DEFAULT}. - * - * @param text is the text of the complex expression. - * - * @param state can be used to specify that the text - * argument is the continuation of text submitted in a - * previous call and/or to receive information to pass to - * continuation calls. - * For more details, see State above. - *

If all calls to this method are independent from one another, - * this argument should be specified as null. - * - * @return the complex expression with directional formatting - * characters added at proper locations to ensure correct - * presentation. - */ - public static String leanToFullText(Object processor, BidiComplexFeatures features, BidiComplexEnvironment environment, String text, int[] state) { - if (processor == null) - return text; - return BidiComplexImpl.leanToFullText(processor, features, environment, text, state); - } - - /** - * Given a lean string, compute the positions of each of its - * characters within the corresponding full string. - * - * @param processor designates one of the registered processors. - * It can be a string containing a keyword according to - * the processor type, or a processor reference. - * For more details, see above - * How to Specify a Processor. - *

If this argument is null, this method - * returns an identity map. - * - * @param features specifies features that affect the processor's - * behavior. - *

This argument may be specified as null, - * in which case the processor will use its standard features - * (as returned by the processor - * {@link custom.IBidiComplexProcessor#getFeatures getFeatures} - * method). - * - * @param environment specifies an environment whose characteristics - * may affect the processor's behavior. - *

This argument may be specified as null, - * in which case the processor will assume a standard - * environment as specified in - * {@link BidiComplexEnvironment#DEFAULT}. - * - * @param text is the text of the complex expression. - * - * @param state can be used to specify that the text - * argument is the continuation of text submitted in a - * previous call and/or to receive information to pass to - * continuation calls. - * For more details, see State above. - *

If all calls to this method are independent from one another, - * this argument should be specified as null. - * - * @return an array of integers with one element for each of the characters - * in the text argument, equal to the offset of the - * corresponding character in the full string. - */ - public static int[] leanToFullMap(Object processor, BidiComplexFeatures features, BidiComplexEnvironment environment, String text, int[] state) { - if (processor == null) { - int[] map = new int[text.length()]; - for (int i = 0; i < map.length; i++) - map[i] = i; - return map; - } - return BidiComplexImpl.leanToFullMap(processor, features, environment, text, state); - } - - /** - * Given a lean string, compute the offsets of characters - * before which directional formatting characters must be added - * in order to ensure correct presentation. - * - *

Only LRMs (for an expression with LTR base direction) and RLMs (for - * an expression with RTL base direction) are considered. Leading and - * trailing LRE, RLE and PDF which might be prefixed or suffixed - * depending on the {@link BidiComplexEnvironment#getOrientation orientation} of the - * GUI component used for display are not reflected in this method. - * - * @param processor designates one of the registered processors. - * It can be a string containing a keyword according to - * the processor type, or a processor reference. - * For more details, see above - * How to Specify a Processor. - *

If this argument is null, this method - * returns an empty array. - * - * @param features specifies features that affect the processor's - * behavior. - *

This argument may be specified as null, - * in which case the processor will use its standard features - * (as returned by the processor - * {@link custom.IBidiComplexProcessor#getFeatures getFeatures} - * method). - * - * @param environment specifies an environment whose characteristics - * may affect the processor's behavior. - *

This argument may be specified as null, - * in which case the processor will assume a standard - * environment as specified in - * {@link BidiComplexEnvironment#DEFAULT}. - * - * @param text is the text of the complex expression. - * - * @param state can be used to specify that the text - * argument is the continuation of text submitted in a - * previous call and/or to receive information to pass to - * continuation calls. - * For more details, see State above. - *

If all calls to this method are independent from one another, - * this argument should be specified as null. - * - * @return an array of offsets to the characters in the text - * argument before which directional marks must be - * added to ensure correct presentation. - * The offsets are sorted in ascending order. - */ - public static int[] leanBidiCharOffsets(Object processor, BidiComplexFeatures features, BidiComplexEnvironment environment, String text, int[] state) { - if (processor == null) - return EMPTY_INT_ARRAY; - return BidiComplexImpl.leanBidiCharOffsets(processor, features, environment, text, state); - } - - /** - * Remove directional formatting characters which were added to a - * complex expression to ensure correct presentation. - * - * @param processor designates one of the registered processors. - * It can be a string containing a keyword according to - * the processor type, or a processor reference. - * For more details, see above - * How to Specify a Processor. - *

If this argument is null, this method - * returns the text string. - * - * @param features specifies features that affect the processor's - * behavior. - *

This argument may be specified as null, - * in which case the processor will use its standard features - * (as returned by the processor - * {@link custom.IBidiComplexProcessor#getFeatures getFeatures} - * method). - * - * @param environment specifies an environment whose characteristics - * may affect the processor's behavior. - *

This argument may be specified as null, - * in which case the processor will assume a standard - * environment as specified in - * {@link BidiComplexEnvironment#DEFAULT}. - * - * @param text is the text of the complex expression including - * directional formatting characters. - * - * @param state can be used to specify that the text - * argument is the continuation of text submitted in a - * previous call and/or to receive information to pass to - * continuation calls. - * For more details, see State above. - *

If all calls to this method are independent from one another, - * this argument should be specified as null. - * - * @return the complex expression without directional formatting - * characters which might have been added by processing it - * with {@link #leanToFullText leanToFullText}. - * - */ - public static String fullToLeanText(Object processor, BidiComplexFeatures features, BidiComplexEnvironment environment, String text, int[] state) { - if (processor == null) - return text; - return BidiComplexImpl.fullToLeanText(processor, features, environment, text, state); - } - - /** - * Given a full string, compute the positions of each of its - * characters within the corresponding lean string. - * - * @param processor designates one of the registered processors. - * It can be a string containing a keyword according to - * the processor type, or a processor reference. - * For more details, see above - * How to Specify a Processor. - *

If this argument is null, this method - * returns an identity map. - * - * @param features specifies features that affect the processor's - * behavior. - *

This argument may be specified as null, - * in which case the processor will use its standard features - * (as returned by the processor - * {@link custom.IBidiComplexProcessor#getFeatures getFeatures} - * method). - * - * @param environment specifies an environment whose characteristics - * may affect the processor's behavior. - *

This argument may be specified as null, - * in which case the processor will assume a standard - * environment as specified in - * {@link BidiComplexEnvironment#DEFAULT}. - * - * @param text is the text of the complex expression including - * directional formatting characters. - * - * @param state can be used to specify that the text - * argument is the continuation of text submitted in a - * previous call and/or to receive information to pass to - * continuation calls. - * For more details, see State above. - *

If all calls to this method are independent from one another, - * this argument should be specified as null. - * - * @return an array of integers with one element for each of the characters - * in the text argument, equal to the offset of the - * corresponding character in the lean string. - * If there is no corresponding - * character in the lean string (because the - * specified character is a directional formatting character - * added when invoking {@link #leanToFullText leanToFullText}), - * the value returned for this character is -1. - */ - public static int[] fullToLeanMap(Object processor, BidiComplexFeatures features, BidiComplexEnvironment environment, String text, int[] state) { - if (processor == null) { - int[] map = new int[text.length()]; - for (int i = 0; i < map.length; i++) - map[i] = i; - return map; - } - return BidiComplexImpl.fullToLeanMap(processor, features, environment, text, state); - } - - /** - * Given a full string, return the offsets of characters - * which are directional formatting characters that have been added - * in order to ensure correct presentation. - * - *

LRMs (for an expression with LTR base direction), RLMs (for - * an expression with RTL base direction) are considered as well as - * leading and trailing LRE, RLE and PDF which might be prefixed or suffixed - * depending on the {@link BidiComplexEnvironment#getOrientation orientation} - * of the GUI component used for display. - * - * @param processor designates one of the registered processors. - * It can be a string containing a keyword according to - * the processor type, or a processor reference. - * For more details, see above - * How to Specify a Processor. - *

If this argument is null, this method - * returns an empty array. - * - * @param features specifies features that affect the processor's - * behavior. - *

This argument may be specified as null, - * in which case the processor will use its standard features - * (as returned by the processor - * {@link custom.IBidiComplexProcessor#getFeatures getFeatures} - * method). - * - * @param environment specifies an environment whose characteristics - * may affect the processor's behavior. - *

This argument may be specified as null, - * in which case the processor will assume a standard - * environment as specified in - * {@link BidiComplexEnvironment#DEFAULT}. - * - * @param text is the text of the complex expression including - * directional formatting characters. - * - * @param state can be used to specify that the text - * argument is the continuation of text submitted in a - * previous call and/or to receive information to pass to - * continuation calls. - * For more details, see State above. - *

If all calls to this method are independent from one another, - * this argument should be specified as null. - * - * @return an array of offsets to the characters in the text - * argument which are directional formatting characters - * added to ensure correct presentation. - * The offsets are sorted in ascending order. - */ - public static int[] fullBidiCharOffsets(Object processor, BidiComplexFeatures features, BidiComplexEnvironment environment, String text, int[] state) { - if (processor == null) - return EMPTY_INT_ARRAY; - return BidiComplexImpl.fullBidiCharOffsets(processor, features, environment, text, state); - } - - /** - * Get the base direction of a complex expression. - * This base direction may depend on - * whether the expression contains Arabic or Hebrew words - * (if it contains both, the first Arabic or Hebrew letter in the - * expression determines which is the governing script) and on - * whether the GUI is {@link BidiComplexEnvironment#getMirrored mirrored}. - * - * @param processor designates one of the registered processors. - * It can be a string containing a keyword according to - * the processor type, or a processor reference. - * For more details, see above - * How to Specify a Processor. - *

If this argument is null, this method - * returns {@link BidiComplexFeatures#DIR_LTR}. - * - * @param features specifies features that affect the processor's - * behavior. - *

This argument may be specified as null, - * in which case the processor will use its standard features - * (as returned by the processor - * {@link custom.IBidiComplexProcessor#getFeatures getFeatures} - * method). - * - * @param environment specifies an environment whose characteristics - * may affect the processor's behavior. - *

This argument may be specified as null, - * in which case the processor will assume a standard - * environment as specified in - * {@link BidiComplexEnvironment#DEFAULT}. - * - * @param text is the text of the complex expression. - * - * @return the base direction of the complex expression. - * It is one of the values {@link BidiComplexFeatures#DIR_LTR} - * or {@link BidiComplexFeatures#DIR_RTL}. - */ - public static int getCurDirection(Object processor, BidiComplexFeatures features, BidiComplexEnvironment environment, String text) { - return BidiComplexImpl.getCurDirection(processor, features, environment, text, null); - } - -} Index: src/org/eclipse/equinox/bidi/BidiComplexEnvironment.java =================================================================== RCS file: src/org/eclipse/equinox/bidi/BidiComplexEnvironment.java diff -N src/org/eclipse/equinox/bidi/BidiComplexEnvironment.java --- src/org/eclipse/equinox/bidi/BidiComplexEnvironment.java 27 Apr 2011 14:53:08 -0000 1.2 +++ /dev/null 1 Jan 1970 00:00:00 -0000 @@ -1,326 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2010, 2011 IBM Corporation and others. - * All rights reserved. This program and the accompanying materials - * are made available under the terms of the Eclipse Public License v1.0 - * which accompanies this distribution, and is available at - * http://www.eclipse.org/legal/epl-v10.html - * - * Contributors: - * IBM Corporation - initial API and implementation - ******************************************************************************/ -package org.eclipse.equinox.bidi; - -import java.util.Locale; -import org.eclipse.equinox.bidi.custom.IBidiComplexProcessor; -import org.eclipse.equinox.bidi.internal.BidiComplexActivator; - -/** - * This class defines certain details of the environment within which - * complex expressions are processed. - *

- * All public fields in this class are final, i.e. cannot be - * changed after creating an instance. - *

- * All methods in {@link BidiComplexEngine} have a BidiComplexEnvironment - * argument. If this argument is specified as null, the - * {@link #DEFAULT} environment is used. - * - *

Code Samples

- *

Example (set all environment parameters) - * 

- *
- *    BidiComplexEnvironment myEnv = new BidiComplexEnvironment("he_IL", true, BidiComplexEnvironment.ORIENT_RTL);
- *
- *
- *

- * This class also provides a number of convenience methods related to the environment. - *

 

- * @see IBidiComplexProcessor#getFeatures - * - * @author Matitiahu Allouche - */ -public class BidiComplexEnvironment { - - /** - * Constant specifying that the orientation of the GUI component - * where a complex expression will be displayed is LTR. - * It can appear as orientation argument for the - * {@link BidiComplexEnvironment#BidiComplexEnvironment BidiComplexEnvironment constructor} - * and as value returned by {@link #getOrientation}. - */ - public static final int ORIENT_LTR = 0; - - /** - * Constant specifying that the orientation of the GUI component - * where a complex expression will be displayed is RTL. - * It can appear as orientation argument for the - * {@link BidiComplexEnvironment#BidiComplexEnvironment BidiComplexEnvironment constructor} - * and as value returned by {@link #getOrientation}. - */ - public static final int ORIENT_RTL = 1; - - /** - * Constant specifying that the orientation of the GUI component - * where a complex expression will be displayed is contextual with - * a default of LTR (if no strong character appears in the text). - * It can appear as orientation argument for the - * {@link BidiComplexEnvironment#BidiComplexEnvironment BidiComplexEnvironment constructor} - * and as value returned by {@link #getOrientation}. - */ - public static final int ORIENT_CONTEXTUAL_LTR = 2; - - /** - * Constant specifying that the orientation of the GUI component - * where a complex expression will be displayed is contextual with - * a default of RTL (if no strong character appears in the text). - * It can appear as orientation argument for the - * {@link BidiComplexEnvironment#BidiComplexEnvironment BidiComplexEnvironment constructor} - * and as value returned by {@link #getOrientation}. - */ - public static final int ORIENT_CONTEXTUAL_RTL = 3; - - /** - * Constant specifying that the orientation of the GUI component - * where a complex expression will be displayed is not known. - * Directional formatting characters must be added as prefix and - * suffix whenever a full text is generated using - * {@link BidiComplexEngine#leanToFullText leanToFullText}. - * It can appear as orientation argument for the - * {@link BidiComplexEnvironment#BidiComplexEnvironment BidiComplexEnvironment constructor} - * and as value returned by {@link #getOrientation}. - */ - public static final int ORIENT_UNKNOWN = 4; - - /** - * Constant specifying that whatever the orientation of the - * GUI component where a complex expression will be displayed, no - * directional formatting characters must be added as prefix or - * suffix when a full text is generated using - * {@link BidiComplexEngine#leanToFullText leanToFullText}. - * It can appear as orientation argument for the - * {@link BidiComplexEnvironment#BidiComplexEnvironment BidiComplexEnvironment constructor} - * and as value returned by {@link #getOrientation}. - */ - public static final int ORIENT_IGNORE = 5; - - /** - * Pre-defined BidiComplexEnvironment instance with values - * for a non-mirrored GUI and a Left-to-Right presentation component.
- * The language is set to null, which defaults to the language - * of the current default locale. - */ - public static final BidiComplexEnvironment DEFAULT = new BidiComplexEnvironment(null, false, ORIENT_LTR); - - /** - * This string is a 2-letters code representing a language as defined by - * ISO-639. If left as null, it defaults to the language - * of the current default locale. - */ - final String language; - - /** - * Flag specifying that complex expressions processed under this environment - * should assume that the GUI is mirrored (globally going from right to left). - */ - final boolean mirrored; - - /** Specify the orientation (a.k.a. base direction) of the GUI - * component in which the full text of the complex expression will - * be displayed. - */ - final int orientation; - - static Locale defaultLocale; - static String defaultLanguage; - static boolean defaultBidi; - boolean bidiFlag; - - /** - * Constructor - * - * @param lang represents the language to be used in this environment. - * It should be specified as a 2-letters code as defined by - * ISO-639.
- * If longer than 2 letters, the extra letters are ignored.
- * If set to null, it defaults to the language - * of the default locale. - * @see #getLanguage - * - * @param mirrored specifies if the GUI is mirrored. - * @see #getMirrored - * - * @param orientation specifies the orientation of the component - * which is to display the complex expression. It must be - * one of the values - * {@link #ORIENT_LTR ORIENT_LTR}, - * {@link #ORIENT_LTR ORIENT_RTL}, - * {@link #ORIENT_CONTEXTUAL_LTR ORIENT_CONTEXTUAL_LTR}, - * {@link #ORIENT_CONTEXTUAL_RTL ORIENT_CONTEXTUAL_RTL}, - * {@link #ORIENT_UNKNOWN ORIENT_UNKNOWN} or - * {@link #ORIENT_IGNORE ORIENT_IGNORE}.
- * If different, it defaults to {@link #ORIENT_UNKNOWN ORIENT_UNKNOWN}. - * @see #getOrientation - */ - public BidiComplexEnvironment(String lang, boolean mirrored, int orientation) { - if (lang == null) { - language = null; - } else { - if (lang.length() > 2) - language = lang.substring(0, 2); - else - language = lang; - bidiFlag = isBidiLanguage(language); - } - this.mirrored = mirrored; - this.orientation = orientation >= ORIENT_LTR && orientation <= ORIENT_IGNORE ? orientation : ORIENT_UNKNOWN; - } - - /** - * Return a 2-letters code representing a language as defined by - * ISO-639. If equal to null, it defaults to the language - * of the current default locale. - */ - public String getLanguage() { - return language; - } - - /** - * Return a flag indicating that complex expressions processed - * within this environment should assume that the GUI is mirrored - * (globally going from right to left). - */ - public boolean getMirrored() { - return mirrored; - } - - /** Return the orientation (a.k.a. base direction) of the GUI - * component in which the full text of the complex expression - * will be displayed.
- * The orientation must have one of the values - * {@link #ORIENT_LTR ORIENT_LTR}, - * {@link #ORIENT_LTR ORIENT_RTL}, - * {@link #ORIENT_CONTEXTUAL_LTR ORIENT_CONTEXTUAL_LTR}, - * {@link #ORIENT_CONTEXTUAL_RTL ORIENT_CONTEXTUAL_RTL}, - * {@link #ORIENT_UNKNOWN ORIENT_UNKNOWN} or - * {@link #ORIENT_IGNORE ORIENT_IGNORE}. - *

- * When the orientation is ORIENT_LTR and the complex - * expression has a RTL base direction, - * {@link BidiComplexEngine#leanToFullText leanToFullText} - * adds RLE+RLM at the head of the full text and RLM+PDF at its - * end. - *

- * When the orientation is ORIENT_RTL and the complex - * expression has a LTR base direction, - * {@link BidiComplexEngine#leanToFullText leanToFullText} - * adds LRE+LRM at the head of the full text and LRM+PDF at its - * end. - *

- * When the orientation is ORIENT_CONTEXTUAL_LTR or - * ORIENT_CONTEXTUAL_RTL and the data content would resolve - * to a RTL orientation while the complex expression has a LTR base - * direction, {@link BidiComplexEngine#leanToFullText leanToFullText} - * adds LRM at the head of the full text. - *

- * When the orientation is ORIENT_CONTEXTUAL_LTR or - * ORIENT_CONTEXTUAL_RTL and the data content would resolve - * to a LTR orientation while the complex expression has a RTL base - * direction, {@link BidiComplexEngine#leanToFullText leanToFullText} - * adds RLM at the head of the full text. - *

- * When the orientation is ORIENT_UNKNOWN and the complex - * expression has a LTR base direction, - * {@link BidiComplexEngine#leanToFullText leanToFullText} - * adds LRE+LRM at the head of the full text and LRM+PDF at its - * end. - *

- * When the orientation is ORIENT_UNKNOWN and the complex - * expression has a RTL base direction, - * {@link BidiComplexEngine#leanToFullText leanToFullText} - * adds RLE+RLM at the head of the full text and RLM+PDF at its - * end. - *

- * When the orientation is ORIENT_IGNORE, - * {@link BidiComplexEngine#leanToFullText leanToFullText} does not add any directional - * formatting characters as either prefix or suffix of the full text. - *

- */ - public int getOrientation() { - return orientation; - } - - /** - * Check whether the current language uses a - * bidi script (Arabic, Hebrew, Farsi or Urdu). - * - * @return true if the current language uses a bidi script. - * The language may have been set explicitly when creating the - * BidiComplexEnvironment instance, or it may have - * defaulted to the language of the current default locale. - * @see #getLanguage - */ - public boolean isBidi() { - if (defaultLanguage != null && defaultLocale.equals(getDefaultLocale())) - return defaultBidi; - - if (language == null) { - defaultLocale = getDefaultLocale(); - defaultLanguage = defaultLocale.getLanguage(); - defaultBidi = isBidiLanguage(defaultLanguage); - return defaultBidi; - } - - return bidiFlag; - } - - static boolean isBidiLanguage(String lang) { - return "iw".equals(lang) || "he".equals(lang) || "ar".equals(lang) || "fa".equals(lang) || "ur".equals(lang); //$NON-NLS-1$ //$NON-NLS-2$ //$NON-NLS-3$ //$NON-NLS-4$ //$NON-NLS-5$ - } - - static String lineSep; - - /** - * Retrieve the string which represents a line separator in this environment. - * - * @return the string which is used as line separator (e.g. CRLF). - */ - public static String getLineSep() { - // use bundle properties - if (lineSep == null) { - // lineSep = System.getProperty("line.separator", "\n"); //$NON-NLS-1$//$NON-NLS-2$ - lineSep = getProperty("line.separator"); //$NON-NLS-1$/ - } - return lineSep; - } - - static String osName; - static boolean flagOS; - - private static String getProperty(String key) { - // use bundle properties - // osName = System.getProperty("os.name").toLowerCase(); //$NON-NLS-1$ - BidiComplexActivator bidiComplexActivator = BidiComplexActivator.getInstance(); - return bidiComplexActivator.getProperty(key); - } - - private Locale getDefaultLocale() { - BidiComplexActivator bidiComplexActivator = BidiComplexActivator.getInstance(); - return bidiComplexActivator.getDefaultLocale(); - } - - /** - * Check if the current OS is supported by the complex expression packages. - * - * @return true if the current OS is supported. - */ - public static boolean isSupportedOS() { - if (osName == null) { - // use bundle properties - // osName = System.getProperty("os.name").toLowerCase(); //$NON-NLS-1$ - osName = getProperty("os.name").toLowerCase(); //$NON-NLS-1$/ - flagOS = osName.startsWith("windows") || osName.startsWith("linux"); //$NON-NLS-1$ //$NON-NLS-2$ - } - return flagOS; - } - -} Index: src/org/eclipse/equinox/bidi/BidiComplexStringRecord.java =================================================================== RCS file: src/org/eclipse/equinox/bidi/BidiComplexStringRecord.java diff -N src/org/eclipse/equinox/bidi/BidiComplexStringRecord.java --- src/org/eclipse/equinox/bidi/BidiComplexStringRecord.java 27 Apr 2011 14:53:08 -0000 1.2 +++ /dev/null 1 Jan 1970 00:00:00 -0000 @@ -1,193 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2010, 2011 IBM Corporation and others. - * All rights reserved. This program and the accompanying materials - * are made available under the terms of the Eclipse Public License v1.0 - * which accompanies this distribution, and is available at - * http://www.eclipse.org/legal/epl-v10.html - * - * Contributors: - * IBM Corporation - initial API and implementation - ******************************************************************************/ -package org.eclipse.equinox.bidi; - -import java.lang.ref.SoftReference; -import org.eclipse.equinox.bidi.custom.BidiComplexStringProcessor; - -/** - * This class records strings which are complex expressions. Several static - * methods in this class allow to record such strings in a pool, and to find if - * a given string is member of the pool. - *

- * Instances of this class are the records which are members of the pool. - *

- * The pool is managed as a cyclic list. When the pool is full, - * each new element overrides the oldest element in the list. - */ -public class BidiComplexStringRecord { - /** - * Number of entries in the pool of recorded strings - */ - public static final int POOLSIZE = 100; - - // maximum index allowed - private static final int MAXINDEX = POOLSIZE - 1; - - // index of the last entered record - private static int last = MAXINDEX; - - // the pool - private static BidiComplexStringRecord[] records = new BidiComplexStringRecord[POOLSIZE]; - - // complex expression types - private static final String[] types = BidiComplexStringProcessor.getKnownTypes(); - - // maximum type index allowed - private static int MAXTYPE = types.length - 1; - - // reference to the recorded string - private SoftReference strRef; - - // hash code of the recorded string - private int hash; - - // reference to the triplets of the recorded string - private SoftReference triRef; - - /** - * Constructor. - * - * @param string the string to record - * - * @param triplets - * array of short integers, the number of elements in the array - * must be a multiple of 3, so that the array is made of one or - * more triplets of short integers. - *

- * The first element in each triplet is the beginning offset of a - * susbstring of string which is a complex - * expression. - *

- * The second element in each triplet is the ending offset of a - * susbstring of string which is a complex - * expression. This offset points to one position beyond the last - * character of the substring. - *

- * The third element in each triplet is the numeric type of the - * complex expression.
- * The type of a complex expression must be one of the string - * values listed in {@link IBidiComplexExpressionTypes}.
- * The corresponding numeric type must be obtained using the - * method {@link #typeStringToShort typeStringToShort}. - */ - public BidiComplexStringRecord(String string, short[] triplets) { - if (string == null || triplets == null) - throw new IllegalArgumentException("The string and triplets argument must not be null!"); //$NON-NLS-1$ - if ((triplets.length % 3) != 0) - throw new IllegalArgumentException("The number of elements in triplets must be a multiple of 3!"); //$NON-NLS-1$ - for (int i = 2; i < triplets.length; i += 3) - if (triplets[i] < 0 || triplets[i] > MAXTYPE) - throw new IllegalArgumentException("Illegal type value in element" + i); - strRef = new SoftReference(string); - triRef = new SoftReference(triplets); - hash = string.hashCode(); - } - - /** - * Get the numeric type of a complex expression given its string type. - * - * @param type type of complex expression as string. It must be one - * of the strings listed in {@link IBidiComplexExpressionTypes}. - * - * @return a value which is the corresponding numeric type. If - * type is invalid, the method returns -1. - */ - public static short typeStringToShort(String type) { - for (int i = 0; i < types.length; i++) - if (types[i].equals(type)) - return (short) i; - return -1; - } - - /** - * Get the string type of a complex expression given its numeric type. - * - * @param shType - * the numeric type of a complex expression. It should be a value - * obtained using {@link #typeStringToShort typeStringToShort}. - * - * @return the corresponding string type. If shType is invalid, - * the method returns null. - */ - public static String typeShortToString(short shType) { - if (shType < 0 || shType > MAXTYPE) - return null; - return types[shType]; - } - - /** - * Add a record to the pool. - * - * @param record a BidiComplexStringRecord instance - */ - public static synchronized void add(BidiComplexStringRecord record) { - if (last < MAXINDEX) - last++; - else - last = 0; - records[last] = record; - } - - /** - * Check if a string is recorded and retrieve its triplets. - * - * @param string the string to check - * - * @return null if the string is not recorded in the pool; - * otherwise, return the triplets associated with this string. - */ - public static short[] getTriplets(String string) { - if (records[0] == null) // no records at all - return null; - if (string == null || string.length() < 1) - return null; - BidiComplexStringRecord rec; - String str; - short[] tri; - int myLast = last; - int hash = string.hashCode(); - for (int i = myLast; i >= 0; i--) { - rec = records[i]; - if (hash == rec.hash && (tri = (short[]) rec.triRef.get()) != null && (str = (String) rec.strRef.get()) != null && string.equals(str)) { - return tri; - } - } - if (records[MAXINDEX] == null) // never recorded past myLast - return null; - for (int i = MAXINDEX; i > myLast; i--) { - rec = records[i]; - if (hash == rec.hash && (tri = (short[]) rec.triRef.get()) != null && (str = (String) rec.strRef.get()) != null && string.equals(str)) { - return tri; - } - } - return null; - } - - /** - * Clear the pool. All elements of the pool are erased and any associated - * memory is freed. - * - */ - public static synchronized void clear() { - for (int i = 0; i <= MAXINDEX; i++) { - BidiComplexStringRecord sr = records[i]; - if (sr == null) - break; - sr.hash = 0; - sr.strRef.clear(); - sr.triRef.clear(); - records[i] = null; - } - last = MAXINDEX; - } - -} Index: src/org/eclipse/equinox/bidi/BidiComplexUtil.java =================================================================== RCS file: src/org/eclipse/equinox/bidi/BidiComplexUtil.java diff -N src/org/eclipse/equinox/bidi/BidiComplexUtil.java --- src/org/eclipse/equinox/bidi/BidiComplexUtil.java 17 May 2011 19:56:17 -0000 1.3 +++ /dev/null 1 Jan 1970 00:00:00 -0000 @@ -1,313 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2010, 2011 IBM Corporation and others. - * All rights reserved. This program and the accompanying materials - * are made available under the terms of the Eclipse Public License v1.0 - * which accompanies this distribution, and is available at - * http://www.eclipse.org/legal/epl-v10.html - * - * Contributors: - * IBM Corporation - initial API and implementation - ******************************************************************************/ -package org.eclipse.equinox.bidi; - -import org.eclipse.equinox.bidi.custom.BidiComplexFeatures; -import org.eclipse.equinox.bidi.custom.BidiComplexProcessor; - -/** - * This class provides a number of convenience functions facilitating the - * processing of complex expressions. - * - * @noextend This class is not intended to be subclassed by clients. - * @noinstantiate This class is not intended to be instantiated by clients. - * - * @author Matitiahu Allouche - */ -final public class BidiComplexUtil { - - /** - * prevent instantiation - */ - private BidiComplexUtil() { - // empty - } - - /** This is a convenience method which can add directional marks in a given - * text before the characters specified in the given array of offsets, - * and can add a prefix and/or a suffix of directional formatting characters. - * This can be used for instance after obtaining offsets by calling - * {@link BidiComplexEngine#leanBidiCharOffsets leanBidiCharOffsets} in order to - * produce a full text corresponding to the source text. - * The directional formatting characters that will be added at the given - * offsets will be LRMs for expressions with LTR base direction and - * RLMs for expressions with RTL base direction. Leading and - * trailing LRE, RLE and PDF which might be needed as prefix or suffix - * depending on the orientation of the GUI component used for display - * may be added depending on argument affix. - * - * @param text is the text of the complex expression. - * - * @param offsets is an array of offsets to characters in text - * before which an LRM or RLM will be inserted. - * Members of the array must be non-negative numbers smaller - * than the length of text. - * The array must be sorted in ascending order without duplicates. - * This argument may be null if there are no marks to add. - * - * @param direction specifies the base direction of the complex expression. - * It must be one of the values {@link BidiComplexFeatures#DIR_LTR} or - * {@link BidiComplexFeatures#DIR_RTL}. - * - * @param affix specifies if a prefix and a suffix should be added to - * the result to make sure that the direction - * specified as third argument is honored even if the expression - * is displayed in a GUI component with a different orientation. - * - * @return a string corresponding to the source text with - * directional marks (LRMs or RLMs) added at the specified offsets, - * and directional formatting characters (LRE, RLE, PDF) added - * as prefix and suffix if so required. - */ - public static String insertMarks(String text, int[] offsets, int direction, boolean affix) { - int textLen = text.length(); - if (textLen == 0) - return ""; //$NON-NLS-1$ - - String curPrefix, curSuffix, full; - char curMark, c; - char[] fullChars; - if (direction == BidiComplexFeatures.DIR_LTR) { - curMark = LRM; - curPrefix = "\u202a\u200e"; /* LRE+LRM *///$NON-NLS-1$ - curSuffix = "\u200e\u202c"; /* LRM+PDF *///$NON-NLS-1$ - } else { - curMark = RLM; - curPrefix = "\u202b\u200f"; /* RLE+RLM *///$NON-NLS-1$ - curSuffix = "\u200f\u202c"; /* RLM+PDF *///$NON-NLS-1$ - } - // add marks at offsets - if ((offsets != null) && (offsets.length > 0)) { - int offLen = offsets.length; - fullChars = new char[textLen + offLen]; - int added = 0; - for (int i = 0, j = 0; i < textLen; i++) { - c = text.charAt(i); - if ((j < offLen) && (i == offsets[j])) { - fullChars[i + added] = curMark; - added++; - j++; - } - fullChars[i + added] = c; - } - full = new String(fullChars); - } else { - full = text; - } - if (affix) - return curPrefix + full + curSuffix; - return full; - } - - /*************************************************************************/ - /* */ - /* The following code is provided for compatibility with TextProcessor */ - /* */ - /*************************************************************************/ - - // The default set of separators to use to segment a string. - private static final String defaultSeparators = ".:/\\"; //$NON-NLS-1$ - // left to right mark - private static final char LRM = '\u200e'; - // left to right mark - private static final char RLM = '\u200f'; - // left to right embedding - private static final char LRE = '\u202a'; - // right to left embedding - private static final char RLE = '\u202b'; - // pop directional format - private static final char PDF = '\u202c'; - - static boolean isProcessingNeeded() { - if (!BidiComplexEnvironment.isSupportedOS()) - return false; - return BidiComplexEnvironment.DEFAULT.isBidi(); - } - - /** - * Process the given text and return a string with appropriate - * directional formatting characters if the locale is a bidi locale. - * This is equivalent to calling - * {@link #process(String str, String separators)} with the default - * set of separators (dot, colon, slash, backslash). - * - * @param str the text to be processed. - * - * @return the processed string. - */ - public static String process(String str) { - return process(str, defaultSeparators); - } - - /** - * Process a string that has a particular semantic meaning to render - * it correctly on bidi locales. This is done by adding directional - * formatting characters so that presentation using the Unicode - * Bidirectional Algorithm will provide the expected result. - * The text is segmented according to the provided separators. - * Each segment has the Unicode Bidi Algorithm applied to it, - * but as a whole, the string is oriented left to right. - *

- * For example, a file path such as d:\myfolder\FOLDER\MYFILE.java - * (where capital letters indicate RTL text) should render as - * d:\myfolder\REDLOF\ELIFYM.java.

- *

- * NOTE: this method inserts directional formatting characters into the - * text. Methods like String.equals(String) and - * String.length() called on the resulting string will not - * return the same values as would be returned for the original string.

- * - * @param str the text to process. - * - * @param separators separators by which the string will be segmented. - * If null, the default separators are used - * (dot, colon, slash, backslash). - * - * @return the processed string. - * If str is null, - * or of length 0, or if the current locale is not a bidi one, - * return the original string. - */ - public static String process(String str, String separators) { - if ((str == null) || (str.length() <= 1) || !isProcessingNeeded()) - return str; - - // do not process a string that has already been processed. - if (str.charAt(0) == LRE && str.charAt(str.length() - 1) == PDF) - return str; - - // do not process a string if all the following conditions are true: - // a) it has no RTL characters - // b) it starts with a LTR character - // c) it ends with a LTR character or a digit - boolean isStringBidi = false; - int strLength = str.length(); - char c; - for (int i = 0; i < strLength; i++) { - c = str.charAt(i); - if (((c >= 0x05d0) && (c <= 0x07b1)) || ((c >= 0xfb1d) && (c <= 0xfefc))) { - isStringBidi = true; - break; - } - } - while (!isStringBidi) { - if (!Character.isLetter(str.charAt(0))) - break; - c = str.charAt(strLength - 1); - if (!Character.isDigit(c) && !Character.isLetter(c)) - break; - return str; - } - - if (separators == null) - separators = defaultSeparators; - - // make sure that LRE/PDF are added around the string - BidiComplexEnvironment env = new BidiComplexEnvironment(null, false, BidiComplexEnvironment.ORIENT_UNKNOWN); - BidiComplexFeatures features = new BidiComplexFeatures(separators, 0, -1, -1, false, false); - return BidiComplexEngine.leanToFullText(new BidiComplexProcessor(), features, env, str, null); - } - - /** - * Process a string that has a particular semantic meaning to render - * it correctly on bidi locales. This is done by adding directional - * formatting characters so that presentation using the Unicode - * Bidirectional Algorithm will provide the expected result.. - * The text is segmented according to the syntax specified in the - * type argument. - * Each segment has the Unicode Bidi Algorithm applied to it, but the - * order of the segments is governed by the type of the complex expression. - *

- * For example, a file path such as d:\myfolder\FOLDER\MYFILE.java - * (where capital letters indicate RTL text) should render as - * d:\myfolder\REDLOF\ELIFYM.java.

- *

- * NOTE: this method inserts directional formatting characters into the - * text. Methods like String.equals(String) and - * String.length() called on the resulting string will not - * return the same values as would be returned for the original string.

- * - * @param str the text to process. - * - * @param type specifies the type of the complex expression. It must - * be one of the values in {@link IBidiComplexExpressionTypes} or a value. - * added by a plug-in extension. - * - * @return the processed string. - * If str is null, - * or of length 0, or if the current locale is not a bidi one, - * return the original string. - */ - public static String processTyped(String str, String type) { - if ((str == null) || (str.length() <= 1) || !isProcessingNeeded()) - return str; - - // do not process a string that has already been processed. - char c = str.charAt(0); - if (((c == LRE) || (c == RLE)) && str.charAt(str.length() - 1) == PDF) - return str; - - // make sure that LRE/PDF are added around the string - BidiComplexEnvironment env = new BidiComplexEnvironment(null, false, BidiComplexEnvironment.ORIENT_UNKNOWN); - return BidiComplexEngine.leanToFullText(type, null, env, str, null); - } - - /** - * Remove directional formatting characters in the given string that - * were inserted by one of the {@link #process process} methods. - * - * @param str string with directional characters to remove. - * - * @return string with no directional formatting characters. - */ - public static String deprocess(String str) { - if ((str == null) || (str.length() <= 1) || !isProcessingNeeded()) - return str; - - StringBuffer buf = new StringBuffer(); - int strLen = str.length(); - for (int i = 0; i < strLen; i++) { - char c = str.charAt(i); - switch (c) { - case LRM : - continue; - case LRE : - continue; - case PDF : - continue; - default : - buf.append(c); - } - } - return buf.toString(); - } - - /** - * Remove directional formatting characters in the given string that - * were inserted by the {@link #processTyped processTyped} method. - * - * @param str string with directional characters to remove. - * - * @param type type of the complex expression as specified when - * calling {@link #processTyped processTyped}. - * - * @return string with no directional formatting characters. - */ - public static String deprocess(String str, String type) { - if ((str == null) || (str.length() <= 1) || !isProcessingNeeded()) - return str; - - // make sure that LRE/PDF are added around the string - BidiComplexEnvironment env = new BidiComplexEnvironment(null, false, BidiComplexEnvironment.ORIENT_UNKNOWN); - return BidiComplexEngine.fullToLeanText(type, null, env, str, null); - } - -} Index: src/org/eclipse/equinox/bidi/IBidiComplexExpressionTypes.java =================================================================== RCS file: src/org/eclipse/equinox/bidi/IBidiComplexExpressionTypes.java diff -N src/org/eclipse/equinox/bidi/IBidiComplexExpressionTypes.java --- src/org/eclipse/equinox/bidi/IBidiComplexExpressionTypes.java 9 Feb 2011 21:28:54 -0000 1.1 +++ /dev/null 1 Jan 1970 00:00:00 -0000 @@ -1,106 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2010, 2011 IBM Corporation and others. - * All rights reserved. This program and the accompanying materials - * are made available under the terms of the Eclipse Public License v1.0 - * which accompanies this distribution, and is available at - * http://www.eclipse.org/legal/epl-v10.html - * - * Contributors: - * IBM Corporation - initial API and implementation - ******************************************************************************/ -package org.eclipse.equinox.bidi; - -/** - * Bidirectional processors supplied in this bundle. - * - * @noextend This interface is not intended to be extended by clients. - */ -public interface IBidiComplexExpressionTypes { - - /** - * Constant indicating a type of complex expression processor adapted - * to processing property file statements. It expects the following - * string format: - * 
-	 *  name=value
-	 *
- */ - public String PROPERTY = "property"; //$NON-NLS-1$ - - /** - * Constant indicating a type of complex expression processor adapted - * to processing compound names. - * This type covers names made of one or more parts separated by underscores: - * 
-	 *  part1_part2_part3
-	 *
- */ - public String UNDERSCORE = "underscore"; //$NON-NLS-1$ - - /** - * Constant indicating a type of complex expression processor adapted - * to processing comma-delimited lists, such as: - * 
-	 *  part1,part2,part3
-	 *
- */ - public String COMMA_DELIMITED = "comma"; //$NON-NLS-1$ - - /** - * Constant indicating a type of complex expression processor adapted - * to processing expressions with the following string format: - * 
-	 *  system(user)
-	 *
- */ - public String SYSTEM_USER = "system"; //$NON-NLS-1$ - - /** - * Constant indicating a type of complex expression processor adapted - * to processing directory and file paths. - */ - public String FILE = "file"; //$NON-NLS-1$ - - /** - * Constant indicating a type of complex expression processor adapted - * to processing e-mail addresses. - */ - public String EMAIL = "email"; //$NON-NLS-1$ - - /** - * Constant indicating a type of complex expression processor adapted - * to processing URLs. - */ - public String URL = "url"; //$NON-NLS-1$ - - /** - * Constant indicating a type of complex expression processor adapted - * to processing regular expressions, possibly spanning more than one - * line. - */ - public String REGEXP = "regex"; //$NON-NLS-1$ - - /** - * Constant indicating a type of complex expression processor adapted - * to processing XPath expressions. - */ - public String XPATH = "xpath"; //$NON-NLS-1$ - - /** - * Constant indicating a type of complex expression processor adapted - * to processing Java code, possibly spanning more than one line. - */ - public String JAVA = "java"; //$NON-NLS-1$ - - /** - * Constant indicating a type of complex expression processor adapted - * to processing SQL statements, possibly spanning more than one line. - */ - public String SQL = "sql"; //$NON-NLS-1$ - - /** - * Constant indicating a type of complex expression processor adapted - * to processing arithmetic expressions with a RTL base direction. - */ - public String RTL_ARITHMETIC = "math"; //$NON-NLS-1$ -} Index: src/org/eclipse/equinox/bidi/ISTextTypes.java =================================================================== RCS file: src/org/eclipse/equinox/bidi/ISTextTypes.java diff -N src/org/eclipse/equinox/bidi/ISTextTypes.java --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ src/org/eclipse/equinox/bidi/ISTextTypes.java 1 Jan 1970 00:00:00 -0000 @@ -0,0 +1,106 @@ +/******************************************************************************* + * Copyright (c) 2010, 2011 IBM Corporation and others. + * All rights reserved. This program and the accompanying materials + * are made available under the terms of the Eclipse Public License v1.0 + * which accompanies this distribution, and is available at + * http://www.eclipse.org/legal/epl-v10.html + * + * Contributors: + * IBM Corporation - initial API and implementation + ******************************************************************************/ +package org.eclipse.equinox.bidi; + +/** + * Bidirectional processors supplied in this bundle. + * + * @noextend This interface is not intended to be extended by clients. + */ +public interface ISTextTypes { + + /** + * Constant indicating a type of structured text processor adapted + * to processing property file statements. It expects the following + * string format: + * 
+	 *  name=value
+	 *
+ */ + public String PROPERTY = "property"; //$NON-NLS-1$ + + /** + * Constant indicating a type of structured text processor adapted + * to processing compound names. + * This type covers names made of one or more parts separated by underscores: + * 
+	 *  part1_part2_part3
+	 *
+ */ + public String UNDERSCORE = "underscore"; //$NON-NLS-1$ + + /** + * Constant indicating a type of structured text processor adapted + * to processing comma-delimited lists, such as: + * 
+	 *  part1,part2,part3
+	 *
+ */ + public String COMMA_DELIMITED = "comma"; //$NON-NLS-1$ + + /** + * Constant indicating a type of structured text processor adapted + * to processing strings with the following format: + * 
+	 *  system(user)
+	 *
+ */ + public String SYSTEM_USER = "system"; //$NON-NLS-1$ + + /** + * Constant indicating a type of structured text processor adapted + * to processing directory and file paths. + */ + public String FILE = "file"; //$NON-NLS-1$ + + /** + * Constant indicating a type of structured text processor adapted + * to processing e-mail addresses. + */ + public String EMAIL = "email"; //$NON-NLS-1$ + + /** + * Constant indicating a type of structured text processor adapted + * to processing URLs. + */ + public String URL = "url"; //$NON-NLS-1$ + + /** + * Constant indicating a type of structured text processor adapted + * to processing regular expressions, possibly spanning more than one + * line. + */ + public String REGEXP = "regex"; //$NON-NLS-1$ + + /** + * Constant indicating a type of structured text processor adapted + * to processing XPath expressions. + */ + public String XPATH = "xpath"; //$NON-NLS-1$ + + /** + * Constant indicating a type of structured text processor adapted + * to processing Java code, possibly spanning more than one line. + */ + public String JAVA = "java"; //$NON-NLS-1$ + + /** + * Constant indicating a type of structured text processor adapted + * to processing SQL statements, possibly spanning more than one line. + */ + public String SQL = "sql"; //$NON-NLS-1$ + + /** + * Constant indicating a type of structured text processor adapted + * to processing arithmetic expressions with a RTL base direction. + */ + public String RTL_ARITHMETIC = "math"; //$NON-NLS-1$ +} Index: src/org/eclipse/equinox/bidi/STextEngine.java =================================================================== RCS file: src/org/eclipse/equinox/bidi/STextEngine.java diff -N src/org/eclipse/equinox/bidi/STextEngine.java --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ src/org/eclipse/equinox/bidi/STextEngine.java 1 Jan 1970 00:00:00 -0000 @@ -0,0 +1,497 @@ +/******************************************************************************* + * Copyright (c) 2010, 2011 IBM Corporation and others. + * All rights reserved. This program and the accompanying materials + * are made available under the terms of the Eclipse Public License v1.0 + * which accompanies this distribution, and is available at + * http://www.eclipse.org/legal/epl-v10.html + * + * Contributors: + * IBM Corporation - initial API and implementation + ******************************************************************************/ +package org.eclipse.equinox.bidi; + +import org.eclipse.equinox.bidi.custom.ISTextProcessor; +import org.eclipse.equinox.bidi.custom.STextFeatures; +import org.eclipse.equinox.bidi.internal.STextImpl; + +/** + * This class acts as a mediator between applications and structured text + * processors. + * The purpose of structured text processors is to add directional + * formatting characters to ensure correct display. + * This class shields applications from the + * intricacies of structured text processors. + *

+ * For a general introduction to structured text, see + * {@link + * the package documentation}. + * + *

How to Specify a Processor

+ * + *

All the methods in this class have a first argument which + * designates a type of processor. + * + *

It can be specified as a string (usually one of the + * literals to be found in {@link ISTextTypes} + * or as an instance of {@link ISTextProcessor}. + * + *

Such an instance can be obtained using the + * {@link org.eclipse.equinox.bidi.custom.STextStringProcessor#getProcessor getProcessor} + * method for the registered processors, or by instantiating a private processor. + * + *

When the same processor is used in multiple calls, it may be + * beneficial to obtain a reference to the processor and to use it + * in following calls, rather than to specify the processor by its + * type expressed as a string, which necessitates a registry search + * for each call. + * + *

A processor reference is also the only way to examine the + * features of a processor by calling its + * {@link ISTextProcessor#getFeatures getFeatures} method. + * + *

Specifying null for the processor as first argument + * of a method causes this method to behave as a no-op. + * + *

State

+ * + *

Most of the methods in this class have a text + * argument which may be just a part of a larger body of text. + * When it is the case that the text is submitted in parts with + * repeated calls, there may be a need to pass information from + * one invocation to the next one. For instance, one invocation + * may detect that a comment or a literal has been started but + * has not been completed. In such cases, a state + * argument must be used. + * + *

The state argument must be an array of integers + * with at least one element. Only the first element is used by + * the methods of this class. + * + *

When submitting the initial part of the text, the first element + * of state must contain the value {@link #STATE_INITIAL} + * or any value <= 0. + * + *

After calling a method with a non-null state argument, + * a value is returned in the first element of state. This + * value should be passed unmodified to the method when calling it again + * with the text which is the continuation of the text submitted in the + * last call. + * + *

When the text submitted to a method is not a continuation and is not + * expected to have a continuation , e.g. it is processed all by itself, + * the state argument should be specified as null. + * + *

Code Samples

+ * + *

The following code shows how to transform a certain type of structured text + * (directory and file paths) in order to obtain the full + * text corresponding to the given lean text. + * + * 

+ *
+ *    String leanText = "D:\\\u05d0\u05d1\\\u05d2\\\u05d3.ext";
+ *    String fullText = STextEngine.leanToFullText(ISTextTypes.FILE, null, null, leanText, null);
+ *    System.out.println("full text = " + fullText);
+ *
+ *
+ * + *

The following code shows how to transform successive lines of Java + * code in order to obtain the full + * text corresponding to the lean text of each line. + * + * 

+ *
+ *    int[] state = new int[1];
+ *    state[0] = STextEngine.STATE_INITIAL;
+ *    String leanText = "int i = 3; // first Java statement";
+ *    String fullText = STextEngine.leanToFullText(ISTextTypes.JAVA, null, null, leanText, state);
+ *    System.out.println("full text = " + fullText);
+ *    leanText = "i += 4; // next Java statement";
+ *    fullText = STextEngine.leanToFullText(ISTextTypes.JAVA, null, null, leanText, state);
+ *    System.out.println("full text = " + fullText);
+ *
+ *
+ * + *

This class provides a user-oriented API but does not provide + * an actual implementation. The real work is done by the class + * {@link STextImpl}. Users of the API need not be concerned by, and + * should not depend upon, details of the implementation in + * STextImpl. + * + * @author Matitiahu Allouche + * + */ +public class STextEngine { + /** + * Constant to use in the first element of the state + * argument when calling most methods of this class + * to indicate that there is no context of previous lines which + * should be initialized before performing the operation. + */ + public static final int STATE_INITIAL = 0; + + private static final int[] EMPTY_INT_ARRAY = new int[0]; + + /** + * Prevent creation of a STextEngine instance + */ + private STextEngine() { + // nothing to do + } + + /** Add directional formatting characters to a structured text + * to ensure correct presentation. + * + * @param processor designates one of the registered processors. + * It can be a string containing a keyword according to + * the processor type, or a processor reference. + * For more details, see above + * How to Specify a Processor. + *

If this argument is null, this method + * returns the text string. + * + * @param features specifies features that affect the processor's + * behavior. + *

This argument may be specified as null, + * in which case the processor will use its standard features + * (as returned by the processor + * {@link ISTextProcessor#getFeatures getFeatures} + * method). + * + * @param environment specifies an environment whose characteristics + * may affect the processor's behavior. + *

This argument may be specified as null, + * in which case the processor will assume a standard + * environment as specified in + * {@link STextEnvironment#DEFAULT}. + * + * @param text is the structured text string. + * + * @param state can be used to specify that the text + * argument is the continuation of text submitted in a + * previous call and/or to receive information to pass to + * continuation calls. + * For more details, see State above. + *

If all calls to this method are independent from one another, + * this argument should be specified as null. + * + * @return the structured text with directional formatting + * characters added at proper locations to ensure correct + * presentation. + */ + public static String leanToFullText(Object processor, STextFeatures features, STextEnvironment environment, String text, int[] state) { + if (processor == null) + return text; + return STextImpl.leanToFullText(processor, features, environment, text, state); + } + + /** + * Given a lean string, compute the positions of each of its + * characters within the corresponding full string. + * + * @param processor designates one of the registered processors. + * It can be a string containing a keyword according to + * the processor type, or a processor reference. + * For more details, see above + * How to Specify a Processor. + *

If this argument is null, this method + * returns an identity map. + * + * @param features specifies features that affect the processor's + * behavior. + *

This argument may be specified as null, + * in which case the processor will use its standard features + * (as returned by the processor + * {@link ISTextProcessor#getFeatures getFeatures} + * method). + * + * @param environment specifies an environment whose characteristics + * may affect the processor's behavior. + *

This argument may be specified as null, + * in which case the processor will assume a standard + * environment as specified in + * {@link STextEnvironment#DEFAULT}. + * + * @param text is the structured text string. + * + * @param state can be used to specify that the text + * argument is the continuation of text submitted in a + * previous call and/or to receive information to pass to + * continuation calls. + * For more details, see State above. + *

If all calls to this method are independent from one another, + * this argument should be specified as null. + * + * @return an array of integers with one element for each of the characters + * in the text argument, equal to the offset of the + * corresponding character in the full string. + */ + public static int[] leanToFullMap(Object processor, STextFeatures features, STextEnvironment environment, String text, int[] state) { + if (processor == null) { + int[] map = new int[text.length()]; + for (int i = 0; i < map.length; i++) + map[i] = i; + return map; + } + return STextImpl.leanToFullMap(processor, features, environment, text, state); + } + + /** + * Given a lean string, compute the offsets of characters + * before which directional formatting characters must be added + * in order to ensure correct presentation. + * + *

Only LRMs (for a string with LTR base direction) and RLMs (for + * a string with RTL base direction) are considered. Leading and + * trailing LRE, RLE and PDF which might be prefixed or suffixed + * depending on the {@link STextEnvironment#getOrientation orientation} of the + * GUI component used for display are not reflected in this method. + * + * @param processor designates one of the registered processors. + * It can be a string containing a keyword according to + * the processor type, or a processor reference. + * For more details, see above + * How to Specify a Processor. + *

If this argument is null, this method + * returns an empty array. + * + * @param features specifies features that affect the processor's + * behavior. + *

This argument may be specified as null, + * in which case the processor will use its standard features + * (as returned by the processor + * {@link ISTextProcessor#getFeatures getFeatures} + * method). + * + * @param environment specifies an environment whose characteristics + * may affect the processor's behavior. + *

This argument may be specified as null, + * in which case the processor will assume a standard + * environment as specified in + * {@link STextEnvironment#DEFAULT}. + * + * @param text is the structured text string. + * + * @param state can be used to specify that the text + * argument is the continuation of text submitted in a + * previous call and/or to receive information to pass to + * continuation calls. + * For more details, see State above. + *

If all calls to this method are independent from one another, + * this argument should be specified as null. + * + * @return an array of offsets to the characters in the text + * argument before which directional marks must be + * added to ensure correct presentation. + * The offsets are sorted in ascending order. + */ + public static int[] leanBidiCharOffsets(Object processor, STextFeatures features, STextEnvironment environment, String text, int[] state) { + if (processor == null) + return EMPTY_INT_ARRAY; + return STextImpl.leanBidiCharOffsets(processor, features, environment, text, state); + } + + /** + * Remove directional formatting characters which were added to a + * structured text string to ensure correct presentation. + * + * @param processor designates one of the registered processors. + * It can be a string containing a keyword according to + * the processor type, or a processor reference. + * For more details, see above + * How to Specify a Processor. + *

If this argument is null, this method + * returns the text string. + * + * @param features specifies features that affect the processor's + * behavior. + *

This argument may be specified as null, + * in which case the processor will use its standard features + * (as returned by the processor + * {@link ISTextProcessor#getFeatures getFeatures} + * method). + * + * @param environment specifies an environment whose characteristics + * may affect the processor's behavior. + *

This argument may be specified as null, + * in which case the processor will assume a standard + * environment as specified in + * {@link STextEnvironment#DEFAULT}. + * + * @param text is the structured text string including + * directional formatting characters. + * + * @param state can be used to specify that the text + * argument is the continuation of text submitted in a + * previous call and/or to receive information to pass to + * continuation calls. + * For more details, see State above. + *

If all calls to this method are independent from one another, + * this argument should be specified as null. + * + * @return the structured text string without directional formatting + * characters which might have been added by processing it + * with {@link #leanToFullText leanToFullText}. + * + */ + public static String fullToLeanText(Object processor, STextFeatures features, STextEnvironment environment, String text, int[] state) { + if (processor == null) + return text; + return STextImpl.fullToLeanText(processor, features, environment, text, state); + } + + /** + * Given a full string, compute the positions of each of its + * characters within the corresponding lean string. + * + * @param processor designates one of the registered processors. + * It can be a string containing a keyword according to + * the processor type, or a processor reference. + * For more details, see above + * How to Specify a Processor. + *

If this argument is null, this method + * returns an identity map. + * + * @param features specifies features that affect the processor's + * behavior. + *

This argument may be specified as null, + * in which case the processor will use its standard features + * (as returned by the processor + * {@link ISTextProcessor#getFeatures getFeatures} + * method). + * + * @param environment specifies an environment whose characteristics + * may affect the processor's behavior. + *

This argument may be specified as null, + * in which case the processor will assume a standard + * environment as specified in + * {@link STextEnvironment#DEFAULT}. + * + * @param text is the structured text string including + * directional formatting characters. + * + * @param state can be used to specify that the text + * argument is the continuation of text submitted in a + * previous call and/or to receive information to pass to + * continuation calls. + * For more details, see State above. + *

If all calls to this method are independent from one another, + * this argument should be specified as null. + * + * @return an array of integers with one element for each of the characters + * in the text argument, equal to the offset of the + * corresponding character in the lean string. + * If there is no corresponding + * character in the lean string (because the + * specified character is a directional formatting character + * added when invoking {@link #leanToFullText leanToFullText}), + * the value returned for this character is -1. + */ + public static int[] fullToLeanMap(Object processor, STextFeatures features, STextEnvironment environment, String text, int[] state) { + if (processor == null) { + int[] map = new int[text.length()]; + for (int i = 0; i < map.length; i++) + map[i] = i; + return map; + } + return STextImpl.fullToLeanMap(processor, features, environment, text, state); + } + + /** + * Given a full string, return the offsets of characters + * which are directional formatting characters that have been added + * in order to ensure correct presentation. + * + *

LRMs (for a string with LTR base direction), RLMs (for + * a string with RTL base direction) are considered as well as + * leading and trailing LRE, RLE and PDF which might be prefixed or suffixed + * depending on the {@link STextEnvironment#getOrientation orientation} + * of the GUI component used for display. + * + * @param processor designates one of the registered processors. + * It can be a string containing a keyword according to + * the processor type, or a processor reference. + * For more details, see above + * How to Specify a Processor. + *

If this argument is null, this method + * returns an empty array. + * + * @param features specifies features that affect the processor's + * behavior. + *

This argument may be specified as null, + * in which case the processor will use its standard features + * (as returned by the processor + * {@link ISTextProcessor#getFeatures getFeatures} + * method). + * + * @param environment specifies an environment whose characteristics + * may affect the processor's behavior. + *

This argument may be specified as null, + * in which case the processor will assume a standard + * environment as specified in + * {@link STextEnvironment#DEFAULT}. + * + * @param text is the structured text string including + * directional formatting characters. + * + * @param state can be used to specify that the text + * argument is the continuation of text submitted in a + * previous call and/or to receive information to pass to + * continuation calls. + * For more details, see State above. + *

If all calls to this method are independent from one another, + * this argument should be specified as null. + * + * @return an array of offsets to the characters in the text + * argument which are directional formatting characters + * added to ensure correct presentation. + * The offsets are sorted in ascending order. + */ + public static int[] fullBidiCharOffsets(Object processor, STextFeatures features, STextEnvironment environment, String text, int[] state) { + if (processor == null) + return EMPTY_INT_ARRAY; + return STextImpl.fullBidiCharOffsets(processor, features, environment, text, state); + } + + /** + * Get the base direction of a structured text. + * This base direction may depend on + * whether the text contains Arabic or Hebrew words + * (if it contains both, the first Arabic or Hebrew letter in the + * text determines which is the governing script) and on + * whether the GUI is {@link STextEnvironment#getMirrored mirrored}. + * + * @param processor designates one of the registered processors. + * It can be a string containing a keyword according to + * the processor type, or a processor reference. + * For more details, see above + * How to Specify a Processor. + *

If this argument is null, this method + * returns {@link STextFeatures#DIR_LTR}. + * + * @param features specifies features that affect the processor's + * behavior. + *

This argument may be specified as null, + * in which case the processor will use its standard features + * (as returned by the processor + * {@link ISTextProcessor#getFeatures getFeatures} + * method). + * + * @param environment specifies an environment whose characteristics + * may affect the processor's behavior. + *

This argument may be specified as null, + * in which case the processor will assume a standard + * environment as specified in + * {@link STextEnvironment#DEFAULT}. + * + * @param text is the structured text string. + * + * @return the base direction of the structured text. + * It is one of the values {@link STextFeatures#DIR_LTR} + * or {@link STextFeatures#DIR_RTL}. + */ + public static int getCurDirection(Object processor, STextFeatures features, STextEnvironment environment, String text) { + return STextImpl.getCurDirection(processor, features, environment, text, null); + } + +} Index: src/org/eclipse/equinox/bidi/STextEnvironment.java =================================================================== RCS file: src/org/eclipse/equinox/bidi/STextEnvironment.java diff -N src/org/eclipse/equinox/bidi/STextEnvironment.java --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ src/org/eclipse/equinox/bidi/STextEnvironment.java 1 Jan 1970 00:00:00 -0000 @@ -0,0 +1,326 @@ +/******************************************************************************* + * Copyright (c) 2010, 2011 IBM Corporation and others. + * All rights reserved. This program and the accompanying materials + * are made available under the terms of the Eclipse Public License v1.0 + * which accompanies this distribution, and is available at + * http://www.eclipse.org/legal/epl-v10.html + * + * Contributors: + * IBM Corporation - initial API and implementation + ******************************************************************************/ +package org.eclipse.equinox.bidi; + +import java.util.Locale; +import org.eclipse.equinox.bidi.custom.ISTextProcessor; +import org.eclipse.equinox.bidi.internal.STextActivator; + +/** + * This class defines certain details of the environment within which + * structured text strings are processed. + *

+ * All public fields in this class are final, i.e. cannot be + * changed after creating an instance. + *

+ * All methods in {@link STextEngine} have a STextEnvironment + * argument. If this argument is specified as null, the + * {@link #DEFAULT} environment is used. + * + *

Code Samples

+ *

Example (set all environment parameters) + * 

+ *
+ *    STextEnvironment myEnv = new STextEnvironment("he_IL", true, STextEnvironment.ORIENT_RTL);
+ *
+ *
+ *

+ * This class also provides a number of convenience methods related to the environment. + *

 

+ * @see ISTextProcessor#getFeatures + * + * @author Matitiahu Allouche + */ +public class STextEnvironment { + + /** + * Constant specifying that the orientation of the GUI component + * where a structured text will be displayed is LTR. + * It can appear as orientation argument for the + * {@link STextEnvironment#STextEnvironment STextEnvironment constructor} + * and as value returned by {@link #getOrientation}. + */ + public static final int ORIENT_LTR = 0; + + /** + * Constant specifying that the orientation of the GUI component + * where a structured text will be displayed is RTL. + * It can appear as orientation argument for the + * {@link STextEnvironment#STextEnvironment STextEnvironment constructor} + * and as value returned by {@link #getOrientation}. + */ + public static final int ORIENT_RTL = 1; + + /** + * Constant specifying that the orientation of the GUI component + * where a structured text will be displayed is contextual with + * a default of LTR (if no strong character appears in the text). + * It can appear as orientation argument for the + * {@link STextEnvironment#STextEnvironment STextEnvironment constructor} + * and as value returned by {@link #getOrientation}. + */ + public static final int ORIENT_CONTEXTUAL_LTR = 2; + + /** + * Constant specifying that the orientation of the GUI component + * where a structured text will be displayed is contextual with + * a default of RTL (if no strong character appears in the text). + * It can appear as orientation argument for the + * {@link STextEnvironment#STextEnvironment STextEnvironment constructor} + * and as value returned by {@link #getOrientation}. + */ + public static final int ORIENT_CONTEXTUAL_RTL = 3; + + /** + * Constant specifying that the orientation of the GUI component + * where a structured text will be displayed is not known. + * Directional formatting characters must be added as prefix and + * suffix whenever a full text is generated using + * {@link STextEngine#leanToFullText leanToFullText}. + * It can appear as orientation argument for the + * {@link STextEnvironment#STextEnvironment STextEnvironment constructor} + * and as value returned by {@link #getOrientation}. + */ + public static final int ORIENT_UNKNOWN = 4; + + /** + * Constant specifying that whatever the orientation of the + * GUI component where a structured text will be displayed, no + * directional formatting characters must be added as prefix or + * suffix when a full text is generated using + * {@link STextEngine#leanToFullText leanToFullText}. + * It can appear as orientation argument for the + * {@link STextEnvironment#STextEnvironment STextEnvironment constructor} + * and as value returned by {@link #getOrientation}. + */ + public static final int ORIENT_IGNORE = 5; + + /** + * Pre-defined STextEnvironment instance with values + * for a non-mirrored GUI and a Left-to-Right presentation component.
+ * The language is set to null, which defaults to the language + * of the current default locale. + */ + public static final STextEnvironment DEFAULT = new STextEnvironment(null, false, ORIENT_LTR); + + /** + * This string is a 2-letters code representing a language as defined by + * ISO-639. If left as null, it defaults to the language + * of the current default locale. + */ + final String language; + + /** + * Flag specifying that structured text processed under this environment + * should assume that the GUI is mirrored (globally going from right to left). + */ + final boolean mirrored; + + /** Specify the orientation (a.k.a. base direction) of the GUI + * component in which the full structured text will + * be displayed. + */ + final int orientation; + + static Locale defaultLocale; + static String defaultLanguage; + static boolean defaultBidi; + boolean bidiFlag; + + /** + * Constructor + * + * @param lang represents the language to be used in this environment. + * It should be specified as a 2-letters code as defined by + * ISO-639.
+ * If longer than 2 letters, the extra letters are ignored.
+ * If set to null, it defaults to the language + * of the default locale. + * @see #getLanguage + * + * @param mirrored specifies if the GUI is mirrored. + * @see #getMirrored + * + * @param orientation specifies the orientation of the component + * which is to display the structured text. It must be + * one of the values + * {@link #ORIENT_LTR ORIENT_LTR}, + * {@link #ORIENT_LTR ORIENT_RTL}, + * {@link #ORIENT_CONTEXTUAL_LTR ORIENT_CONTEXTUAL_LTR}, + * {@link #ORIENT_CONTEXTUAL_RTL ORIENT_CONTEXTUAL_RTL}, + * {@link #ORIENT_UNKNOWN ORIENT_UNKNOWN} or + * {@link #ORIENT_IGNORE ORIENT_IGNORE}.
+ * If different, it defaults to {@link #ORIENT_UNKNOWN ORIENT_UNKNOWN}. + * @see #getOrientation + */ + public STextEnvironment(String lang, boolean mirrored, int orientation) { + if (lang == null) { + language = null; + } else { + if (lang.length() > 2) + language = lang.substring(0, 2); + else + language = lang; + bidiFlag = isBidiLanguage(language); + } + this.mirrored = mirrored; + this.orientation = orientation >= ORIENT_LTR && orientation <= ORIENT_IGNORE ? orientation : ORIENT_UNKNOWN; + } + + /** + * Return a 2-letters code representing a language as defined by + * ISO-639. If equal to null, it defaults to the language + * of the current default locale. + */ + public String getLanguage() { + return language; + } + + /** + * Return a flag indicating that structured text processed + * within this environment should assume that the GUI is mirrored + * (globally going from right to left). + */ + public boolean getMirrored() { + return mirrored; + } + + /** Return the orientation (a.k.a. base direction) of the GUI + * component in which the full structured text + * will be displayed.
+ * The orientation must have one of the values + * {@link #ORIENT_LTR ORIENT_LTR}, + * {@link #ORIENT_LTR ORIENT_RTL}, + * {@link #ORIENT_CONTEXTUAL_LTR ORIENT_CONTEXTUAL_LTR}, + * {@link #ORIENT_CONTEXTUAL_RTL ORIENT_CONTEXTUAL_RTL}, + * {@link #ORIENT_UNKNOWN ORIENT_UNKNOWN} or + * {@link #ORIENT_IGNORE ORIENT_IGNORE}. + *

+ * When the orientation is ORIENT_LTR and the + * structured text has a RTL base direction, + * {@link STextEngine#leanToFullText leanToFullText} + * adds RLE+RLM at the head of the full text and RLM+PDF at its + * end. + *

+ * When the orientation is ORIENT_RTL and the + * structured text has a LTR base direction, + * {@link STextEngine#leanToFullText leanToFullText} + * adds LRE+LRM at the head of the full text and LRM+PDF at its + * end. + *

+ * When the orientation is ORIENT_CONTEXTUAL_LTR or + * ORIENT_CONTEXTUAL_RTL and the data content would resolve + * to a RTL orientation while the structured text has a LTR base + * direction, {@link STextEngine#leanToFullText leanToFullText} + * adds LRM at the head of the full text. + *

+ * When the orientation is ORIENT_CONTEXTUAL_LTR or + * ORIENT_CONTEXTUAL_RTL and the data content would resolve + * to a LTR orientation while the structured text has a RTL base + * direction, {@link STextEngine#leanToFullText leanToFullText} + * adds RLM at the head of the full text. + *

+ * When the orientation is ORIENT_UNKNOWN and the + * structured text has a LTR base direction, + * {@link STextEngine#leanToFullText leanToFullText} + * adds LRE+LRM at the head of the full text and LRM+PDF at its + * end. + *

+ * When the orientation is ORIENT_UNKNOWN and the + * structured text has a RTL base direction, + * {@link STextEngine#leanToFullText leanToFullText} + * adds RLE+RLM at the head of the full text and RLM+PDF at its + * end. + *

+ * When the orientation is ORIENT_IGNORE, + * {@link STextEngine#leanToFullText leanToFullText} does not add any directional + * formatting characters as either prefix or suffix of the full text. + *

+ */ + public int getOrientation() { + return orientation; + } + + /** + * Check whether the current language uses a + * bidi script (Arabic, Hebrew, Farsi or Urdu). + * + * @return true if the current language uses a bidi script. + * The language may have been set explicitly when creating the + * STextEnvironment instance, or it may have + * defaulted to the language of the current default locale. + * @see #getLanguage + */ + public boolean isBidi() { + if (defaultLanguage != null && defaultLocale.equals(getDefaultLocale())) + return defaultBidi; + + if (language == null) { + defaultLocale = getDefaultLocale(); + defaultLanguage = defaultLocale.getLanguage(); + defaultBidi = isBidiLanguage(defaultLanguage); + return defaultBidi; + } + + return bidiFlag; + } + + static boolean isBidiLanguage(String lang) { + return "iw".equals(lang) || "he".equals(lang) || "ar".equals(lang) || "fa".equals(lang) || "ur".equals(lang); //$NON-NLS-1$ //$NON-NLS-2$ //$NON-NLS-3$ //$NON-NLS-4$ //$NON-NLS-5$ + } + + static String lineSep; + + /** + * Retrieve the string which represents a line separator in this environment. + * + * @return the string which is used as line separator (e.g. CRLF). + */ + public static String getLineSep() { + // use bundle properties + if (lineSep == null) { + // lineSep = System.getProperty("line.separator", "\n"); //$NON-NLS-1$//$NON-NLS-2$ + lineSep = getProperty("line.separator"); //$NON-NLS-1$/ + } + return lineSep; + } + + static String osName; + static boolean flagOS; + + private static String getProperty(String key) { + // use bundle properties + // osName = System.getProperty("os.name").toLowerCase(); //$NON-NLS-1$ + STextActivator sTextActivator = STextActivator.getInstance(); + return sTextActivator.getProperty(key); + } + + private Locale getDefaultLocale() { + STextActivator sTextActivator = STextActivator.getInstance(); + return sTextActivator.getDefaultLocale(); + } + + /** + * Check if the current OS is supported by the structured text packages. + * + * @return true if the current OS is supported. + */ + public static boolean isSupportedOS() { + if (osName == null) { + // use bundle properties + // osName = System.getProperty("os.name").toLowerCase(); //$NON-NLS-1$ + osName = getProperty("os.name").toLowerCase(); //$NON-NLS-1$/ + flagOS = osName.startsWith("windows") || osName.startsWith("linux"); //$NON-NLS-1$ //$NON-NLS-2$ + } + return flagOS; + } + +} Index: src/org/eclipse/equinox/bidi/STextStringRecord.java =================================================================== RCS file: src/org/eclipse/equinox/bidi/STextStringRecord.java diff -N src/org/eclipse/equinox/bidi/STextStringRecord.java --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ src/org/eclipse/equinox/bidi/STextStringRecord.java 1 Jan 1970 00:00:00 -0000 @@ -0,0 +1,192 @@ +/******************************************************************************* + * Copyright (c) 2010, 2011 IBM Corporation and others. + * All rights reserved. This program and the accompanying materials + * are made available under the terms of the Eclipse Public License v1.0 + * which accompanies this distribution, and is available at + * http://www.eclipse.org/legal/epl-v10.html + * + * Contributors: + * IBM Corporation - initial API and implementation + ******************************************************************************/ +package org.eclipse.equinox.bidi; + +import java.lang.ref.SoftReference; +import org.eclipse.equinox.bidi.custom.STextStringProcessor; + +/** + * This class records strings which are structured text. Several static + * methods in this class allow to record such strings in a pool, and to find if + * a given string is member of the pool. + *

+ * Instances of this class are the records which are members of the pool. + *

+ * The pool is managed as a cyclic list. When the pool is full, + * each new element overrides the oldest element in the list. + */ +public class STextStringRecord { + /** + * Number of entries in the pool of recorded strings + */ + public static final int POOLSIZE = 100; + + // maximum index allowed + private static final int MAXINDEX = POOLSIZE - 1; + + // index of the last entered record + private static int last = MAXINDEX; + + // the pool + private static STextStringRecord[] records = new STextStringRecord[POOLSIZE]; + + // structured text types + private static final String[] types = STextStringProcessor.getKnownTypes(); + + // maximum type index allowed + private static int MAXTYPE = types.length - 1; + + // reference to the recorded string + private SoftReference strRef; + + // hash code of the recorded string + private int hash; + + // reference to the triplets of the recorded string + private SoftReference triRef; + + /** + * Constructor. + * + * @param string the string to record + * + * @param triplets + * array of short integers, the number of elements in the array + * must be a multiple of 3, so that the array is made of one or + * more triplets of short integers. + *

+ * The first element in each triplet is the beginning offset of a + * susbstring of string which is a structured text. + *

+ * The second element in each triplet is the ending offset of a + * susbstring of string which is a structured text. + * This offset points to one position beyond the last + * character of the substring. + *

+ * The third element in each triplet is the numeric type of the + * structured text.
+ * The type of a structured text must be one of the string + * values listed in {@link ISTextTypes}.
+ * The corresponding numeric type must be obtained using the + * method {@link #typeStringToShort typeStringToShort}. + */ + public STextStringRecord(String string, short[] triplets) { + if (string == null || triplets == null) + throw new IllegalArgumentException("The string and triplets argument must not be null!"); //$NON-NLS-1$ + if ((triplets.length % 3) != 0) + throw new IllegalArgumentException("The number of elements in triplets must be a multiple of 3!"); //$NON-NLS-1$ + for (int i = 2; i < triplets.length; i += 3) + if (triplets[i] < 0 || triplets[i] > MAXTYPE) + throw new IllegalArgumentException("Illegal type value in element" + i); //$NON-NLS-1$ + strRef = new SoftReference(string); + triRef = new SoftReference(triplets); + hash = string.hashCode(); + } + + /** + * Get the numeric type of a structured text given its string type. + * + * @param type type of structured text as string. It must be one + * of the strings listed in {@link ISTextTypes}. + * + * @return a value which is the corresponding numeric type. If + * type is invalid, the method returns -1. + */ + public static short typeStringToShort(String type) { + for (int i = 0; i < types.length; i++) + if (types[i].equals(type)) + return (short) i; + return -1; + } + + /** + * Get the string type of a structured text given its numeric type. + * + * @param shType + * the numeric type of a structured text. It should be a value + * obtained using {@link #typeStringToShort typeStringToShort}. + * + * @return the corresponding string type. If shType is invalid, + * the method returns null. + */ + public static String typeShortToString(short shType) { + if (shType < 0 || shType > MAXTYPE) + return null; + return types[shType]; + } + + /** + * Add a record to the pool. + * + * @param record a STextStringRecord instance + */ + public static synchronized void add(STextStringRecord record) { + if (last < MAXINDEX) + last++; + else + last = 0; + records[last] = record; + } + + /** + * Check if a string is recorded and retrieve its triplets. + * + * @param string the string to check + * + * @return null if the string is not recorded in the pool; + * otherwise, return the triplets associated with this string. + */ + public static short[] getTriplets(String string) { + if (records[0] == null) // no records at all + return null; + if (string == null || string.length() < 1) + return null; + STextStringRecord rec; + String str; + short[] tri; + int myLast = last; + int hash = string.hashCode(); + for (int i = myLast; i >= 0; i--) { + rec = records[i]; + if (hash == rec.hash && (tri = (short[]) rec.triRef.get()) != null && (str = (String) rec.strRef.get()) != null && string.equals(str)) { + return tri; + } + } + if (records[MAXINDEX] == null) // never recorded past myLast + return null; + for (int i = MAXINDEX; i > myLast; i--) { + rec = records[i]; + if (hash == rec.hash && (tri = (short[]) rec.triRef.get()) != null && (str = (String) rec.strRef.get()) != null && string.equals(str)) { + return tri; + } + } + return null; + } + + /** + * Clear the pool. All elements of the pool are erased and any associated + * memory is freed. + * + */ + public static synchronized void clear() { + for (int i = 0; i <= MAXINDEX; i++) { + STextStringRecord sr = records[i]; + if (sr == null) + break; + sr.hash = 0; + sr.strRef.clear(); + sr.triRef.clear(); + records[i] = null; + } + last = MAXINDEX; + } + +} Index: src/org/eclipse/equinox/bidi/STextUtil.java =================================================================== RCS file: src/org/eclipse/equinox/bidi/STextUtil.java diff -N src/org/eclipse/equinox/bidi/STextUtil.java --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ src/org/eclipse/equinox/bidi/STextUtil.java 1 Jan 1970 00:00:00 -0000 @@ -0,0 +1,313 @@ +/******************************************************************************* + * Copyright (c) 2010, 2011 IBM Corporation and others. + * All rights reserved. This program and the accompanying materials + * are made available under the terms of the Eclipse Public License v1.0 + * which accompanies this distribution, and is available at + * http://www.eclipse.org/legal/epl-v10.html + * + * Contributors: + * IBM Corporation - initial API and implementation + ******************************************************************************/ +package org.eclipse.equinox.bidi; + +import org.eclipse.equinox.bidi.custom.STextFeatures; +import org.eclipse.equinox.bidi.custom.STextProcessor; + +/** + * This class provides a number of convenience functions facilitating the + * processing of structured text. + * + * @noextend This class is not intended to be subclassed by clients. + * @noinstantiate This class is not intended to be instantiated by clients. + * + * @author Matitiahu Allouche + */ +final public class STextUtil { + + /** + * prevent instantiation + */ + private STextUtil() { + // empty + } + + /** This is a convenience method which can add directional marks in a given + * text before the characters specified in the given array of offsets, + * and can add a prefix and/or a suffix of directional formatting characters. + * This can be used for instance after obtaining offsets by calling + * {@link STextEngine#leanBidiCharOffsets leanBidiCharOffsets} in order to + * produce a full text corresponding to the source text. + * The directional formatting characters that will be added at the given + * offsets will be LRMs for structured text strings with LTR base direction + * and RLMs for strings with RTL base direction. Leading and + * trailing LRE, RLE and PDF which might be needed as prefix or suffix + * depending on the orientation of the GUI component used for display + * may be added depending on argument affix. + * + * @param text is the structured text string + * + * @param offsets is an array of offsets to characters in text + * before which an LRM or RLM will be inserted. + * Members of the array must be non-negative numbers smaller + * than the length of text. + * The array must be sorted in ascending order without duplicates. + * This argument may be null if there are no marks to add. + * + * @param direction specifies the base direction of the structured text. + * It must be one of the values {@link STextFeatures#DIR_LTR} or + * {@link STextFeatures#DIR_RTL}. + * + * @param affix specifies if a prefix and a suffix should be added to + * the result to make sure that the direction + * specified as third argument is honored even if the string + * is displayed in a GUI component with a different orientation. + * + * @return a string corresponding to the source text with + * directional marks (LRMs or RLMs) added at the specified offsets, + * and directional formatting characters (LRE, RLE, PDF) added + * as prefix and suffix if so required. + */ + public static String insertMarks(String text, int[] offsets, int direction, boolean affix) { + int textLen = text.length(); + if (textLen == 0) + return ""; //$NON-NLS-1$ + + String curPrefix, curSuffix, full; + char curMark, c; + char[] fullChars; + if (direction == STextFeatures.DIR_LTR) { + curMark = LRM; + curPrefix = "\u202a\u200e"; /* LRE+LRM *///$NON-NLS-1$ + curSuffix = "\u200e\u202c"; /* LRM+PDF *///$NON-NLS-1$ + } else { + curMark = RLM; + curPrefix = "\u202b\u200f"; /* RLE+RLM *///$NON-NLS-1$ + curSuffix = "\u200f\u202c"; /* RLM+PDF *///$NON-NLS-1$ + } + // add marks at offsets + if ((offsets != null) && (offsets.length > 0)) { + int offLen = offsets.length; + fullChars = new char[textLen + offLen]; + int added = 0; + for (int i = 0, j = 0; i < textLen; i++) { + c = text.charAt(i); + if ((j < offLen) && (i == offsets[j])) { + fullChars[i + added] = curMark; + added++; + j++; + } + fullChars[i + added] = c; + } + full = new String(fullChars); + } else { + full = text; + } + if (affix) + return curPrefix + full + curSuffix; + return full; + } + + /*************************************************************************/ + /* */ + /* The following code is provided for compatibility with TextProcessor */ + /* */ + /*************************************************************************/ + + // The default set of separators to use to segment a string. + private static final String defaultSeparators = ".:/\\"; //$NON-NLS-1$ + // left to right mark + private static final char LRM = '\u200e'; + // left to right mark + private static final char RLM = '\u200f'; + // left to right embedding + private static final char LRE = '\u202a'; + // right to left embedding + private static final char RLE = '\u202b'; + // pop directional format + private static final char PDF = '\u202c'; + + static boolean isProcessingNeeded() { + if (!STextEnvironment.isSupportedOS()) + return false; + return STextEnvironment.DEFAULT.isBidi(); + } + + /** + * Process the given text and return a string with appropriate + * directional formatting characters if the locale is a bidi locale. + * This is equivalent to calling + * {@link #process(String str, String separators)} with the default + * set of separators (dot, colon, slash, backslash). + * + * @param str the text to be processed. + * + * @return the processed string. + */ + public static String process(String str) { + return process(str, defaultSeparators); + } + + /** + * Process a string that has a particular semantic meaning to render + * it correctly on bidi locales. This is done by adding directional + * formatting characters so that presentation using the Unicode + * Bidirectional Algorithm will provide the expected result. + * The text is segmented according to the provided separators. + * Each segment has the Unicode Bidi Algorithm applied to it, + * but as a whole, the string is oriented left to right. + *

+ * For example, a file path such as d:\myfolder\FOLDER\MYFILE.java + * (where capital letters indicate RTL text) should render as + * d:\myfolder\REDLOF\ELIFYM.java.

+ *

+ * NOTE: this method inserts directional formatting characters into the + * text. Methods like String.equals(String) and + * String.length() called on the resulting string will not + * return the same values as would be returned for the original string.

+ * + * @param str the text to process. + * + * @param separators separators by which the string will be segmented. + * If null, the default separators are used + * (dot, colon, slash, backslash). + * + * @return the processed string. + * If str is null, + * or of length 0, or if the current locale is not a bidi one, + * return the original string. + */ + public static String process(String str, String separators) { + if ((str == null) || (str.length() <= 1) || !isProcessingNeeded()) + return str; + + // do not process a string that has already been processed. + if (str.charAt(0) == LRE && str.charAt(str.length() - 1) == PDF) + return str; + + // do not process a string if all the following conditions are true: + // a) it has no RTL characters + // b) it starts with a LTR character + // c) it ends with a LTR character or a digit + boolean isStringBidi = false; + int strLength = str.length(); + char c; + for (int i = 0; i < strLength; i++) { + c = str.charAt(i); + if (((c >= 0x05d0) && (c <= 0x07b1)) || ((c >= 0xfb1d) && (c <= 0xfefc))) { + isStringBidi = true; + break; + } + } + while (!isStringBidi) { + if (!Character.isLetter(str.charAt(0))) + break; + c = str.charAt(strLength - 1); + if (!Character.isDigit(c) && !Character.isLetter(c)) + break; + return str; + } + + if (separators == null) + separators = defaultSeparators; + + // make sure that LRE/PDF are added around the string + STextEnvironment env = new STextEnvironment(null, false, STextEnvironment.ORIENT_UNKNOWN); + STextFeatures features = new STextFeatures(separators, 0, -1, -1, false, false); + return STextEngine.leanToFullText(new STextProcessor(), features, env, str, null); + } + + /** + * Process a string that has a particular semantic meaning to render + * it correctly on bidi locales. This is done by adding directional + * formatting characters so that presentation using the Unicode + * Bidirectional Algorithm will provide the expected result.. + * The text is segmented according to the syntax specified in the + * type argument. + * Each segment has the Unicode Bidi Algorithm applied to it, but the + * order of the segments is governed by the type of the structured text. + *

+ * For example, a file path such as d:\myfolder\FOLDER\MYFILE.java + * (where capital letters indicate RTL text) should render as + * d:\myfolder\REDLOF\ELIFYM.java.

+ *

+ * NOTE: this method inserts directional formatting characters into the + * text. Methods like String.equals(String) and + * String.length() called on the resulting string will not + * return the same values as would be returned for the original string.

+ * + * @param str the text to process. + * + * @param type specifies the type of the structured text. It must + * be one of the values in {@link ISTextTypes} or a value. + * added by a plug-in extension. + * + * @return the processed string. + * If str is null, + * or of length 0, or if the current locale is not a bidi one, + * return the original string. + */ + public static String processTyped(String str, String type) { + if ((str == null) || (str.length() <= 1) || !isProcessingNeeded()) + return str; + + // do not process a string that has already been processed. + char c = str.charAt(0); + if (((c == LRE) || (c == RLE)) && str.charAt(str.length() - 1) == PDF) + return str; + + // make sure that LRE/PDF are added around the string + STextEnvironment env = new STextEnvironment(null, false, STextEnvironment.ORIENT_UNKNOWN); + return STextEngine.leanToFullText(type, null, env, str, null); + } + + /** + * Remove directional formatting characters in the given string that + * were inserted by one of the {@link #process process} methods. + * + * @param str string with directional characters to remove. + * + * @return string with no directional formatting characters. + */ + public static String deprocess(String str) { + if ((str == null) || (str.length() <= 1) || !isProcessingNeeded()) + return str; + + StringBuffer buf = new StringBuffer(); + int strLen = str.length(); + for (int i = 0; i < strLen; i++) { + char c = str.charAt(i); + switch (c) { + case LRM : + continue; + case LRE : + continue; + case PDF : + continue; + default : + buf.append(c); + } + } + return buf.toString(); + } + + /** + * Remove directional formatting characters in the given string that + * were inserted by the {@link #processTyped processTyped} method. + * + * @param str string with directional characters to remove. + * + * @param type type of the structured text as specified when + * calling {@link #processTyped processTyped}. + * + * @return string with no directional formatting characters. + */ + public static String deprocess(String str, String type) { + if ((str == null) || (str.length() <= 1) || !isProcessingNeeded()) + return str; + + // make sure that LRE/PDF are added around the string + STextEnvironment env = new STextEnvironment(null, false, STextEnvironment.ORIENT_UNKNOWN); + return STextEngine.fullToLeanText(type, null, env, str, null); + } + +} Index: src/org/eclipse/equinox/bidi/custom/BidiComplexFeatures.java =================================================================== RCS file: src/org/eclipse/equinox/bidi/custom/BidiComplexFeatures.java diff -N src/org/eclipse/equinox/bidi/custom/BidiComplexFeatures.java --- src/org/eclipse/equinox/bidi/custom/BidiComplexFeatures.java 17 May 2011 19:56:16 -0000 1.2 +++ /dev/null 1 Jan 1970 00:00:00 -0000 @@ -1,250 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2010, 2011 IBM Corporation and others. - * All rights reserved. This program and the accompanying materials - * are made available under the terms of the Eclipse Public License v1.0 - * which accompanies this distribution, and is available at - * http://www.eclipse.org/legal/epl-v10.html - * - * Contributors: - * IBM Corporation - initial API and implementation - ******************************************************************************/ -package org.eclipse.equinox.bidi.custom; - -import org.eclipse.equinox.bidi.BidiComplexEnvironment; - -/** - * This class defines features of a complex expression processor. - *

- * All public fields in this class are final, i.e. cannot be - * changed after creating an instance. - * - *

Code Sample

- *

Example 1 (set all features) - * 

- *
- *    BidiComplexFeatures f1 = new BidiComplexFeatures("+-=", 0, -1, -1, false, false);
- *
- *
- *

Example 2 (change only the separators) - * 

- *
- *    BidiComplexFeatures f2 = new BidiComplexFeatures("[]|()", f1.getSpecialsCount(),
- *                                     f1.getDirArabic(), f1.getDirHebrew(),
- *                                     f1.getIgnoreArabic(), f1.getIgnoreHebrew());
- *
- *
- * - * @see IBidiComplexProcessor#getFeatures - * - * @author Matitiahu Allouche - */ -public class BidiComplexFeatures { - - /** - * Constant specifying that the base direction of a complex expression is LTR. - * The base direction may depend on whether the GUI is - * {@link BidiComplexEnvironment#getMirrored mirrored} and may - * may be different for Arabic and for Hebrew. - * This constant can appear as dirArabic - * or dirHebrew argument for the - * {@link BidiComplexFeatures#BidiComplexFeatures BidiComplexFeatures constructor} - * and as value returned by {@link #getDirArabic} or {@link #getDirHebrew} - * methods. - */ - public static final int DIR_LTR = 0; - - /** - * Constant specifying that the base direction of a complex expression is RTL. - * The base direction may depend on whether the GUI is - * {@link BidiComplexEnvironment#getMirrored mirrored} and may - * may be different for Arabic and for Hebrew. - * This constant can appear as dirArabic - * or dirHebrew argument for the - * {@link BidiComplexFeatures#BidiComplexFeatures BidiComplexFeatures constructor} - * and as value returned by {@link #getDirArabic} or {@link #getDirHebrew} - * methods. - */ - public static final int DIR_RTL = 1; - - /** - * Pre-defined BidiComplexFeatures instance with values for no - * separators, no special processing, all directions LTR - * and support for neither Arabic nor Hebrew.
- * Since there are no separators and no special processing, a complex - * expression processor with such features would do nothing.
- * It is more efficient to do nothing with a null processor. - */ - public static final BidiComplexFeatures DEFAULT = new BidiComplexFeatures(null, 0, -1, -1, true, true); - - /** - * String grouping one-character separators which - * separate the text of the complex expression into tokens. - */ - final String separators; - - /** - * Number of special cases for the associated processor. - * Special cases exist for some types of complex expression processors. - * They are implemented by overriding methods - * {@link IBidiComplexProcessor#indexOfSpecial indexOfSpecial} and - * {@link IBidiComplexProcessor#processSpecial processSpecial}. - * Examples of special cases are comments, literals, or anything which - * is not identified by a one-character separator. - */ - final int specialsCount; - - /** - * Base direction of the complex expression for Arabic. - * If a complex expression contains both Arabic and - * Hebrew words, the first Arabic or Hebrew letter in the - * expression determines which is the governing script).
- * The value of this field must be one of - * {@link #DIR_LTR} or {@link #DIR_RTL}. - * - * @see #dirHebrew - */ - final int dirArabic; - - /** - * Base direction of the complex expression for Hebrew. - * If a complex expression contains both Arabic and - * Hebrew words, the first Arabic or Hebrew letter in the - * expression determines which is the governing script).
- * The value of this field must be one of - * {@link #DIR_LTR} or {@link #DIR_RTL}. - * - * @see #dirArabic - */ - final int dirHebrew; - - /** - * Flag indicating that Arabic letters will not be considered for - * processing complex expressions. If both this flag and - * ignoreHebrew are set to true, the - * processor will do nothing (but some overhead can be expected). - */ - final boolean ignoreArabic; - - /** - * Flag indicating that Hebrew letters will not be considered for - * processing complex expressions. If both this flag and - * ignoreArabic are set to true, the - * processor will do nothing (but some overhead can be expected). - */ - final boolean ignoreHebrew; - - /** - * Constructor - * - * @param separators is a string where each character is a separator - * which separates the complex expression into tokens. - * @see #getSeparators - * - * @param specialsCount specifies the number of special cases handled - * by the processor. This value must be identical to the - * number of special cases handled by the processor with which - * this BidiComplexFeatures instance is associated. - * @see #getSpecialsCount - * - * @param dirArabic specifies the base direction of the complex expression - * for Arabic. It must be {@link #DIR_LTR} or {@link #DIR_RTL}. - * If it is not (for instance if it is a negative value), it - * defaults to DIR_LTR. - * @see #getDirArabic - * - * @param dirHebrew specifies the base direction of the complex expression - * for Hebrew. It must be {@link #DIR_LTR} or {@link #DIR_RTL}. - * If it is not (for instance if it is a negative value), it - * defaults to DIR_LTR. - * @see #getDirHebrew - * - * @param ignoreArabic indicates that Arabic letters will not be - * considered for processing complex expressions. - * If both this flag and ignoreHebrew - * are set to true, the processor will do - * nothing (but some overhead can be expected). - * @see #getIgnoreArabic - * - * @param ignoreHebrew indicates that Hebrew letters will not be - * considered for processing complex expressions. - * If both this flag and ignoreArabic - * are set to true, the processor will do - * nothing (but some overhead can be expected). - * @see #getIgnoreHebrew - */ - public BidiComplexFeatures(String separators, int specialsCount, int dirArabic, int dirHebrew, boolean ignoreArabic, boolean ignoreHebrew) { - - this.separators = separators == null ? "" : separators; //$NON-NLS-1$ - this.specialsCount = specialsCount; - this.dirArabic = dirArabic == DIR_LTR || dirArabic == DIR_RTL ? dirArabic : DIR_LTR; - this.dirHebrew = dirHebrew == DIR_LTR || dirHebrew == DIR_RTL ? dirHebrew : DIR_LTR; - this.ignoreArabic = ignoreArabic; - this.ignoreHebrew = ignoreHebrew; - } - - /** - * @return a string grouping one-character separators which separate - * the text of the complex expression into tokens. - */ - public String getSeparators() { - return separators; - } - - /** - * @return the number of special cases for the associated processor. - * Special cases exist for some types of complex expression - * processors. They are implemented by overriding methods - * {@link IBidiComplexProcessor#indexOfSpecial indexOfSpecial} and - * {@link IBidiComplexProcessor#processSpecial processSpecial}. - * Examples of special cases are comments, literals, or - * anything which is not identified by a one-character separator. - */ - public int getSpecialsCount() { - return specialsCount; - } - - /** - * @return the base direction of the complex expression for Arabic. - * If a complex expression contains both Arabic and - * Hebrew words, the first Arabic or Hebrew letter in the - * expression determines which is the governing script.
- * The value of this field is one of - * {@link #DIR_LTR} or {@link #DIR_RTL}. - * - * @see #getDirHebrew - */ - public int getDirArabic() { - return dirArabic; - } - - /** - * @return the base direction of the complex expression for Hebrew. - * If a complex expression contains both Arabic and - * Hebrew words, the first Arabic or Hebrew letter in the - * expression determines which is the governing script.
- * The value of this field is one of - * {@link #DIR_LTR} or {@link #DIR_RTL}. - * - * @see #getDirArabic - */ - public int getDirHebrew() { - return dirHebrew; - } - - /** - * @return a flag indicating that Arabic letters will not be considered - * for processing complex expressions. - */ - public boolean getIgnoreArabic() { - return ignoreArabic; - } - - /** - * Flag indicating that Hebrew letters will not be considered for - * processing complex expressions. - */ - public boolean getIgnoreHebrew() { - return ignoreHebrew; - } - -} Index: src/org/eclipse/equinox/bidi/custom/BidiComplexProcessor.java =================================================================== RCS file: src/org/eclipse/equinox/bidi/custom/BidiComplexProcessor.java diff -N src/org/eclipse/equinox/bidi/custom/BidiComplexProcessor.java --- src/org/eclipse/equinox/bidi/custom/BidiComplexProcessor.java 17 May 2011 19:56:16 -0000 1.3 +++ /dev/null 1 Jan 1970 00:00:00 -0000 @@ -1,235 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2010, 2011 IBM Corporation and others. - * All rights reserved. This program and the accompanying materials - * are made available under the terms of the Eclipse Public License v1.0 - * which accompanies this distribution, and is available at - * http://www.eclipse.org/legal/epl-v10.html - * - * Contributors: - * IBM Corporation - initial API and implementation - ******************************************************************************/ -package org.eclipse.equinox.bidi.custom; - -import org.eclipse.equinox.bidi.BidiComplexEngine; -import org.eclipse.equinox.bidi.BidiComplexEnvironment; -import org.eclipse.equinox.bidi.internal.BidiComplexImpl; - -/** - * Generic processor which can be used as superclass (base class) - * for specific complex expression processors. - *

- * Here are some guidelines about how to write complex expression - * processors. - *

    - *
  • Processor instances may be accessed simultaneously by - * several threads. They should have no instance variables.
    • - *
  • Each use of a processor is associated with a set of - * {@link BidiComplexFeatures features}. - * All processors must have a default set of features which may be - * queried with the {@link #getFeatures getFeatures} method. - * These default features may be overridden by specifying a - * features argument when calling a method. - * See for instance - * {@link BidiComplexEngine#leanToFullText leanToFullText}. - *
  • The behavior of a processor is governed by 3 factors, all included - * in associated {@link BidiComplexFeatures features} data. - *
      - *
    • The separators specified for its - * {@link BidiComplexFeatures features} determines how submitted - * complex expressions are split into tokens.
      • - *
    • The tokens are displayed one after the other according - * to the appropriate direction, which can be different for - * Arabic and for Hebrew.
      • - *
    • The number of special cases which need to be handled by - * code specific to that processor.
      • - *
    • - *
- * - * @see BidiComplexFeatures#getSeparators - * @see BidiComplexFeatures#getDirArabic - * @see BidiComplexFeatures#getDirHebrew - * @see BidiComplexFeatures#getSpecialsCount - * - * @author Matitiahu Allouche - */ -public class BidiComplexProcessor implements IBidiComplexProcessor { - - /** - * In BidiComplexProcessor this method returns a - * {@link BidiComplexFeatures#DEFAULT DEFAULT} value which - * directs the processor to do nothing. - * - *

A processor which extends this class must override this method - * and return a BidiComplexFeatures instance representing its specific - * features. - * - * @see IBidiComplexProcessor#getFeatures the corresponding interface method - */ - public BidiComplexFeatures getFeatures(BidiComplexEnvironment environment) { - throw new IllegalStateException("A processor must have a getFeatures() method."); //$NON-NLS-1$ - } - - /** - * In BidiComplexProcessor this method throws an - * IllegalStateException. This is appropriate behavior - * (and does not need to be overridden) for processors whose - * number of special cases is zero, which means that - * indexOfSpecial should never be called for them. - * - *

A processor handling special cases must override this method. - * - * @see IBidiComplexProcessor#indexOfSpecial the corresponding interface method - */ - public int indexOfSpecial(BidiComplexFeatures features, String text, byte[] dirProps, int[] offsets, int caseNumber, int fromIndex) { - // This method must be overridden by all subclasses with special cases. - throw new IllegalStateException("A processor must have an indexOfSpecial() method."); //$NON-NLS-1$ - } - - /** - * In BidiComplexProcessor this method throws an - * IllegalStateException. This is appropriate behavior - * (and does not need to be overridden) for processors whose - * number of special cases is zero, which means that - * processSpecial should never be called for them. - * - *

A processor handling special cases must override this method. - * - * @see IBidiComplexProcessor#processSpecial the corresponding interface method - */ - public int processSpecial(BidiComplexFeatures features, String text, byte[] dirProps, int[] offsets, int[] state, int caseNumber, int separLocation) { - // This method must be overridden by all subclasses with any special case. - throw new IllegalStateException("A processor must have a processSpecial() method."); //$NON-NLS-1$ - } - - /** - * This method can be called from within - * {@link IBidiComplexProcessor#indexOfSpecial indexOfSpecial} or - * {@link IBidiComplexProcessor#processSpecial processSpecial} in - * implementations of {@link IBidiComplexProcessor} to retrieve the - * bidirectional class of characters in the lean text. - * - * @param text is the text of the complex expression received as - * parameter to indexOfSpecial or - * processSpecial. - * - * @param dirProps is a parameter received by indexOfSpecial - * or processSpecial, uniquely to be used as argument - * for calls to getDirProp and other methods used - * by processors. - * - * @param index position of the character in the lean text. - * It must be a non-negative number smaller than the length - * of the lean text. - * - * @return the bidirectional class of the character. It is one of the - * values which can be returned by - * java.lang.Character.getDirectionality. - * However, it is recommended to use getDirProp - * rather than java.lang.Character.getDirectionality - * since getDirProp manages a cache of character - * properties and so can be more efficient than calling the - * java.lang.Character method. - */ - public static byte getDirProp(String text, byte[] dirProps, int index) { - return BidiComplexImpl.getDirProp(text, dirProps, index); - } - - /** - * This method can be called from within - * {@link IBidiComplexProcessor#indexOfSpecial indexOfSpecial} or - * {@link IBidiComplexProcessor#processSpecial processSpecial} in - * implementations of {@link IBidiComplexProcessor} to set or - * override the bidirectional class of characters in the lean text. - * - * @param dirProps is a parameter received by indexOfSpecial - * or processSpecial, uniquely to be used as argument - * for calls to setDirProp and other methods used - * by processors. - * - * @param index position of the character in the lean text. - * It must be a non-negative number smaller than the length - * of the lean text. - * - * @param dirProp bidirectional class of the character. It is one of the - * values which can be returned by - * java.lang.Character.getDirectionality. - */ - public static void setDirProp(byte[] dirProps, int index, byte dirProp) { - BidiComplexImpl.setDirProp(dirProps, index, dirProp); - } - - /** - * This method can be called from within - * {@link IBidiComplexProcessor#indexOfSpecial indexOfSpecial} or - * {@link IBidiComplexProcessor#processSpecial processSpecial} in - * implementations of {@link IBidiComplexProcessor} - * to specify that a mark character must be added before the character - * at the specified position of the lean text when generating the - * full text. The mark character will be LRM for complex expressions - * with a LTR base direction, and RLM for complex expressions with RTL - * base direction. The mark character is not added physically by this - * method, but its position is noted and will be used when generating - * the full text. - * - * @param text is the text of the complex expression received as - * parameter to indexOfSpecial or - * processSpecial. - * - * @param dirProps is a parameter received by indexOfSpecial - * or processSpecial, uniquely to be used as argument - * for calls to insertMark and other methods used - * by processors. - * - * @param offsets is a parameter received by indexOfSpecial - * or processSpecial, uniquely to be used as argument - * for calls to insertMark and other methods used - * by processors. - * - * @param offset position of the character in the lean text. - * It must be a non-negative number smaller than the length - * of the lean text. - * For the benefit of efficiency, it is better to insert - * multiple marks in ascending order of the offsets. - */ - public static void insertMark(String text, byte[] dirProps, int[] offsets, int offset) { - BidiComplexImpl.insertMark(text, dirProps, offsets, offset); - } - - /** - * This method can be called from within - * {@link IBidiComplexProcessor#indexOfSpecial indexOfSpecial} or - * {@link IBidiComplexProcessor#processSpecial processSpecial} in - * implementations of {@link IBidiComplexProcessor} to add a - * directional mark before a - * separator if needed for correct display, depending on the - * base direction of the expression and on the class of the - * characters in the lean text preceding and following - * the separator itself. - * - * @param features is the {@link BidiComplexFeatures} instance - * received as parameter to indexOfSpecial or - * processSpecial. - * - * @param text is the text of the complex expression received as - * parameter to indexOfSpecial or - * processSpecial. - * - * @param dirProps is a parameter received by indexOfSpecial - * or processSpecial, uniquely to be used as argument - * for calls to processSeparator and other methods used - * by processors. - * - * @param offsets is a parameter received by indexOfSpecial - * or processSpecial, uniquely to be used as argument - * for calls to processSeparator and other methods used - * by processors. - * - * @param separLocation offset of the separator in the lean text. - * It must be a non-negative number smaller than the length - * of the lean text. - */ - public static void processSeparator(BidiComplexFeatures features, String text, byte[] dirProps, int[] offsets, int separLocation) { - BidiComplexImpl.processSeparator(features, text, dirProps, offsets, separLocation); - } - -} Index: src/org/eclipse/equinox/bidi/custom/BidiComplexStringProcessor.java =================================================================== RCS file: src/org/eclipse/equinox/bidi/custom/BidiComplexStringProcessor.java diff -N src/org/eclipse/equinox/bidi/custom/BidiComplexStringProcessor.java --- src/org/eclipse/equinox/bidi/custom/BidiComplexStringProcessor.java 9 Feb 2011 21:28:55 -0000 1.1 +++ /dev/null 1 Jan 1970 00:00:00 -0000 @@ -1,40 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2010, 2011 IBM Corporation and others. - * All rights reserved. This program and the accompanying materials - * are made available under the terms of the Eclipse Public License v1.0 - * which accompanies this distribution, and is available at - * http://www.eclipse.org/legal/epl-v10.html - * - * Contributors: - * IBM Corporation - initial API and implementation - ******************************************************************************/ -package org.eclipse.equinox.bidi.custom; - -import org.eclipse.equinox.bidi.internal.BidiComplexTypesCollector; - -/** - * This class provides access to registered complex expression processors. - */ -public class BidiComplexStringProcessor { - /** - * Retrieve all registered types of complex expression processors. - * - * @return an array of strings, each string identifying a type of - * complex expression processor. - */ - static public String[] getKnownTypes() { - return BidiComplexTypesCollector.getInstance().getTypes(); - } - - /** - * Get access to a complex expression processor of a given type. - * - * @param type string identifying a type of processor - * - * @return a reference to an instance of a processor of the - * required type. If the type is unknown, return null. - */ - static public IBidiComplexProcessor getProcessor(String type) { - return BidiComplexTypesCollector.getInstance().getProcessor(type); - } -} Index: src/org/eclipse/equinox/bidi/custom/IBidiComplexProcessor.java =================================================================== RCS file: src/org/eclipse/equinox/bidi/custom/IBidiComplexProcessor.java diff -N src/org/eclipse/equinox/bidi/custom/IBidiComplexProcessor.java --- src/org/eclipse/equinox/bidi/custom/IBidiComplexProcessor.java 17 May 2011 19:56:16 -0000 1.3 +++ /dev/null 1 Jan 1970 00:00:00 -0000 @@ -1,170 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2010, 2011 IBM Corporation and others. - * All rights reserved. This program and the accompanying materials - * are made available under the terms of the Eclipse Public License v1.0 - * which accompanies this distribution, and is available at - * http://www.eclipse.org/legal/epl-v10.html - * - * Contributors: - * IBM Corporation - initial API and implementation - ******************************************************************************/ -package org.eclipse.equinox.bidi.custom; - -import org.eclipse.equinox.bidi.BidiComplexEnvironment; -import org.eclipse.equinox.bidi.BidiComplexEngine; - -/** - * Interface for all complex expression processors. - * For guidelines about implementation, see - * {@link BidiComplexProcessor}. - * - * @author Matitiahu Allouche - */ -public interface IBidiComplexProcessor { - - /** - * return the - * {@link BidiComplexFeatures} characterizing the processor. - * - * @param env the current environment, which may affect the behavior of - * the processor. This parameter may be specified as - * null, in which case the - * {@link BidiComplexEnvironment#DEFAULT DEFAULT} - * environment should be assumed. - * - * @return the features in use for this processor. - */ - public abstract BidiComplexFeatures getFeatures(BidiComplexEnvironment env); - - /** - * Locate occurrences of special strings within a complex expression - * and return their indexes one after the other in successive calls. - *

- * This method is called repeatedly from the code implementing - * {@link BidiComplexEngine#leanToFullText leanToFullText} if the - * number of special cases appearing in the associated features - * parameter is greater than zero. - *

- * The code implementing this method may use the following methods - * in {@link BidiComplexProcessor}: - *

    - *
  • {@link BidiComplexProcessor#getDirProp getDirProp}
    • - *
  • {@link BidiComplexProcessor#setDirProp setDirProp}
    • - *
  • {@link BidiComplexProcessor#insertMark insertMark}
    • - *
  • {@link BidiComplexProcessor#processSeparator processSeparator}
    • - *
- * - * @param features is the {@link BidiComplexFeatures} instance - * currently associated with this processor. - * - * @param text is the text of the complex expression before - * addition of any directional formatting characters. - * - * @param dirProps is a parameter received by indexOfSpecial - * uniquely to be used as argument for calls to methods which - * need it. - * - * @param offsets is a parameter received by indexOfSpecial - * uniquely to be used as argument for calls to methods which - * need it. - * - * @param caseNumber number of the special case to locate. - * This number varies from 1 to the number of special cases - * in the features associated with this processor. - * The meaning of this number is internal to the class - * implementing indexOfSpecial. - * - * @param fromIndex the index within text to start - * the search from. - * - * @return the position where the start of the special case was located. - * The method must return the first occurrence of whatever - * identifies the start of the special case starting from - * fromIndex. The method does not have to check if - * this occurrence appears within the scope of another special - * case (e.g. a comment starting delimiter within the scope of - * a literal or vice-versa). - *
If no occurrence is found, the method must return -1. - */ - public int indexOfSpecial(BidiComplexFeatures features, String text, byte[] dirProps, int[] offsets, int caseNumber, int fromIndex); - - /** - * This method handles special cases specific to this processor. - * It is called by {@link BidiComplexEngine#leanToFullText leanToFullText} - * when a special case occurrence is located by - * {@link #indexOfSpecial indexOfSpecial}. - *

- * The code implementing this method may use the following methods - * in {@link BidiComplexProcessor}: - *

    - *
  • {@link BidiComplexProcessor#getDirProp getDirProp}
    • - *
  • {@link BidiComplexProcessor#setDirProp setDirProp}
    • - *
  • {@link BidiComplexProcessor#insertMark insertMark}
    • - *
  • {@link BidiComplexProcessor#processSeparator processSeparator}
    • - *
- *

- * If a special processing cannot be completed within a current call to - * processSpecial (for instance, a comment has been started - * in the current line but its end appears in a following line), - * processSpecial should specify a final state by - * putting its value in the first element of the state - * parameter. - * The meaning of this state is internal to the processor. - * On a later call to - * {@link BidiComplexEngine#leanToFullText leanToFullText} - * specifying that state value, processSpecial will be - * called with that value for parameter caseNumber and - * -1 for parameter separLocation and should - * perform whatever initializations are required depending on the state. - * - * @param features is the {@link BidiComplexFeatures} instance - * currently associated with this processor. - * - * @param text is the text of the complex expression before - * addition of any directional formatting characters. - * - * @param dirProps is a parameter received by processSpecial - * uniquely to be used as argument for calls to methods which - * need it. - * - * @param offsets is a parameter received by processSpecial - * uniquely to be used as argument for calls to methods which - * need it. - * - * @param state is an integer array with at least one element. - * If the processor needs to signal the occurrence of a - * special case which must be passed to the next call to - * leanToFullText (for instance, a comment or a - * literal started but not closed in the current - * text), it must put a value in the first element - * of the state parameter. - * This value must be a number between 1 and the number of - * special cases appearing in the features associated with - * this processor. This number is passed back to the caller - * and should be specified as state argument - * in the next call to leanToFullText together - * with the continuation text. - * The meaning of this number is internal to the processor. - * - * @param caseNumber number of the special case to handle. - * - * @param separLocation the position returned by - * {@link #indexOfSpecial indexOfSpecial}. In calls to - * {@link BidiComplexEngine#leanToFullText leanToFullText} and other - * methods of {@link BidiComplexEngine} specifying a non-null - * state parameter, processSpecial is - * called when initializing the processing with the value of - * caseNumber equal to the value returned in the - * first element of state and the value of - * separLocation equal to -1. - * - * @return the position after the scope of the special case ends. - * For instance, the position after the end of a comment, - * the position after the end of a literal. - *
A value greater or equal to the length of text - * means that there is no further occurrence of this case in the - * current complex expression. - */ - public int processSpecial(BidiComplexFeatures features, String text, byte[] dirProps, int[] offsets, int[] state, int caseNumber, int separLocation); - -} Index: src/org/eclipse/equinox/bidi/custom/ISTextProcessor.java =================================================================== RCS file: src/org/eclipse/equinox/bidi/custom/ISTextProcessor.java diff -N src/org/eclipse/equinox/bidi/custom/ISTextProcessor.java --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ src/org/eclipse/equinox/bidi/custom/ISTextProcessor.java 1 Jan 1970 00:00:00 -0000 @@ -0,0 +1,170 @@ +/******************************************************************************* + * Copyright (c) 2010, 2011 IBM Corporation and others. + * All rights reserved. This program and the accompanying materials + * are made available under the terms of the Eclipse Public License v1.0 + * which accompanies this distribution, and is available at + * http://www.eclipse.org/legal/epl-v10.html + * + * Contributors: + * IBM Corporation - initial API and implementation + ******************************************************************************/ +package org.eclipse.equinox.bidi.custom; + +import org.eclipse.equinox.bidi.STextEngine; +import org.eclipse.equinox.bidi.STextEnvironment; + +/** + * Interface for all structured text processors. + * For guidelines about implementation, see + * {@link STextProcessor}. + * + * @author Matitiahu Allouche + */ +public interface ISTextProcessor { + + /** + * return the + * {@link STextFeatures} characterizing the processor. + * + * @param env the current environment, which may affect the behavior of + * the processor. This parameter may be specified as + * null, in which case the + * {@link STextEnvironment#DEFAULT DEFAULT} + * environment should be assumed. + * + * @return the features in use for this processor. + */ + public abstract STextFeatures getFeatures(STextEnvironment env); + + /** + * Locate occurrences of special strings within a structured text + * and return their indexes one after the other in successive calls. + *

+ * This method is called repeatedly from the code implementing + * {@link STextEngine#leanToFullText leanToFullText} if the + * number of special cases appearing in the associated features + * parameter is greater than zero. + *

+ * The code implementing this method may use the following methods + * in {@link STextProcessor}: + *

    + *
  • {@link STextProcessor#getDirProp getDirProp}
    • + *
  • {@link STextProcessor#setDirProp setDirProp}
    • + *
  • {@link STextProcessor#insertMark insertMark}
    • + *
  • {@link STextProcessor#processSeparator processSeparator}
    • + *
+ * + * @param features is the {@link STextFeatures} instance + * currently associated with this processor. + * + * @param text is the structured text string before + * addition of any directional formatting characters. + * + * @param dirProps is a parameter received by indexOfSpecial + * uniquely to be used as argument for calls to methods which + * need it. + * + * @param offsets is a parameter received by indexOfSpecial + * uniquely to be used as argument for calls to methods which + * need it. + * + * @param caseNumber number of the special case to locate. + * This number varies from 1 to the number of special cases + * in the features associated with this processor. + * The meaning of this number is internal to the class + * implementing indexOfSpecial. + * + * @param fromIndex the index within text to start + * the search from. + * + * @return the position where the start of the special case was located. + * The method must return the first occurrence of whatever + * identifies the start of the special case starting from + * fromIndex. The method does not have to check if + * this occurrence appears within the scope of another special + * case (e.g. a comment starting delimiter within the scope of + * a literal or vice-versa). + *
If no occurrence is found, the method must return -1. + */ + public int indexOfSpecial(STextFeatures features, String text, byte[] dirProps, int[] offsets, int caseNumber, int fromIndex); + + /** + * This method handles special cases specific to this processor. + * It is called by {@link STextEngine#leanToFullText leanToFullText} + * when a special case occurrence is located by + * {@link #indexOfSpecial indexOfSpecial}. + *

+ * The code implementing this method may use the following methods + * in {@link STextProcessor}: + *

    + *
  • {@link STextProcessor#getDirProp getDirProp}
    • + *
  • {@link STextProcessor#setDirProp setDirProp}
    • + *
  • {@link STextProcessor#insertMark insertMark}
    • + *
  • {@link STextProcessor#processSeparator processSeparator}
    • + *
+ *

+ * If a special processing cannot be completed within a current call to + * processSpecial (for instance, a comment has been started + * in the current line but its end appears in a following line), + * processSpecial should specify a final state by + * putting its value in the first element of the state + * parameter. + * The meaning of this state is internal to the processor. + * On a later call to + * {@link STextEngine#leanToFullText leanToFullText} + * specifying that state value, processSpecial will be + * called with that value for parameter caseNumber and + * -1 for parameter separLocation and should + * perform whatever initializations are required depending on the state. + * + * @param features is the {@link STextFeatures} instance + * currently associated with this processor. + * + * @param text is the structured text string before + * addition of any directional formatting characters. + * + * @param dirProps is a parameter received by processSpecial + * uniquely to be used as argument for calls to methods which + * need it. + * + * @param offsets is a parameter received by processSpecial + * uniquely to be used as argument for calls to methods which + * need it. + * + * @param state is an integer array with at least one element. + * If the processor needs to signal the occurrence of a + * special case which must be passed to the next call to + * leanToFullText (for instance, a comment or a + * literal started but not closed in the current + * text), it must put a value in the first element + * of the state parameter. + * This value must be a number between 1 and the number of + * special cases appearing in the features associated with + * this processor. This number is passed back to the caller + * and should be specified as state argument + * in the next call to leanToFullText together + * with the continuation text. + * The meaning of this number is internal to the processor. + * + * @param caseNumber number of the special case to handle. + * + * @param separLocation the position returned by + * {@link #indexOfSpecial indexOfSpecial}. In calls to + * {@link STextEngine#leanToFullText leanToFullText} and other + * methods of {@link STextEngine} specifying a non-null + * state parameter, processSpecial is + * called when initializing the processing with the value of + * caseNumber equal to the value returned in the + * first element of state and the value of + * separLocation equal to -1. + * + * @return the position after the scope of the special case ends. + * For instance, the position after the end of a comment, + * the position after the end of a literal. + *
A value greater or equal to the length of text + * means that there is no further occurrence of this case in the + * current structured text. + */ + public int processSpecial(STextFeatures features, String text, byte[] dirProps, int[] offsets, int[] state, int caseNumber, int separLocation); + +} Index: src/org/eclipse/equinox/bidi/custom/STextFeatures.java =================================================================== RCS file: src/org/eclipse/equinox/bidi/custom/STextFeatures.java diff -N src/org/eclipse/equinox/bidi/custom/STextFeatures.java --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ src/org/eclipse/equinox/bidi/custom/STextFeatures.java 1 Jan 1970 00:00:00 -0000 @@ -0,0 +1,250 @@ +/******************************************************************************* + * Copyright (c) 2010, 2011 IBM Corporation and others. + * All rights reserved. This program and the accompanying materials + * are made available under the terms of the Eclipse Public License v1.0 + * which accompanies this distribution, and is available at + * http://www.eclipse.org/legal/epl-v10.html + * + * Contributors: + * IBM Corporation - initial API and implementation + ******************************************************************************/ +package org.eclipse.equinox.bidi.custom; + +import org.eclipse.equinox.bidi.STextEnvironment; + +/** + * This class defines features of a structured text processor. + *

+ * All public fields in this class are final, i.e. cannot be + * changed after creating an instance. + * + *

Code Sample

+ *

Example 1 (set all features) + * 

+ *
+ *    STextFeatures f1 = new STextFeatures("+-=", 0, -1, -1, false, false);
+ *
+ *
+ *

Example 2 (change only the separators) + * 

+ *
+ *    STextFeatures f2 = new STextFeatures("[]|()", f1.getSpecialsCount(),
+ *                                     f1.getDirArabic(), f1.getDirHebrew(),
+ *                                     f1.getIgnoreArabic(), f1.getIgnoreHebrew());
+ *
+ *
+ * + * @see ISTextProcessor#getFeatures + * + * @author Matitiahu Allouche + */ +public class STextFeatures { + + /** + * Constant specifying that the base direction of a structured text is LTR. + * The base direction may depend on whether the GUI is + * {@link STextEnvironment#getMirrored mirrored} and may + * may be different for Arabic and for Hebrew. + * This constant can appear as dirArabic + * or dirHebrew argument for the + * {@link STextFeatures#STextFeatures STextFeatures constructor} + * and as value returned by {@link #getDirArabic} or {@link #getDirHebrew} + * methods. + */ + public static final int DIR_LTR = 0; + + /** + * Constant specifying that the base direction of a structured text is RTL. + * The base direction may depend on whether the GUI is + * {@link STextEnvironment#getMirrored mirrored} and may + * may be different for Arabic and for Hebrew. + * This constant can appear as dirArabic + * or dirHebrew argument for the + * {@link STextFeatures#STextFeatures STextFeatures constructor} + * and as value returned by {@link #getDirArabic} or {@link #getDirHebrew} + * methods. + */ + public static final int DIR_RTL = 1; + + /** + * Pre-defined STextFeatures instance with values for no + * separators, no special processing, all directions LTR + * and support for neither Arabic nor Hebrew.
+ * Since there are no separators and no special processing, a structured text + * processor with such features would do nothing.
+ * It is more efficient to do nothing with a null processor. + */ + public static final STextFeatures DEFAULT = new STextFeatures(null, 0, -1, -1, true, true); + + /** + * String grouping one-character separators which + * separate the structured text into tokens. + */ + final String separators; + + /** + * Number of special cases for the associated processor. + * Special cases exist for some types of structured text processors. + * They are implemented by overriding methods + * {@link ISTextProcessor#indexOfSpecial indexOfSpecial} and + * {@link ISTextProcessor#processSpecial processSpecial}. + * Examples of special cases are comments, literals, or anything which + * is not identified by a one-character separator. + */ + final int specialsCount; + + /** + * Base direction of the structured text for Arabic. + * If a structured text contains both Arabic and + * Hebrew words, the first Arabic or Hebrew letter in the + * text determines which is the governing script).
+ * The value of this field must be one of + * {@link #DIR_LTR} or {@link #DIR_RTL}. + * + * @see #dirHebrew + */ + final int dirArabic; + + /** + * Base direction of the structured text for Hebrew. + * If a structured text contains both Arabic and + * Hebrew words, the first Arabic or Hebrew letter in the + * text determines which is the governing script).
+ * The value of this field must be one of + * {@link #DIR_LTR} or {@link #DIR_RTL}. + * + * @see #dirArabic + */ + final int dirHebrew; + + /** + * Flag indicating that Arabic letters will not be considered for + * processing structured text. If both this flag and + * ignoreHebrew are set to true, the + * processor will do nothing (but some overhead can be expected). + */ + final boolean ignoreArabic; + + /** + * Flag indicating that Hebrew letters will not be considered for + * processing structured text. If both this flag and + * ignoreArabic are set to true, the + * processor will do nothing (but some overhead can be expected). + */ + final boolean ignoreHebrew; + + /** + * Constructor + * + * @param separators is a string where each character is a separator + * which separates the structured text into tokens. + * @see #getSeparators + * + * @param specialsCount specifies the number of special cases handled + * by the processor. This value must be identical to the + * number of special cases handled by the processor with which + * this STextFeatures instance is associated. + * @see #getSpecialsCount + * + * @param dirArabic specifies the base direction of the structured text + * for Arabic. It must be {@link #DIR_LTR} or {@link #DIR_RTL}. + * If it is not (for instance if it is a negative value), it + * defaults to DIR_LTR. + * @see #getDirArabic + * + * @param dirHebrew specifies the base direction of the structured text + * for Hebrew. It must be {@link #DIR_LTR} or {@link #DIR_RTL}. + * If it is not (for instance if it is a negative value), it + * defaults to DIR_LTR. + * @see #getDirHebrew + * + * @param ignoreArabic indicates that Arabic letters will not be + * considered for processing structured text. + * If both this flag and ignoreHebrew + * are set to true, the processor will do + * nothing (but some overhead can be expected). + * @see #getIgnoreArabic + * + * @param ignoreHebrew indicates that Hebrew letters will not be + * considered for processing structured text. + * If both this flag and ignoreArabic + * are set to true, the processor will do + * nothing (but some overhead can be expected). + * @see #getIgnoreHebrew + */ + public STextFeatures(String separators, int specialsCount, int dirArabic, int dirHebrew, boolean ignoreArabic, boolean ignoreHebrew) { + + this.separators = separators == null ? "" : separators; //$NON-NLS-1$ + this.specialsCount = specialsCount; + this.dirArabic = dirArabic == DIR_LTR || dirArabic == DIR_RTL ? dirArabic : DIR_LTR; + this.dirHebrew = dirHebrew == DIR_LTR || dirHebrew == DIR_RTL ? dirHebrew : DIR_LTR; + this.ignoreArabic = ignoreArabic; + this.ignoreHebrew = ignoreHebrew; + } + + /** + * @return a string grouping one-character separators which separate + * the structured text into tokens. + */ + public String getSeparators() { + return separators; + } + + /** + * @return the number of special cases for the associated processor. + * Special cases exist for some types of structured text + * processors. They are implemented by overriding methods + * {@link ISTextProcessor#indexOfSpecial indexOfSpecial} and + * {@link ISTextProcessor#processSpecial processSpecial}. + * Examples of special cases are comments, literals, or + * anything which is not identified by a one-character separator. + */ + public int getSpecialsCount() { + return specialsCount; + } + + /** + * @return the base direction of the structured text for Arabic. + * If a structured text contains both Arabic and + * Hebrew words, the first Arabic or Hebrew letter in the + * text determines which is the governing script.
+ * The value of this field is one of + * {@link #DIR_LTR} or {@link #DIR_RTL}. + * + * @see #getDirHebrew + */ + public int getDirArabic() { + return dirArabic; + } + + /** + * @return the base direction of the structured text for Hebrew. + * If a structured text contains both Arabic and + * Hebrew words, the first Arabic or Hebrew letter in the + * text determines which is the governing script.
+ * The value of this field is one of + * {@link #DIR_LTR} or {@link #DIR_RTL}. + * + * @see #getDirArabic + */ + public int getDirHebrew() { + return dirHebrew; + } + + /** + * @return a flag indicating that Arabic letters will not be considered + * for processing structured text. + */ + public boolean getIgnoreArabic() { + return ignoreArabic; + } + + /** + * Flag indicating that Hebrew letters will not be considered for + * processing structured text. + */ + public boolean getIgnoreHebrew() { + return ignoreHebrew; + } + +} Index: src/org/eclipse/equinox/bidi/custom/STextProcessor.java =================================================================== RCS file: src/org/eclipse/equinox/bidi/custom/STextProcessor.java diff -N src/org/eclipse/equinox/bidi/custom/STextProcessor.java --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ src/org/eclipse/equinox/bidi/custom/STextProcessor.java 1 Jan 1970 00:00:00 -0000 @@ -0,0 +1,235 @@ +/******************************************************************************* + * Copyright (c) 2010, 2011 IBM Corporation and others. + * All rights reserved. This program and the accompanying materials + * are made available under the terms of the Eclipse Public License v1.0 + * which accompanies this distribution, and is available at + * http://www.eclipse.org/legal/epl-v10.html + * + * Contributors: + * IBM Corporation - initial API and implementation + ******************************************************************************/ +package org.eclipse.equinox.bidi.custom; + +import org.eclipse.equinox.bidi.STextEngine; +import org.eclipse.equinox.bidi.STextEnvironment; +import org.eclipse.equinox.bidi.internal.STextImpl; + +/** + * Generic processor which can be used as superclass (base class) + * for specific structured text processors. + *

+ * Here are some guidelines about how to write structured text + * processors. + *

    + *
  • Processor instances may be accessed simultaneously by + * several threads. They should have no instance variables.
    • + *
  • Each use of a processor is associated with a set of + * {@link STextFeatures features}. + * All processors must have a default set of features which may be + * queried with the {@link #getFeatures getFeatures} method. + * These default features may be overridden by specifying a + * features argument when calling a method. + * See for instance + * {@link STextEngine#leanToFullText leanToFullText}. + *
  • The behavior of a processor is governed by 3 factors, all included + * in associated {@link STextFeatures features} data. + *
      + *
    • The separators specified in its + * {@link STextFeatures features} determine how submitted + * structured text is split into tokens.
      • + *
    • The tokens are displayed one after the other according + * to the appropriate direction, which can be different for + * Arabic and for Hebrew.
      • + *
    • The number of special cases which need to be handled by + * code specific to that processor.
      • + *
    • + *
+ * + * @see STextFeatures#getSeparators + * @see STextFeatures#getDirArabic + * @see STextFeatures#getDirHebrew + * @see STextFeatures#getSpecialsCount + * + * @author Matitiahu Allouche + */ +public class STextProcessor implements ISTextProcessor { + + /** + * In STextProcessor this method returns a + * {@link STextFeatures#DEFAULT DEFAULT} value which + * directs the processor to do nothing. + * + *

A processor which extends this class must override this method + * and return a STextFeatures instance representing its specific + * features. + * + * @see ISTextProcessor#getFeatures the corresponding interface method + */ + public STextFeatures getFeatures(STextEnvironment environment) { + throw new IllegalStateException("A processor must have a getFeatures() method."); //$NON-NLS-1$ + } + + /** + * In STextProcessor this method throws an + * IllegalStateException. This is appropriate behavior + * (and does not need to be overridden) for processors whose + * number of special cases is zero, which means that + * indexOfSpecial should never be called for them. + * + *

A processor handling special cases must override this method. + * + * @see ISTextProcessor#indexOfSpecial the corresponding interface method + */ + public int indexOfSpecial(STextFeatures features, String text, byte[] dirProps, int[] offsets, int caseNumber, int fromIndex) { + // This method must be overridden by all subclasses with special cases. + throw new IllegalStateException("A processor must have an indexOfSpecial() method."); //$NON-NLS-1$ + } + + /** + * In STextProcessor this method throws an + * IllegalStateException. This is appropriate behavior + * (and does not need to be overridden) for processors whose + * number of special cases is zero, which means that + * processSpecial should never be called for them. + * + *

A processor handling special cases must override this method. + * + * @see ISTextProcessor#processSpecial the corresponding interface method + */ + public int processSpecial(STextFeatures features, String text, byte[] dirProps, int[] offsets, int[] state, int caseNumber, int separLocation) { + // This method must be overridden by all subclasses with any special case. + throw new IllegalStateException("A processor must have a processSpecial() method."); //$NON-NLS-1$ + } + + /** + * This method can be called from within + * {@link ISTextProcessor#indexOfSpecial indexOfSpecial} or + * {@link ISTextProcessor#processSpecial processSpecial} in + * implementations of {@link ISTextProcessor} to retrieve the + * bidirectional class of characters in the lean text. + * + * @param text is the structured text string received as + * parameter to indexOfSpecial or + * processSpecial. + * + * @param dirProps is a parameter received by indexOfSpecial + * or processSpecial, uniquely to be used as argument + * for calls to getDirProp and other methods used + * by processors. + * + * @param index position of the character in the lean text. + * It must be a non-negative number smaller than the length + * of the lean text. + * + * @return the bidirectional class of the character. It is one of the + * values which can be returned by + * java.lang.Character.getDirectionality. + * However, it is recommended to use getDirProp + * rather than java.lang.Character.getDirectionality + * since getDirProp manages a cache of character + * properties and so can be more efficient than calling the + * java.lang.Character method. + */ + public static byte getDirProp(String text, byte[] dirProps, int index) { + return STextImpl.getDirProp(text, dirProps, index); + } + + /** + * This method can be called from within + * {@link ISTextProcessor#indexOfSpecial indexOfSpecial} or + * {@link ISTextProcessor#processSpecial processSpecial} in + * implementations of {@link ISTextProcessor} to set or + * override the bidirectional class of characters in the lean text. + * + * @param dirProps is a parameter received by indexOfSpecial + * or processSpecial, uniquely to be used as argument + * for calls to setDirProp and other methods used + * by processors. + * + * @param index position of the character in the lean text. + * It must be a non-negative number smaller than the length + * of the lean text. + * + * @param dirProp bidirectional class of the character. It is one of the + * values which can be returned by + * java.lang.Character.getDirectionality. + */ + public static void setDirProp(byte[] dirProps, int index, byte dirProp) { + STextImpl.setDirProp(dirProps, index, dirProp); + } + + /** + * This method can be called from within + * {@link ISTextProcessor#indexOfSpecial indexOfSpecial} or + * {@link ISTextProcessor#processSpecial processSpecial} in + * implementations of {@link ISTextProcessor} + * to specify that a mark character must be added before the character + * at the specified position of the lean text when generating the + * full text. The mark character will be LRM for structured text + * with a LTR base direction, and RLM for structured text with RTL + * base direction. The mark character is not added physically by this + * method, but its position is noted and will be used when generating + * the full text. + * + * @param text is the structured text string received as + * parameter to indexOfSpecial or + * processSpecial. + * + * @param dirProps is a parameter received by indexOfSpecial + * or processSpecial, uniquely to be used as argument + * for calls to insertMark and other methods used + * by processors. + * + * @param offsets is a parameter received by indexOfSpecial + * or processSpecial, uniquely to be used as argument + * for calls to insertMark and other methods used + * by processors. + * + * @param offset position of the character in the lean text. + * It must be a non-negative number smaller than the length + * of the lean text. + * For the benefit of efficiency, it is better to insert + * multiple marks in ascending order of the offsets. + */ + public static void insertMark(String text, byte[] dirProps, int[] offsets, int offset) { + STextImpl.insertMark(text, dirProps, offsets, offset); + } + + /** + * This method can be called from within + * {@link ISTextProcessor#indexOfSpecial indexOfSpecial} or + * {@link ISTextProcessor#processSpecial processSpecial} in + * implementations of {@link ISTextProcessor} to add a + * directional mark before a + * separator if needed for correct display, depending on the + * base direction of the text and on the class of the + * characters in the lean text preceding and following + * the separator itself. + * + * @param features is the {@link STextFeatures} instance + * received as parameter to indexOfSpecial or + * processSpecial. + * + * @param text is the structured text string received as + * parameter to indexOfSpecial or + * processSpecial. + * + * @param dirProps is a parameter received by indexOfSpecial + * or processSpecial, uniquely to be used as argument + * for calls to processSeparator and other methods used + * by processors. + * + * @param offsets is a parameter received by indexOfSpecial + * or processSpecial, uniquely to be used as argument + * for calls to processSeparator and other methods used + * by processors. + * + * @param separLocation offset of the separator in the lean text. + * It must be a non-negative number smaller than the length + * of the lean text. + */ + public static void processSeparator(STextFeatures features, String text, byte[] dirProps, int[] offsets, int separLocation) { + STextImpl.processSeparator(features, text, dirProps, offsets, separLocation); + } + +} Index: src/org/eclipse/equinox/bidi/custom/STextStringProcessor.java =================================================================== RCS file: src/org/eclipse/equinox/bidi/custom/STextStringProcessor.java diff -N src/org/eclipse/equinox/bidi/custom/STextStringProcessor.java --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ src/org/eclipse/equinox/bidi/custom/STextStringProcessor.java 1 Jan 1970 00:00:00 -0000 @@ -0,0 +1,40 @@ +/******************************************************************************* + * Copyright (c) 2010, 2011 IBM Corporation and others. + * All rights reserved. This program and the accompanying materials + * are made available under the terms of the Eclipse Public License v1.0 + * which accompanies this distribution, and is available at + * http://www.eclipse.org/legal/epl-v10.html + * + * Contributors: + * IBM Corporation - initial API and implementation + ******************************************************************************/ +package org.eclipse.equinox.bidi.custom; + +import org.eclipse.equinox.bidi.internal.STextTypesCollector; + +/** + * This class provides access to registered structured text processors. + */ +public class STextStringProcessor { + /** + * Retrieve all registered types of structured text processors. + * + * @return an array of strings, each string identifying a type of + * structured text processor. + */ + static public String[] getKnownTypes() { + return STextTypesCollector.getInstance().getTypes(); + } + + /** + * Get access to a structured text processor of a given type. + * + * @param type string identifying a type of processor + * + * @return a reference to an instance of a processor of the + * required type. If the type is unknown, return null. + */ + static public ISTextProcessor getProcessor(String type) { + return STextTypesCollector.getInstance().getProcessor(type); + } +} Index: src/org/eclipse/equinox/bidi/custom/package.html =================================================================== RCS file: /cvsroot/rt/org.eclipse.equinox/components/bundles/org.eclipse.equinox.bidi/src/org/eclipse/equinox/bidi/custom/package.html,v retrieving revision 1.2 diff -u -r1.2 package.html --- src/org/eclipse/equinox/bidi/custom/package.html 27 Apr 2011 14:53:08 -0000 1.2 +++ src/org/eclipse/equinox/bidi/custom/package.html 25 May 2011 16:24:14 -0000 @@ -6,18 +6,18 @@ This package provides an interface and classes for -developing complex expressions processors. +developing structured text processors.

Index: src/org/eclipse/equinox/bidi/internal/BidiComplexActivator.java =================================================================== RCS file: src/org/eclipse/equinox/bidi/internal/BidiComplexActivator.java diff -N src/org/eclipse/equinox/bidi/internal/BidiComplexActivator.java --- src/org/eclipse/equinox/bidi/internal/BidiComplexActivator.java 9 Feb 2011 21:28:55 -0000 1.1 +++ /dev/null 1 Jan 1970 00:00:00 -0000 @@ -1,91 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2010, 2011 IBM Corporation and others. - * All rights reserved. This program and the accompanying materials - * are made available under the terms of the Eclipse Public License v1.0 - * which accompanies this distribution, and is available at - * http://www.eclipse.org/legal/epl-v10.html - * - * Contributors: - * IBM Corporation - initial API and implementation - *******************************************************************************/ -package org.eclipse.equinox.bidi.internal; - -import java.util.Locale; -import org.eclipse.osgi.framework.log.FrameworkLog; -import org.eclipse.osgi.framework.log.FrameworkLogEntry; -import org.eclipse.osgi.service.localization.LocaleProvider; -import org.osgi.framework.*; -import org.osgi.util.tracker.ServiceTracker; - -public class BidiComplexActivator implements BundleActivator { - - private ServiceTracker logTracker = null; - private BundleContext bundleContext; - private static BidiComplexActivator instance; - - public BidiComplexActivator() { - instance = this; // there is only one bundle activator - } - - public void start(BundleContext context) throws Exception { - bundleContext = context; - instance = this; - } - - public void stop(BundleContext context) throws Exception { - if (logTracker != null) { - logTracker.close(); - logTracker = null; - } - bundleContext = null; - } - - public static BidiComplexActivator getInstance() { - return instance; - } - - public String getProperty(String key) { - return bundleContext.getProperty(key); - } - - public Locale getDefaultLocale() { - // use OSGi service - ServiceReference[] references = null; - try { - references = bundleContext.getAllServiceReferences(null, LocaleProvider.class.getName()); - } catch (InvalidSyntaxException e) { - // do nothing - } - if (references == null || references.length < 1) - return Locale.getDefault(); - Object service = bundleContext.getService(references[0]); - LocaleProvider localeProvider = (LocaleProvider) service; - if (localeProvider != null) { - Locale currentLocale = localeProvider.getLocale(); - bundleContext.ungetService(references[0]); - if (currentLocale != null) - return currentLocale; - } - return Locale.getDefault(); - } - - private FrameworkLog getFrameworkLog() { - if (logTracker == null) { - logTracker = new ServiceTracker(bundleContext, FrameworkLog.class.getName(), null); - logTracker.open(); - } - return (FrameworkLog) logTracker.getService(); - } - - static public void logError(String message, Exception e) { - FrameworkLog frameworkLog = instance.getFrameworkLog(); - if (frameworkLog != null) { - frameworkLog.log(new FrameworkLogEntry("org.eclipse.equinox.bidi", FrameworkLogEntry.ERROR, 1, message, 0, e, null)); //$NON-NLS-1$ - return; - } - System.err.println(message); - if (e != null) - e.printStackTrace(); - } - -} Index: src/org/eclipse/equinox/bidi/internal/BidiComplexDelims.java =================================================================== RCS file: src/org/eclipse/equinox/bidi/internal/BidiComplexDelims.java diff -N src/org/eclipse/equinox/bidi/internal/BidiComplexDelims.java --- src/org/eclipse/equinox/bidi/internal/BidiComplexDelims.java 17 May 2011 19:56:16 -0000 1.3 +++ /dev/null 1 Jan 1970 00:00:00 -0000 @@ -1,61 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2010, 2011 IBM Corporation and others. - * All rights reserved. This program and the accompanying materials - * are made available under the terms of the Eclipse Public License v1.0 - * which accompanies this distribution, and is available at - * http://www.eclipse.org/legal/epl-v10.html - * - * Contributors: - * IBM Corporation - initial API and implementation - ******************************************************************************/ -package org.eclipse.equinox.bidi.internal; - -import org.eclipse.equinox.bidi.custom.BidiComplexFeatures; -import org.eclipse.equinox.bidi.custom.BidiComplexProcessor; - -/** - * BidiComplexDelims is a processor for complex expressions - * composed of text segments separated by separators where the text segments - * may include delimited parts within which separators are treated like - * regular characters. - * - * @author Matitiahu Allouche - */ -public abstract class BidiComplexDelims extends BidiComplexProcessor { - - /** - * This method locates occurrences of start delimiters. - */ - public int indexOfSpecial(BidiComplexFeatures features, String text, byte[] dirProps, int[] offsets, int caseNumber, int fromIndex) { - char delim = getDelimiters().charAt((caseNumber - 1) * 2); - return text.indexOf(delim, fromIndex); - } - - /** - * This method skips until after the matching end delimiter. - */ - public int processSpecial(BidiComplexFeatures features, String text, byte[] dirProps, int[] offsets, int[] state, int caseNumber, int separLocation) { - BidiComplexProcessor.processSeparator(features, text, dirProps, offsets, separLocation); - int loc = separLocation + 1; - char delim = getDelimiters().charAt((caseNumber * 2) - 1); - loc = text.indexOf(delim, loc); - if (loc < 0) - return text.length(); - return loc + 1; - } - - /** - * @return a string containing the delimiters implemented in this class - * instance. This string must include an even - * number of characters. The first 2 characters of a string - * constitute a pair, the next 2 characters are a second pair, etc... - * In each pair, the first character is a start delimiter and - * the second character is an end delimiter. In the lean - * text, any part starting with a start delimiter and ending with - * the corresponding end delimiter is a delimited part. Within a - * delimited part, separators are treated like regular characters, - * which means that they do not define new segments. - */ - protected abstract String getDelimiters(); - -} Index: src/org/eclipse/equinox/bidi/internal/BidiComplexDelimsEsc.java =================================================================== RCS file: src/org/eclipse/equinox/bidi/internal/BidiComplexDelimsEsc.java diff -N src/org/eclipse/equinox/bidi/internal/BidiComplexDelimsEsc.java --- src/org/eclipse/equinox/bidi/internal/BidiComplexDelimsEsc.java 17 May 2011 19:56:16 -0000 1.3 +++ /dev/null 1 Jan 1970 00:00:00 -0000 @@ -1,56 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2010, 2011 IBM Corporation and others. - * All rights reserved. This program and the accompanying materials - * are made available under the terms of the Eclipse Public License v1.0 - * which accompanies this distribution, and is available at - * http://www.eclipse.org/legal/epl-v10.html - * - * Contributors: - * IBM Corporation - initial API and implementation - ******************************************************************************/ -package org.eclipse.equinox.bidi.internal; - -import org.eclipse.equinox.bidi.custom.BidiComplexFeatures; -import org.eclipse.equinox.bidi.custom.BidiComplexProcessor; - -/** - * BidiComplexDelims is a processor for complex expressions - * composed of text segments separated by separators where the text segments - * may include delimited parts within which separators are treated like - * regular characters and the delimiters may be escaped. - * This is similar to {@link BidiComplexDelims} except - * that delimiters can be escaped using the backslash character. - *
    - *
  • Two consecutive backslashes in a delimited part are treated like - * one regular character.
    • - *
  • An ending delimiter preceded by an odd number of backslashes is - * treated like a regular character within the delimited part.
    • - *
- * - * @author Matitiahu Allouche - */ -public abstract class BidiComplexDelimsEsc extends BidiComplexDelims { - - /** - * This method skips until after the matching end delimiter, - * ignoring possibly escaped end delimiters. - */ - public int processSpecial(BidiComplexFeatures features, String text, byte[] dirProps, int[] offsets, int[] state, int caseNumber, int separLocation) { - BidiComplexProcessor.processSeparator(features, text, dirProps, offsets, separLocation); - int location = separLocation + 1; - char delim = getDelimiters().charAt((caseNumber * 2) - 1); - while (true) { - location = text.indexOf(delim, location); - if (location < 0) - return text.length(); - int cnt = 0; - for (int i = location - 1; text.charAt(i) == '\\'; i--) { - cnt++; - } - location++; - if ((cnt & 1) == 0) - return location; - } - } - -} Index: src/org/eclipse/equinox/bidi/internal/BidiComplexImpl.java =================================================================== RCS file: src/org/eclipse/equinox/bidi/internal/BidiComplexImpl.java diff -N src/org/eclipse/equinox/bidi/internal/BidiComplexImpl.java --- src/org/eclipse/equinox/bidi/internal/BidiComplexImpl.java 17 May 2011 19:56:16 -0000 1.3 +++ /dev/null 1 Jan 1970 00:00:00 -0000 @@ -1,700 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2010, 2011 IBM Corporation and others. - * All rights reserved. This program and the accompanying materials - * are made available under the terms of the Eclipse Public License v1.0 - * which accompanies this distribution, and is available at - * http://www.eclipse.org/legal/epl-v10.html - * - * Contributors: - * IBM Corporation - initial API and implementation - ******************************************************************************/ -package org.eclipse.equinox.bidi.internal; - -import org.eclipse.equinox.bidi.BidiComplexEngine; -import org.eclipse.equinox.bidi.BidiComplexEnvironment; -import org.eclipse.equinox.bidi.custom.*; - -/** - * BidiComplexImpl provides the code which implements the API in - * {@link BidiComplexEngine}. All its public methods are shadows of similarly - * signed methods of BidiComplexEngine, and their documentation - * is by reference to the methods in BidiComplexEngine. - * - * @author Matitiahu Allouche - */ -public class BidiComplexImpl { - - static final String EMPTY_STRING = ""; //$NON-NLS-1$ - static final byte B = Character.DIRECTIONALITY_PARAGRAPH_SEPARATOR; - static final byte L = Character.DIRECTIONALITY_LEFT_TO_RIGHT; - static final byte R = Character.DIRECTIONALITY_RIGHT_TO_LEFT; - static final byte AL = Character.DIRECTIONALITY_RIGHT_TO_LEFT_ARABIC; - static final byte AN = Character.DIRECTIONALITY_ARABIC_NUMBER; - static final byte EN = Character.DIRECTIONALITY_EUROPEAN_NUMBER; - static final char LRM = 0x200E; - static final char RLM = 0x200F; - static final char LRE = 0x202A; - static final char RLE = 0x202B; - static final char PDF = 0x202C; - static final char[] MARKS = {LRM, RLM}; - static final char[] EMBEDS = {LRE, RLE}; - static final byte[] STRONGS = {L, R}; - static final int PREFIX_LENGTH = 2; - static final int SUFFIX_LENGTH = 2; - static final int FIXES_LENGTH = PREFIX_LENGTH + SUFFIX_LENGTH; - static final int DIRPROPS_ADD = 2; - static final int OFFSETS_SHIFT = 3; - static final int[] EMPTY_INT_ARRAY = new int[0]; - static final BidiComplexEnvironment IGNORE_ENVIRONMENT = new BidiComplexEnvironment(null, false, BidiComplexEnvironment.ORIENT_IGNORE); - - /** - * Prevent creation of a BidiComplexEngine instance - */ - private BidiComplexImpl() { - // nothing to do - } - - /* - // keep private copy of specialsCount to avoid later modification - specialsCount = features.getSpecialsCount(); - locations = new int[features.getSeparators().length() + specialsCount]; - } - */ - static long computeNextLocation(IBidiComplexProcessor processor, BidiComplexFeatures features, String text, byte[] dirProps, int[] offsets, int[] locations, int[] state, int curPos) { - String separators = features.getSeparators(); - int separCount = separators.length(); - int specialsCount = features.getSpecialsCount(); - int len = text.length(); - int nextLocation = len; - int idxLocation = 0; - // Start with special sequences to give them precedence over simple - // separators. This may apply to cases like slash+asterisk versus slash. - for (int i = 0; i < specialsCount; i++) { - int location = locations[separCount + i]; - if (location < curPos) { - offsets = ensureRoomInOffsets(offsets); - location = processor.indexOfSpecial(features, text, dirProps, offsets, i + 1, curPos); - if (location < 0) - location = len; - locations[separCount + i] = location; - } - if (location < nextLocation) { - nextLocation = location; - idxLocation = separCount + i; - } - } - for (int i = 0; i < separCount; i++) { - int location = locations[i]; - if (location < curPos) { - location = text.indexOf(separators.charAt(i), curPos); - if (location < 0) - location = len; - locations[i] = location; - } - if (location < nextLocation) { - nextLocation = location; - idxLocation = i; - } - } - return nextLocation + (((long) idxLocation) << 32); - } - - static int getCurOrient(BidiComplexEnvironment environment, String text, byte[] dirProps) { - int orient = environment.getOrientation(); - if ((orient & BidiComplexEnvironment.ORIENT_CONTEXTUAL_LTR) == 0) { - // absolute orientation - return orient; - } - // contextual orientation - int len = text.length(); - byte dirProp; - for (int i = 0; i < len; i++) { - // In the following lines, B, L, R and AL represent bidi categories - // as defined in the Unicode Bidirectional Algorithm - // ( http://www.unicode.org/reports/tr9/ ). - // B represents the category Block Separator. - // L represents the category Left to Right character. - // R represents the category Right to Left character. - // AL represents the category Arabic Letter. - dirProp = dirProps[i]; - if (dirProp == 0) { - dirProp = Character.getDirectionality(text.charAt(i)); - if (dirProp == B) // B char resolves to L or R depending on orientation - continue; - dirProps[i] = (byte) (dirProp + DIRPROPS_ADD); - } else { - dirProp -= DIRPROPS_ADD; - } - if (dirProp == L) - return BidiComplexEnvironment.ORIENT_LTR; - if (dirProp == R || dirProp == AL) - return BidiComplexEnvironment.ORIENT_RTL; - } - // return the default orientation corresponding to the contextual orientation - return orient & 1; - } - - /** - * @see BidiComplexEngine#getCurDirection BidiComplexEngine.getCurDirection - */ - public static int getCurDirection(Object _processor, BidiComplexFeatures features, BidiComplexEnvironment environment, String text, byte[] dirProps) { - if (environment == null) - environment = BidiComplexEnvironment.DEFAULT; - if (features == null) { - if (_processor == null) - return BidiComplexFeatures.DIR_LTR; - IBidiComplexProcessor processor; - if (_processor instanceof java.lang.String) { - processor = BidiComplexStringProcessor.getProcessor((String) _processor); - if (processor == null) - throw new IllegalArgumentException("Invalid processor type!"); //$NON-NLS-1$ - } else if (_processor instanceof IBidiComplexProcessor) - processor = (IBidiComplexProcessor) _processor; - else - throw new IllegalArgumentException("Invalid processor argument!"); //$NON-NLS-1$ - features = processor.getFeatures(environment); - } - int dirArabic = features.getDirArabic(); - int dirHebrew = features.getDirHebrew(); - // same direction for Arabic and Hebrew? - if (dirArabic == dirHebrew) - return dirArabic; - // check if Arabic or Hebrew letter comes first - int len = text.length(); - if (dirProps == null) - dirProps = new byte[len + 1]; - byte dirProp; - for (int i = 0; i < len; i++) { - // In the following lines, R and AL represent bidi categories - // as defined in the Unicode Bidirectional Algorithm - // ( http://www.unicode.org/reports/tr9/ ). - // R represents the category Right to Left character. - // AL represents the category Arabic Letter. - byte saveOrient = dirProps[len]; - dirProps[len] = -1; // make getDirProp return B - dirProp = getDirProp(text, dirProps, i); - dirProps[len] = saveOrient; - if (dirProp == AL) - return dirArabic; - if (dirProp == R) - return dirHebrew; - } - // found no Arabic or Hebrew character - return BidiComplexFeatures.DIR_LTR; - } - - /** - * @see BidiComplexProcessor#getDirProp BidiComplexProcessor.getDirProp - */ - public static byte getDirProp(String text, byte[] dirProps, int index) { - byte dirProp = dirProps[index]; - if (dirProp == 0) { - // In the following lines, B, L and R represent bidi categories - // as defined in the Unicode Bidirectional Algorithm - // ( http://www.unicode.org/reports/tr9/ ). - // B represents the category Block Separator. - // L represents the category Left to Right character. - // R represents the category Right to Left character. - dirProp = Character.getDirectionality(text.charAt(index)); - if (dirProp == B) { - // the last entry of dirProps contains the current component orientation - byte orient = dirProps[dirProps.length - 1]; - if (orient == -1) - return B; - dirProp = orient == BidiComplexEnvironment.ORIENT_RTL ? R : L; - } - dirProps[index] = (byte) (dirProp + DIRPROPS_ADD); - return dirProp; - } - return (byte) (dirProp - DIRPROPS_ADD); - } - - /** - * @see BidiComplexProcessor#setDirProp BidiComplexProcessor.setDirProp - */ - public static void setDirProp(byte[] dirProps, int index, byte dirProp) { - dirProps[index] = (byte) (dirProp + DIRPROPS_ADD); - } - - /** - * @see BidiComplexProcessor#processSeparator BidiComplexProcessor.processSeparator - */ - public static void processSeparator(BidiComplexFeatures features, String text, byte[] dirProps, int[] offsets, int separLocation) { - // In this method, L, R, AL, AN and EN represent bidi categories - // as defined in the Unicode Bidirectional Algorithm - // ( http://www.unicode.org/reports/tr9/ ). - // L represents the category Left to Right character. - // R represents the category Right to Left character. - // AL represents the category Arabic Letter. - // AN represents the category Arabic Number. - // EN represents the category European Number. - int len = text.length(); - // offsets[2] contains the complex expression direction - if (offsets[2] == BidiComplexFeatures.DIR_RTL) { - // the expression base direction is RTL - for (int i = separLocation - 1; i >= 0; i--) { - byte dirProp = getDirProp(text, dirProps, i); - if (dirProp == R || dirProp == AL) - return; - if (dirProp == L) { - for (int j = separLocation; j < len; j++) { - dirProp = getDirProp(text, dirProps, j); - if (dirProp == R || dirProp == AL) - return; - if (dirProp == L || dirProp == EN) { - insertMark(text, dirProps, offsets, separLocation); - return; - } - } - return; - } - } - return; - } - - // the expression base direction is LTR - boolean doneAN = false; - boolean ignoreArabic = features.getIgnoreArabic(); - boolean ignoreHebrew = features.getIgnoreHebrew(); - if (ignoreArabic && ignoreHebrew) - return; - byte _R, _AL, _AN; - if (ignoreArabic) { - _AL = Byte.MIN_VALUE; // not a real value - _AN = Byte.MIN_VALUE; - } else { - _AL = AL; - _AN = AN; - } - if (ignoreHebrew) - _R = Byte.MIN_VALUE; - else - _R = R; - for (int i = separLocation - 1; i >= 0; i--) { - byte dirProp = getDirProp(text, dirProps, i); - if (dirProp == L) - return; - if (dirProp == _R || dirProp == _AL) { - for (int j = separLocation; j < len; j++) { - dirProp = getDirProp(text, dirProps, j); - if (dirProp == L) - return; - if (dirProp == _R || dirProp == EN || dirProp == _AL || dirProp == _AN) { - insertMark(text, dirProps, offsets, separLocation); - return; - } - } - return; - } - if (dirProp == _AN && !doneAN) { - for (int j = separLocation; j < len; j++) { - dirProp = getDirProp(text, dirProps, j); - if (dirProp == L) - return; - if (dirProp == _AL || dirProp == _AN || dirProp == _R) { - insertMark(text, dirProps, offsets, separLocation); - return; - } - } - doneAN = true; - } - } - } - - /** - * @see BidiComplexEngine#leanToFullText BidiComplexEngine.leanToFullText - */ - public static String leanToFullText(Object processor, BidiComplexFeatures features, BidiComplexEnvironment environment, String text, int[] state) { - int len = text.length(); - if (len == 0) - return text; - byte[] dirProps = new byte[len + 1]; - int[] offsets = leanToFullCommon(processor, features, environment, text, state, dirProps); - int prefixLength = offsets[1]; - int count = offsets[0] - OFFSETS_SHIFT; - if (count == 0 && prefixLength == 0) - return text; - int newLen = len + count; - if (prefixLength == 1) - newLen++; /* +1 for a mark char */ - else if (prefixLength == 2) - newLen += FIXES_LENGTH; - char[] fullChars = new char[newLen]; - int added = prefixLength; - // add marks at offsets - int direction = offsets[2]; - char curMark = MARKS[direction]; - for (int i = 0, j = OFFSETS_SHIFT; i < len; i++) { - char c = text.charAt(i); - // offsets[0] contains the number of used entries - if (j < offsets[0] && i == offsets[j]) { - fullChars[i + added] = curMark; - added++; - j++; - } - fullChars[i + added] = c; - } - if (prefixLength > 0) { /* add prefix/suffix ? */ - if (prefixLength == 1) { /* contextual orientation */ - fullChars[0] = curMark; - } else { - // When the orientation is RTL, we need to add EMBED at the - // start of the text and PDF at its end. - // However, because of a bug in Windows' handling of LRE/PDF, - // we add EMBED_PREFIX at the start and EMBED_SUFFIX at the end. - char curEmbed = EMBEDS[direction]; - fullChars[0] = curEmbed; - fullChars[1] = curMark; - fullChars[newLen - 1] = PDF; - fullChars[newLen - 2] = curMark; - } - } - return new String(fullChars); - } - - /** - * @see BidiComplexEngine#leanToFullMap BidiComplexEngine.leanToFullMap - */ - public static int[] leanToFullMap(Object processor, BidiComplexFeatures features, BidiComplexEnvironment environment, String text, int[] state) { - int len = text.length(); - if (len == 0) - return EMPTY_INT_ARRAY; - byte[] dirProps = new byte[len + 1]; - int[] offsets = leanToFullCommon(processor, features, environment, text, state, dirProps); - int prefixLength = offsets[1]; - int[] map = new int[len]; - int count = offsets[0]; // number of used entries - int added = prefixLength; - for (int pos = 0, i = OFFSETS_SHIFT; pos < len; pos++) { - if (i < count && pos == offsets[i]) { - added++; - i++; - } - map[pos] = pos + added; - } - return map; - } - - /** - * @see BidiComplexEngine#leanBidiCharOffsets BidiComplexEngine.leanBidiCharOffsets - */ - public static int[] leanBidiCharOffsets(Object processor, BidiComplexFeatures features, BidiComplexEnvironment environment, String text, int[] state) { - int len = text.length(); - if (len == 0) - return EMPTY_INT_ARRAY; - byte[] dirProps = new byte[len + 1]; - int[] offsets = leanToFullCommon(processor, features, environment, text, state, dirProps); - // offsets[0] contains the number of used entries - int count = offsets[0] - OFFSETS_SHIFT; - int[] result = new int[count]; - System.arraycopy(offsets, OFFSETS_SHIFT, result, 0, count); - return result; - } - - static int[] leanToFullCommon(Object _processor, BidiComplexFeatures features, BidiComplexEnvironment environment, String text, int[] state, byte[] dirProps) { - IBidiComplexProcessor processor; - if (_processor instanceof java.lang.String) { - processor = BidiComplexStringProcessor.getProcessor((String) _processor); - if (processor == null) - throw new IllegalArgumentException("Invalid processor type!"); //$NON-NLS-1$ - } else if (_processor instanceof IBidiComplexProcessor) - processor = (IBidiComplexProcessor) _processor; - else - throw new IllegalArgumentException("Invalid processor argument!"); //$NON-NLS-1$ - if (environment == null) - environment = BidiComplexEnvironment.DEFAULT; - if (features == null) - features = processor.getFeatures(environment); - if (state == null) { - state = new int[1]; - state[0] = BidiComplexEngine.STATE_INITIAL; - } - int len = text.length(); - // dirProps: 1 byte for each char in text, + 1 byte = current orientation - int orient = getCurOrient(environment, text, dirProps); - dirProps[len] = (byte) orient; - int separCount = features.getSeparators().length(); - int direction = getCurDirection(processor, features, environment, text, dirProps); - // current position - int curPos = 0; - // offsets of marks to add. Entry 0 is the number of used slots; - // entry 1 is reserved to pass prefixLength. - // entry 2 is reserved to pass direction.. - int[] offsets = new int[20]; - offsets[0] = OFFSETS_SHIFT; - offsets[2] = direction; - // initialize locations - int[] locations = new int[separCount + features.getSpecialsCount()]; - for (int i = 0, k = locations.length; i < k; i++) { - locations[i] = -1; - } - if (state[0] > BidiComplexEngine.STATE_INITIAL) { - offsets = ensureRoomInOffsets(offsets); - int initState = state[0]; - state[0] = BidiComplexEngine.STATE_INITIAL; - curPos = processor.processSpecial(features, text, dirProps, offsets, state, initState, -1); - } - while (true) { - // location of next token to handle - int nextLocation; - // index of next token to handle (if < separCount, this is a separator; otherwise a special case - int idxLocation; - long res = computeNextLocation(processor, features, text, dirProps, offsets, locations, state, curPos); - nextLocation = (int) (res & 0x00000000FFFFFFFF); /* low word */ - if (nextLocation >= len) - break; - idxLocation = (int) (res >> 32); /* high word */ - if (idxLocation < separCount) { - offsets = ensureRoomInOffsets(offsets); - processSeparator(features, text, dirProps, offsets, nextLocation); - curPos = nextLocation + 1; - } else { - offsets = ensureRoomInOffsets(offsets); - idxLocation -= (separCount - 1); // because caseNumber starts from 1 - curPos = processor.processSpecial(features, text, dirProps, offsets, state, idxLocation, nextLocation); - } - } - if (orient == BidiComplexEnvironment.ORIENT_IGNORE) - offsets[1] = 0; - else { - // recompute orient since it may have changed if contextual - orient = getCurOrient(environment, text, dirProps); - dirProps[len] = (byte) orient; - if (orient == direction && orient != BidiComplexEnvironment.ORIENT_UNKNOWN) - offsets[1] = 0; - else if ((environment.getOrientation() & BidiComplexEnvironment.ORIENT_CONTEXTUAL_LTR) != 0) - offsets[1] = 1; - else - offsets[1] = 2; - } - return offsets; - } - - /** - * @see BidiComplexEngine#fullToLeanText BidiComplexEngine.fullToLeanText - */ - public static String fullToLeanText(Object _processor, BidiComplexFeatures features, BidiComplexEnvironment environment, String text, int[] state) { - if (text.length() == 0) - return text; - IBidiComplexProcessor processor; - if (_processor instanceof java.lang.String) { - processor = BidiComplexStringProcessor.getProcessor((String) _processor); - if (processor == null) - throw new IllegalArgumentException("Invalid processor type!"); //$NON-NLS-1$ - } else if (_processor instanceof IBidiComplexProcessor) - processor = (IBidiComplexProcessor) _processor; - else - throw new IllegalArgumentException("Invalid processor argument!"); //$NON-NLS-1$ - if (environment == null) - environment = BidiComplexEnvironment.DEFAULT; - if (features == null) - features = processor.getFeatures(environment); - if (state == null) { - state = new int[1]; - state[0] = BidiComplexEngine.STATE_INITIAL; - } - int dir = getCurDirection(processor, features, environment, text, null); - char curMark = MARKS[dir]; - char curEmbed = EMBEDS[dir]; - int i; // used as loop index - // remove any prefix and leading mark - int lenText = text.length(); - for (i = 0; i < lenText; i++) { - char c = text.charAt(i); - if (c != curEmbed && c != curMark) - break; - } - if (i > 0) { // found at least one prefix or leading mark - text = text.substring(i); - lenText = text.length(); - } - // remove any suffix and trailing mark - for (i = lenText - 1; i >= 0; i--) { - char c = text.charAt(i); - if (c != PDF && c != curMark) - break; - } - if (i < 0) // only suffix and trailing marks, no real data - return EMPTY_STRING; - if (i < (lenText - 1)) { // found at least one suffix or trailing mark - text = text.substring(0, i + 1); - lenText = text.length(); - } - char[] chars = text.toCharArray(); - // remove marks from chars - int cnt = 0; - for (i = 0; i < lenText; i++) { - char c = chars[i]; - if (c == curMark) - cnt++; - else if (cnt > 0) - chars[i - cnt] = c; - } - String lean = new String(chars, 0, lenText - cnt); - String full = leanToFullText(processor, features, IGNORE_ENVIRONMENT, lean, state); - if (full.equals(text)) - return lean; - - // There are some marks in full which are not in text and/or vice versa. - // We need to add to lean any mark appearing in text and not in full. - // The completed lean can never be longer than text itself. - char[] newChars = new char[lenText]; - char cFull, cText; - int idxFull, idxText, idxLean, newCharsPos; - int lenFull = full.length(); - idxFull = idxText = idxLean = newCharsPos = 0; - while (idxText < lenText && idxFull < lenFull) { - cFull = full.charAt(idxFull); - cText = text.charAt(idxText); - if (cFull == cText) { /* chars are equal, proceed */ - if (cFull != curMark) - newChars[newCharsPos++] = chars[idxLean++]; - idxText++; - idxFull++; - continue; - } - if (cFull == curMark) { /* extra Mark in full text */ - idxFull++; - continue; - } - if (cText == curMark) { /* extra Mark in source full text */ - idxText++; - // idxText-2 always >= 0 since leading Marks were removed from text - if (text.charAt(idxText - 2) == curMark) - continue; // ignore successive Marks in text after the first one - newChars[newCharsPos++] = curMark; - continue; - } - // we should never get here (extra char which is not a Mark) - throw new IllegalStateException("Internal error: extra character not a Mark."); //$NON-NLS-1$ - } - if (idxText < lenText) /* full ended before text - this should never happen since - we removed all marks and PDFs at the end of text */ - throw new IllegalStateException("Internal error: unexpected EOL."); //$NON-NLS-1$ - - lean = new String(newChars, 0, newCharsPos); - return lean; - } - - /** - * @see BidiComplexEngine#fullToLeanMap BidiComplexEngine.fullToLeanMap - */ - public static int[] fullToLeanMap(Object processor, BidiComplexFeatures features, BidiComplexEnvironment environment, String full, int[] state) { - int lenFull = full.length(); - if (lenFull == 0) - return EMPTY_INT_ARRAY; - String lean = fullToLeanText(processor, features, environment, full, state); - int lenLean = lean.length(); - int dir = getCurDirection(processor, features, environment, lean, null); - char curMark = MARKS[dir]; - char curEmbed = EMBEDS[dir]; - int[] map = new int[lenFull]; - int idxFull, idxLean; - // skip any prefix and leading mark - for (idxFull = 0; idxFull < lenFull; idxFull++) { - char c = full.charAt(idxFull); - if (c != curEmbed && c != curMark) - break; - map[idxFull] = -1; - } - // lean must be a subset of Full, so we only check on iLean < leanLen - for (idxLean = 0; idxLean < lenLean; idxFull++) { - if (full.charAt(idxFull) == lean.charAt(idxLean)) { - map[idxFull] = idxLean; - idxLean++; - } else - map[idxFull] = -1; - } - for (; idxFull < lenFull; idxFull++) - map[idxFull] = -1; - return map; - } - - /** - * @see BidiComplexEngine#fullBidiCharOffsets BidiComplexEngine.fullBidiCharOffsets - */ - public static int[] fullBidiCharOffsets(Object processor, BidiComplexFeatures features, BidiComplexEnvironment environment, String full, int[] state) { - int lenFull = full.length(); - if (lenFull == 0) - return EMPTY_INT_ARRAY; - String lean = fullToLeanText(processor, features, environment, full, state); - int[] offsets = new int[20]; - offsets[0] = OFFSETS_SHIFT; - int lenLean = lean.length(); - int idxLean, idxFull; - // lean must be a subset of Full, so we only check on iLean < leanLen - for (idxLean = idxFull = 0; idxLean < lenLean; idxFull++) { - if (full.charAt(idxFull) == lean.charAt(idxLean)) - idxLean++; - else { - offsets = ensureRoomInOffsets(offsets); - insertMark(lean, null, offsets, idxFull); - } - } - for (; idxFull < lenFull; idxFull++) { - offsets = ensureRoomInOffsets(offsets); - insertMark(lean, null, offsets, idxFull); - } - int[] result = new int[offsets[0] - OFFSETS_SHIFT]; - System.arraycopy(offsets, OFFSETS_SHIFT, result, 0, result.length); - return result; - } - - static int[] ensureRoomInOffsets(int[] offsets) { - // make sure - if ((offsets.length - offsets[0]) < 3) { - int[] newOffsets = new int[offsets.length * 2]; - System.arraycopy(offsets, 0, newOffsets, 0, offsets[0]); - return newOffsets; - } - return offsets; - } - - /** - * @see BidiComplexProcessor#insertMark BidiComplexProcessor.insertMark - */ - public static void insertMark(String text, byte[] dirProps, int[] offsets, int offset) { - int count = offsets[0];// number of used entries - int index = count - 1; // index of greatest member <= offset - // look up after which member the new offset should be inserted - while (index >= OFFSETS_SHIFT) { - int wrkOffset = offsets[index]; - if (offset > wrkOffset) - break; - if (offset == wrkOffset) - return; // avoid duplicates - index--; - } - index++; // index now points at where to insert - int length = count - index; // number of members to move up - if (length > 0) // shift right all members greater than offset - System.arraycopy(offsets, index, offsets, index + 1, length); - offsets[index] = offset; - offsets[0]++; // number of used entries - // if the offset is 0, adding a mark does not change anything - if (dirProps == null || offset < 1) - return; - - byte dirProp = getDirProp(text, dirProps, offset); - // if the current char is a strong one or a digit, we change the - // dirProp of the previous char to account for the inserted mark. - // In the following lines, L, R, AL, AN and EN represent bidi categories - // as defined in the Unicode Bidirectional Algorithm - // ( http://www.unicode.org/reports/tr9/ ). - // L represents the category Left to Right character. - // R represents the category Right to Left character. - // AL represents the category Arabic Letter. - // AN represents the category Arabic Number. - // EN represents the category European Number. - if (dirProp == L || dirProp == R || dirProp == AL || dirProp == EN || dirProp == AN) - index = offset - 1; - else - // if the current char is a neutral, we change its own dirProp - index = offset; - - int dir = offsets[2]; // current expression direction - setDirProp(dirProps, index, STRONGS[dir]); - return; - } - -} Index: src/org/eclipse/equinox/bidi/internal/BidiComplexSingle.java =================================================================== RCS file: src/org/eclipse/equinox/bidi/internal/BidiComplexSingle.java diff -N src/org/eclipse/equinox/bidi/internal/BidiComplexSingle.java --- src/org/eclipse/equinox/bidi/internal/BidiComplexSingle.java 17 May 2011 19:56:16 -0000 1.3 +++ /dev/null 1 Jan 1970 00:00:00 -0000 @@ -1,52 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2010, 2011 IBM Corporation and others. - * All rights reserved. This program and the accompanying materials - * are made available under the terms of the Eclipse Public License v1.0 - * which accompanies this distribution, and is available at - * http://www.eclipse.org/legal/epl-v10.html - * - * Contributors: - * IBM Corporation - initial API and implementation - ******************************************************************************/ -package org.eclipse.equinox.bidi.internal; - -import org.eclipse.equinox.bidi.custom.BidiComplexFeatures; -import org.eclipse.equinox.bidi.custom.BidiComplexProcessor; - -/** - * BidiComplexSingle is a processor for complex expressions - * composed of two parts separated by a separator. - * The first occurrence of the separator delimits the end of the first part - * and the start of the second part. Further occurrences of the separator, - * if any, are treated like regular characters of the second text part. - * The processor makes sure that the expression be presented in the form - * (assuming that the equal sign is the separator): - * 
- *  part1=part2
- *
- * The {@link BidiComplexFeatures#getSeparators separators} - * field in the {@link BidiComplexFeatures features} - * of this processor should contain exactly one character. - * Additional characters will be ignored. - * - * @author Matitiahu Allouche - */ -public abstract class BidiComplexSingle extends BidiComplexProcessor { - - /** - * This method locates occurrences of the separator. - */ - public int indexOfSpecial(BidiComplexFeatures features, String text, byte[] dirProps, int[] offsets, int caseNumber, int fromIndex) { - return text.indexOf(features.getSeparators().charAt(0), fromIndex); - } - - /** - * This method inserts a mark before the separator if needed and - * skips to the end of the source string. - */ - public int processSpecial(BidiComplexFeatures features, String text, byte[] dirProps, int[] offsets, int[] state, int caseNumber, int separLocation) { - BidiComplexProcessor.processSeparator(features, text, dirProps, offsets, separLocation); - return text.length(); - } - -} Index: src/org/eclipse/equinox/bidi/internal/BidiComplexTypesCollector.java =================================================================== RCS file: src/org/eclipse/equinox/bidi/internal/BidiComplexTypesCollector.java diff -N src/org/eclipse/equinox/bidi/internal/BidiComplexTypesCollector.java --- src/org/eclipse/equinox/bidi/internal/BidiComplexTypesCollector.java 9 Feb 2011 21:28:55 -0000 1.1 +++ /dev/null 1 Jan 1970 00:00:00 -0000 @@ -1,111 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2010, 2011 IBM Corporation and others. - * All rights reserved. This program and the accompanying materials - * are made available under the terms of the Eclipse Public License v1.0 - * which accompanies this distribution, and is available at - * http://www.eclipse.org/legal/epl-v10.html - * - * Contributors: - * IBM Corporation - initial API and implementation - *******************************************************************************/ -package org.eclipse.equinox.bidi.internal; - -import java.util.HashMap; -import java.util.Map; -import org.eclipse.core.runtime.*; -import org.eclipse.equinox.bidi.custom.IBidiComplexProcessor; - -public class BidiComplexTypesCollector implements IRegistryEventListener { - - private static final String EXT_POINT = "org.eclipse.equinox.bidi.bidiTypes"; //$NON-NLS-1$ - - private static final String CE_NAME = "typeDescription"; //$NON-NLS-1$ - private static final String ATTR_TYPE = "type"; //$NON-NLS-1$ - private static final String ATTR_PROCESSOR = "class"; //$NON-NLS-1$ - - private Map types; - private Map factories; - - static private BidiComplexTypesCollector instance = new BidiComplexTypesCollector(); - - private BidiComplexTypesCollector() { - IExtensionRegistry registry = RegistryFactory.getRegistry(); - registry.addListener(this, EXT_POINT); - } - - static public BidiComplexTypesCollector getInstance() { - return instance; - } - - public String[] getTypes() { - if (types == null) - read(); - int size = types.size(); - String[] result = new String[size]; - types.keySet().toArray(result); - return result; - } - - public IBidiComplexProcessor getProcessor(String type) { - if (types == null) - read(); - Object processor = types.get(type); - if (processor instanceof IBidiComplexProcessor) - return (IBidiComplexProcessor) processor; - return null; - } - - private void read() { - if (types == null) - types = new HashMap(); - else - types.clear(); - - if (factories == null) - factories = new HashMap(); - else - factories.clear(); - - IExtensionRegistry registry = RegistryFactory.getRegistry(); - IExtensionPoint extPoint = registry.getExtensionPoint(EXT_POINT); - IExtension[] extensions = extPoint.getExtensions(); - - for (int i = 0; i < extensions.length; i++) { - IConfigurationElement[] confElements = extensions[i].getConfigurationElements(); - for (int j = 0; j < confElements.length; j++) { - if (CE_NAME != confElements[j].getName()) - BidiComplexActivator.logError("BiDi types: unexpected element name " + confElements[j].getName(), new IllegalArgumentException()); //$NON-NLS-1$ - String type = confElements[j].getAttribute(ATTR_TYPE); - Object processor; - try { - processor = confElements[j].createExecutableExtension(ATTR_PROCESSOR); - } catch (CoreException e) { - BidiComplexActivator.logError("BiDi types: unable to create processor for " + type, e); //$NON-NLS-1$ - continue; - } - types.put(type, processor); - factories.put(type, confElements[j]); - } - } - } - - public void added(IExtension[] extensions) { - types = null; - factories = null; - } - - public void removed(IExtension[] extensions) { - types = null; - factories = null; - } - - public void added(IExtensionPoint[] extensionPoints) { - types = null; - factories = null; - } - - public void removed(IExtensionPoint[] extensionPoints) { - types = null; - factories = null; - } -} Index: src/org/eclipse/equinox/bidi/internal/STextActivator.java =================================================================== RCS file: src/org/eclipse/equinox/bidi/internal/STextActivator.java diff -N src/org/eclipse/equinox/bidi/internal/STextActivator.java --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ src/org/eclipse/equinox/bidi/internal/STextActivator.java 1 Jan 1970 00:00:00 -0000 @@ -0,0 +1,91 @@ +/******************************************************************************* + * Copyright (c) 2010, 2011 IBM Corporation and others. + * All rights reserved. This program and the accompanying materials + * are made available under the terms of the Eclipse Public License v1.0 + * which accompanies this distribution, and is available at + * http://www.eclipse.org/legal/epl-v10.html + * + * Contributors: + * IBM Corporation - initial API and implementation + *******************************************************************************/ +package org.eclipse.equinox.bidi.internal; + +import java.util.Locale; +import org.eclipse.osgi.framework.log.FrameworkLog; +import org.eclipse.osgi.framework.log.FrameworkLogEntry; +import org.eclipse.osgi.service.localization.LocaleProvider; +import org.osgi.framework.*; +import org.osgi.util.tracker.ServiceTracker; + +public class STextActivator implements BundleActivator { + + private ServiceTracker logTracker = null; + private BundleContext bundleContext; + private static STextActivator instance; + + public STextActivator() { + instance = this; // there is only one bundle activator + } + + public void start(BundleContext context) throws Exception { + bundleContext = context; + instance = this; + } + + public void stop(BundleContext context) throws Exception { + if (logTracker != null) { + logTracker.close(); + logTracker = null; + } + bundleContext = null; + } + + public static STextActivator getInstance() { + return instance; + } + + public String getProperty(String key) { + return bundleContext.getProperty(key); + } + + public Locale getDefaultLocale() { + // use OSGi service + ServiceReference[] references = null; + try { + references = bundleContext.getAllServiceReferences(null, LocaleProvider.class.getName()); + } catch (InvalidSyntaxException e) { + // do nothing + } + if (references == null || references.length < 1) + return Locale.getDefault(); + Object service = bundleContext.getService(references[0]); + LocaleProvider localeProvider = (LocaleProvider) service; + if (localeProvider != null) { + Locale currentLocale = localeProvider.getLocale(); + bundleContext.ungetService(references[0]); + if (currentLocale != null) + return currentLocale; + } + return Locale.getDefault(); + } + + private FrameworkLog getFrameworkLog() { + if (logTracker == null) { + logTracker = new ServiceTracker(bundleContext, FrameworkLog.class.getName(), null); + logTracker.open(); + } + return (FrameworkLog) logTracker.getService(); + } + + static public void logError(String message, Exception e) { + FrameworkLog frameworkLog = instance.getFrameworkLog(); + if (frameworkLog != null) { + frameworkLog.log(new FrameworkLogEntry("org.eclipse.equinox.bidi", FrameworkLogEntry.ERROR, 1, message, 0, e, null)); //$NON-NLS-1$ + return; + } + System.err.println(message); + if (e != null) + e.printStackTrace(); + } + +} Index: src/org/eclipse/equinox/bidi/internal/STextDelims.java =================================================================== RCS file: src/org/eclipse/equinox/bidi/internal/STextDelims.java diff -N src/org/eclipse/equinox/bidi/internal/STextDelims.java --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ src/org/eclipse/equinox/bidi/internal/STextDelims.java 1 Jan 1970 00:00:00 -0000 @@ -0,0 +1,61 @@ +/******************************************************************************* + * Copyright (c) 2010, 2011 IBM Corporation and others. + * All rights reserved. This program and the accompanying materials + * are made available under the terms of the Eclipse Public License v1.0 + * which accompanies this distribution, and is available at + * http://www.eclipse.org/legal/epl-v10.html + * + * Contributors: + * IBM Corporation - initial API and implementation + ******************************************************************************/ +package org.eclipse.equinox.bidi.internal; + +import org.eclipse.equinox.bidi.custom.STextFeatures; +import org.eclipse.equinox.bidi.custom.STextProcessor; + +/** + * STextDelims is a processor for structured text + * composed of text segments separated by separators where the text segments + * may include delimited parts within which separators are treated like + * regular characters. + * + * @author Matitiahu Allouche + */ +public abstract class STextDelims extends STextProcessor { + + /** + * This method locates occurrences of start delimiters. + */ + public int indexOfSpecial(STextFeatures features, String text, byte[] dirProps, int[] offsets, int caseNumber, int fromIndex) { + char delim = getDelimiters().charAt((caseNumber - 1) * 2); + return text.indexOf(delim, fromIndex); + } + + /** + * This method skips until after the matching end delimiter. + */ + public int processSpecial(STextFeatures features, String text, byte[] dirProps, int[] offsets, int[] state, int caseNumber, int separLocation) { + STextProcessor.processSeparator(features, text, dirProps, offsets, separLocation); + int loc = separLocation + 1; + char delim = getDelimiters().charAt((caseNumber * 2) - 1); + loc = text.indexOf(delim, loc); + if (loc < 0) + return text.length(); + return loc + 1; + } + + /** + * @return a string containing the delimiters implemented in this class + * instance. This string must include an even + * number of characters. The first 2 characters of a string + * constitute a pair, the next 2 characters are a second pair, etc... + * In each pair, the first character is a start delimiter and + * the second character is an end delimiter. In the lean + * text, any part starting with a start delimiter and ending with + * the corresponding end delimiter is a delimited part. Within a + * delimited part, separators are treated like regular characters, + * which means that they do not define new segments. + */ + protected abstract String getDelimiters(); + +} Index: src/org/eclipse/equinox/bidi/internal/STextDelimsEsc.java =================================================================== RCS file: src/org/eclipse/equinox/bidi/internal/STextDelimsEsc.java diff -N src/org/eclipse/equinox/bidi/internal/STextDelimsEsc.java --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ src/org/eclipse/equinox/bidi/internal/STextDelimsEsc.java 1 Jan 1970 00:00:00 -0000 @@ -0,0 +1,56 @@ +/******************************************************************************* + * Copyright (c) 2010, 2011 IBM Corporation and others. + * All rights reserved. This program and the accompanying materials + * are made available under the terms of the Eclipse Public License v1.0 + * which accompanies this distribution, and is available at + * http://www.eclipse.org/legal/epl-v10.html + * + * Contributors: + * IBM Corporation - initial API and implementation + ******************************************************************************/ +package org.eclipse.equinox.bidi.internal; + +import org.eclipse.equinox.bidi.custom.STextFeatures; +import org.eclipse.equinox.bidi.custom.STextProcessor; + +/** + * STextDelims is a processor for structured text + * composed of text segments separated by separators where the text segments + * may include delimited parts within which separators are treated like + * regular characters and the delimiters may be escaped. + * This is similar to {@link STextDelims} except + * that delimiters can be escaped using the backslash character. + *
    + *
  • Two consecutive backslashes in a delimited part are treated like + * one regular character.
    • + *
  • An ending delimiter preceded by an odd number of backslashes is + * treated like a regular character within the delimited part.
    • + *
+ * + * @author Matitiahu Allouche + */ +public abstract class STextDelimsEsc extends STextDelims { + + /** + * This method skips until after the matching end delimiter, + * ignoring possibly escaped end delimiters. + */ + public int processSpecial(STextFeatures features, String text, byte[] dirProps, int[] offsets, int[] state, int caseNumber, int separLocation) { + STextProcessor.processSeparator(features, text, dirProps, offsets, separLocation); + int location = separLocation + 1; + char delim = getDelimiters().charAt((caseNumber * 2) - 1); + while (true) { + location = text.indexOf(delim, location); + if (location < 0) + return text.length(); + int cnt = 0; + for (int i = location - 1; text.charAt(i) == '\\'; i--) { + cnt++; + } + location++; + if ((cnt & 1) == 0) + return location; + } + } + +} Index: src/org/eclipse/equinox/bidi/internal/STextImpl.java =================================================================== RCS file: src/org/eclipse/equinox/bidi/internal/STextImpl.java diff -N src/org/eclipse/equinox/bidi/internal/STextImpl.java --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ src/org/eclipse/equinox/bidi/internal/STextImpl.java 1 Jan 1970 00:00:00 -0000 @@ -0,0 +1,700 @@ +/******************************************************************************* + * Copyright (c) 2010, 2011 IBM Corporation and others. + * All rights reserved. This program and the accompanying materials + * are made available under the terms of the Eclipse Public License v1.0 + * which accompanies this distribution, and is available at + * http://www.eclipse.org/legal/epl-v10.html + * + * Contributors: + * IBM Corporation - initial API and implementation + ******************************************************************************/ +package org.eclipse.equinox.bidi.internal; + +import org.eclipse.equinox.bidi.STextEngine; +import org.eclipse.equinox.bidi.STextEnvironment; +import org.eclipse.equinox.bidi.custom.*; + +/** + * STextImpl provides the code which implements the API in + * {@link STextEngine}. All its public methods are shadows of similarly + * signed methods of STextEngine, and their documentation + * is by reference to the methods in STextEngine. + * + * @author Matitiahu Allouche + */ +public class STextImpl { + + static final String EMPTY_STRING = ""; //$NON-NLS-1$ + static final byte B = Character.DIRECTIONALITY_PARAGRAPH_SEPARATOR; + static final byte L = Character.DIRECTIONALITY_LEFT_TO_RIGHT; + static final byte R = Character.DIRECTIONALITY_RIGHT_TO_LEFT; + static final byte AL = Character.DIRECTIONALITY_RIGHT_TO_LEFT_ARABIC; + static final byte AN = Character.DIRECTIONALITY_ARABIC_NUMBER; + static final byte EN = Character.DIRECTIONALITY_EUROPEAN_NUMBER; + static final char LRM = 0x200E; + static final char RLM = 0x200F; + static final char LRE = 0x202A; + static final char RLE = 0x202B; + static final char PDF = 0x202C; + static final char[] MARKS = {LRM, RLM}; + static final char[] EMBEDS = {LRE, RLE}; + static final byte[] STRONGS = {L, R}; + static final int PREFIX_LENGTH = 2; + static final int SUFFIX_LENGTH = 2; + static final int FIXES_LENGTH = PREFIX_LENGTH + SUFFIX_LENGTH; + static final int DIRPROPS_ADD = 2; + static final int OFFSETS_SHIFT = 3; + static final int[] EMPTY_INT_ARRAY = new int[0]; + static final STextEnvironment IGNORE_ENVIRONMENT = new STextEnvironment(null, false, STextEnvironment.ORIENT_IGNORE); + + /** + * Prevent creation of a STextEngine instance + */ + private STextImpl() { + // nothing to do + } + + /* + // keep private copy of specialsCount to avoid later modification + specialsCount = features.getSpecialsCount(); + locations = new int[features.getSeparators().length() + specialsCount]; + } + */ + static long computeNextLocation(ISTextProcessor processor, STextFeatures features, String text, byte[] dirProps, int[] offsets, int[] locations, int[] state, int curPos) { + String separators = features.getSeparators(); + int separCount = separators.length(); + int specialsCount = features.getSpecialsCount(); + int len = text.length(); + int nextLocation = len; + int idxLocation = 0; + // Start with special sequences to give them precedence over simple + // separators. This may apply to cases like slash+asterisk versus slash. + for (int i = 0; i < specialsCount; i++) { + int location = locations[separCount + i]; + if (location < curPos) { + offsets = ensureRoomInOffsets(offsets); + location = processor.indexOfSpecial(features, text, dirProps, offsets, i + 1, curPos); + if (location < 0) + location = len; + locations[separCount + i] = location; + } + if (location < nextLocation) { + nextLocation = location; + idxLocation = separCount + i; + } + } + for (int i = 0; i < separCount; i++) { + int location = locations[i]; + if (location < curPos) { + location = text.indexOf(separators.charAt(i), curPos); + if (location < 0) + location = len; + locations[i] = location; + } + if (location < nextLocation) { + nextLocation = location; + idxLocation = i; + } + } + return nextLocation + (((long) idxLocation) << 32); + } + + static int getCurOrient(STextEnvironment environment, String text, byte[] dirProps) { + int orient = environment.getOrientation(); + if ((orient & STextEnvironment.ORIENT_CONTEXTUAL_LTR) == 0) { + // absolute orientation + return orient; + } + // contextual orientation + int len = text.length(); + byte dirProp; + for (int i = 0; i < len; i++) { + // In the following lines, B, L, R and AL represent bidi categories + // as defined in the Unicode Bidirectional Algorithm + // ( http://www.unicode.org/reports/tr9/ ). + // B represents the category Block Separator. + // L represents the category Left to Right character. + // R represents the category Right to Left character. + // AL represents the category Arabic Letter. + dirProp = dirProps[i]; + if (dirProp == 0) { + dirProp = Character.getDirectionality(text.charAt(i)); + if (dirProp == B) // B char resolves to L or R depending on orientation + continue; + dirProps[i] = (byte) (dirProp + DIRPROPS_ADD); + } else { + dirProp -= DIRPROPS_ADD; + } + if (dirProp == L) + return STextEnvironment.ORIENT_LTR; + if (dirProp == R || dirProp == AL) + return STextEnvironment.ORIENT_RTL; + } + // return the default orientation corresponding to the contextual orientation + return orient & 1; + } + + /** + * @see STextEngine#getCurDirection STextEngine.getCurDirection + */ + public static int getCurDirection(Object _processor, STextFeatures features, STextEnvironment environment, String text, byte[] dirProps) { + if (environment == null) + environment = STextEnvironment.DEFAULT; + if (features == null) { + if (_processor == null) + return STextFeatures.DIR_LTR; + ISTextProcessor processor; + if (_processor instanceof java.lang.String) { + processor = STextStringProcessor.getProcessor((String) _processor); + if (processor == null) + throw new IllegalArgumentException("Invalid processor type!"); //$NON-NLS-1$ + } else if (_processor instanceof ISTextProcessor) + processor = (ISTextProcessor) _processor; + else + throw new IllegalArgumentException("Invalid processor argument!"); //$NON-NLS-1$ + features = processor.getFeatures(environment); + } + int dirArabic = features.getDirArabic(); + int dirHebrew = features.getDirHebrew(); + // same direction for Arabic and Hebrew? + if (dirArabic == dirHebrew) + return dirArabic; + // check if Arabic or Hebrew letter comes first + int len = text.length(); + if (dirProps == null) + dirProps = new byte[len + 1]; + byte dirProp; + for (int i = 0; i < len; i++) { + // In the following lines, R and AL represent bidi categories + // as defined in the Unicode Bidirectional Algorithm + // ( http://www.unicode.org/reports/tr9/ ). + // R represents the category Right to Left character. + // AL represents the category Arabic Letter. + byte saveOrient = dirProps[len]; + dirProps[len] = -1; // make getDirProp return B + dirProp = getDirProp(text, dirProps, i); + dirProps[len] = saveOrient; + if (dirProp == AL) + return dirArabic; + if (dirProp == R) + return dirHebrew; + } + // found no Arabic or Hebrew character + return STextFeatures.DIR_LTR; + } + + /** + * @see STextProcessor#getDirProp STextProcessor.getDirProp + */ + public static byte getDirProp(String text, byte[] dirProps, int index) { + byte dirProp = dirProps[index]; + if (dirProp == 0) { + // In the following lines, B, L and R represent bidi categories + // as defined in the Unicode Bidirectional Algorithm + // ( http://www.unicode.org/reports/tr9/ ). + // B represents the category Block Separator. + // L represents the category Left to Right character. + // R represents the category Right to Left character. + dirProp = Character.getDirectionality(text.charAt(index)); + if (dirProp == B) { + // the last entry of dirProps contains the current component orientation + byte orient = dirProps[dirProps.length - 1]; + if (orient == -1) + return B; + dirProp = orient == STextEnvironment.ORIENT_RTL ? R : L; + } + dirProps[index] = (byte) (dirProp + DIRPROPS_ADD); + return dirProp; + } + return (byte) (dirProp - DIRPROPS_ADD); + } + + /** + * @see STextProcessor#setDirProp STextProcessor.setDirProp + */ + public static void setDirProp(byte[] dirProps, int index, byte dirProp) { + dirProps[index] = (byte) (dirProp + DIRPROPS_ADD); + } + + /** + * @see STextProcessor#processSeparator STextProcessor.processSeparator + */ + public static void processSeparator(STextFeatures features, String text, byte[] dirProps, int[] offsets, int separLocation) { + // In this method, L, R, AL, AN and EN represent bidi categories + // as defined in the Unicode Bidirectional Algorithm + // ( http://www.unicode.org/reports/tr9/ ). + // L represents the category Left to Right character. + // R represents the category Right to Left character. + // AL represents the category Arabic Letter. + // AN represents the category Arabic Number. + // EN represents the category European Number. + int len = text.length(); + // offsets[2] contains the structured text direction + if (offsets[2] == STextFeatures.DIR_RTL) { + // the structured text base direction is RTL + for (int i = separLocation - 1; i >= 0; i--) { + byte dirProp = getDirProp(text, dirProps, i); + if (dirProp == R || dirProp == AL) + return; + if (dirProp == L) { + for (int j = separLocation; j < len; j++) { + dirProp = getDirProp(text, dirProps, j); + if (dirProp == R || dirProp == AL) + return; + if (dirProp == L || dirProp == EN) { + insertMark(text, dirProps, offsets, separLocation); + return; + } + } + return; + } + } + return; + } + + // the structured text base direction is LTR + boolean doneAN = false; + boolean ignoreArabic = features.getIgnoreArabic(); + boolean ignoreHebrew = features.getIgnoreHebrew(); + if (ignoreArabic && ignoreHebrew) + return; + byte _R, _AL, _AN; + if (ignoreArabic) { + _AL = Byte.MIN_VALUE; // not a real value + _AN = Byte.MIN_VALUE; + } else { + _AL = AL; + _AN = AN; + } + if (ignoreHebrew) + _R = Byte.MIN_VALUE; + else + _R = R; + for (int i = separLocation - 1; i >= 0; i--) { + byte dirProp = getDirProp(text, dirProps, i); + if (dirProp == L) + return; + if (dirProp == _R || dirProp == _AL) { + for (int j = separLocation; j < len; j++) { + dirProp = getDirProp(text, dirProps, j); + if (dirProp == L) + return; + if (dirProp == _R || dirProp == EN || dirProp == _AL || dirProp == _AN) { + insertMark(text, dirProps, offsets, separLocation); + return; + } + } + return; + } + if (dirProp == _AN && !doneAN) { + for (int j = separLocation; j < len; j++) { + dirProp = getDirProp(text, dirProps, j); + if (dirProp == L) + return; + if (dirProp == _AL || dirProp == _AN || dirProp == _R) { + insertMark(text, dirProps, offsets, separLocation); + return; + } + } + doneAN = true; + } + } + } + + /** + * @see STextEngine#leanToFullText STextEngine.leanToFullText + */ + public static String leanToFullText(Object processor, STextFeatures features, STextEnvironment environment, String text, int[] state) { + int len = text.length(); + if (len == 0) + return text; + byte[] dirProps = new byte[len + 1]; + int[] offsets = leanToFullCommon(processor, features, environment, text, state, dirProps); + int prefixLength = offsets[1]; + int count = offsets[0] - OFFSETS_SHIFT; + if (count == 0 && prefixLength == 0) + return text; + int newLen = len + count; + if (prefixLength == 1) + newLen++; /* +1 for a mark char */ + else if (prefixLength == 2) + newLen += FIXES_LENGTH; + char[] fullChars = new char[newLen]; + int added = prefixLength; + // add marks at offsets + int direction = offsets[2]; + char curMark = MARKS[direction]; + for (int i = 0, j = OFFSETS_SHIFT; i < len; i++) { + char c = text.charAt(i); + // offsets[0] contains the number of used entries + if (j < offsets[0] && i == offsets[j]) { + fullChars[i + added] = curMark; + added++; + j++; + } + fullChars[i + added] = c; + } + if (prefixLength > 0) { /* add prefix/suffix ? */ + if (prefixLength == 1) { /* contextual orientation */ + fullChars[0] = curMark; + } else { + // When the orientation is RTL, we need to add EMBED at the + // start of the text and PDF at its end. + // However, because of a bug in Windows' handling of LRE/PDF, + // we add EMBED_PREFIX at the start and EMBED_SUFFIX at the end. + char curEmbed = EMBEDS[direction]; + fullChars[0] = curEmbed; + fullChars[1] = curMark; + fullChars[newLen - 1] = PDF; + fullChars[newLen - 2] = curMark; + } + } + return new String(fullChars); + } + + /** + * @see STextEngine#leanToFullMap STextEngine.leanToFullMap + */ + public static int[] leanToFullMap(Object processor, STextFeatures features, STextEnvironment environment, String text, int[] state) { + int len = text.length(); + if (len == 0) + return EMPTY_INT_ARRAY; + byte[] dirProps = new byte[len + 1]; + int[] offsets = leanToFullCommon(processor, features, environment, text, state, dirProps); + int prefixLength = offsets[1]; + int[] map = new int[len]; + int count = offsets[0]; // number of used entries + int added = prefixLength; + for (int pos = 0, i = OFFSETS_SHIFT; pos < len; pos++) { + if (i < count && pos == offsets[i]) { + added++; + i++; + } + map[pos] = pos + added; + } + return map; + } + + /** + * @see STextEngine#leanBidiCharOffsets STextEngine.leanBidiCharOffsets + */ + public static int[] leanBidiCharOffsets(Object processor, STextFeatures features, STextEnvironment environment, String text, int[] state) { + int len = text.length(); + if (len == 0) + return EMPTY_INT_ARRAY; + byte[] dirProps = new byte[len + 1]; + int[] offsets = leanToFullCommon(processor, features, environment, text, state, dirProps); + // offsets[0] contains the number of used entries + int count = offsets[0] - OFFSETS_SHIFT; + int[] result = new int[count]; + System.arraycopy(offsets, OFFSETS_SHIFT, result, 0, count); + return result; + } + + static int[] leanToFullCommon(Object _processor, STextFeatures features, STextEnvironment environment, String text, int[] state, byte[] dirProps) { + ISTextProcessor processor; + if (_processor instanceof java.lang.String) { + processor = STextStringProcessor.getProcessor((String) _processor); + if (processor == null) + throw new IllegalArgumentException("Invalid processor type!"); //$NON-NLS-1$ + } else if (_processor instanceof ISTextProcessor) + processor = (ISTextProcessor) _processor; + else + throw new IllegalArgumentException("Invalid processor argument!"); //$NON-NLS-1$ + if (environment == null) + environment = STextEnvironment.DEFAULT; + if (features == null) + features = processor.getFeatures(environment); + if (state == null) { + state = new int[1]; + state[0] = STextEngine.STATE_INITIAL; + } + int len = text.length(); + // dirProps: 1 byte for each char in text, + 1 byte = current orientation + int orient = getCurOrient(environment, text, dirProps); + dirProps[len] = (byte) orient; + int separCount = features.getSeparators().length(); + int direction = getCurDirection(processor, features, environment, text, dirProps); + // current position + int curPos = 0; + // offsets of marks to add. Entry 0 is the number of used slots; + // entry 1 is reserved to pass prefixLength. + // entry 2 is reserved to pass direction.. + int[] offsets = new int[20]; + offsets[0] = OFFSETS_SHIFT; + offsets[2] = direction; + // initialize locations + int[] locations = new int[separCount + features.getSpecialsCount()]; + for (int i = 0, k = locations.length; i < k; i++) { + locations[i] = -1; + } + if (state[0] > STextEngine.STATE_INITIAL) { + offsets = ensureRoomInOffsets(offsets); + int initState = state[0]; + state[0] = STextEngine.STATE_INITIAL; + curPos = processor.processSpecial(features, text, dirProps, offsets, state, initState, -1); + } + while (true) { + // location of next token to handle + int nextLocation; + // index of next token to handle (if < separCount, this is a separator; otherwise a special case + int idxLocation; + long res = computeNextLocation(processor, features, text, dirProps, offsets, locations, state, curPos); + nextLocation = (int) (res & 0x00000000FFFFFFFF); /* low word */ + if (nextLocation >= len) + break; + idxLocation = (int) (res >> 32); /* high word */ + if (idxLocation < separCount) { + offsets = ensureRoomInOffsets(offsets); + processSeparator(features, text, dirProps, offsets, nextLocation); + curPos = nextLocation + 1; + } else { + offsets = ensureRoomInOffsets(offsets); + idxLocation -= (separCount - 1); // because caseNumber starts from 1 + curPos = processor.processSpecial(features, text, dirProps, offsets, state, idxLocation, nextLocation); + } + } + if (orient == STextEnvironment.ORIENT_IGNORE) + offsets[1] = 0; + else { + // recompute orient since it may have changed if contextual + orient = getCurOrient(environment, text, dirProps); + dirProps[len] = (byte) orient; + if (orient == direction && orient != STextEnvironment.ORIENT_UNKNOWN) + offsets[1] = 0; + else if ((environment.getOrientation() & STextEnvironment.ORIENT_CONTEXTUAL_LTR) != 0) + offsets[1] = 1; + else + offsets[1] = 2; + } + return offsets; + } + + /** + * @see STextEngine#fullToLeanText STextEngine.fullToLeanText + */ + public static String fullToLeanText(Object _processor, STextFeatures features, STextEnvironment environment, String text, int[] state) { + if (text.length() == 0) + return text; + ISTextProcessor processor; + if (_processor instanceof java.lang.String) { + processor = STextStringProcessor.getProcessor((String) _processor); + if (processor == null) + throw new IllegalArgumentException("Invalid processor type!"); //$NON-NLS-1$ + } else if (_processor instanceof ISTextProcessor) + processor = (ISTextProcessor) _processor; + else + throw new IllegalArgumentException("Invalid processor argument!"); //$NON-NLS-1$ + if (environment == null) + environment = STextEnvironment.DEFAULT; + if (features == null) + features = processor.getFeatures(environment); + if (state == null) { + state = new int[1]; + state[0] = STextEngine.STATE_INITIAL; + } + int dir = getCurDirection(processor, features, environment, text, null); + char curMark = MARKS[dir]; + char curEmbed = EMBEDS[dir]; + int i; // used as loop index + // remove any prefix and leading mark + int lenText = text.length(); + for (i = 0; i < lenText; i++) { + char c = text.charAt(i); + if (c != curEmbed && c != curMark) + break; + } + if (i > 0) { // found at least one prefix or leading mark + text = text.substring(i); + lenText = text.length(); + } + // remove any suffix and trailing mark + for (i = lenText - 1; i >= 0; i--) { + char c = text.charAt(i); + if (c != PDF && c != curMark) + break; + } + if (i < 0) // only suffix and trailing marks, no real data + return EMPTY_STRING; + if (i < (lenText - 1)) { // found at least one suffix or trailing mark + text = text.substring(0, i + 1); + lenText = text.length(); + } + char[] chars = text.toCharArray(); + // remove marks from chars + int cnt = 0; + for (i = 0; i < lenText; i++) { + char c = chars[i]; + if (c == curMark) + cnt++; + else if (cnt > 0) + chars[i - cnt] = c; + } + String lean = new String(chars, 0, lenText - cnt); + String full = leanToFullText(processor, features, IGNORE_ENVIRONMENT, lean, state); + if (full.equals(text)) + return lean; + + // There are some marks in full which are not in text and/or vice versa. + // We need to add to lean any mark appearing in text and not in full. + // The completed lean can never be longer than text itself. + char[] newChars = new char[lenText]; + char cFull, cText; + int idxFull, idxText, idxLean, newCharsPos; + int lenFull = full.length(); + idxFull = idxText = idxLean = newCharsPos = 0; + while (idxText < lenText && idxFull < lenFull) { + cFull = full.charAt(idxFull); + cText = text.charAt(idxText); + if (cFull == cText) { /* chars are equal, proceed */ + if (cFull != curMark) + newChars[newCharsPos++] = chars[idxLean++]; + idxText++; + idxFull++; + continue; + } + if (cFull == curMark) { /* extra Mark in full text */ + idxFull++; + continue; + } + if (cText == curMark) { /* extra Mark in source full text */ + idxText++; + // idxText-2 always >= 0 since leading Marks were removed from text + if (text.charAt(idxText - 2) == curMark) + continue; // ignore successive Marks in text after the first one + newChars[newCharsPos++] = curMark; + continue; + } + // we should never get here (extra char which is not a Mark) + throw new IllegalStateException("Internal error: extra character not a Mark."); //$NON-NLS-1$ + } + if (idxText < lenText) /* full ended before text - this should never happen since + we removed all marks and PDFs at the end of text */ + throw new IllegalStateException("Internal error: unexpected EOL."); //$NON-NLS-1$ + + lean = new String(newChars, 0, newCharsPos); + return lean; + } + + /** + * @see STextEngine#fullToLeanMap STextEngine.fullToLeanMap + */ + public static int[] fullToLeanMap(Object processor, STextFeatures features, STextEnvironment environment, String full, int[] state) { + int lenFull = full.length(); + if (lenFull == 0) + return EMPTY_INT_ARRAY; + String lean = fullToLeanText(processor, features, environment, full, state); + int lenLean = lean.length(); + int dir = getCurDirection(processor, features, environment, lean, null); + char curMark = MARKS[dir]; + char curEmbed = EMBEDS[dir]; + int[] map = new int[lenFull]; + int idxFull, idxLean; + // skip any prefix and leading mark + for (idxFull = 0; idxFull < lenFull; idxFull++) { + char c = full.charAt(idxFull); + if (c != curEmbed && c != curMark) + break; + map[idxFull] = -1; + } + // lean must be a subset of Full, so we only check on iLean < leanLen + for (idxLean = 0; idxLean < lenLean; idxFull++) { + if (full.charAt(idxFull) == lean.charAt(idxLean)) { + map[idxFull] = idxLean; + idxLean++; + } else + map[idxFull] = -1; + } + for (; idxFull < lenFull; idxFull++) + map[idxFull] = -1; + return map; + } + + /** + * @see STextEngine#fullBidiCharOffsets STextEngine.fullBidiCharOffsets + */ + public static int[] fullBidiCharOffsets(Object processor, STextFeatures features, STextEnvironment environment, String full, int[] state) { + int lenFull = full.length(); + if (lenFull == 0) + return EMPTY_INT_ARRAY; + String lean = fullToLeanText(processor, features, environment, full, state); + int[] offsets = new int[20]; + offsets[0] = OFFSETS_SHIFT; + int lenLean = lean.length(); + int idxLean, idxFull; + // lean must be a subset of Full, so we only check on iLean < leanLen + for (idxLean = idxFull = 0; idxLean < lenLean; idxFull++) { + if (full.charAt(idxFull) == lean.charAt(idxLean)) + idxLean++; + else { + offsets = ensureRoomInOffsets(offsets); + insertMark(lean, null, offsets, idxFull); + } + } + for (; idxFull < lenFull; idxFull++) { + offsets = ensureRoomInOffsets(offsets); + insertMark(lean, null, offsets, idxFull); + } + int[] result = new int[offsets[0] - OFFSETS_SHIFT]; + System.arraycopy(offsets, OFFSETS_SHIFT, result, 0, result.length); + return result; + } + + static int[] ensureRoomInOffsets(int[] offsets) { + // make sure + if ((offsets.length - offsets[0]) < 3) { + int[] newOffsets = new int[offsets.length * 2]; + System.arraycopy(offsets, 0, newOffsets, 0, offsets[0]); + return newOffsets; + } + return offsets; + } + + /** + * @see STextProcessor#insertMark STextProcessor.insertMark + */ + public static void insertMark(String text, byte[] dirProps, int[] offsets, int offset) { + int count = offsets[0];// number of used entries + int index = count - 1; // index of greatest member <= offset + // look up after which member the new offset should be inserted + while (index >= OFFSETS_SHIFT) { + int wrkOffset = offsets[index]; + if (offset > wrkOffset) + break; + if (offset == wrkOffset) + return; // avoid duplicates + index--; + } + index++; // index now points at where to insert + int length = count - index; // number of members to move up + if (length > 0) // shift right all members greater than offset + System.arraycopy(offsets, index, offsets, index + 1, length); + offsets[index] = offset; + offsets[0]++; // number of used entries + // if the offset is 0, adding a mark does not change anything + if (dirProps == null || offset < 1) + return; + + byte dirProp = getDirProp(text, dirProps, offset); + // if the current char is a strong one or a digit, we change the + // dirProp of the previous char to account for the inserted mark. + // In the following lines, L, R, AL, AN and EN represent bidi categories + // as defined in the Unicode Bidirectional Algorithm + // ( http://www.unicode.org/reports/tr9/ ). + // L represents the category Left to Right character. + // R represents the category Right to Left character. + // AL represents the category Arabic Letter. + // AN represents the category Arabic Number. + // EN represents the category European Number. + if (dirProp == L || dirProp == R || dirProp == AL || dirProp == EN || dirProp == AN) + index = offset - 1; + else + // if the current char is a neutral, we change its own dirProp + index = offset; + + int dir = offsets[2]; // current structured text direction + setDirProp(dirProps, index, STRONGS[dir]); + return; + } + +} Index: src/org/eclipse/equinox/bidi/internal/STextSingle.java =================================================================== RCS file: src/org/eclipse/equinox/bidi/internal/STextSingle.java diff -N src/org/eclipse/equinox/bidi/internal/STextSingle.java --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ src/org/eclipse/equinox/bidi/internal/STextSingle.java 1 Jan 1970 00:00:00 -0000 @@ -0,0 +1,52 @@ +/******************************************************************************* + * Copyright (c) 2010, 2011 IBM Corporation and others. + * All rights reserved. This program and the accompanying materials + * are made available under the terms of the Eclipse Public License v1.0 + * which accompanies this distribution, and is available at + * http://www.eclipse.org/legal/epl-v10.html + * + * Contributors: + * IBM Corporation - initial API and implementation + ******************************************************************************/ +package org.eclipse.equinox.bidi.internal; + +import org.eclipse.equinox.bidi.custom.STextFeatures; +import org.eclipse.equinox.bidi.custom.STextProcessor; + +/** + * STextSingle is a processor for structured text + * composed of two parts separated by a separator. + * The first occurrence of the separator delimits the end of the first part + * and the start of the second part. Further occurrences of the separator, + * if any, are treated like regular characters of the second text part. + * The processor makes sure that the text be presented in the form + * (assuming that the equal sign is the separator): + * 
+ *  part1=part2
+ *
+ * The {@link STextFeatures#getSeparators separators} + * field in the {@link STextFeatures features} + * of this processor should contain exactly one character. + * Additional characters will be ignored. + * + * @author Matitiahu Allouche + */ +public abstract class STextSingle extends STextProcessor { + + /** + * This method locates occurrences of the separator. + */ + public int indexOfSpecial(STextFeatures features, String text, byte[] dirProps, int[] offsets, int caseNumber, int fromIndex) { + return text.indexOf(features.getSeparators().charAt(0), fromIndex); + } + + /** + * This method inserts a mark before the separator if needed and + * skips to the end of the source string. + */ + public int processSpecial(STextFeatures features, String text, byte[] dirProps, int[] offsets, int[] state, int caseNumber, int separLocation) { + STextProcessor.processSeparator(features, text, dirProps, offsets, separLocation); + return text.length(); + } + +} Index: src/org/eclipse/equinox/bidi/internal/STextTypesCollector.java =================================================================== RCS file: src/org/eclipse/equinox/bidi/internal/STextTypesCollector.java diff -N src/org/eclipse/equinox/bidi/internal/STextTypesCollector.java --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ src/org/eclipse/equinox/bidi/internal/STextTypesCollector.java 1 Jan 1970 00:00:00 -0000 @@ -0,0 +1,111 @@ +/******************************************************************************* + * Copyright (c) 2010, 2011 IBM Corporation and others. + * All rights reserved. This program and the accompanying materials + * are made available under the terms of the Eclipse Public License v1.0 + * which accompanies this distribution, and is available at + * http://www.eclipse.org/legal/epl-v10.html + * + * Contributors: + * IBM Corporation - initial API and implementation + *******************************************************************************/ +package org.eclipse.equinox.bidi.internal; + +import java.util.HashMap; +import java.util.Map; +import org.eclipse.core.runtime.*; +import org.eclipse.equinox.bidi.custom.ISTextProcessor; + +public class STextTypesCollector implements IRegistryEventListener { + + private static final String EXT_POINT = "org.eclipse.equinox.bidi.bidiTypes"; //$NON-NLS-1$ + + private static final String CE_NAME = "typeDescription"; //$NON-NLS-1$ + private static final String ATTR_TYPE = "type"; //$NON-NLS-1$ + private static final String ATTR_PROCESSOR = "class"; //$NON-NLS-1$ + + private Map types; + private Map factories; + + static private STextTypesCollector instance = new STextTypesCollector(); + + private STextTypesCollector() { + IExtensionRegistry registry = RegistryFactory.getRegistry(); + registry.addListener(this, EXT_POINT); + } + + static public STextTypesCollector getInstance() { + return instance; + } + + public String[] getTypes() { + if (types == null) + read(); + int size = types.size(); + String[] result = new String[size]; + types.keySet().toArray(result); + return result; + } + + public ISTextProcessor getProcessor(String type) { + if (types == null) + read(); + Object processor = types.get(type); + if (processor instanceof ISTextProcessor) + return (ISTextProcessor) processor; + return null; + } + + private void read() { + if (types == null) + types = new HashMap(); + else + types.clear(); + + if (factories == null) + factories = new HashMap(); + else + factories.clear(); + + IExtensionRegistry registry = RegistryFactory.getRegistry(); + IExtensionPoint extPoint = registry.getExtensionPoint(EXT_POINT); + IExtension[] extensions = extPoint.getExtensions(); + + for (int i = 0; i < extensions.length; i++) { + IConfigurationElement[] confElements = extensions[i].getConfigurationElements(); + for (int j = 0; j < confElements.length; j++) { + if (CE_NAME != confElements[j].getName()) + STextActivator.logError("BiDi types: unexpected element name " + confElements[j].getName(), new IllegalArgumentException()); //$NON-NLS-1$ + String type = confElements[j].getAttribute(ATTR_TYPE); + Object processor; + try { + processor = confElements[j].createExecutableExtension(ATTR_PROCESSOR); + } catch (CoreException e) { + STextActivator.logError("BiDi types: unable to create processor for " + type, e); //$NON-NLS-1$ + continue; + } + types.put(type, processor); + factories.put(type, confElements[j]); + } + } + } + + public void added(IExtension[] extensions) { + types = null; + factories = null; + } + + public void removed(IExtension[] extensions) { + types = null; + factories = null; + } + + public void added(IExtensionPoint[] extensionPoints) { + types = null; + factories = null; + } + + public void removed(IExtensionPoint[] extensionPoints) { + types = null; + factories = null; + } +} Index: src/org/eclipse/equinox/bidi/internal/consumable/BidiComplexComma.java =================================================================== RCS file: src/org/eclipse/equinox/bidi/internal/consumable/BidiComplexComma.java diff -N src/org/eclipse/equinox/bidi/internal/consumable/BidiComplexComma.java --- src/org/eclipse/equinox/bidi/internal/consumable/BidiComplexComma.java 17 May 2011 19:56:16 -0000 1.3 +++ /dev/null 1 Jan 1970 00:00:00 -0000 @@ -1,36 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2010, 2011 IBM Corporation and others. - * All rights reserved. This program and the accompanying materials - * are made available under the terms of the Eclipse Public License v1.0 - * which accompanies this distribution, and is available at - * http://www.eclipse.org/legal/epl-v10.html - * - * Contributors: - * IBM Corporation - initial API and implementation - ******************************************************************************/ -package org.eclipse.equinox.bidi.internal.consumable; - -import org.eclipse.equinox.bidi.BidiComplexEnvironment; -import org.eclipse.equinox.bidi.custom.BidiComplexFeatures; -import org.eclipse.equinox.bidi.custom.BidiComplexProcessor; - -/** - * Processor adapted to processing comma-delimited lists, such as: - * 
- *    part1,part2,part3
- *
- */ -public class BidiComplexComma extends BidiComplexProcessor { - static final BidiComplexFeatures FEATURES = new BidiComplexFeatures(",", 0, -1, -1, false, false); //$NON-NLS-1$ - - /** - * This method retrieves the features specific to this processor. - * - * @return features with one separator (comma), no special cases, - * LTR direction for Arabic and Hebrew, and support for both. - */ - public BidiComplexFeatures getFeatures(BidiComplexEnvironment env) { - return FEATURES; - } - -} Index: src/org/eclipse/equinox/bidi/internal/consumable/BidiComplexEmail.java =================================================================== RCS file: src/org/eclipse/equinox/bidi/internal/consumable/BidiComplexEmail.java diff -N src/org/eclipse/equinox/bidi/internal/consumable/BidiComplexEmail.java --- src/org/eclipse/equinox/bidi/internal/consumable/BidiComplexEmail.java 17 May 2011 19:56:16 -0000 1.3 +++ /dev/null 1 Jan 1970 00:00:00 -0000 @@ -1,48 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2010, 2011 IBM Corporation and others. - * All rights reserved. This program and the accompanying materials - * are made available under the terms of the Eclipse Public License v1.0 - * which accompanies this distribution, and is available at - * http://www.eclipse.org/legal/epl-v10.html - * - * Contributors: - * IBM Corporation - initial API and implementation - ******************************************************************************/ -package org.eclipse.equinox.bidi.internal.consumable; - -import org.eclipse.equinox.bidi.BidiComplexEnvironment; -import org.eclipse.equinox.bidi.custom.BidiComplexFeatures; -import org.eclipse.equinox.bidi.internal.BidiComplexDelimsEsc; - -/** - * Processor adapted to processing e-mail addresses. - */ -public class BidiComplexEmail extends BidiComplexDelimsEsc { - static final int LTR = BidiComplexFeatures.DIR_LTR; - static final int RTL = BidiComplexFeatures.DIR_RTL; - static final BidiComplexFeatures MIRRORED = new BidiComplexFeatures("<>.:,;@", 2, RTL, LTR, false, false); //$NON-NLS-1$ - static final BidiComplexFeatures NOT_MIRRORED = new BidiComplexFeatures("<>.:,;@", 2, LTR, LTR, false, false); //$NON-NLS-1$ - - /** - * This method retrieves the features specific to this processor. - * - * @return features with separators "<>.:,;@", 2 special cases, - * LTR direction for Arabic when the GUI is not mirrored, - * RTL direction for Arabic when the GUI is mirrored, - * LTR direction for Hebrew in all cases, - * and support for both Arabic and Hebrew. - */ - public BidiComplexFeatures getFeatures(BidiComplexEnvironment env) { - if (env == null) - env = BidiComplexEnvironment.DEFAULT; - return env.getMirrored() ? MIRRORED : NOT_MIRRORED; - } - - /** - * @return parentheses and quotation marks as delimiters. - */ - protected String getDelimiters() { - return "()\"\""; //$NON-NLS-1$ - } - -} Index: src/org/eclipse/equinox/bidi/internal/consumable/BidiComplexFile.java =================================================================== RCS file: src/org/eclipse/equinox/bidi/internal/consumable/BidiComplexFile.java diff -N src/org/eclipse/equinox/bidi/internal/consumable/BidiComplexFile.java --- src/org/eclipse/equinox/bidi/internal/consumable/BidiComplexFile.java 17 May 2011 19:56:16 -0000 1.3 +++ /dev/null 1 Jan 1970 00:00:00 -0000 @@ -1,33 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2010, 2011 IBM Corporation and others. - * All rights reserved. This program and the accompanying materials - * are made available under the terms of the Eclipse Public License v1.0 - * which accompanies this distribution, and is available at - * http://www.eclipse.org/legal/epl-v10.html - * - * Contributors: - * IBM Corporation - initial API and implementation - ******************************************************************************/ -package org.eclipse.equinox.bidi.internal.consumable; - -import org.eclipse.equinox.bidi.BidiComplexEnvironment; -import org.eclipse.equinox.bidi.custom.BidiComplexFeatures; -import org.eclipse.equinox.bidi.custom.BidiComplexProcessor; - -/** - * Processor adapted to processing directory and file paths. - */ -public class BidiComplexFile extends BidiComplexProcessor { - static final BidiComplexFeatures FEATURES = new BidiComplexFeatures(":/\\.", 0, -1, -1, false, false); //$NON-NLS-1$ - - /** - * This method retrieves the features specific to this processor. - * - * @return features with separators ":/\.", no special cases, - * LTR direction for Arabic and Hebrew, and support for both. - */ - public BidiComplexFeatures getFeatures(BidiComplexEnvironment env) { - return FEATURES; - } - -} Index: src/org/eclipse/equinox/bidi/internal/consumable/BidiComplexJava.java =================================================================== RCS file: src/org/eclipse/equinox/bidi/internal/consumable/BidiComplexJava.java diff -N src/org/eclipse/equinox/bidi/internal/consumable/BidiComplexJava.java --- src/org/eclipse/equinox/bidi/internal/consumable/BidiComplexJava.java 17 May 2011 19:56:16 -0000 1.3 +++ /dev/null 1 Jan 1970 00:00:00 -0000 @@ -1,134 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2010, 2011 IBM Corporation and others. - * All rights reserved. This program and the accompanying materials - * are made available under the terms of the Eclipse Public License v1.0 - * which accompanies this distribution, and is available at - * http://www.eclipse.org/legal/epl-v10.html - * - * Contributors: - * IBM Corporation - initial API and implementation - ******************************************************************************/ -package org.eclipse.equinox.bidi.internal.consumable; - -import org.eclipse.equinox.bidi.BidiComplexEngine; -import org.eclipse.equinox.bidi.BidiComplexEnvironment; -import org.eclipse.equinox.bidi.custom.BidiComplexFeatures; -import org.eclipse.equinox.bidi.custom.BidiComplexProcessor; - -/** - * BidiComplexJava is a processor for complex expressions - * composed of Java statements. Such a complex expression may span - * multiple lines. - *

- * In applications like an editor where parts of the text might be modified - * while other parts are not, the user may want to call - * {@link BidiComplexEngine#leanToFullText leanToFullText} - * separately on each line and save the initial state of each line (this is - * the final state of the previous line which can be retrieved using - * the value returned in the first element of the state argument). - * If both the content - * of a line and its initial state have not changed, the user can be sure that - * the last full text computed for this line has not changed either. - * - * @see BidiComplexEngine#leanToFullText explanation of state in leanToFullText - * - * @author Matitiahu Allouche - */ -public class BidiComplexJava extends BidiComplexProcessor { - private static final byte WS = Character.DIRECTIONALITY_WHITESPACE; - static final BidiComplexFeatures FEATURES = new BidiComplexFeatures("[](){}.+-<>=~!&*/%^|?:,;\t", 4, -1, -1, false, false); //$NON-NLS-1$ - static final String lineSep = BidiComplexEnvironment.getLineSep(); - - /** - * This method retrieves the features specific to this processor. - * - * @return features with separators "[](){}.+-<>=~!&/*%^|?:,;\t", - * 4 special cases, LTR direction for Arabic and Hebrew, - * and support for both. - */ - public BidiComplexFeatures getFeatures(BidiComplexEnvironment env) { - return FEATURES; - } - - /** - * This method looks for occurrences of 4 special strings: - *

    - *
  1. spaces
    2. - *
  2. literals starting with quotation mark
    3. - *
  3. comments starting with slash-asterisk
    4. - *
  4. comments starting with slash-slash
    5. - *
- */ - public int indexOfSpecial(BidiComplexFeatures features, String text, byte[] dirProps, int[] offsets, int caseNumber, int fromIndex) { - switch (caseNumber) { - case 1 : /* space */ - return text.indexOf(' ', fromIndex); - case 2 : /* literal */ - return text.indexOf('"', fromIndex); - case 3 : /* slash-aster comment */ - return text.indexOf("/*", fromIndex); //$NON-NLS-1$ - case 4 : /* slash-slash comment */ - return text.indexOf("//", fromIndex); //$NON-NLS-1$ - } - // we should never get here - return -1; - } - - /** - * This method processes the 4 special cases as follows. - *
    - *
  1. skip the run of spaces
    2. - *
  2. look for a matching quotation mark and skip until after it
    3. - *
  3. skip until after the closing asterisk-slash
    4. - *
  4. skip until after a line separator
    5. - *
- */ - public int processSpecial(BidiComplexFeatures features, String text, byte[] dirProps, int[] offsets, int[] state, int caseNumber, int separLocation) { - int location, counter, i; - - BidiComplexProcessor.processSeparator(features, text, dirProps, offsets, separLocation); - switch (caseNumber) { - case 1 : /* space */ - separLocation++; - while (separLocation < text.length() && text.charAt(separLocation) == ' ') { - BidiComplexProcessor.setDirProp(dirProps, separLocation, WS); - separLocation++; - } - return separLocation; - case 2 : /* literal */ - location = separLocation + 1; - while (true) { - location = text.indexOf('"', location); - if (location < 0) - return text.length(); - for (counter = 0, i = location - 1; text.charAt(i) == '\\'; i--) { - counter++; - } - location++; - if ((counter & 1) == 0) - return location; - } - case 3 : /* slash-aster comment */ - if (separLocation < 0) { // continuation line - location = 0; - } else - location = separLocation + 2; // skip the opening slash-aster - location = text.indexOf("*/", location); //$NON-NLS-1$ - if (location < 0) { - state[0] = caseNumber; - return text.length(); - } - // we need to call processSeparator since text may follow the - // end of comment immediately without even a space - BidiComplexProcessor.processSeparator(features, text, dirProps, offsets, location); - return location + 2; - case 4 : /* slash-slash comment */ - location = text.indexOf(lineSep, separLocation + 2); - if (location < 0) - return text.length(); - return location + lineSep.length(); - } - // we should never get here - return text.length(); - } -} Index: src/org/eclipse/equinox/bidi/internal/consumable/BidiComplexMath.java =================================================================== RCS file: src/org/eclipse/equinox/bidi/internal/consumable/BidiComplexMath.java diff -N src/org/eclipse/equinox/bidi/internal/consumable/BidiComplexMath.java --- src/org/eclipse/equinox/bidi/internal/consumable/BidiComplexMath.java 17 May 2011 19:56:16 -0000 1.3 +++ /dev/null 1 Jan 1970 00:00:00 -0000 @@ -1,35 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2010, 2011 IBM Corporation and others. - * All rights reserved. This program and the accompanying materials - * are made available under the terms of the Eclipse Public License v1.0 - * which accompanies this distribution, and is available at - * http://www.eclipse.org/legal/epl-v10.html - * - * Contributors: - * IBM Corporation - initial API and implementation - ******************************************************************************/ -package org.eclipse.equinox.bidi.internal.consumable; - -import org.eclipse.equinox.bidi.BidiComplexEnvironment; -import org.eclipse.equinox.bidi.custom.BidiComplexFeatures; -import org.eclipse.equinox.bidi.custom.BidiComplexProcessor; - -/** - * Processor adapted to processing arithmetic expressions with right-to-left - * base direction. - */ -public class BidiComplexMath extends BidiComplexProcessor { - static final int RTL = BidiComplexFeatures.DIR_RTL; - static final BidiComplexFeatures FEATURES = new BidiComplexFeatures("+-/*()=", 0, RTL, RTL, false, false); //$NON-NLS-1$ - - /** - * This method retrieves the features specific to this processor. - * - * @return features with separators "+-/*()=", no special cases, - * RTL direction for Arabic and Hebrew, and support for both. - */ - public BidiComplexFeatures getFeatures(BidiComplexEnvironment env) { - return FEATURES; - } - -} Index: src/org/eclipse/equinox/bidi/internal/consumable/BidiComplexProperty.java =================================================================== RCS file: src/org/eclipse/equinox/bidi/internal/consumable/BidiComplexProperty.java diff -N src/org/eclipse/equinox/bidi/internal/consumable/BidiComplexProperty.java --- src/org/eclipse/equinox/bidi/internal/consumable/BidiComplexProperty.java 17 May 2011 19:56:16 -0000 1.3 +++ /dev/null 1 Jan 1970 00:00:00 -0000 @@ -1,37 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2010, 2011 IBM Corporation and others. - * All rights reserved. This program and the accompanying materials - * are made available under the terms of the Eclipse Public License v1.0 - * which accompanies this distribution, and is available at - * http://www.eclipse.org/legal/epl-v10.html - * - * Contributors: - * IBM Corporation - initial API and implementation - ******************************************************************************/ -package org.eclipse.equinox.bidi.internal.consumable; - -import org.eclipse.equinox.bidi.BidiComplexEnvironment; -import org.eclipse.equinox.bidi.custom.BidiComplexFeatures; -import org.eclipse.equinox.bidi.internal.BidiComplexSingle; - -/** - * Processor adapted to processing property file statements. - * It expects the following string format: - * 
- *    name=value
- *
- */ -public class BidiComplexProperty extends BidiComplexSingle { - static final BidiComplexFeatures FEATURES = new BidiComplexFeatures("=", 1, -1, -1, false, false); //$NON-NLS-1$ - - /** - * This method retrieves the features specific to this processor. - * - * @return features with one separator (equal sign), 1 special case, - * LTR direction for Arabic and Hebrew, and support for both. - */ - public BidiComplexFeatures getFeatures(BidiComplexEnvironment env) { - return FEATURES; - } - -} Index: src/org/eclipse/equinox/bidi/internal/consumable/BidiComplexRegex.java =================================================================== RCS file: src/org/eclipse/equinox/bidi/internal/consumable/BidiComplexRegex.java diff -N src/org/eclipse/equinox/bidi/internal/consumable/BidiComplexRegex.java --- src/org/eclipse/equinox/bidi/internal/consumable/BidiComplexRegex.java 17 May 2011 19:56:16 -0000 1.3 +++ /dev/null 1 Jan 1970 00:00:00 -0000 @@ -1,221 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2010, 2011 IBM Corporation and others. - * All rights reserved. This program and the accompanying materials - * are made available under the terms of the Eclipse Public License v1.0 - * which accompanies this distribution, and is available at - * http://www.eclipse.org/legal/epl-v10.html - * - * Contributors: - * IBM Corporation - initial API and implementation - ******************************************************************************/ -package org.eclipse.equinox.bidi.internal.consumable; - -import org.eclipse.equinox.bidi.BidiComplexEngine; -import org.eclipse.equinox.bidi.BidiComplexEnvironment; -import org.eclipse.equinox.bidi.custom.*; - -/** - * BidiComplexRegex is a processor for regular expressions. - * Such expressions may span multiple lines. - *

- * In applications like an editor where parts of the text might be modified - * while other parts are not, the user may want to call - * {@link BidiComplexEngine#leanToFullText leanToFullText} - * separately on each line and save the initial state of each line (this is - * the final state of the previous line which can be retrieved using - * the value returned in the first element of the state argument). - * If both the content - * of a line and its initial state have not changed, the user can be sure that - * the last full text computed for this line has not changed either. - * - * @see BidiComplexEngine#leanToFullText explanation of state in leanToFullText - * - * @author Matitiahu Allouche - */ -public class BidiComplexRegex extends BidiComplexProcessor { - static final String[] startStrings = {"", /* 0 *//* dummy *///$NON-NLS-1$ - "(?#", /* 1 *//* comment (?#...) *///$NON-NLS-1$ - "(?<", /* 2 *//* named group (? *///$NON-NLS-1$ - "(?'", /* 3 *//* named group (?'name' *///$NON-NLS-1$ - "(?(<", /* 4 *//* conditional named back reference (?() *///$NON-NLS-1$ - "(?('", /* 5 *//* conditional named back reference (?('name') *///$NON-NLS-1$ - "(?(", /* 6 *//* conditional named back reference (?(name) *///$NON-NLS-1$ - "(?&", /* 7 *//* named parentheses reference (?&name) *///$NON-NLS-1$ - "(?P<", /* 8 *//* named group (?P *///$NON-NLS-1$ - "\\k<", /* 9 *//* named back reference \k *///$NON-NLS-1$ - "\\k'", /* 10 *//* named back reference \k'name' *///$NON-NLS-1$ - "\\k{", /* 11 *//* named back reference \k{name} *///$NON-NLS-1$ - "(?P=", /* 12 *//* named back reference (?P=name) *///$NON-NLS-1$ - "\\g{", /* 13 *//* named back reference \g{name} *///$NON-NLS-1$ - "\\g<", /* 14 *//* subroutine call \g *///$NON-NLS-1$ - "\\g'", /* 15 *//* subroutine call \g'name' *///$NON-NLS-1$ - "(?(R&", /* 16 *//* named back reference recursion (?(R&name) *///$NON-NLS-1$ - "\\Q" /* 17 *//* quoted sequence \Q...\E *///$NON-NLS-1$ - }; - static final char[] endChars = { - // 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 - '.', ')', '>', '\'', ')', ')', ')', ')', '>', '>', '\'', '}', ')', '}', '>', '\'', ')'}; - static final int numberOfStrings = startStrings.length; /* 18 */ - static final int maxSpecial = numberOfStrings; - static final BidiComplexFeatures FEATURES = new BidiComplexFeatures(null, maxSpecial, -1, -1, false, false); - static final byte L = Character.DIRECTIONALITY_LEFT_TO_RIGHT; - static final byte R = Character.DIRECTIONALITY_RIGHT_TO_LEFT; - static final byte AL = Character.DIRECTIONALITY_RIGHT_TO_LEFT_ARABIC; - static final byte AN = Character.DIRECTIONALITY_ARABIC_NUMBER; - static final byte EN = Character.DIRECTIONALITY_EUROPEAN_NUMBER; - - /** - * This method retrieves the features specific to this processor. - * - * @see IBidiComplexProcessor#getFeatures - * - * @return features with no separators , special cases for each kind of - * regular expression syntactic string, - * LTR direction for Arabic and Hebrew, and support for both. - */ - public BidiComplexFeatures getFeatures(BidiComplexEnvironment env) { - return FEATURES; - } - - /** - * This method locates occurrences of the syntactic strings and of - * R, AL, EN, AN characters. - */ - public int indexOfSpecial(BidiComplexFeatures features, String text, byte[] dirProps, int[] offsets, int caseNumber, int fromIndex) { - // In this method, L, R, AL, AN and EN represent bidi categories - // as defined in the Unicode Bidirectional Algorithm - // ( http://www.unicode.org/reports/tr9/ ). - // L represents the category Left to Right character. - // R represents the category Right to Left character. - // AL represents the category Arabic Letter. - // AN represents the category Arabic Number. - // EN represents the category European Number. - byte dirProp; - - if (caseNumber < numberOfStrings) { - /* 1 *//* comment (?#...) */ - /* 2 *//* named group (? */ - /* 3 *//* named group (?'name' */ - /* 4 *//* conditional named back reference (?(name) */ - /* 5 *//* conditional named back reference (?() */ - /* 6 *//* conditional named back reference (?('name') */ - /* 7 *//* named parentheses reference (?&name) */ - /* 8 *//* named group (?P */ - /* 9 *//* named back reference \k */ - /* 10 *//* named back reference \k'name' */ - /* 11 *//* named back reference \k{name} */ - /* 12 *//* named back reference (?P=name) */ - /* 13 *//* named back reference \g{name} */ - /* 14 *//* subroutine call \g */ - /* 15 *//* subroutine call \g'name' */ - /* 16 *//* named back reference recursion (?(R&name) */ - /* 17 *//* quoted sequence \Q...\E */ - return text.indexOf(startStrings[caseNumber], fromIndex); - } - // there never is a need for a mark before the first char - if (fromIndex <= 0) - fromIndex = 1; - // look for R, AL, AN, EN which are potentially needing a mark - for (; fromIndex < text.length(); fromIndex++) { - dirProp = BidiComplexProcessor.getDirProp(text, dirProps, fromIndex); - // R and AL will always be examined using processSeparator() - if (dirProp == R || dirProp == AL) - return fromIndex; - - if (dirProp == EN || dirProp == AN) { - // no need for a mark after the first digit in a number - if (BidiComplexProcessor.getDirProp(text, dirProps, fromIndex - 1) == dirProp) - continue; - - for (int i = fromIndex - 1; i >= 0; i--) { - dirProp = BidiComplexProcessor.getDirProp(text, dirProps, i); - // after a L char, no need for a mark - if (dirProp == L) - continue; - - // digit after R or AL or AN need a mark, except for EN - // following AN, but this is a contrived case, so we - // don't check for it (and calling processSeparator() - // for it will do no harm) - if (dirProp == R || dirProp == AL || dirProp == AN) - return fromIndex; - } - continue; - } - } - return -1; - } - - /** - * This method process the special cases. - */ - public int processSpecial(BidiComplexFeatures features, String text, byte[] dirProps, int[] offsets, int[] state, int caseNumber, int separLocation) { - int location; - - switch (caseNumber) { - case 1 : /* comment (?#...) */ - if (separLocation < 0) { - // initial state from previous line - location = 0; - } else { - BidiComplexProcessor.processSeparator(features, text, dirProps, offsets, separLocation); - // skip the opening "(?#" - location = separLocation + 3; - } - location = text.indexOf(')', location); - if (location < 0) { - state[0] = caseNumber; - return text.length(); - } - return location + 1; - case 2 : /* named group (? */ - case 3 : /* named group (?'name' */ - case 4 : /* conditional named back reference (?(name) */ - case 5 : /* conditional named back reference (?() */ - case 6 : /* conditional named back reference (?('name') */ - case 7 : /* named parentheses reference (?&name) */ - BidiComplexProcessor.processSeparator(features, text, dirProps, offsets, separLocation); - // no need for calling processSeparator() for the following cases - // since the starting string contains a L char - case 8 : /* named group (?P */ - case 9 : /* named back reference \k */ - case 10 : /* named back reference \k'name' */ - case 11 : /* named back reference \k{name} */ - case 12 : /* named back reference (?P=name) */ - case 13 : /* named back reference \g{name} */ - case 14 : /* subroutine call \g */ - case 15 : /* subroutine call \g'name' */ - case 16 : /* named back reference recursion (?(R&name) */ - // skip the opening string - location = separLocation + startStrings[caseNumber].length(); - // look for ending character - location = text.indexOf(endChars[caseNumber], location); - if (location < 0) - return text.length(); - return location + 1; - case 17 : /* quoted sequence \Q...\E */ - if (separLocation < 0) { - // initial state from previous line - location = 0; - } else { - BidiComplexProcessor.processSeparator(features, text, dirProps, offsets, separLocation); - // skip the opening "\Q" - location = separLocation + 2; - } - location = text.indexOf("\\E", location); //$NON-NLS-1$ - if (location < 0) { - state[0] = caseNumber; - return text.length(); - } - // set the dirProp for the "E" to L (Left to Right character) - BidiComplexProcessor.setDirProp(dirProps, location + 1, L); - return location + 2; - case 18 : /* R, AL, AN, EN */ - BidiComplexProcessor.processSeparator(features, text, dirProps, offsets, separLocation); - return separLocation + 1; - - } - // we should never get here - return text.length(); - } -} Index: src/org/eclipse/equinox/bidi/internal/consumable/BidiComplexSql.java =================================================================== RCS file: src/org/eclipse/equinox/bidi/internal/consumable/BidiComplexSql.java diff -N src/org/eclipse/equinox/bidi/internal/consumable/BidiComplexSql.java --- src/org/eclipse/equinox/bidi/internal/consumable/BidiComplexSql.java 17 May 2011 19:56:16 -0000 1.3 +++ /dev/null 1 Jan 1970 00:00:00 -0000 @@ -1,152 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2010, 2011 IBM Corporation and others. - * All rights reserved. This program and the accompanying materials - * are made available under the terms of the Eclipse Public License v1.0 - * which accompanies this distribution, and is available at - * http://www.eclipse.org/legal/epl-v10.html - * - * Contributors: - * IBM Corporation - initial API and implementation - ******************************************************************************/ -package org.eclipse.equinox.bidi.internal.consumable; - -import org.eclipse.equinox.bidi.BidiComplexEngine; -import org.eclipse.equinox.bidi.BidiComplexEnvironment; -import org.eclipse.equinox.bidi.custom.BidiComplexFeatures; -import org.eclipse.equinox.bidi.custom.BidiComplexProcessor; - -/** - * BidiComplexSql is a processor for complex expressions - * composed of SQL statements. Such a complex expression may span - * multiple lines. - *

- * In applications like an editor where parts of the text might be modified - * while other parts are not, the user may want to call - * {@link BidiComplexEngine#leanToFullText leanToFullText} - * separately on each line and save the initial state of each line (this is - * the final state of the previous line which can be retrieved from the - * value returned in the first element of the state argument). - * If both the content - * of a line and its initial state have not changed, the user can be sure that - * the last full text computed for this line has not changed either. - * - * @see BidiComplexEngine#leanToFullText explanation of state in leanToFullText - * - * @author Matitiahu Allouche - */ -public class BidiComplexSql extends BidiComplexProcessor { - private static final byte WS = Character.DIRECTIONALITY_WHITESPACE; - static final String separators = "\t!#%&()*+,-./:;<=>?|[]{}"; //$NON-NLS-1$ - static final BidiComplexFeatures FEATURES = new BidiComplexFeatures(separators, 5, -1, -1, false, false); - static final String lineSep = BidiComplexEnvironment.getLineSep(); - - /** - * This method retrieves the features specific to this processor. - * - * @return features with separators "\t!#%&()*+,-./:;<=>?|[]{}", 5 special cases, - * LTR direction for Arabic and Hebrew, and support for both. - */ - public BidiComplexFeatures getFeatures(BidiComplexEnvironment env) { - return FEATURES; - } - - /** - * This method looks for occurrences of 5 special strings: - *

    - *
  1. spaces
    2. - *
  2. literals starting with apostrophe
    3. - *
  3. identifiers starting with quotation mark
    4. - *
  4. comments starting with slash-asterisk
    5. - *
  5. comments starting with hyphen-hyphen
    6. - *
- */ - public int indexOfSpecial(BidiComplexFeatures features, String text, byte[] dirProps, int[] offsets, int caseNumber, int fromIndex) { - switch (caseNumber) { - case 1 : /* space */ - return text.indexOf(" ", fromIndex); //$NON-NLS-1$ - case 2 : /* literal */ - return text.indexOf('\'', fromIndex); - case 3 : /* delimited identifier */ - return text.indexOf('"', fromIndex); - case 4 : /* slash-aster comment */ - return text.indexOf("/*", fromIndex); //$NON-NLS-1$ - case 5 : /* hyphen-hyphen comment */ - return text.indexOf("--", fromIndex); //$NON-NLS-1$ - } - // we should never get here - return -1; - } - - /** - * This method processes the 5 special cases as follows. - *
    - *
  1. skip the run of spaces
    2. - *
  2. look for a matching apostrophe and skip until after it
    3. - *
  3. look for a matching quotation mark and skip until after it
    4. - *
  4. skip until after the closing asterisk-slash
    5. - *
  5. skip until after a line separator
    6. - *
- */ - public int processSpecial(BidiComplexFeatures features, String text, byte[] dirProps, int[] offsets, int[] state, int caseNumber, int separLocation) { - int location; - - BidiComplexProcessor.processSeparator(features, text, dirProps, offsets, separLocation); - switch (caseNumber) { - case 1 : /* space */ - separLocation++; - while (separLocation < text.length() && text.charAt(separLocation) == ' ') { - BidiComplexProcessor.setDirProp(dirProps, separLocation, WS); - separLocation++; - } - return separLocation; - case 2 : /* literal */ - location = separLocation + 1; - while (true) { - location = text.indexOf('\'', location); - if (location < 0) { - state[0] = caseNumber; - return text.length(); - } - if ((location + 1) < text.length() && text.charAt(location + 1) == '\'') { - location += 2; - continue; - } - return location + 1; - } - case 3 : /* delimited identifier */ - location = separLocation + 1; - while (true) { - location = text.indexOf('"', location); - if (location < 0) - return text.length(); - - if ((location + 1) < text.length() && text.charAt(location + 1) == '"') { - location += 2; - continue; - } - return location + 1; - } - case 4 : /* slash-aster comment */ - if (separLocation < 0) // continuation line - location = 0; - else - location = separLocation + 2; // skip the opening slash-aster - location = text.indexOf("*/", location); //$NON-NLS-1$ - if (location < 0) { - state[0] = caseNumber; - return text.length(); - } - // we need to call processSeparator since text may follow the - // end of comment immediately without even a space - BidiComplexProcessor.processSeparator(features, text, dirProps, offsets, location); - return location + 2; - case 5 : /* hyphen-hyphen comment */ - location = text.indexOf(lineSep, separLocation + 2); - if (location < 0) - return text.length(); - return location + lineSep.length(); - } - // we should never get here - return text.length(); - } -} Index: src/org/eclipse/equinox/bidi/internal/consumable/BidiComplexSystem.java =================================================================== RCS file: src/org/eclipse/equinox/bidi/internal/consumable/BidiComplexSystem.java diff -N src/org/eclipse/equinox/bidi/internal/consumable/BidiComplexSystem.java --- src/org/eclipse/equinox/bidi/internal/consumable/BidiComplexSystem.java 17 May 2011 19:56:16 -0000 1.3 +++ /dev/null 1 Jan 1970 00:00:00 -0000 @@ -1,36 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2010, 2011 IBM Corporation and others. - * All rights reserved. This program and the accompanying materials - * are made available under the terms of the Eclipse Public License v1.0 - * which accompanies this distribution, and is available at - * http://www.eclipse.org/legal/epl-v10.html - * - * Contributors: - * IBM Corporation - initial API and implementation - ******************************************************************************/ -package org.eclipse.equinox.bidi.internal.consumable; - -import org.eclipse.equinox.bidi.BidiComplexEnvironment; -import org.eclipse.equinox.bidi.custom.BidiComplexFeatures; -import org.eclipse.equinox.bidi.internal.BidiComplexSingle; - -/** - * Processor adapted to processing expressions with the following string format: - * 
- *    system(user)
- *
- */ -public class BidiComplexSystem extends BidiComplexSingle { - static final BidiComplexFeatures FEATURES = new BidiComplexFeatures("(", 0, -1, -1, false, false); //$NON-NLS-1$ - - /** - * This method retrieves the features specific to this processor. - * - * @return features with one separator (opening parenthesis), - * no special cases, LTR direction for Arabic and Hebrew, - * and support for both. - */ - public BidiComplexFeatures getFeatures(BidiComplexEnvironment env) { - return FEATURES; - } -} Index: src/org/eclipse/equinox/bidi/internal/consumable/BidiComplexURL.java =================================================================== RCS file: src/org/eclipse/equinox/bidi/internal/consumable/BidiComplexURL.java diff -N src/org/eclipse/equinox/bidi/internal/consumable/BidiComplexURL.java --- src/org/eclipse/equinox/bidi/internal/consumable/BidiComplexURL.java 17 May 2011 19:56:16 -0000 1.3 +++ /dev/null 1 Jan 1970 00:00:00 -0000 @@ -1,34 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2010, 2011 IBM Corporation and others. - * All rights reserved. This program and the accompanying materials - * are made available under the terms of the Eclipse Public License v1.0 - * which accompanies this distribution, and is available at - * http://www.eclipse.org/legal/epl-v10.html - * - * Contributors: - * IBM Corporation - initial API and implementation - ******************************************************************************/ -package org.eclipse.equinox.bidi.internal.consumable; - -import org.eclipse.equinox.bidi.BidiComplexEnvironment; -import org.eclipse.equinox.bidi.custom.BidiComplexFeatures; -import org.eclipse.equinox.bidi.custom.BidiComplexProcessor; - -/** - * Processor adapted to processing URLs. - */ -public class BidiComplexURL extends BidiComplexProcessor { - static final BidiComplexFeatures FEATURES = new BidiComplexFeatures(":?#/@.[]", 0, -1, -1, false, false); //$NON-NLS-1$ - - /** - * This method retrieves the features specific to this processor. - * - * @return features with separators ":?#/@.[]", - * no special cases, LTR direction for Arabic and Hebrew, - * and support for both. - */ - public BidiComplexFeatures getFeatures(BidiComplexEnvironment env) { - return FEATURES; - } - -} Index: src/org/eclipse/equinox/bidi/internal/consumable/BidiComplexUnderscore.java =================================================================== RCS file: src/org/eclipse/equinox/bidi/internal/consumable/BidiComplexUnderscore.java diff -N src/org/eclipse/equinox/bidi/internal/consumable/BidiComplexUnderscore.java --- src/org/eclipse/equinox/bidi/internal/consumable/BidiComplexUnderscore.java 17 May 2011 19:56:16 -0000 1.3 +++ /dev/null 1 Jan 1970 00:00:00 -0000 @@ -1,38 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2010, 2011 IBM Corporation and others. - * All rights reserved. This program and the accompanying materials - * are made available under the terms of the Eclipse Public License v1.0 - * which accompanies this distribution, and is available at - * http://www.eclipse.org/legal/epl-v10.html - * - * Contributors: - * IBM Corporation - initial API and implementation - ******************************************************************************/ -package org.eclipse.equinox.bidi.internal.consumable; - -import org.eclipse.equinox.bidi.BidiComplexEnvironment; -import org.eclipse.equinox.bidi.custom.BidiComplexFeatures; -import org.eclipse.equinox.bidi.custom.BidiComplexProcessor; - -/** - * Processor adapted to processing compound names. - * This type covers names made of one or more parts, separated by underscores: - * 
- *    part1_part2_part3
- *
- */ -public class BidiComplexUnderscore extends BidiComplexProcessor { - static final BidiComplexFeatures FEATURES = new BidiComplexFeatures("_", 0, -1, -1, false, false); //$NON-NLS-1$ - - /** - * This method retrieves the features specific to this processor. - * - * @return features with one separator (underscore), - * no special cases, LTR direction for Arabic and Hebrew, - * and support for both. - */ - public BidiComplexFeatures getFeatures(BidiComplexEnvironment env) { - return FEATURES; - } - -} Index: src/org/eclipse/equinox/bidi/internal/consumable/BidiComplexXPath.java =================================================================== RCS file: src/org/eclipse/equinox/bidi/internal/consumable/BidiComplexXPath.java diff -N src/org/eclipse/equinox/bidi/internal/consumable/BidiComplexXPath.java --- src/org/eclipse/equinox/bidi/internal/consumable/BidiComplexXPath.java 17 May 2011 19:56:16 -0000 1.3 +++ /dev/null 1 Jan 1970 00:00:00 -0000 @@ -1,41 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2010, 2011 IBM Corporation and others. - * All rights reserved. This program and the accompanying materials - * are made available under the terms of the Eclipse Public License v1.0 - * which accompanies this distribution, and is available at - * http://www.eclipse.org/legal/epl-v10.html - * - * Contributors: - * IBM Corporation - initial API and implementation - ******************************************************************************/ -package org.eclipse.equinox.bidi.internal.consumable; - -import org.eclipse.equinox.bidi.BidiComplexEnvironment; -import org.eclipse.equinox.bidi.custom.BidiComplexFeatures; -import org.eclipse.equinox.bidi.internal.BidiComplexDelims; - -/** - * Processor adapted to processing XPath expressions. - */ -public class BidiComplexXPath extends BidiComplexDelims { - static final BidiComplexFeatures FEATURES = new BidiComplexFeatures(" /[]<>=!:@.|()+-*", 2, -1, -1, false, false); //$NON-NLS-1$ - - /** - * This method retrieves the features specific to this processor. - * - * @return features with separators " /[]<>=!:@.|()+-*", - * 2 special cases, LTR direction for Arabic and Hebrew, - * and support for both. - */ - public BidiComplexFeatures getFeatures(BidiComplexEnvironment env) { - return FEATURES; - } - - /** - * @return apostrophe and quotation mark as delimiters. - */ - protected String getDelimiters() { - return "''\"\""; //$NON-NLS-1$ - } - -} Index: src/org/eclipse/equinox/bidi/internal/consumable/STextComma.java =================================================================== RCS file: src/org/eclipse/equinox/bidi/internal/consumable/STextComma.java diff -N src/org/eclipse/equinox/bidi/internal/consumable/STextComma.java --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ src/org/eclipse/equinox/bidi/internal/consumable/STextComma.java 1 Jan 1970 00:00:00 -0000 @@ -0,0 +1,36 @@ +/******************************************************************************* + * Copyright (c) 2010, 2011 IBM Corporation and others. + * All rights reserved. This program and the accompanying materials + * are made available under the terms of the Eclipse Public License v1.0 + * which accompanies this distribution, and is available at + * http://www.eclipse.org/legal/epl-v10.html + * + * Contributors: + * IBM Corporation - initial API and implementation + ******************************************************************************/ +package org.eclipse.equinox.bidi.internal.consumable; + +import org.eclipse.equinox.bidi.STextEnvironment; +import org.eclipse.equinox.bidi.custom.STextFeatures; +import org.eclipse.equinox.bidi.custom.STextProcessor; + +/** + * Processor adapted to processing comma-delimited lists, such as: + * 
+ *    part1,part2,part3
+ *
+ */ +public class STextComma extends STextProcessor { + static final STextFeatures FEATURES = new STextFeatures(",", 0, -1, -1, false, false); //$NON-NLS-1$ + + /** + * This method retrieves the features specific to this processor. + * + * @return features with one separator (comma), no special cases, + * LTR direction for Arabic and Hebrew, and support for both. + */ + public STextFeatures getFeatures(STextEnvironment env) { + return FEATURES; + } + +} Index: src/org/eclipse/equinox/bidi/internal/consumable/STextEmail.java =================================================================== RCS file: src/org/eclipse/equinox/bidi/internal/consumable/STextEmail.java diff -N src/org/eclipse/equinox/bidi/internal/consumable/STextEmail.java --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ src/org/eclipse/equinox/bidi/internal/consumable/STextEmail.java 1 Jan 1970 00:00:00 -0000 @@ -0,0 +1,48 @@ +/******************************************************************************* + * Copyright (c) 2010, 2011 IBM Corporation and others. + * All rights reserved. This program and the accompanying materials + * are made available under the terms of the Eclipse Public License v1.0 + * which accompanies this distribution, and is available at + * http://www.eclipse.org/legal/epl-v10.html + * + * Contributors: + * IBM Corporation - initial API and implementation + ******************************************************************************/ +package org.eclipse.equinox.bidi.internal.consumable; + +import org.eclipse.equinox.bidi.STextEnvironment; +import org.eclipse.equinox.bidi.custom.STextFeatures; +import org.eclipse.equinox.bidi.internal.STextDelimsEsc; + +/** + * Processor adapted to processing e-mail addresses. + */ +public class STextEmail extends STextDelimsEsc { + static final int LTR = STextFeatures.DIR_LTR; + static final int RTL = STextFeatures.DIR_RTL; + static final STextFeatures MIRRORED = new STextFeatures("<>.:,;@", 2, RTL, LTR, false, false); //$NON-NLS-1$ + static final STextFeatures NOT_MIRRORED = new STextFeatures("<>.:,;@", 2, LTR, LTR, false, false); //$NON-NLS-1$ + + /** + * This method retrieves the features specific to this processor. + * + * @return features with separators "<>.:,;@", 2 special cases, + * LTR direction for Arabic when the GUI is not mirrored, + * RTL direction for Arabic when the GUI is mirrored, + * LTR direction for Hebrew in all cases, + * and support for both Arabic and Hebrew. + */ + public STextFeatures getFeatures(STextEnvironment env) { + if (env == null) + env = STextEnvironment.DEFAULT; + return env.getMirrored() ? MIRRORED : NOT_MIRRORED; + } + + /** + * @return parentheses and quotation marks as delimiters. + */ + protected String getDelimiters() { + return "()\"\""; //$NON-NLS-1$ + } + +} Index: src/org/eclipse/equinox/bidi/internal/consumable/STextFile.java =================================================================== RCS file: src/org/eclipse/equinox/bidi/internal/consumable/STextFile.java diff -N src/org/eclipse/equinox/bidi/internal/consumable/STextFile.java --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ src/org/eclipse/equinox/bidi/internal/consumable/STextFile.java 1 Jan 1970 00:00:00 -0000 @@ -0,0 +1,33 @@ +/******************************************************************************* + * Copyright (c) 2010, 2011 IBM Corporation and others. + * All rights reserved. This program and the accompanying materials + * are made available under the terms of the Eclipse Public License v1.0 + * which accompanies this distribution, and is available at + * http://www.eclipse.org/legal/epl-v10.html + * + * Contributors: + * IBM Corporation - initial API and implementation + ******************************************************************************/ +package org.eclipse.equinox.bidi.internal.consumable; + +import org.eclipse.equinox.bidi.STextEnvironment; +import org.eclipse.equinox.bidi.custom.STextFeatures; +import org.eclipse.equinox.bidi.custom.STextProcessor; + +/** + * Processor adapted to processing directory and file paths. + */ +public class STextFile extends STextProcessor { + static final STextFeatures FEATURES = new STextFeatures(":/\\.", 0, -1, -1, false, false); //$NON-NLS-1$ + + /** + * This method retrieves the features specific to this processor. + * + * @return features with separators ":/\.", no special cases, + * LTR direction for Arabic and Hebrew, and support for both. + */ + public STextFeatures getFeatures(STextEnvironment env) { + return FEATURES; + } + +} Index: src/org/eclipse/equinox/bidi/internal/consumable/STextJava.java =================================================================== RCS file: src/org/eclipse/equinox/bidi/internal/consumable/STextJava.java diff -N src/org/eclipse/equinox/bidi/internal/consumable/STextJava.java --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ src/org/eclipse/equinox/bidi/internal/consumable/STextJava.java 1 Jan 1970 00:00:00 -0000 @@ -0,0 +1,134 @@ +/******************************************************************************* + * Copyright (c) 2010, 2011 IBM Corporation and others. + * All rights reserved. This program and the accompanying materials + * are made available under the terms of the Eclipse Public License v1.0 + * which accompanies this distribution, and is available at + * http://www.eclipse.org/legal/epl-v10.html + * + * Contributors: + * IBM Corporation - initial API and implementation + ******************************************************************************/ +package org.eclipse.equinox.bidi.internal.consumable; + +import org.eclipse.equinox.bidi.STextEngine; +import org.eclipse.equinox.bidi.STextEnvironment; +import org.eclipse.equinox.bidi.custom.STextFeatures; +import org.eclipse.equinox.bidi.custom.STextProcessor; + +/** + * STextJava is a processor for structured text + * composed of Java statements. Such a structured text may span + * multiple lines. + *

+ * In applications like an editor where parts of the text might be modified + * while other parts are not, the user may want to call + * {@link STextEngine#leanToFullText leanToFullText} + * separately on each line and save the initial state of each line (this is + * the final state of the previous line which can be retrieved using + * the value returned in the first element of the state argument). + * If both the content + * of a line and its initial state have not changed, the user can be sure that + * the last full text computed for this line has not changed either. + * + * @see STextEngine#leanToFullText explanation of state in leanToFullText + * + * @author Matitiahu Allouche + */ +public class STextJava extends STextProcessor { + private static final byte WS = Character.DIRECTIONALITY_WHITESPACE; + static final STextFeatures FEATURES = new STextFeatures("[](){}.+-<>=~!&*/%^|?:,;\t", 4, -1, -1, false, false); //$NON-NLS-1$ + static final String lineSep = STextEnvironment.getLineSep(); + + /** + * This method retrieves the features specific to this processor. + * + * @return features with separators "[](){}.+-<>=~!&/*%^|?:,;\t", + * 4 special cases, LTR direction for Arabic and Hebrew, + * and support for both. + */ + public STextFeatures getFeatures(STextEnvironment env) { + return FEATURES; + } + + /** + * This method looks for occurrences of 4 special strings: + *

    + *
  1. spaces
    2. + *
  2. literals starting with quotation mark
    3. + *
  3. comments starting with slash-asterisk
    4. + *
  4. comments starting with slash-slash
    5. + *
+ */ + public int indexOfSpecial(STextFeatures features, String text, byte[] dirProps, int[] offsets, int caseNumber, int fromIndex) { + switch (caseNumber) { + case 1 : /* space */ + return text.indexOf(' ', fromIndex); + case 2 : /* literal */ + return text.indexOf('"', fromIndex); + case 3 : /* slash-aster comment */ + return text.indexOf("/*", fromIndex); //$NON-NLS-1$ + case 4 : /* slash-slash comment */ + return text.indexOf("//", fromIndex); //$NON-NLS-1$ + } + // we should never get here + return -1; + } + + /** + * This method processes the 4 special cases as follows. + *
    + *
  1. skip the run of spaces
    2. + *
  2. look for a matching quotation mark and skip until after it
    3. + *
  3. skip until after the closing asterisk-slash
    4. + *
  4. skip until after a line separator
    5. + *
+ */ + public int processSpecial(STextFeatures features, String text, byte[] dirProps, int[] offsets, int[] state, int caseNumber, int separLocation) { + int location, counter, i; + + STextProcessor.processSeparator(features, text, dirProps, offsets, separLocation); + switch (caseNumber) { + case 1 : /* space */ + separLocation++; + while (separLocation < text.length() && text.charAt(separLocation) == ' ') { + STextProcessor.setDirProp(dirProps, separLocation, WS); + separLocation++; + } + return separLocation; + case 2 : /* literal */ + location = separLocation + 1; + while (true) { + location = text.indexOf('"', location); + if (location < 0) + return text.length(); + for (counter = 0, i = location - 1; text.charAt(i) == '\\'; i--) { + counter++; + } + location++; + if ((counter & 1) == 0) + return location; + } + case 3 : /* slash-aster comment */ + if (separLocation < 0) { // continuation line + location = 0; + } else + location = separLocation + 2; // skip the opening slash-aster + location = text.indexOf("*/", location); //$NON-NLS-1$ + if (location < 0) { + state[0] = caseNumber; + return text.length(); + } + // we need to call processSeparator since text may follow the + // end of comment immediately without even a space + STextProcessor.processSeparator(features, text, dirProps, offsets, location); + return location + 2; + case 4 : /* slash-slash comment */ + location = text.indexOf(lineSep, separLocation + 2); + if (location < 0) + return text.length(); + return location + lineSep.length(); + } + // we should never get here + return text.length(); + } +} Index: src/org/eclipse/equinox/bidi/internal/consumable/STextMath.java =================================================================== RCS file: src/org/eclipse/equinox/bidi/internal/consumable/STextMath.java diff -N src/org/eclipse/equinox/bidi/internal/consumable/STextMath.java --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ src/org/eclipse/equinox/bidi/internal/consumable/STextMath.java 1 Jan 1970 00:00:00 -0000 @@ -0,0 +1,35 @@ +/******************************************************************************* + * Copyright (c) 2010, 2011 IBM Corporation and others. + * All rights reserved. This program and the accompanying materials + * are made available under the terms of the Eclipse Public License v1.0 + * which accompanies this distribution, and is available at + * http://www.eclipse.org/legal/epl-v10.html + * + * Contributors: + * IBM Corporation - initial API and implementation + ******************************************************************************/ +package org.eclipse.equinox.bidi.internal.consumable; + +import org.eclipse.equinox.bidi.STextEnvironment; +import org.eclipse.equinox.bidi.custom.STextFeatures; +import org.eclipse.equinox.bidi.custom.STextProcessor; + +/** + * Processor adapted to processing arithmetic expressions with right-to-left + * base direction. + */ +public class STextMath extends STextProcessor { + static final int RTL = STextFeatures.DIR_RTL; + static final STextFeatures FEATURES = new STextFeatures("+-/*()=", 0, RTL, RTL, false, false); //$NON-NLS-1$ + + /** + * This method retrieves the features specific to this processor. + * + * @return features with separators "+-/*()=", no special cases, + * RTL direction for Arabic and Hebrew, and support for both. + */ + public STextFeatures getFeatures(STextEnvironment env) { + return FEATURES; + } + +} Index: src/org/eclipse/equinox/bidi/internal/consumable/STextProperty.java =================================================================== RCS file: src/org/eclipse/equinox/bidi/internal/consumable/STextProperty.java diff -N src/org/eclipse/equinox/bidi/internal/consumable/STextProperty.java --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ src/org/eclipse/equinox/bidi/internal/consumable/STextProperty.java 1 Jan 1970 00:00:00 -0000 @@ -0,0 +1,37 @@ +/******************************************************************************* + * Copyright (c) 2010, 2011 IBM Corporation and others. + * All rights reserved. This program and the accompanying materials + * are made available under the terms of the Eclipse Public License v1.0 + * which accompanies this distribution, and is available at + * http://www.eclipse.org/legal/epl-v10.html + * + * Contributors: + * IBM Corporation - initial API and implementation + ******************************************************************************/ +package org.eclipse.equinox.bidi.internal.consumable; + +import org.eclipse.equinox.bidi.STextEnvironment; +import org.eclipse.equinox.bidi.custom.STextFeatures; +import org.eclipse.equinox.bidi.internal.STextSingle; + +/** + * Processor adapted to processing property file statements. + * It expects the following string format: + * 
+ *    name=value
+ *
+ */ +public class STextProperty extends STextSingle { + static final STextFeatures FEATURES = new STextFeatures("=", 1, -1, -1, false, false); //$NON-NLS-1$ + + /** + * This method retrieves the features specific to this processor. + * + * @return features with one separator (equal sign), 1 special case, + * LTR direction for Arabic and Hebrew, and support for both. + */ + public STextFeatures getFeatures(STextEnvironment env) { + return FEATURES; + } + +} Index: src/org/eclipse/equinox/bidi/internal/consumable/STextRegex.java =================================================================== RCS file: src/org/eclipse/equinox/bidi/internal/consumable/STextRegex.java diff -N src/org/eclipse/equinox/bidi/internal/consumable/STextRegex.java --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ src/org/eclipse/equinox/bidi/internal/consumable/STextRegex.java 1 Jan 1970 00:00:00 -0000 @@ -0,0 +1,221 @@ +/******************************************************************************* + * Copyright (c) 2010, 2011 IBM Corporation and others. + * All rights reserved. This program and the accompanying materials + * are made available under the terms of the Eclipse Public License v1.0 + * which accompanies this distribution, and is available at + * http://www.eclipse.org/legal/epl-v10.html + * + * Contributors: + * IBM Corporation - initial API and implementation + ******************************************************************************/ +package org.eclipse.equinox.bidi.internal.consumable; + +import org.eclipse.equinox.bidi.STextEngine; +import org.eclipse.equinox.bidi.STextEnvironment; +import org.eclipse.equinox.bidi.custom.*; + +/** + * STextRegex is a processor for regular expressions. + * Such expressions may span multiple lines. + *

+ * In applications like an editor where parts of the text might be modified + * while other parts are not, the user may want to call + * {@link STextEngine#leanToFullText leanToFullText} + * separately on each line and save the initial state of each line (this is + * the final state of the previous line which can be retrieved using + * the value returned in the first element of the state argument). + * If both the content + * of a line and its initial state have not changed, the user can be sure that + * the last full text computed for this line has not changed either. + * + * @see STextEngine#leanToFullText explanation of state in leanToFullText + * + * @author Matitiahu Allouche + */ +public class STextRegex extends STextProcessor { + static final String[] startStrings = {"", /* 0 *//* dummy *///$NON-NLS-1$ + "(?#", /* 1 *//* comment (?#...) *///$NON-NLS-1$ + "(?<", /* 2 *//* named group (? *///$NON-NLS-1$ + "(?'", /* 3 *//* named group (?'name' *///$NON-NLS-1$ + "(?(<", /* 4 *//* conditional named back reference (?() *///$NON-NLS-1$ + "(?('", /* 5 *//* conditional named back reference (?('name') *///$NON-NLS-1$ + "(?(", /* 6 *//* conditional named back reference (?(name) *///$NON-NLS-1$ + "(?&", /* 7 *//* named parentheses reference (?&name) *///$NON-NLS-1$ + "(?P<", /* 8 *//* named group (?P *///$NON-NLS-1$ + "\\k<", /* 9 *//* named back reference \k *///$NON-NLS-1$ + "\\k'", /* 10 *//* named back reference \k'name' *///$NON-NLS-1$ + "\\k{", /* 11 *//* named back reference \k{name} *///$NON-NLS-1$ + "(?P=", /* 12 *//* named back reference (?P=name) *///$NON-NLS-1$ + "\\g{", /* 13 *//* named back reference \g{name} *///$NON-NLS-1$ + "\\g<", /* 14 *//* subroutine call \g *///$NON-NLS-1$ + "\\g'", /* 15 *//* subroutine call \g'name' *///$NON-NLS-1$ + "(?(R&", /* 16 *//* named back reference recursion (?(R&name) *///$NON-NLS-1$ + "\\Q" /* 17 *//* quoted sequence \Q...\E *///$NON-NLS-1$ + }; + static final char[] endChars = { + // 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 + '.', ')', '>', '\'', ')', ')', ')', ')', '>', '>', '\'', '}', ')', '}', '>', '\'', ')'}; + static final int numberOfStrings = startStrings.length; /* 18 */ + static final int maxSpecial = numberOfStrings; + static final STextFeatures FEATURES = new STextFeatures(null, maxSpecial, -1, -1, false, false); + static final byte L = Character.DIRECTIONALITY_LEFT_TO_RIGHT; + static final byte R = Character.DIRECTIONALITY_RIGHT_TO_LEFT; + static final byte AL = Character.DIRECTIONALITY_RIGHT_TO_LEFT_ARABIC; + static final byte AN = Character.DIRECTIONALITY_ARABIC_NUMBER; + static final byte EN = Character.DIRECTIONALITY_EUROPEAN_NUMBER; + + /** + * This method retrieves the features specific to this processor. + * + * @see ISTextProcessor#getFeatures + * + * @return features with no separators , special cases for each kind of + * regular expression syntactic string, + * LTR direction for Arabic and Hebrew, and support for both. + */ + public STextFeatures getFeatures(STextEnvironment env) { + return FEATURES; + } + + /** + * This method locates occurrences of the syntactic strings and of + * R, AL, EN, AN characters. + */ + public int indexOfSpecial(STextFeatures features, String text, byte[] dirProps, int[] offsets, int caseNumber, int fromIndex) { + // In this method, L, R, AL, AN and EN represent bidi categories + // as defined in the Unicode Bidirectional Algorithm + // ( http://www.unicode.org/reports/tr9/ ). + // L represents the category Left to Right character. + // R represents the category Right to Left character. + // AL represents the category Arabic Letter. + // AN represents the category Arabic Number. + // EN represents the category European Number. + byte dirProp; + + if (caseNumber < numberOfStrings) { + /* 1 *//* comment (?#...) */ + /* 2 *//* named group (? */ + /* 3 *//* named group (?'name' */ + /* 4 *//* conditional named back reference (?(name) */ + /* 5 *//* conditional named back reference (?() */ + /* 6 *//* conditional named back reference (?('name') */ + /* 7 *//* named parentheses reference (?&name) */ + /* 8 *//* named group (?P */ + /* 9 *//* named back reference \k */ + /* 10 *//* named back reference \k'name' */ + /* 11 *//* named back reference \k{name} */ + /* 12 *//* named back reference (?P=name) */ + /* 13 *//* named back reference \g{name} */ + /* 14 *//* subroutine call \g */ + /* 15 *//* subroutine call \g'name' */ + /* 16 *//* named back reference recursion (?(R&name) */ + /* 17 *//* quoted sequence \Q...\E */ + return text.indexOf(startStrings[caseNumber], fromIndex); + } + // there never is a need for a mark before the first char + if (fromIndex <= 0) + fromIndex = 1; + // look for R, AL, AN, EN which are potentially needing a mark + for (; fromIndex < text.length(); fromIndex++) { + dirProp = STextProcessor.getDirProp(text, dirProps, fromIndex); + // R and AL will always be examined using processSeparator() + if (dirProp == R || dirProp == AL) + return fromIndex; + + if (dirProp == EN || dirProp == AN) { + // no need for a mark after the first digit in a number + if (STextProcessor.getDirProp(text, dirProps, fromIndex - 1) == dirProp) + continue; + + for (int i = fromIndex - 1; i >= 0; i--) { + dirProp = STextProcessor.getDirProp(text, dirProps, i); + // after a L char, no need for a mark + if (dirProp == L) + continue; + + // digit after R or AL or AN need a mark, except for EN + // following AN, but this is a contrived case, so we + // don't check for it (and calling processSeparator() + // for it will do no harm) + if (dirProp == R || dirProp == AL || dirProp == AN) + return fromIndex; + } + continue; + } + } + return -1; + } + + /** + * This method process the special cases. + */ + public int processSpecial(STextFeatures features, String text, byte[] dirProps, int[] offsets, int[] state, int caseNumber, int separLocation) { + int location; + + switch (caseNumber) { + case 1 : /* comment (?#...) */ + if (separLocation < 0) { + // initial state from previous line + location = 0; + } else { + STextProcessor.processSeparator(features, text, dirProps, offsets, separLocation); + // skip the opening "(?#" + location = separLocation + 3; + } + location = text.indexOf(')', location); + if (location < 0) { + state[0] = caseNumber; + return text.length(); + } + return location + 1; + case 2 : /* named group (? */ + case 3 : /* named group (?'name' */ + case 4 : /* conditional named back reference (?(name) */ + case 5 : /* conditional named back reference (?() */ + case 6 : /* conditional named back reference (?('name') */ + case 7 : /* named parentheses reference (?&name) */ + STextProcessor.processSeparator(features, text, dirProps, offsets, separLocation); + // no need for calling processSeparator() for the following cases + // since the starting string contains a L char + case 8 : /* named group (?P */ + case 9 : /* named back reference \k */ + case 10 : /* named back reference \k'name' */ + case 11 : /* named back reference \k{name} */ + case 12 : /* named back reference (?P=name) */ + case 13 : /* named back reference \g{name} */ + case 14 : /* subroutine call \g */ + case 15 : /* subroutine call \g'name' */ + case 16 : /* named back reference recursion (?(R&name) */ + // skip the opening string + location = separLocation + startStrings[caseNumber].length(); + // look for ending character + location = text.indexOf(endChars[caseNumber], location); + if (location < 0) + return text.length(); + return location + 1; + case 17 : /* quoted sequence \Q...\E */ + if (separLocation < 0) { + // initial state from previous line + location = 0; + } else { + STextProcessor.processSeparator(features, text, dirProps, offsets, separLocation); + // skip the opening "\Q" + location = separLocation + 2; + } + location = text.indexOf("\\E", location); //$NON-NLS-1$ + if (location < 0) { + state[0] = caseNumber; + return text.length(); + } + // set the dirProp for the "E" to L (Left to Right character) + STextProcessor.setDirProp(dirProps, location + 1, L); + return location + 2; + case 18 : /* R, AL, AN, EN */ + STextProcessor.processSeparator(features, text, dirProps, offsets, separLocation); + return separLocation + 1; + + } + // we should never get here + return text.length(); + } +} Index: src/org/eclipse/equinox/bidi/internal/consumable/STextSql.java =================================================================== RCS file: src/org/eclipse/equinox/bidi/internal/consumable/STextSql.java diff -N src/org/eclipse/equinox/bidi/internal/consumable/STextSql.java --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ src/org/eclipse/equinox/bidi/internal/consumable/STextSql.java 1 Jan 1970 00:00:00 -0000 @@ -0,0 +1,152 @@ +/******************************************************************************* + * Copyright (c) 2010, 2011 IBM Corporation and others. + * All rights reserved. This program and the accompanying materials + * are made available under the terms of the Eclipse Public License v1.0 + * which accompanies this distribution, and is available at + * http://www.eclipse.org/legal/epl-v10.html + * + * Contributors: + * IBM Corporation - initial API and implementation + ******************************************************************************/ +package org.eclipse.equinox.bidi.internal.consumable; + +import org.eclipse.equinox.bidi.STextEngine; +import org.eclipse.equinox.bidi.STextEnvironment; +import org.eclipse.equinox.bidi.custom.STextFeatures; +import org.eclipse.equinox.bidi.custom.STextProcessor; + +/** + * STextSql is a processor for structured text + * composed of SQL statements. Such a structured text may span + * multiple lines. + *

+ * In applications like an editor where parts of the text might be modified + * while other parts are not, the user may want to call + * {@link STextEngine#leanToFullText leanToFullText} + * separately on each line and save the initial state of each line (this is + * the final state of the previous line which can be retrieved from the + * value returned in the first element of the state argument). + * If both the content + * of a line and its initial state have not changed, the user can be sure that + * the last full text computed for this line has not changed either. + * + * @see STextEngine#leanToFullText explanation of state in leanToFullText + * + * @author Matitiahu Allouche + */ +public class STextSql extends STextProcessor { + private static final byte WS = Character.DIRECTIONALITY_WHITESPACE; + static final String separators = "\t!#%&()*+,-./:;<=>?|[]{}"; //$NON-NLS-1$ + static final STextFeatures FEATURES = new STextFeatures(separators, 5, -1, -1, false, false); + static final String lineSep = STextEnvironment.getLineSep(); + + /** + * This method retrieves the features specific to this processor. + * + * @return features with separators "\t!#%&()*+,-./:;<=>?|[]{}", 5 special cases, + * LTR direction for Arabic and Hebrew, and support for both. + */ + public STextFeatures getFeatures(STextEnvironment env) { + return FEATURES; + } + + /** + * This method looks for occurrences of 5 special strings: + *

    + *
  1. spaces
    2. + *
  2. literals starting with apostrophe
    3. + *
  3. identifiers starting with quotation mark
    4. + *
  4. comments starting with slash-asterisk
    5. + *
  5. comments starting with hyphen-hyphen
    6. + *
+ */ + public int indexOfSpecial(STextFeatures features, String text, byte[] dirProps, int[] offsets, int caseNumber, int fromIndex) { + switch (caseNumber) { + case 1 : /* space */ + return text.indexOf(" ", fromIndex); //$NON-NLS-1$ + case 2 : /* literal */ + return text.indexOf('\'', fromIndex); + case 3 : /* delimited identifier */ + return text.indexOf('"', fromIndex); + case 4 : /* slash-aster comment */ + return text.indexOf("/*", fromIndex); //$NON-NLS-1$ + case 5 : /* hyphen-hyphen comment */ + return text.indexOf("--", fromIndex); //$NON-NLS-1$ + } + // we should never get here + return -1; + } + + /** + * This method processes the 5 special cases as follows. + *
    + *
  1. skip the run of spaces
    2. + *
  2. look for a matching apostrophe and skip until after it
    3. + *
  3. look for a matching quotation mark and skip until after it
    4. + *
  4. skip until after the closing asterisk-slash
    5. + *
  5. skip until after a line separator
    6. + *
+ */ + public int processSpecial(STextFeatures features, String text, byte[] dirProps, int[] offsets, int[] state, int caseNumber, int separLocation) { + int location; + + STextProcessor.processSeparator(features, text, dirProps, offsets, separLocation); + switch (caseNumber) { + case 1 : /* space */ + separLocation++; + while (separLocation < text.length() && text.charAt(separLocation) == ' ') { + STextProcessor.setDirProp(dirProps, separLocation, WS); + separLocation++; + } + return separLocation; + case 2 : /* literal */ + location = separLocation + 1; + while (true) { + location = text.indexOf('\'', location); + if (location < 0) { + state[0] = caseNumber; + return text.length(); + } + if ((location + 1) < text.length() && text.charAt(location + 1) == '\'') { + location += 2; + continue; + } + return location + 1; + } + case 3 : /* delimited identifier */ + location = separLocation + 1; + while (true) { + location = text.indexOf('"', location); + if (location < 0) + return text.length(); + + if ((location + 1) < text.length() && text.charAt(location + 1) == '"') { + location += 2; + continue; + } + return location + 1; + } + case 4 : /* slash-aster comment */ + if (separLocation < 0) // continuation line + location = 0; + else + location = separLocation + 2; // skip the opening slash-aster + location = text.indexOf("*/", location); //$NON-NLS-1$ + if (location < 0) { + state[0] = caseNumber; + return text.length(); + } + // we need to call processSeparator since text may follow the + // end of comment immediately without even a space + STextProcessor.processSeparator(features, text, dirProps, offsets, location); + return location + 2; + case 5 : /* hyphen-hyphen comment */ + location = text.indexOf(lineSep, separLocation + 2); + if (location < 0) + return text.length(); + return location + lineSep.length(); + } + // we should never get here + return text.length(); + } +} Index: src/org/eclipse/equinox/bidi/internal/consumable/STextSystem.java =================================================================== RCS file: src/org/eclipse/equinox/bidi/internal/consumable/STextSystem.java diff -N src/org/eclipse/equinox/bidi/internal/consumable/STextSystem.java --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ src/org/eclipse/equinox/bidi/internal/consumable/STextSystem.java 1 Jan 1970 00:00:00 -0000 @@ -0,0 +1,36 @@ +/******************************************************************************* + * Copyright (c) 2010, 2011 IBM Corporation and others. + * All rights reserved. This program and the accompanying materials + * are made available under the terms of the Eclipse Public License v1.0 + * which accompanies this distribution, and is available at + * http://www.eclipse.org/legal/epl-v10.html + * + * Contributors: + * IBM Corporation - initial API and implementation + ******************************************************************************/ +package org.eclipse.equinox.bidi.internal.consumable; + +import org.eclipse.equinox.bidi.STextEnvironment; +import org.eclipse.equinox.bidi.custom.STextFeatures; +import org.eclipse.equinox.bidi.internal.STextSingle; + +/** + * Processor adapted to processing structured text with the following format: + * 
+ *    system(user)
+ *
+ */ +public class STextSystem extends STextSingle { + static final STextFeatures FEATURES = new STextFeatures("(", 0, -1, -1, false, false); //$NON-NLS-1$ + + /** + * This method retrieves the features specific to this processor. + * + * @return features with one separator (opening parenthesis), + * no special cases, LTR direction for Arabic and Hebrew, + * and support for both. + */ + public STextFeatures getFeatures(STextEnvironment env) { + return FEATURES; + } +} Index: src/org/eclipse/equinox/bidi/internal/consumable/STextURL.java =================================================================== RCS file: src/org/eclipse/equinox/bidi/internal/consumable/STextURL.java diff -N src/org/eclipse/equinox/bidi/internal/consumable/STextURL.java --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ src/org/eclipse/equinox/bidi/internal/consumable/STextURL.java 1 Jan 1970 00:00:00 -0000 @@ -0,0 +1,34 @@ +/******************************************************************************* + * Copyright (c) 2010, 2011 IBM Corporation and others. + * All rights reserved. This program and the accompanying materials + * are made available under the terms of the Eclipse Public License v1.0 + * which accompanies this distribution, and is available at + * http://www.eclipse.org/legal/epl-v10.html + * + * Contributors: + * IBM Corporation - initial API and implementation + ******************************************************************************/ +package org.eclipse.equinox.bidi.internal.consumable; + +import org.eclipse.equinox.bidi.STextEnvironment; +import org.eclipse.equinox.bidi.custom.STextFeatures; +import org.eclipse.equinox.bidi.custom.STextProcessor; + +/** + * Processor adapted to processing URLs. + */ +public class STextURL extends STextProcessor { + static final STextFeatures FEATURES = new STextFeatures(":?#/@.[]", 0, -1, -1, false, false); //$NON-NLS-1$ + + /** + * This method retrieves the features specific to this processor. + * + * @return features with separators ":?#/@.[]", + * no special cases, LTR direction for Arabic and Hebrew, + * and support for both. + */ + public STextFeatures getFeatures(STextEnvironment env) { + return FEATURES; + } + +} Index: src/org/eclipse/equinox/bidi/internal/consumable/STextUnderscore.java =================================================================== RCS file: src/org/eclipse/equinox/bidi/internal/consumable/STextUnderscore.java diff -N src/org/eclipse/equinox/bidi/internal/consumable/STextUnderscore.java --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ src/org/eclipse/equinox/bidi/internal/consumable/STextUnderscore.java 1 Jan 1970 00:00:00 -0000 @@ -0,0 +1,38 @@ +/******************************************************************************* + * Copyright (c) 2010, 2011 IBM Corporation and others. + * All rights reserved. This program and the accompanying materials + * are made available under the terms of the Eclipse Public License v1.0 + * which accompanies this distribution, and is available at + * http://www.eclipse.org/legal/epl-v10.html + * + * Contributors: + * IBM Corporation - initial API and implementation + ******************************************************************************/ +package org.eclipse.equinox.bidi.internal.consumable; + +import org.eclipse.equinox.bidi.STextEnvironment; +import org.eclipse.equinox.bidi.custom.STextFeatures; +import org.eclipse.equinox.bidi.custom.STextProcessor; + +/** + * Processor adapted to processing compound names. + * This type covers names made of one or more parts, separated by underscores: + * 
+ *    part1_part2_part3
+ *
+ */ +public class STextUnderscore extends STextProcessor { + static final STextFeatures FEATURES = new STextFeatures("_", 0, -1, -1, false, false); //$NON-NLS-1$ + + /** + * This method retrieves the features specific to this processor. + * + * @return features with one separator (underscore), + * no special cases, LTR direction for Arabic and Hebrew, + * and support for both. + */ + public STextFeatures getFeatures(STextEnvironment env) { + return FEATURES; + } + +} Index: src/org/eclipse/equinox/bidi/internal/consumable/STextXPath.java =================================================================== RCS file: src/org/eclipse/equinox/bidi/internal/consumable/STextXPath.java diff -N src/org/eclipse/equinox/bidi/internal/consumable/STextXPath.java --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ src/org/eclipse/equinox/bidi/internal/consumable/STextXPath.java 1 Jan 1970 00:00:00 -0000 @@ -0,0 +1,41 @@ +/******************************************************************************* + * Copyright (c) 2010, 2011 IBM Corporation and others. + * All rights reserved. This program and the accompanying materials + * are made available under the terms of the Eclipse Public License v1.0 + * which accompanies this distribution, and is available at + * http://www.eclipse.org/legal/epl-v10.html + * + * Contributors: + * IBM Corporation - initial API and implementation + ******************************************************************************/ +package org.eclipse.equinox.bidi.internal.consumable; + +import org.eclipse.equinox.bidi.STextEnvironment; +import org.eclipse.equinox.bidi.custom.STextFeatures; +import org.eclipse.equinox.bidi.internal.STextDelims; + +/** + * Processor adapted to processing XPath expressions. + */ +public class STextXPath extends STextDelims { + static final STextFeatures FEATURES = new STextFeatures(" /[]<>=!:@.|()+-*", 2, -1, -1, false, false); //$NON-NLS-1$ + + /** + * This method retrieves the features specific to this processor. + * + * @return features with separators " /[]<>=!:@.|()+-*", + * 2 special cases, LTR direction for Arabic and Hebrew, + * and support for both. + */ + public STextFeatures getFeatures(STextEnvironment env) { + return FEATURES; + } + + /** + * @return apostrophe and quotation mark as delimiters. + */ + protected String getDelimiters() { + return "''\"\""; //$NON-NLS-1$ + } + +} Index: src/org/eclipse/equinox/bidi/package.html =================================================================== RCS file: /cvsroot/rt/org.eclipse.equinox/components/bundles/org.eclipse.equinox.bidi/src/org/eclipse/equinox/bidi/package.html,v retrieving revision 1.2 diff -u -r1.2 package.html --- src/org/eclipse/equinox/bidi/package.html 27 Apr 2011 14:53:08 -0000 1.2 +++ src/org/eclipse/equinox/bidi/package.html 25 May 2011 16:24:21 -0000 @@ -6,10 +6,10 @@ This package provides interfaces and classes for -processing complex expressions. +processing structured text. -

Introduction to Complex Expressions

+

Introduction to Structured Text

Bidirectional text offers interesting challenges to presentation systems. For plain text, the Unicode Bidirectional Algorithm @@ -23,28 +23,28 @@ has no awareness of these special cases, often gives incorrect results when displaying such structured text.

-The general idea in handling complex expressions is to add directional +The general idea in handling structured text is to add directional formatting characters at proper locations in the text to supplement the standard algorithm, so that the final result is correctly displayed using the UBA.

-A class which handles complex expressions is thus essentially a +A class which handles structured text is thus essentially a transformation engine which receives text without directional formatting characters as input and produces as output the same text with added directional formatting characters, hopefully in the minimum quantity which is sufficient to ensure correct display, considering the type of -complex expression involved. +structured text involved.

In this package, text without directional formatting characters is called lean text while the text with added directional formatting characters is called full text.

The class {@link -BidiComplexEngine} -is the main tool for processing complex expressions. It facilitates -handling several types of complex expressions, each type being handled +STextEngine} +is the main tool for processing structured text. It facilitates +handling several types of structured text, each type being handled by a specific -{@link processor} :

+{@link processor} :

  • property (name=value)
  • compound name (xxx_yy_zzzz)
    • @@ -62,43 +62,43 @@

    Other classes and one interface in this package may be used to complement and facilitate the action of -{@link BidiComplexEngine}: +{@link STextEngine}:

      -
    • {@link BidiComplexEnvironment} +
    • {@link STextEnvironment} regroups details about the environment
      • -
    • {@link BidiComplexUtil} +
    • {@link STextUtil} provides a number of convenience methods to process some common types of - complex expressions. When using methods in this class, there is no need + structured text. When using methods in this class, there is no need to use other classes of this package. However, the other classes allow more precise control and possibly better performance.
      • -
    • {@link IBidiComplexExpressionTypes} +
    • {@link ISTextTypes} is an interface which contains only literals identifying the various types - of complex expressions currently supported.
      • -
    • {@link BidiComplexStringRecord} - allows to record strings which are complex expressions with their + of structured text currently supported.
      • +
    • {@link STextStringRecord} + allows to record strings which are structured text with their type, so that another module can check if a given string has been recorded - as a complex expression and retrieve its type.
      • + as a structured text string and retrieve its type.

    -{@link BidiComplexEngine} and the +{@link STextEngine} and the other classes and interface mentioned above are intended for users who -need to process complex expressions for which there already exist +need to process structured text for which there already exist processors.

    -Developers who want to develop new processors to support types of complex -expressions not currently supported can use the following components of the +Developers who want to develop new processors to support types of structured text +not currently supported can use the following components of the package {@link org.eclipse.equinox.bidi.custom}:

    @@ -111,12 +111,12 @@ org.eclipse.equinox.bidi.internal.consumable}
The tools in the first package can serve as example of how to develop -processors for currently unsupported types of complex expressions.
+processors for currently unsupported types of structured text.
The latter package contains classes for the processors implementing -the currently supported types of complex expressions. +the currently supported types of structured text.

However, users wishing to process the currently supported types of -complex expressions typically don't need to interact with these +structured text typically don't need to interact with these two packages.

 

@@ -169,7 +169,7 @@ PDF directional formatting characters.
  • Because some presentation engines are not strictly conformant to the -UBA, the implementation of complex expressions in this package adds LRM +UBA, the implementation of structured text in this package adds LRM or RLM characters in association with LRE, RLE or PDF in cases where this would not be needed if the presentation engine was fully conformant to the UBA. Such added marks will not have harmful effects on