Community
Participate
Working Groups
Our automated tool, chkpii, has detected a number of problems with quote handling in java property files being processed by the MessageFormat class. These problems need to be fixed in the english property files before they are sent for translation, and the bug is propogated to the translated version as well. I'm including info from the tool on how to fix the problems for those who are outside of ibm and may not have access to it. In addition, when we process the files we add the line: # NLS_MESSAGEFORMAT_VAR to the beginning of each file. This insturcts chkpii on how we are handling property strings in the code. ECLIPSE\PLUGINS\ORG.ECLIPSE.JDT.CORE\JDTCORE_JAR\ORG\ECLIPSE\JDT\INTERNAL\COMPIL ER\BATCH\ MESSAGES.PROPERTIES JAVA2-PRB 918 Found 1 single quote in text handled by Java MessageFormat class. Double this quote. Line: 63 ECLIPSE\PLUGINS\ORG.ECLIPSE.JDT.CORE\JDTCORE_JAR\ORG\ECLIPSE\JDT\INTERNAL\COMPIL ER\PROBLEM\ MESSAGES.PROPERTIES JAVA2-PRB 919 Found 2 consecutive single quotes in text not handled by Java MessageFormat class. Remove 1 quote. Line: 108 919 Found 2 consecutive single quotes in text not handled by Java MessageFormat class. Remove 1 quote. Line: 203 919 Found 2 consecutive single quotes in text not handled by Java MessageFormat class. Remove 1 quote. Line: 333 ECLIPSE\PLUGINS\ORG.ECLIPSE.JDT.CORE\JDTCORE_JAR\ORG\ECLIPSE\JDT\INTERNAL\CORE\U TIL\ MESSAGES.PROPERTIES JAVA2-PRB 919 Found 2 consecutive single quotes in text not handled by Java MessageFormat class. Remove 1 quote. Line: 188 ECLIPSE\PLUGINS\ORG.ECLIPSE.JDT.JUNIT\JUNITSUPPORT_JAR\ORG\ECLIPSE\JDT\INTERNAL\ JUNIT\UI\ JUNITMESSAGES.PROPERTIES JAVA2-PRB 918 Found 1 single quote in text handled by Java MessageFormat class. Double this quote. Line: 167 ECLIPSE\PLUGINS\ORG.ECLIPSE.JDT.UI\JDT_JAR\ORG\ECLIPSE\JDT\INTERNAL\UI\ JAVAUIMESSAGES.PROPERTIES JAVA2-PRB 918 Found 1 single quote in text handled by Java MessageFormat class. Double this quote. Line: 90 ECLIPSE\PLUGINS\ORG.ECLIPSE.JDT.UI\JDT_JAR\ORG\ECLIPSE\JDT\INTERNAL\COREXT\REFAC TORING\ REFACTORING.PROPERTIES JAVA2-PRB 919 Found 2 consecutive single quotes in text not handled by Java MessageFormat class. Remove 1 quote. Line: 465 918 Found 1 single quote in text handled by Java MessageFormat class. Double this quote. Line: 768 ECLIPSE\PLUGINS\ORG.ECLIPSE.JDT.UI\JDT_JAR\ORG\ECLIPSE\JDT\INTERNAL\UI\REFACTORI NG\ REFACTORINGUI.PROPERTIES JAVA2-PRB 918 Found 1 single quote in text handled by Java MessageFormat class. Double this quote. Line: 74 ------------DESCRIPTION-------------- 918 Found 1 single quote in text handled by Java MessageFormat class. Double this quote. Line: nnn Description: In November, 2004, the translation process changed to allow development to indicate whether the message text will be processed by the Java MessageFormat class or not. If the text will be processed by MessageFormat, then all single quotes in the text must be doubled in order for the end user to see one single quote. MessageFormat assumes that the single quote character is an escape character. A single quote has been found which is not immediately followed by another single quote ('), a left brace ({), or a right brace (}). Possible causes are: The developer incorrectly coded the English text. The single quote should have been coded as 2 consecutive single quotes because the text will be processed by MessageFormat. The translator incorrectly entered a single quote in the text and the translation tools did not double this character. The wrong NLS_MESSAGEFORMAT_xxx comment is in effect for this text. This has caused CHKPII to check for the wrong usage of single quotes. The NLS_MESSAGEFORMAT_VAR comment is in effect for this text which incorrectly contains a replacement variable. When the text contains replacement variables, NLS_MESSAGEFORMAT_VAR indicates that all single quotes must exist as 2 consecutive single quotes. When this message is listed as a warning (W), then the IBMJDK21 set of markup tables was used to translate this file, which assumes that NLS_MESSAGEFORMAT_VAR is the default process. A single quote exists in text which contains a replacement variable, and possibly should be doubled. Action: Locate the listed line number in the file and locate the incorrectly coded single quote. If the text will be processed by MessageFormat, then change the one single quote (') to 2 consecutive single quotes (''). Consider using a different character if this will not change the meaning of the text (like double quote ("), parenthesis (), bracket []). If the wrong NLS_MESSAGEFORMAT_xxx comment is in effect, contact development to determine if a different comment should be used for this text. If the NLS_MESSAGEFORMAT_VAR comment is in effect and the text contains text which is interpreted as a replacement variable, check the English to determine if a variable should exist in this text or not. Remove it if it should not be in this text. If the lab instructed you to use IBMJDK21, then change the one single quote (') to 2 consecutive single quotes (''). Otherwise, determine if IBMJDK11 should have been used to analyze this file or verify that 1 single quote is appropriate in this text. 919 Found 2 consecutive single quotes in text not handled by Java MessageFormat class. Remove 1 quote. Line: nnn Description: In November, 2004, the translation process changed to allow development to indicate whether the message text will be processed by the Java MessageFormat class or not. If the text will be processed by MessageFormat, then all single quotes in the text must be doubled in order for the end user to see one single quote. MessageFormat assumes that the single quote character is an escape character. 2 consecutive single quotes have been found in text which will not be processed by MessageFormat. Possible causes are: The developer incorrectly coded the English text. The 2 consecutive single quotes should have been coded as 1 single quote because the text will not be processed by MessageFormat. The translator incorrectly entered 2 consecutive single quotes in the text. The wrong NLS_MESSAGEFORMAT_xxx comment is in effect for this text. This has caused CHKPII to check for the wrong usage of single quotes. The NLS_MESSAGEFORMAT_VAR comment is in effect for this text which incorrectly does not contains a replacement variable. When the text does not contain replacement variables, NLS_MESSAGEFORMAT_VAR indicates that all single quotes must exist as 1 single quote. When this message is listed as a warning (W), then either (1) the IBMJDK21 set of markup tables was used to translate this file (which assumes that NLS_MESSAGEFORMAT_VAR is the default process) and 2 consecutive single quotes exist in text which does not contains a replacement variable and possibly should not be doubled, or (2) the IBMJDK11 set of markup tables was used to translate this file (which assumes that NLS_MESSAGEFORMAT_NONE is the default process) and 2 consecutive single quotes exist in the text and possibly should not be doubled. Action: Locate the listed line number in the file and locate the incorrectly coded single quote. If the text will not be processed by MessageFormat, then change the 2 consecutive single quotes ('') to 1 single quote ('). Consider using a different character if this will not change the meaning of the text (like double quote ("), parenthesis (), bracket []). If the wrong NLS_MESSAGEFORMAT_xxx comment is in effect, contact development to determine if a different comment should be used for this text. If the NLS_MESSAGEFORMAT_VAR comment is in effect and the text does not a replacement variable, check the English to determine if a variable should exist in this text or not. Add it if it should be in this text. If the lab instructed you to use IBMJDK21, then change the 2 consecutive single quote ('') to 1 single quote ('). Otherwise, verify that 2 consecutive single quotes is appropriate in this text.
Changing version to 3.1. These fixes aren't necessary for the maintenance stream.
What version was used to check this?
We actually ran chkpii on the maintenance stream, but when we ran it on the 3.1 stream, the output appeared to be the same.
I meant what version of the chkpii tool was used.
Version 6.12 (the latest)
What would be the option to specify to get these warnings?
chkpii *.* /s /jsq /s for subdirectories /jsq for the quote problem
So you mean that the double-single quote should only be used if the line contains one {0} or more?
Yes, you should only double the single quotes if there is variable substitution in the string.
Fix applied for JDT/Core. Moving to JDT/UI for fixes in JDT/UI files.
Created attachment 16955 [details] Apply on org.eclipse.jdt.junit project in HEAD
Created attachment 16956 [details] Apply on org.eclipse.jdt.ui project in HEAD
Created attachment 16968 [details] Patch to fix releng chkpii test I've tested the patch under WinXp and Linux.
I've updated the ZRH chkpii test to report those failures. Fixed the problems in HEAD ==> chkpii test is green again. The releng chkpii test also does not report those problems. I've attached a patch which fixes this. Moving to releng to consider the patch.
Jonathan - The JDT/Core message catalogs are not processed by MessageFormat but by our custom implementation.
In that case you should place the comment # NLS_MESSAGEFORMAT_NONE on the first line of all applicable properties files. When chkpii sees this comment, it should not check for any doubling of single quotes. Try it and let me know if it works. Its important to add this to all of your files because when we send the files for translation, we add the NLS_MESSAGEFORMAT_VAR line to all files which don't specify a policy. The translators will use this comment to determine how to handle single quotes in their translations.
This problem is fixed in the JDT files in N20050110-0010. The following files remain: ECLIPSE\PLUGINS\ORG.ECLIPSE.COMPARE_3.1.0 \COMPARE_JAR\ORG\ECLIPSE\COMPARE\CONTENTMERGEVIEWER\ TEXTMERGEVIEWERRESOURCES.PROPERTIES JAVA2-PRB 919 Found 2 consecutive single quotes in text not handled by Java MessageFormat class. Remove 1 quote. Line: 24 ECLIPSE\PLUGINS\ORG.ECLIPSE.PDE.UI_3.1.0\PDEUI_JAR\ORG\ECLIPSE\PDE\INTERNAL\UI\ PDERESOURCES.PROPERTIES JAVA2-PRB 919 Found 2 consecutive single quotes in text not handled by Java MessageFormat class. Remove 1 quote. Line: 589 918 Found 1 single quote in text handled by Java MessageFormat class. Double this quote. Line: 623 918 Found 1 single quote in text handled by Java MessageFormat class. Double this quote. Line: 1331 ECLIPSE\PLUGINS\ORG.ECLIPSE.PDE_3.1.0\PDE_JAR\ORG\ECLIPSE\PDE\INTERNAL\ PDERESOURCES.PROPERTIES JAVA2-PRB 918 Found 1 single quote in text handled by Java MessageFormat class. Double this quote. Line: 41 918 Found 1 single quote in text handled by Java MessageFormat class. Double this quote. Line: 43
Philippe, with regard to comment #15, why does JDT/Core us its own implementation of MessageFormat? This may cause you problems down the road. MessageFormat is one of the JRE classes that benefits from improvements in Unicode support (and other NL improvements) in the JREs we support. We get regular pressure to move to using ICU4J for improved Unicode support. One of the key points we make to fend this off is that we get the same benefit (admitedly later on) as these improvements are roled into the JREs themselves. If JDT/Core is not using these JRE facilities, you may not get the benefits even after the improvements arrive in our supported JREs. I believe the OSGI layer also has their own implementation, but that is because they need to keep J2ME as their core JRE prereq rather than J2SE. java.text.MessageFormat is not in J2ME. In any event, we need to get rid of the CHKPII errors and warnings. This should be possible even if you continue to use your own implementation and have different rules for when to double up on single quotes. There are NLS_MESSAGEFORMAT_<VAR,NONE,ALL> directives that can be added to properties files to work around this. Jonathan or David can provide the details.
We fixed the errors, as our implementation doesn't really care about single/double quotes. Thus if we evolve to MessageFormat it should be a no brainer. For historical reasons we had to define our own implementation, since we had to be able to run our batch compiler (outside Eclipse) on tiny class libraries which did not provide this type. There was never a good reason to move on, seems like we should consider this (time permitting).
