Community
Participate
Working Groups
The following code demonstrates two problems with mapped positions: 1) position at end of source gets mapped to 0 2) positions are not mapped correctly if the source contains multi-line comments. For 2), it seems that the wrong offset of a position is equal to the number of line delimiters in the multi-line comment. I will workaround this by escaping multi-line comments with single-line comments, but it would be nice to have this fixed. ---8<--- source ---8<--- import org.eclipse.jdt.internal.formatter.CodeFormatter; public class CodeFormatterBug { private static void print(String string, int[] positions) { for (int i= 0; i < positions.length; i++) System.out.print(positions[i] + " "); System.out.println(""); for (int i= 0; i < positions.length - 1; i++) { System.out.print("#"); // mark position System.out.print(string.substring(positions[i], positions[i + 1])); } } public static void main(String arguments[]) { String string= "/**\n" + " * foo\n" + " */\n" + "Foo Bar\n" + "blah blah\n"; int[] positions= { 0, string.indexOf("foo"), string.indexOf("Foo"), string.indexOf("Bar"), string.length() }; CodeFormatter formatter= new CodeFormatter(); System.out.println("before:"); print(string, positions); formatter.setPositionsToMap(positions); string= formatter.formatSourceString(string); positions= formatter.getMappedPositions(); System.out.println("after:"); print(string, positions); } } ---8<--- output ---8<--- before: 0 7 15 19 33 #/** * #foo */ #Foo #Bar blah blah after: 0 12 20 24 0 #/** * foo # */ Fo#o Ba#java.lang.StringIndexOutOfBoundsException: String index out of range: - 24 at java.lang.String.substring(String.java:1504) at CodeFormatterBug.print(CodeFormatterBug.java:12) at CodeFormatterBug.main(CodeFormatterBug.java:42) Exception in thread "main"
Additional information: I need this feature to support multi-line comments in templates. I'd like to create templates for e.g. public methods: /** * */ public ${return_type} ${name}(${arguments}) { ${cursor} } Right now, I have no good way to catch this situation.
Fix done in the CodeFormatter class. Several problems were found with the multiple lines comments. All code formatter tests passed.
I was wrong with my verification. The bug is not fixed. I run the program above again and got the same result.
The fix is done only in HEAD. It hasn't been integrated in any build yet. If you want me to set it up on your machine let me know. You simply need to replace the CodeFormatter file.
Oh, I see. Can you send me the file, or would it be already in the build 20011114?
Unfortunately it seems that this fix is not included in the 20011115 build according to the release note. I will send you the file. It is only one class to change. You need to patch your jdtcore.jar file in your install.
I have problems testing because of another bug. It seems the fixed CodeFormatter is still not in the repository. When is it going to be released?
Ok, I can verify that the CodeFormatter is fixed for the test case. Still waiting for it to be checked into the repository... :)
According to the build notes provided for the next integration build (20011120), it should be included in this build. It should be available this afternoon (Ottawa's time).
Unfortunately, I found another test case, where positions are not mapped correctly. The position in front of comment gets mapped into the comment. Note that if you remove the last line, the mapped position is correct. ---8<--- package foo; import org.eclipse.jdt.internal.formatter.CodeFormatter; public class CodeFormatterBug2 { private static void print(String string, int[] positions) { for (int i= 0; i < positions.length; i++) System.out.print(positions[i] + " "); System.out.println(""); for (int i= 0; i < positions.length - 1; i++) { System.out.print("#"); // mark position System.out.print(string.substring(positions[i], positions[i + 1])); } } public static void main(String arguments[]) { String string= "foo\n" + "/*${cursor}*/\n" + "Foo\n"; // bug doesn't occur if you remove this line int[] positions= { 0, string.indexOf("/"), string.length() }; CodeFormatter formatter= new CodeFormatter(); System.out.println("before:"); print(string, positions); formatter.setPositionsToMap(positions); string= formatter.formatSourceString(string); positions= formatter.getMappedPositions(); System.out.println("after:"); print(string, positions); while(true); } } ---8<--- output --- before: 0 4 22 #foo #/*${cursor}*/ Foo after: 0 6 23 #foo /#*${cursor}*/ Foo
This is a issue with the line separator. When the line separator (\n) is consumed after the foo token it is replaced by \r\n which contains an extra character. The offset is actually one and not two. Right now I take two instead of one and this explain the offset of one. I am looking at a way to set properly the offset between formatted source and original source. If you change your test case to set the line separator of the code formatter to be \n, then the positions are right even if you keep the last line.
This problem did not occure befor the other bug fix. Was it some coincidence that it happened to work? I'll use the workaround of specifying the same line delimiter used in the string for now.
Yes, this was a side-effect. But before the formatter could never map properly if the line delimiter has 2 characters and the string you want to format contained only one character line delimiter and the string contained a multiple line comment and it was a coincidence that this code was formatted properly. That change had to be done. I found the problem. I simply forgot to map the positions inside the comment in case there is not line delimiters in the comment. I fixed it, but I have trouble to run the tests inside my 20011116 workspace. As soon as I can do it and I don't find any regression, I release it.
The fix is released in HEAD. I don't know when it will be integrated in a build.
It's me again :-) I have a case where the position before a multi-byte line-break ends up being inside a multi-byte line-break. This has severe consequences when used as a selection in a StyledText. Although I think StyledText should not choke on selection inside multi-byte line-breaks, the CodeFormatter should not produce these off-by-one positions... So far, this only happened within a multi-line comment. ---8>--- package foo; import org.eclipse.jdt.internal.formatter.CodeFormatter; public class CodeFormatterBug3 { private static void print(String string, int[] positions) { for (int i= 0; i < positions.length; i++) System.out.print(positions[i] + " "); System.out.println(""); for (int i= 0; i < positions.length - 1; i++) { System.out.print("#"); // mark position System.out.print(string.substring(positions[i], positions[i + 1])); } } public static void main(String arguments[]) { String string= "foo\r\n" + "/**X\r\n" + " * foo\r\n" + " */\r\n" + "bar"; int[] positions= { 0, string.indexOf("X") + 1, string.length() }; CodeFormatter formatter= new CodeFormatter(); System.out.println("before:"); print(string, positions); formatter.setPositionsToMap(positions); string= formatter.formatSourceString(string); positions= formatter.getMappedPositions(); System.out.println("after:"); print(string, positions); } } ---8<---output--- before: 0 9 27 #foo /**X# * foo */ barafter: 0 10 27 #foo /**X # * foo */ bar
Why is it possible to select a position inside a line break? Of course there is no protection against this. The line breaks are written in an atomic manner. This explains the offset by one after it. I will try to find a solution for that, but I think such a position doesn't make sense to me. Did you find another bug except this position inside a line break?
I have a fix for it and I added tests in the formatter tests suite. I simply didn't handle this case, because I thought it was not possible to put a marker inside a line break. Now it is handled and let me know if you have other problems with the mapping positions. Release in HEAD.
My position is before mapping is the *beginning* of the line-break, while after the format, the mapped position happens to be *inside* the line-break. The correct behaviour would be for the position to stay at the beginning of the line-break. My positionsToMap are valid, but not the mappedPositions.
ignore my last comment, I didn't see your second comment. I'll check the jar and tell you if it works.
According to your test case, it should work. I map to a position that makes sense. The '#' is at the same place using your test class. Let me know if it doesn't work for you.
I verified that this bug is fixed. Go ahead and release it. However, I have seen other positions which didn't map correctly either... I think I'm going to write a small position map test for the code formatter...
Sure, I'd like to know when the positions are not mapped properly.
