Community
Participate
Working Groups
I20090113-0918 Description Resource Path Type Location The default encoding of 'Cp1252' does not match the project specific encoding of 'ISO-8859-1' build.properties org.eclipse.jdt.ui.tests Plug-in Problem line 1
Wrong? See: http://eclipsesource.com/blogs/2009/01/09/tip-encoding-issues-with-plug-ins/ We think it's better to warn the user if he forgot to set the javacCustomEncodings directive in the build.properties when a specific encoding has been set at project level. Of course you can choose to ignore this compiler error More info in bug 155015.
The encoding of build.properties is defined to be ISO-8859-1 by its content type. Are you saying that PDE build changes the bits in build.properties depending on the project encoding? That would be a bug that should be fixed in PDE build, and not something the user should be warned about. I also don't understand why PDE build cannot just read the encodings from .settings/org.eclipse.core.resources.prefs .
Oh, or is this problem not about the encoding of the build.properties file at all? Then I think the problem message should explicitly tell that this is a problem with PDE build, e.g.: The default encoding used by PDE build ('Cp1252') does not match the project specific encoding ('ISO-8859-1'). And even better would of course be if PDE build understood the encoding specified in the workspace.
(In reply to comment #3) > Oh, or is this problem not about the encoding of the build.properties file at > all? Then I think the problem message should explicitly tell that this is a > problem with PDE build, e.g.: > > The default encoding used by PDE build ('Cp1252') does not match the project > specific encoding ('ISO-8859-1'). > > And even better would of course be if PDE build understood the encoding > specified in the workspace. So is the request here to improve the message? The message is very valid and helps people catch issues when their project encoding differs to what PDE Build would produce in an export or headless build.
> So is the request here to improve the message? Yes. I first though it was an invalid problem about the encoding of the build.properties file (since the problem is reported on that file). But actually, I just didn't understand the problem message, so I think it should specifically tell that this is a problem for PDE build.
It could be a coincidence, but in all builds where the quick-fix generated line javacDefaultEncoding.javauitests.jar = ISO-8859-1 was in /org.eclipse.jdt.ui.tests/build.properties and /org.eclipse.jdt.ui.tests.refactoring/build.properties, the builds failed because the generated jars in the two test plug-ins were almost empty (except for MANIFEST.MF and a few non-Java resources). I've reverted the build.properties for N20090118-2000, and that build was OK again.
I hit that problem once [1] , when I wanted to force UTF-8 encoding in the pde.ui.tests plug-in (because a DBCS related test case needed that...), this may be a problem with PDE Build itself? [1] see bug 225047 comment 10 and following...
Increasing to blocker as applying the quick fix causes the plug-in to fail during the official build which is very bad. Either PDE Build gets a fix asap or the problem must not be reported until the build issue is understood and fixed. Darin, can you please take care that this gets resolved for the M-build? Thanks.
I commented out the validation code for now. Andrew and I will discuss the best approach on how to move forward.
The algorithm was modified to only check project specific encoding settings versus the project AND the workspace (this is the default behavior of getDefaultCharset()...) After talking with Andrew, this seems like the best solution. The reason we were hitting other issues was because the project wouldn't have a specific encoding but the workspace did. Sorry for the churn. > 20090120
(In reply to comment #10) > The algorithm was modified to only check project specific encoding settings > versus the project AND the workspace (this is the default behavior of > getDefaultCharset()...) The reason for this is that there is no workspace in the releng build. Also, I believe the workspace default depends on the platform/vm, which will be different in the releng build.
Sorry guys, I have to reopen this again. I just downloaded N20090120-2000 (WinXP) and rebuilt my workspace and exactly the same errors as reported in comment 0 still appear on: org.eclipse.jdt.ui.tests org.eclipse.jdt.ui.tests.refactoring and the quick fix inserts the same entry which causes the build to fail: javacDefaultEncoding.refactoringtests.jar = ISO-8859-1 May I suggest that you test your next fix using the two mentioned projects checked out from CVS with tag: "v20090120-0800"? Thanks. Andrew, shouldn't PDE Build be improved to give more information when the build fails in such situations? We (releng and I) had no hint in any build result or .log file i.e. we could only guess about recent changes and try to undo them.
(In reply to comment #12) > Andrew, shouldn't PDE Build be improved to give more information when the build > fails in such situations? We (releng and I) had no hint in any build result or > .log file i.e. we could only guess about recent changes and try to undo them. > Build is simply passing the encoding to the compiler, it never reads the source itself. Any additional information about what is going wrong needs to come from the compiler.
With compiler you mean the ECJ?
Generally yes, we are just calling the javac ant task, and by default this goes to org.eclipse.jdt.core.JDTCompilerAdapter. The file specific encodings are actually set with help from the JDTCompilerAdapter and are not supported on other compilers. Default encodings are set on javac's "encoding" attribute.
>The file specific encodings are >actually set with help from the JDTCompilerAdapter and are not supported on >other compilers. Default encodings are set on javac's "encoding" attribute. When you say "are not supported" you mean by Eclipse PDE build? Because javac well supports this via -encoding command line option. Also, I'm not convinced that this caused the problem because almost the affected projects do compile with the explicit given ISO-8859-1 encoding (on both WindowsXP and Linux). Rather I guess passing along that property breaks something during PDE build.
We need to come to a resolution for the M5 build. Currently, the UI is reporting an error (which seems valid). However, using the quick fix causes the build to fail. Exporting the plug-in from the UI appears to work with or without the encoding property. Andrew pointed out that we should be specifying encodings for all files in the plug-in that do not match the top level encoding (in case there are mixed encodings). This can be done something like: javacCustomEncodings.library.jar = src/org/foo[US-ASCII], src/org/foo/bar.java[ISO-8859-1] I propose to suppress this warning for M5, and revisit in M6. Chris?
I've commented out the validation for now. I'm not sure why things are breaking, I figure most people would set UTF-8 as a project specific setting and this would get people to correctly ensure their plug-ins are built to that encoding. As you can get different results if you build your plug-ins on Linux, Mac and Windows if this isn't set.
>I figure most people would set UTF-8 as a project specific setting Right, but some don't. That's why this warning and the setting got introduced ;-) Unfortunately, specifying the encoding seems to break PDE build or one of the tools it uses.
moving to M7 for investigation
removing milestone and setting as enhancement since functionality was removed
*** Bug 274786 has been marked as a duplicate of this bug. ***
Does the WARNING also verifies different encoding within a project (eg., some files UTF-8, some ISO...)?
>Does the WARNING also verifies different encoding within a project (eg., some >files UTF-8, some ISO...)? Interesting question. It is a very common setup to e.g. have UTF-8 Java source files (as set on the project) along with Java properties files that are ISO-8859-1 encoded. How does PDE build handle this?
PDE/Build lets you specify encodings per file or folder, I doubt UI is generating these properties.
>PDE/Build lets you specify encodings per file or folder, I doubt UI is >generating these properties. But by default it uses the project or file specific settings? If not, shouldn't that be the case? Otherwise we'd have to duplicate all the information for each custom encoding on a folder or file in the build.properties.
There are two properties: javacDefaultEncoding.<library> - set the default for the whole library. This is passed to the ant javac task's "encoding" attribute and applies to all the source files. Then there is javacCustomEncodings.<library>, these are passed with help from the JDTCompilerAdapter and specify encodings (for files or folders) that override the default above. If neither are set, then it is whatever batch compiler uses by default.
So, is this only about the COMPILER encoding for reading *.java source files and does e.g. not affect *.properties files?
Yes, we really only care about *.java files. The only other files I can think of that get read during the build are: Manifest.mf : spec'ed to be UTF-8 feature.xml, .product file: handled by xml parsing build.properties, p2.inf : spec'ed by Properties class to be ISO 8859-1
Good. This proves that the problem message must be changed indeed, since "The default encoding of 'Cp1252' does not match the project specific encoding of 'ISO-8859-1'" is confusing and actually it is irrelevant in this case that the current default encoding is 'Cp1252' since it might be completely different on build time (e.g. when built on a Linux box). How about: Compiler encoding must be specified since the project defines 'xyz' as its encoding ?
(In reply to comment #30) > Compiler encoding must be specified since the project defines 'xyz' as its > encoding +1 Would also work with "Compile encoding for file 'xyz' must be specified since the file defines 'ABC' as its encoding. However, I still think the warning is pretty annoying because it's avoidable most of the time. The custom encoding is already specified in the ".settings/org.eclipse.core.resources.pref" file. PDE Build could parse/read that file at build file generation time and set the properties for the compiler. Most projects which do use PDE Build also commit the ".settings" folder into the SCM. Thus, it's there at build time. Maybe the build.properties just needs an entry that instructs PDE Build to re-use that information? javacCustomEncodings.library.jar = .settings/org.eclipse.core.resources.pref[_PREF_]
(In reply to comment #31) > However, I still think the warning is pretty annoying because it's avoidable > most of the time. The custom encoding is already specified in the > ".settings/org.eclipse.core.resources.pref" file. PDE Build could parse/read > that file at build file generation time and set the properties for the > compiler. Most projects which do use PDE Build also commit the ".settings" > folder into the SCM. Thus, it's there at build time. Maybe the build.properties > just needs an entry that instructs PDE Build to re-use that information? > > javacCustomEncodings.library.jar = > .settings/org.eclipse.core.resources.pref[_PREF_] The problem in general is that the contents of those preference files are implementation details of whoever wrote them. We don't want to keep track of other bundle's internals. Bug 291528 is going to do this with org.eclipse.jdt.core.pref, however in that case PDE/Build is just passing the file to jdt.core who happens to be the one who wrote it. For the encodings, I don't expect that jdt.core know hows to interpret org.eclipse.core.resources.pref any more than Build does.
(In reply to comment #32) > The problem in general is that the contents of those preference files are > implementation details of whoever wrote them. We don't want to keep track of > other bundle's internals. IIRC, the problem in general is that PDE/Build does not know what a workspace or an Eclipse project is, so it can't use the existing APIs to read the existing settings (IFile#getCharset(..) and IContainer#getDefaultCharset(..)).
I still think this is wrong and forcing the user to duplicate all that information (I hate duplication ! ;-) just to make PDE Build happy is lame - we can do better. core.resource could make the format of the encoding entries in the settings file public, so that PDE can parse the file.
The problems with the bundles missing class files are because the releng build system sets the following as default compiler arguments: "-inlineJSR -enableJavadoc -encoding ISO-8859-1 -warn:-discouraged,forbidden,warningToken" And, setting the "encoding" property on the javac task (which is what the javacDefaultEncoding.<library> property does) adds a second "-encoding ISO-8859-1" to the command line generated by the JDTCompilerAdapter. We then have two "-encoding" arguments on the compiler command line. The Batch compiler throws an IllegalArgumentException on the second one. This will be a problem in our releng setup for any bundle setting the "javacDefaultEncoding.<library>" property. Presumably jdt.ui.tests is one of the few bundles that (1) has custom encodings set on the project and (2) has committers that pay attention to warnings and made the change :) I see two options: - The JDTCompilerAdapter gets changed to accept multiple -encoding arguments and to perhaps favor the <javac> task encoding attribute over any other <compilerargs>. (Favor the last -encoding encountered, and when creating the command line, add the encoding attribute after adding the compiler args). - The releng build gets changed to not specify -encoding as a default argument
Moving to JCore for comment. My feeling is that it is less risky to have the compiler accept mutlitple encoding arguments rather than change the build. Olivier, how do you feel about this? Perhaps it would make sense to log a warning/information message that multiple encodings were specified (if they conflict), and which one was accepted. In the case that all are the same, no warning is neccessary. Pending the outcome of this fix, we could re-release the feature/fix in bug 155015 to validate/synch encoding settings in build.properties. I think we could have the default severity level at "ignore".
(In reply to comment #36) > Moving to JCore for comment. My feeling is that it is less risky to have the > compiler accept mutlitple encoding arguments rather than change the build. > Olivier, how do you feel about this? Perhaps it would make sense to log a > warning/information message that multiple encodings were specified (if they > conflict), and which one was accepted. In the case that all are the same, no > warning is neccessary. I can change the compiler to accept more than once -encoding on the command line. I only need to update the documentation to explain that the last one will actually set the encoding for the project. I'll work on a fix.
comment 36 has been implemented in bug 310330. *** This bug has been marked as a duplicate of bug 310330 ***
I just wanted to note that this one looks like it's the same as my bug 261395. Glad to see that its being addressed as I wasn't happy with the original solution either. (Since the Cocoa build doesn't use MacRoman it hasn't come up as an issue for me anymore.)
Verified for 3.6M7
