Community
Participate
Working Groups
OS: windows 2003, RHEL 3.0, SLES 9 Beta3.5 Language: Japanese Build level: I20040422-200404231600 JDK version: IBM JDK 1.4.2 Beta Test case #: Steps to recreate problem: 1-on Eclipse Windows2003 (default encoding MS932), check out a project on RHEL made by Eclipse Linux-gtk (default encoding EUC-JP) 2-change encoding to EUC-JP at the project properties Info page 3- .... Error: come up many errors after/befor changing the encoding of project Expected Result: no erros come up. it seems that the compilation works fine when the EUC-JP project includes .encoding file that content is :EUC-JP, I find it can be generated by setting EUC-JP at Other combobox of Project propertiess Info page. I'd like to know whetehr it's an expected behavior.
Created attachment 9931 [details] screen shot of errors
CVS has nothing to do with the project encoding support. Moving to JDT/UI since the errors are Java compile errors
Andre, can you please comment on this.
This problem still exists in I200405211200. On SLES9 (UTF-8 locale), these errors were not displayed but unable to run the java applications, there was no main method error was displayed. When I change the Default encoding to EUC-JP at Other form Inherit from container (EUC-JP) on the project properties Info page on RHEL, the errors on Windows2003 didn't occure and the applications ran successfully on SLES9. The changing at Other field seems to create .settings/org.eclipse.core.resources.prefs file. I'm going to increase the severity since this scenario is important
Created attachment 11031 [details] zipped cvs directory created on EUC-JP That's not have live org.eclipse.core.resources file. All of files under english package contain English only and these files don't have the error. while files under japanese packge contains Japanese text and have the error
Please note that we do not send out resource deltas after the encoding of the enclosing project has been changed. This behavior is conistent with the workspace encoding setting found in Window > Preferences > Editors. Can you try to manually build the project after changing the project's encoding.
I tried to build manually but the situation was same. I tried to relaunch the Eclipse and rebuild, some of errors were gone but some errors were remained.
Please confirm that you forced a full rebuild by: - turning off 'Build Automatically" in Project menu - you performed a "Clean..." on the project in question - you did a 'Build Project' Some more Questions: - does the editor always show correct glyphs? - if a file had compile errors, could you get rid of the errors by modifying the file and saving it back (with auto build on)?
I didn't perform the "Clean...". Now all of th error were gone. Thanks! a quick question - Do I have to do the steps of compile manually for that scenario?
Yes, as long as we are not sending out resource-deltas when changing the encoding for a resource (file, folder, project, workspace), you'll have to manually rebuild the affected projects once, that is you'll have to remove the old build state via the "Clean..." command. I'm moving this bug to platform.core, however I don't think we need to change this behavior for 3.0. But we should document it somewhere.
ok thanks. Please document this.
Original summary: "[encoding] DBCS: improper compilation for check out different encoding project" Will make sure it gets addressed by doc.
*** Bug 67488 has been marked as a duplicate of this bug. ***
Actually, we would need a delta so as to force to recreate our index files and update the JavaModel. Index files and Java Model are updated on resource deltas using standard resource change listeners; not using a builder, since we want all the model and search to operate accurately independently from compiler actions. Likewise, if only one resource encoding got changed, it feels overkill to rebuild entire workspace. I would vote for platform to issue a content change on affected resources (playing the encoding propagation rules), all existing clients would then refresh normally as they should already.
*** Bug 64716 has been marked as a duplicate of this bug. ***
If not addressed for 3.0, then doc should mention the need to close/reopen all projects, plus manually find and discard all index files in metadata.
Moreover all rsc change listener implementors who did persist some metadata are out of sync once encoding has changed, and likely have no way to recover unless a delta is issued. Naively, a contents change would be all I would expect. Encoded contents are likely denoting sources, if so: contents == f(bytes, encoding), thus it doesn't strike me that either when bytes or encoding change, I would get a contents change delta.
To clarify my expectations. If we want backward compatibility with 2.1 change listeners, encoding changes should be materialized as a content changes on *every* affected children. So the delta should include all children. If not, then all listeners would have to be rewritten to do the propagation by themselves, which isn't easy since they would have to play the inheritance rules.
Raising severity.
Thought I should add: backward compatibility with 2.1 plugins that did the reading/decoding themselves is not an issue because we are talking about new 3.0 API here. 2.1 clients would only check the workspace global encoding anyway (they are not aware of IFile#getCharset).
I agree. My claim was rather that if you broadcast CONTENT change event, you will guarantee to be backward compatible with any plugin written for 2.1; as opposed to require all plugins to listen to a new type of event so as to understand encoding modification inducing some content changes.
Old Summary: [doc] suggest "Project->Clean..." after encoding is changed Changing summary to better describe problem.
Added bug 67884 to cover the missing implementation for the proposed content type notification in the runtime.
Created attachment 12525 [details] patch for org.eclipse.core.runtime
Created attachment 12526 [details] patch for org.eclipse.core.resources
Created attachment 12534 [details] patch for org.eclipse.core.runtime This is another patch for runtime with a clause on the content type listeners telling clients who uses the Resources bundle that they should not use this mechanism. Instead, the Resources bundle will react to content type changes and propagate them to clients via resource change notification.
Summary: For 3.0: 1). We have added a constant IResourceDelta.ENCODING which is spec'd as "a change in the resource's encoding". This flag is not yet used in resource deltas. 2). We added the setCharset/setDefaultCharset APIs to IFile and IContainer that take a progress monitor. API is spec'd that it will cause a workspace operation and changes will be reported in subsequent resource notification. 3). We deprecated the non-progress monitor setCharset/setDefaultCharset APIs on IFile and IContainer. Users are recommended to use the new APIs instead. People calling the old APIs will NOT be done within a workspace operation. 4). We added a listener mechanism to the content type manager. Clients are able to add/remove a listener that will get notification when content type changes happen. Adding and removing listeners are no-ops for 3.0; notification is not currently sent. 5). We added a clause to the content type listener telling clients who uses the Resources bundle that they should not use this mechanism. Instead, the Resources bundle will react to content type changes and propagate them to clients via resource change notification. post 3.0: 1). We will hook in the notification for the content type change listeners. (bug 67884) 2). We will hook the IResourceDelta.ENCODING flag into resource deltas. The resource bundle will also react to changes in the content type manager and propagate those via resource changes. (this bug) Phillipe and Andre, please comment on the proposed patches.
Do you really want to make this API change at this time? I'm scared. Wouldn't it be possible to introduce a UI action that scrubs all state that might be affected by encoding changes? Changing the encoding of a folder, project, or workspace would present a dialog that would ask whether to "Scrub all State". I'm not sure whether encoding changes can and should be handled automatically by core.resources. In addition to changing the encoding attribute of a resource hierarchy there is also the issue of *converting the contents* of resources to reflect the encoding attribute (e.g. convert ISO files to UTF-8 because it becomes necessary to deal with Japanese names). So sometimes only the attribute is changed, sometimes the contents, and in other cases both. Since these use cases won't be handled by core.resources, I don't think that core.resources needs to provide one partial solution.
I also believe that the addition of an API for 3.0 feels a bit late in the game, and I am not convinced it will resist to being implemented post 3.0. However, I would still favor having the platform broadcast a content change when encoding got changed. So as for our tools to properly react to these in an efficient manner. A full cleanup sounds like a workaround. Question about the proposed API: why introducing a new flag ? Couldn't it be retrofitted into CONTENT flag ? The ENCODING flag would pretty much require all tools to suddenly be made aware of encoding support, where they could have pretty much ignored it in the past, if all they were doing was to read resources having a default encoding. Also, is the delta being propagated down to all affected children (which do not have a specific encoding) ?
Trust me, it won't be implemented for 3.0. :-) If we want this fixed for 3.0.1, then we must introduce the new API now since we can't add API for a maintenance release. Content and encoding changes are not the same for all clients, that is why we created a new constant. The intent is to make the delta a deep change but it is spec'd vaguely enough such that it can be discussed at a later point in time when we have more time.
I understand the new flag, but this means you will require all existing plugins to be rewritten to consider this new flag. This means all 2.1 and 3.0 plugins will be out of sync and will not properly react to encoding changes. Just to makes things clear.
I confess I've only briefly followed much of this discussion, but I am confused about the design and intended "fix". I realize "encoding" is one of the most important, new "content description" parameters, but there could be others added downstream (or upstream, I guess). Ideally, for all of them, a "contents changed" event could be used to signal a change, be backwards compatible with existing listeners, and, ideally, allow a more detailed "query" of the event for those new pieces of code that know how to deal with more specific types of "content change". Is this the gist of the design that being proposed?
David - this is what I was also asking for, but not what the platform is currently intending to deliver.
The motivation behind using a new constant rather than the existing "CONTENT" bit is that not all clients are interested in changes to encoding.
Can you clarify with an example of a client that's interested in content, but not interested in its encoding? My intuition is that anyone who's gotten content (or content changes) would have to do so again if the encoding was changed. (Otherwise what they have is basically in error). But, as usual, I may not understand exactly what you mean, so thought an example would help. Thanks.
Team providers are an example of a client who is only interested in the bytes and does not try to interpret them.
David, this PR is about the use of IFile#setCharset / IContainer#setDefaultCharset, and generating events for them. Changes to the content type specific default encoding will be seamlessly supported, but that is not the focus here. Changes to any properties embedded in the contents will cause resource change events to be sent, so there is no need to to provide special suoport for that. For instance, if someone changes the "encoding" attribute contained in the XML declaration section (<?xml...?>), a content change would be reported, not an encoding one (although the encoding for the file has actually changed). Philippe, re: comment 31 - I believe one of us is not understanding the other's point (and I hope it is you <g>): 2.1 plug-ins *cannot* benefit of encoding changes being reported as content change events. They don't know about IFile#getCharset, so even if you propagate an event for a file encoding that has changed, the stupid plugin will still use the workspace's global charset when re-reading the file, instead of the right one. As you may have noticed, the only change they will be able to react properly is a change to the workspace's global encoding, which is something very rare (will be even rarer on 3.0), and for what there were no events in the past, anyway. For 3.0 plug-ins: I believe most of them would be able to react to this API change after 3.0 is released, although I am sure many wouldn't.
I think my claim could be summarized as: if you did broadcast a CONTENT change, then I would have nothing to change, since encoding/bytes change is pretty much the same throughout JDTCore: we have to rebuild, reread etc... So for most tools out there, which already react to content change, there would be nothing to do at all. I understand that there are a few special clients out there, but these would have to narrow down the change to further see whether bytes or encoding did change. So maybe having the CONTENTS flag positionned, plus an extra ONLY_ENCODING flag in case the bytes did not change would do it I believe. [Note: If you could change the workspace encoding in 2.1, then this would be enough to recreate the issue without using resource specific charsets.]
It is a new feature (clients being able to react to changes in the encoding), so it is not too much to ask those willing to benefit from this change to adapt their code. Those who are not interested in the feature, don't have to react. That seems fair in my opinion. Re: changing the workspace encoding - I commented on that in the second paragraph of comment 37.
Just wanted to say thanks for all the clarifications and think I understand it enough now to understand both points of view. It appears, in 3.0, that anyone who is interested in the encoding as determined by content, MUST also pay attention to changes in this encoding flag/change. In some cases I'm familiar with (e.g. XML), this might just result in a user error message, or something, that says the encoding in contents does not agree with the "set" encoding (resulting in a "what do you really want to do?" sort of message or preference). So, I hadn't realized that and appreciate the clarifications (and suggest this be well "cross documented" in both APIs :) It'll unfortunately be a while before we implement this so not sure of suitability or ease until then.
Actually, it seems you got it wrong: the special encoding change events will only be generated if the encoding setting is changed anywhere else other than the content (for instance, using the new (IFile#setCharset/IContainer#setDefaultCharset, exposed to the user by the resource properties dialog). If the encoding change happens due to a change in the content itself, a regular content change event is issued (and encoding change events are not).
I have released the patches in HEAD (with minor javadoc touch-ups) and they will be in today's 4pm EST build. I will add entries to the release notes re: these new APIs not being implemented yet. Changing the target milestone to 3.1. (will adjust to be 3.0.1 once that milestone is created in Bugzilla)
*** Bug 68585 has been marked as a duplicate of this bug. ***
*** Bug 56312 has been marked as a duplicate of this bug. ***
*** Bug 64077 has been marked as a duplicate of this bug. ***
A fix for the basic scenarios (clients calling IFile#setCharset/IContainer#setDefaultCharset) has been released to the maintenance branch and will be in next maintenance build. The code will be released to HEAD shortly after M1 is declared. Not closing yet because we still need to hook it up with content type/project preference change events. Entering bug 71742 against UI to ensure the API that takes progress monitors is used in the resource properties dialog and the editors preference page.
This fix seems not to be in 3.1 builds (I200408241200) yet. Any date (build) when this will be available?
Indeed, it is in the 3.0.1 builds but not in the 3.1 (although it is in HEAD). The full fix will be in next week's integration build. Sorry if that caused any confusion. Note, however, that you can use the non-deprecated API right now. Its current behavior will be similar to the original (deprecated) version, which does not generate any deltas. When the fix gets integrated, you will get the deltas.
My problem was that I started to add encoding delta handling and it did not get triggered ;-)
I missed one caller in JDT/UI. Opened bug 72689 to address it. Thanks for pointing it out, Daniel!
Full fix went into both maintenance and HEAD streams. Available in next weeks integration/maintenance builds. Written with DJ, reviewed by JohnA.
Verified in M200409010800. Thanks to all of you for resolution of this PR!
