Comment 1 DJ Houghton 2003-01-16 10:28:02 EST

Rafael, please investigate. 
Co-ordinate efforts with John as he may be working in that space.
Thanks.

Comment 2 Rafael Chaves 2003-01-16 13:23:53 EST

Jed, could you provide the unicode codes for that file name? 

Comment 3 Jed Anderson 2003-01-16 14:09:40 EST

Copied from the Eclispe variables view with "display hexidecimal values" turned 
on.

[0]= c [\\u0063]
[1]= : [\\u003a]
[2]= \\ [\\u005c]
[3]= n [\\u006e]
[4]= a [\\u0061]
[5]= t [\\u0074]
[6]= i [\\u0069]
[7]= v [\\u0076]
[8]= e [\\u0065]
[9]= t [\\u0074]
[10]= e [\\u0065]
[11]= s [\\u0073]
[12]= t [\\u0074]
[13]= \\ [\\u005c]
[14]= t [\\u0074]
[15]= e [\\u0065]
[16]= s [\\u0073]
[17]= t [\\u0074]
[18]= \\ [\\u005c]
[19]= あ [\\u3042]

Comment 4 Jed Anderson 2003-01-16 14:30:38 EST

The following two bugs were found using my Autorefresh support.  I haven't
proven if either of these bugs are my fault or core problems, otherwise I would
have forwarded them on, but they may help you find this bug.

http://dev.eclipse.org/bugs/show_bug.cgi?id=27813 [autorefresh] NPE in
UnifiedTree during filesystem poll
http://dev.eclipse.org/bugs/show_bug.cgi?id=27844 [autorefresh] ClassCastExpression

Comment 5 Rafael Chaves 2003-01-21 17:46:09 EST

I tried to reproduce this error with no success (everything went fine).

Steps:
1 - created external folder c:\nativetest\test
2 - created project test
3 - created folder named nativetest in project test pointing to c:\nativetest.
4 - created externally file with name "\u3042" in c:\nativetest\test
5 - refreshed the project - the file just created appears with correct name and 
producing no errors.

Jed, can you reproduce the problem with a recent build? Am I missing any step? 
By the way, which VM are you using when having this problem? 

Something that is really weird is that the stack trace shows that the resource 
already existed before refreshing but it was known as a folder and has been 
discovered as a file in the file system.

Comment 6 Jed Anderson 2003-01-22 18:18:20 EST

Try this path:

1. Start Eclipse on a fresh workspace, c:\UnicodeTestWorkspace.
2. Create a simple project named test (no japanese characters in the pathname 
yet).
3. In the windows explorer, create a _folder_ named \u3042.
4. Refresh the project, the folder will appear as a file!  

When you double click the file you will see a simple little editor that tells 
you, "Resource /Test/\u3042 does not exist." (it actually uses the japanese 
character instead of \u3042).  The file will be removed from the project in the 
navigator.

Comment 7 John Arthorne 2003-01-23 09:46:38 EST

This looks like a duplicate of bug 13463.

Comment 8 Rafael Chaves 2003-01-23 11:19:04 EST

John, but isn't that bug related to filenames containing characters that are 
not supported using the default encoding?

It seems here there is a different issue: the problem Jed is facing is 
happening in a Japanese box, with file whose filename is made of as Japanese 
character (\u3042 is properly recognized using the Windows Japanese (MS932) 
encoding).

Unless Jed's configuration does not have MS932 as the default encoding. Could 
you run the program ShowEncoding (will attach) to see what the default encoding 
in your machine is, Jed (should be MS932)? Thanks, again.

Comment 9 Rafael Chaves 2003-01-23 11:20:30 EST

Created attachment 3104 [details]
Source/.class for ShowEncoding program

Comment 10 Jed Anderson 2003-01-23 12:29:49 EST

John, excellent advice!

I took out the core natives, and this bug disappears.  I will annotate 13463 as
well to indicate this.

I took a look at the natives.  I am surprised that they work on DBCS boxes.  On
NT/2K/XP, you must prepend "\\?\" to any path to force windows to make a unicode
call.  We are not doing this at all.  Also, the getPlatformBytes might not be
necessary at all (I am still wading through MSDN looking for clues on this). 
Any info the core team has on this would be most helpful for me (I am writing
natives for the the autorefresh support).

If no one on your team has experience writing natives (I know Rodrigo is working
on something else), I would like to take a stab at re-writing the natives to see
if I can make this problem go away.

Comment 11 DJ Houghton 2003-01-23 12:39:56 EST

I have been looking into this a little and talking with SWT folks.

Basically what is happening is that in our DLL we are calling the FindFirstFile 
windows call in all cases. We should be attempting to call the Unicode version 
of this function if it is available.

How to determine which function to call? SWT has code which figures out if we 
can make the Unicode call or not. See the static initializer of the 
org.eclipse.swt.internal.win32.OS class.

Comment 12 Rafael Chaves 2003-01-23 13:47:04 EST

Jed, I actually don't have experience with it but I'll be working on this issue 
as well. But of course we would appreciate if you share your findings, and feel 
free to propose any patches to the Core DLL regarding this issue.

By the way, have you figured out if your default encoding was incorrect?

Comment 13 Jed Anderson 2003-01-23 13:54:09 EST

I have a method which determines if the user is running on NT/2K/XP.  This was a
quick hack I wrote until I could figure out the "correct" way to write this
method (which is to query whether the OS has unicode support installed (all
NT/2K/XP boxes should)).

/*
 * Class:     com_example_autorefresh_win32_internal_Win32Natives
 * Method:    supportsLongFilenames
 * Signature: ()Z
 */
JNIEXPORT jboolean JNICALL
Java_com_example_autorefresh_win32_internal_Win32Natives_LongFilenamesSupported
  (JNIEnv *env, jclass this) {
  	OSVERSIONINFO osvi;
  	memset(&osvi, 0, sizeof(OSVERSIONINFO));
  	osvi.dwOSVersionInfoSize = sizeof(OSVERSIONINFO);
  	if (! GetVersionEx (&osvi) ) 
    	return JNI_FALSE;
    if (osvi.dwMajorVersion >= 5)
    	return JNI_TRUE;
    return JNI_FALSE;
}

Comment 14 DJ Houghton 2003-02-04 17:50:59 EST

New DLL has been created and released.
Marking as duplicate.

*** This bug has been marked as a duplicate of 13463 ***

Comment 15 Jed Anderson 2003-02-04 18:57:17 EST

DJ, please see
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/fileio/base/getfileattributes.asp

The doc indicates that we need to prepend \\?\ to the paths.  I didn't see this
in your fix, but then this advice could be a red herring.

Comment 16 Rafael Chaves 2003-02-04 21:19:20 EST

Jed, you need to prepend \\?\ only if you want to extend the path string limit 
to 32K -1 chars. Despite it can be interesting, it is not mandatory. What maybe 
is confusing you is the fact that you only can use this feature when using the 
Unicode version of the functions.

By the way, if you looked at the code, you probably noticed we borrowed your 
snippet for platform identification. Thanks!

Comment 17 Jed Anderson 2003-02-05 12:49:40 EST

Hey, thanks for the credit!

Do we not want to extend the maximum length of the filename to 2^15 - 1?  I
remember running into problems years ago in VAJ with filename maximums.  This
feels like an easy fix for much improved functionality.

Comment 18 Rafael Chaves 2003-02-05 13:14:13 EST

I thought about it, but I wonder if there are any additional performance costs 
when using that notation. I haven't read/heard anything about it, nor run any 
performance tests, but I tried to keep the new code as close to what we had 
before as possible so we don't get any surprises now that we are starting to 
build 2.1 RCs.

I agree, we should consider using the \\?\ notation for extending path length 
limits. I just opened a feature request for it: bug 30995.