Community
Participate
Working Groups
Hopefully this has been submitted in the right place and with the right priority. (It is a stopper for me.) I have an RCP application that uses SWT and a JNI package of my own. I also have a stand-alone SWT version that does essentially the same thing. In both of these, the JNI part can be replaced with a pure Java implementation. Apart from a switch that uses the full Java version or not, they all use the same package. The JNI implementation is old code that has been working for some time, both in and out of Eclipse. All of these versions work except the RCP application using JNI. That is, it is not a bug in the package nor in making the RCP product. The RCP JNI version fails, eventually giving an exception, because the C routine FindClass in the following C++ code returns null: extern "C" { static int theMessageCallback(const char* pFormat, va_list args) { jobject callback= (jobject) epicsThreadPrivateGet(messageCallbackID); if (callback==NULL) return ECA_NORMAL; JNIEnv* env; _jvm->AttachCurrentThread((void**)&env, NULL); char buf[1024]; sprintf(buf, pFormat, args); jstring msg= env->NewStringUTF(buf); jclass clazz = env->FindClass("gov/aps/jca/jni/JNIContextMessageCallback"); jmethodID methodID = env->GetMethodID(clazz, "fire", "(Ljava/lang/String;)V"); env->CallVoidMethod(callback, methodID, msg); _jvm->DetachCurrentThread(); return ECA_NORMAL; } } This is a C callback, and the idea is to call the Java callback with the string from the arguments. (The problem is that clazz is null. env is not null.) The callback gets called from code in another DLL that is independent of the JNI wrappers. That library knows nothing about Java, and the callback prototype doesn't have an env as an argument. It calls the Java routine that will handle the callback in Java, getting the env from the thread. This is different from what is more commonly done, which is to have a C implementation of a Java class, that gets passed the env, which (indirectly) knows about the class. The reason it fails is that the C function, FindClass, which is purely J2SDK, not Eclipse, uses the classpath to find the class. With an RCP application, Eclipse apparently uses <app-dir>/startup.jar as the classpath. This is determined by printing out java.class.path from the running app. startup.jar has very little in it and little, if anything, that is specific to the particular RCP app. Consequently FindClass does not find the class in this case (because it is not in the classpath). The RCP app does find the other classes in whatever manner Eclipse uses to locate jars in plug-ins, presumably using its own classloaders. It works OK until the above C callback gets called. The reason it works standalone (not a plug-in in an RCP app) is that I specify the classpath specifically to include that jar, and FindClass uses that classpath and finds it. (The reason it works with the pure Java version is that FindClass is not used.) I was unable to get the RCP app to use my classpath, for instance by specifying it in vmargs or myapp.ini. Eclipse apparently *overwrites* the classpath with the above. I would consider this also to be a bug. It should append or use what is specified on the command line (which could include startup.jar). I can make it work by manually putting the package in startup.jar. This allows FindClass to locate it in the classpath (while Eclipse apparently finds it in the plugin directory, since the jar files are really in another plug-in). This is not a good solution, (if for no other reason than that the class is in two places, now) but it demonstrates the problem, which seems to be that FindClass does not work in the Eclipse environment when called from C or C++ code that is not directly part of the native implementation of a Java method and that the reason is because the J2SDK FindClass cannot find the class as Eclipse can. This is a stopper for me. It is a reasonable and straightforward thing to do. I think it is an oversight in the Eclipse implementation. I cannot change the implementation, as it is not my code. In addition, it is a correct JNI implementation. I apologize for this being long. The problem is somewhat deep and hard to easily understand. I would be happy to elucidate, if necessary.
Your analysis appears complete. Thanks! Yes, the classpath used/expected by Eclipse is just startup.jar. You can put more stuff there but it will never be picked up by anything in Eclipse itself. This is desired behaviour and is true of several (if not all) OSGi implementations. OSGi bundles are exposed only to the types they import/require. Adding things to the classpath does not export/provide them to OSGi so they will never be found. Having said that, it would seem to be overkill for us to be overwriting anything you did happen to put on the classpath when running Eclipes. For example, did you try eclipse.exe -classpath startup.jar;foo.jar and have it not work? I'm pretty sure that you can do this sort of thing (ie., I believe i have done it in the past). If you can't please open a bug report with details. If it is a problem, try running Eclipse directly using java.exe rather than eclipse.exe. There is no real magic there. Just something like java -cp startup.jar;foo.jar org.eclipse.core.launcher.Main -application/product <id> You can likely get what you want if you add to the bootclasspath either on the command line or in eclipse.ini. Note that this will force all bundles to have direct access to the classes you put there regardless any import/export statements in the manifests. That is, the duplicate copy in your plugin will be ignored in favour of this since the boot loader (not the app loader) is alwasy consulted first. Finally, you can eliminate some duplication by putting the foo.jar on the appclasspath and use either osgi.parentClassloader=app or the app buddy loading policy (see the 3.1 help) in your bundles that use the classes on the app classpath. Note that most of these approaches are somewhat hard to manage since they involve globally providing a set of classes and messing with global configuration information. As for FindClass, not sure what so say there. The function should really take a classloader or object to start from rather than just assuming the that it should use the app classloader. There are lots of other systems with which this approahc will not work (JBoss, context classloaders, ...)
Thanks for your reply. 1. It is my experience that the app.exe (eclipse.exe) overwrites anything you specify on the classpath with <app-dir>/startup.jar. I had previously tried setting -classpath in -vmargs both through the app.ini and via entering it as an argument to app.exe. I verified that javaw is started as a separate process with -classpath as I specify. After that, if I print out java.class.path in the program or display it in a dialog, it is always <app-dir>/startup.jar. I just tried using -classpath instead of -vmargs -classpath, as you suggested, and it still prints <app-dir>/startup.jar. (I was not aware that -classpath was an eclipse.exe option.) That it behaves this way should be easy for you to confirm. 2. I don't want to do that anyway. I want the jars for the JNI package to be in a plug-in (or eventually plug-ins). Trying to overwrite the classpath is a kludge, and, as you said, hard to manage. I only tried it to verify my diagnosis of the problem was accurate. 3. The problem is really that FindClass does not work under Eclipse (under these conditions) and that FindClass is an essential part of JNI. I cannot rewrite FindClass. It is part of Java. I cannot even rewrite the package as I do not own it. Moreover, it is a working Java package outside of Eclipse, and what is done is standard practice. This package is part of control system software used to control accelerators and other large physics experiments around the world, and I work in that community. In addition to control systems, I would like to use Eclipse to manage X-ray scattering data analysis. Most of the analysis codes (literally hundreds) are written by scientists, often in FORTRAN or C, seldom in Java. Thus, I need to be able to access native code, whether that is elegant or not. If FindClass and JNI does not work, I am pretty much out of business in using Eclipse, which would be unfortunate. I am not sufficiently familiar with the internal workings of Eclipse (or perhaps even Java) to see a way to get around this problem. I think there is incentive for the appropriate developers to look at it. It is a generic problem with Eclipse and JNI.
I understand the problem and appreciate your situation but fundamentally there are many cases for which the FindClass approach is broken. At a very basic level Eclipse the classpaths managed by Eclipse form dynamic acyclic digraph, not a fixed linear progression as expected by the app class loader. We can, through some of the mechanisms mentioned allow plugins to see things on the app classpath but what you want is for someone looking at the app classpath from C to see classes in plugins. I hope I am missing something here but don't think I am.
The big picture is (if what you say is correct and cannot be fixed) that JNI is not usable with Eclipse, or more accurately, that it is only usable in a limited fashion. >> what you want is for someone looking at the app classpath from C to >> see classes in plugins Yes. A significant part of JNI is that you can call Java from C. If my package is implemented in a plug-in, then I should be able to call back into the methods in its classes.
>>As for FindClass, not sure what so say there. The function should >>really take a classloader or object to start from rather than just >>assuming the that it should use the app classloader. What I have found is the following: (http://java.sun.com/j2se/1.4.2/docs/guide/jni/jni-12.html#FindClass) FindClass locates the class loader associated with the current native method. If the native code belongs to a system class, no class loader will be involved. Otherwise, the proper class loader will be invoked to load and link the named class. (We are not a system class, so the latter should happen.) On the other hand, it appears that the JNI defaults to JNI_VERSION_1_1, which doesn't know about class loaders and uses the classpath. This may be the problem, and I will check and get back. If so, it is my bug.
Great! Please to let us know. This could go in a FAQ somewhere.
I have checked, and the version is not the problem. There is a GetVersion method (C code) and it returns JNI_VERSION_1_4. Thus, it is well beyond JNI_VERSION_1_1 AND should invoke "the proper class loader" as in the reference I quoted (and others). The only thing *I* can change is to add an OnLoad method that returns the required version. However, this is only used to allow access to new functions at the end of the JNI function tables, and I do not use any of the new functions, just basic ones like FindClass(). I have tried all of the three possibilities (JNI_VERSION_1_n, with n = 1, 2, and 4) anyway. I am using j2sdk1.4.2_06. The jdk1.5.0_05 jni.h has the same options. I have also tried the jdk1.5.0_05 runtime. The same thing happens: It cannot find the class using FindClass() from the callback. I have also debugged on it. If I do: this.getClass().getClassLoader().loadClass("MyClass"); then it uses org.eclipse.core.runtime.adaptor.EclipseClassLoader. However, it appears to be in java.net.URLClassLoader, when it cannot find the class from the JNI FindClass. Thus, Java's idea of "the proper class loader" is different from ours. It thus looks like JNI cannot be made to work with jar files in a plug-in under Eclipse, at least a long as Eclipse overwrites the java.class.path with one that does not contain the requested class. The JNI implementation relies on the class being in the java.class.path. This is getting deeper into the internal workings of Eclipse and Java than my expertise warrants. I think it is an important shortcoming of Eclipse, and should be fixed, or perhaps there is a work around. To fix it or work around it probably takes more knowledge and access than I have. That is why I have submitted it as a bug. I will be happy to elaborate if this is not clear.
Hi Kenneth, I've been following this bug and might be able to help. I've been using JNI and FindClass callbacks in Eclipse without the problems you're having. I want to clarify something about your circumstances. Is the callback happening in the second DLL (e.g. a DLL other than the one with the JNI wrappers loaded via System.loadLibrary(...))? If so I suspect that's the problem; only the first DLL will be associated with the plugin's classloader. This isn't really just an Eclipse thing, it's a Java/JNI thing. The other DLL will normally be loaded by the OS process (outside of the java world) and won't have an associated classloader to provide context. This is why it ends up using the App Classloader. One dirty trick that you might try is using System.loadLibrary(xxx) to load that 2nd DLL. In this case the process shouldn't have to load the DLL again and "maybe" the DLL will still do its class lookup with the plugin's classloader. Beyond this I think Jeff's suggestions in comment #1 should work and are probably the best way to go.
Simon, Thanks for your reply and suggestions. Yes, you are correct that the callback that uses FindClass is coming from a 2nd DLL loaded by the JNI DLL. Moreover, it is in its own C thread, so it's not clear what AttachCurrentThread is doing (except that it is not what we want). The JNI documentation does not go into much depth, and I haven't been able to find if the source code is available. In addition, I am not very experienced with JNI, though I probably will be if I continue on my current path: I need to incorporate FORTRAN code into RCP apps, which should be even more fun. Your suggestion may work. The C callback in question will also take a void * clientData when created and will return that to the callback. I am considering passing the env from the routine that creates it and using that env->FindClass. If that doesn't work, there should be something else I can pass that gets us back to the EclipseClassLoader. The solutions in Comment #1 are less attractive. I would, of course, like to just use the Product Export Wizard and other tools provided by Eclipse without extra manual steps. (KISS) It is becoming clear that it is more of a problem in using Eclipse than a bug. I do, however, think Eclipse should not overwrite a classpath specified after vmargs. I believe in enabling the user. If he shoots himself in the foot, that is his problem. The good ones don't. In my defense, it is tested code that works well outside Eclipse.
We need a bug report for the overwriting of the classpath when using eclipse.exe. Pleaes add the details of your usecase (e.g., examples)
To demonstrate the overwriting of the classpath: 1. Make a Hello World RCP application via the New Project wizard. 2. Add the following lines to Application.java in the run method: // Get Properties String info = ""; info += "\nProperty Information\n"; String [] name = { "java.library.path", "java.class.path", }; for(int i=0; i < name.length; i++) { String value = System.getProperty(name[i]); String line = name[i] + ": " + value + "\n"; info += line; } System.out.print(info); 3. Make a Run configuration and put the following in the VM arguments: -cp junk1 -Djava.library.path=junk2 -Djava.class.path=junk3 I get: Property Information java.library.path: junk2 java.class.path: c:\eclipse\startup.jar It ignores the classpath. You can also try to specify the classpath in your own way. The classpath is ignored in any way I have tried. If this is not sufficient, I can send the project, but it doesn't seem necessary. The above is the only change. The problem is easy to demonstrate.
I opened bug 126499 to deal with the launcher overwrite problem.
I believe I have worked around the problem. The workaround seems to be to cache the method ID and to use the cached value instead of calling FindClass in the callback. (This is also more efficient.) In my case the callback comes from a native thread that is different from, and otherwise independent of, the thread in which the JNI was loaded. I do not know if this is a necessary condition or if FindClass, called in a function that needs AttachCurrentThread, always uses the classpath, as it does in my case. The JNI documentation (http://java.sun.com/j2se/1.5.0/docs/guide/jni/spec/functions.html) is ambiguous: It says: This function loads a locally-defined class. It searches the directories and zip files specified by the CLASSPATH environment variable for the class with the specified name. It also says: The Java security model has been extended to allow non-system classes to load and call native methods. In the Java 2 Platform, FindClass locates the class loader associated with the current native method. If the native code belongs to a system class, no class loader will be involved. Otherwise, the proper class loader will be invoked to load and link the named class. and: When FindClass is called through the Invocation Interface, there is no current native method or its associated class loader. In that case, the result of ClassLoader.getBaseClassLoader is used. This is the class loader the virtual machine creates for applications, and is able to locate classes listed in the java.class.path property. My experience is that if the desired function is in the classpath, there is no problem. However, under Eclipse, it is not in the classpath, it does not use the Eclipse class loader, and it does not find the class. I presume the JNI folks would say it is using the Invocation Interface in this case. It is probably not feasible to change the Eclipse behavior; however, it is a problem. It was mentioned that a FAQ might be appropriate. This would be my crack at one: JNI implementations that work outside of Eclipse, may not work under Eclipse. In particular, there is a problem using Eclipse with a JNI implementation that uses FindClass in a function where the JNIEnv pointer is not available, such as in a C callback. The reason is that FindClass, in this case, seems to use the classpath to find the class. If the desired function is in the classpath, as it would typically be in a stand-alone application, there is no problem. However, under Eclipse, it is not in the classpath, because Eclipse, by itself or as an RCP application, overwrites the classpath with startup.jar, which only includes generic launcher classes. Eclipse then uses its own class loader to find classes. The Eclipse class loader does appear to be used by FindClass in JNI functions which are passed the JNIEnv pointer, but not when you have to use AttachCurrentThread to get the JNIEnv pointer. Example: static JavaVM* jvm; // Global variable void myCallback(void) { JNIEnv* env; jvm->AttachCurrentThread((void**)&env, NULL); // Fails if "some/class" is not in the classpath: jclass cls = env->FindClass("some/class"); jmethodID methodID = env->GetMethodID(cls, "methodName", "(Ljava/lang/String;)V or whatever signature"); env->CallVoidMethod(callback, methodID, ...); jvm->DetachCurrentThread(); } } A solution is to cache the method ID, for example: static jmethodID mid; // Global variable JNIEXPORT jint JNICALL JNI_OnLoad(JavaVM *vm, void *reserved) { ... // Store the JavaVM pointer jvm = vm; // Find the class and store the method ID // Will use the class loader that loaded the JNI library jclass cls = env->FindClass(className"some/class"); if(!cls) goto ERR; mid = env->GetMethodID(cls, "methodName", "(Ljava/lang/String;)V or whatever signature"); if(!mid) goto ERR; ... } void myCallback(void) { JNIEnv* env; jvm->AttachCurrentThread((void**)&env, NULL); env->CallVoidMethod(callback, mid, ...); // Handle error ... jvm->DetachCurrentThread(); } }
Thanks for all the time everyone has spent investigating this. Thanks also for the FAQ doc. I'm downgrading this bug and marking as a doc bug so that we are sure to include the text somewhere. Perhaps we can put it on the wiki? I'm not sure where it would go in the formal doc.
As noted on bug 126499, the fact that the classpath is 'overwritten' is not a bug, and that's not what's happening. Eclipse runs two VMs; one is the one you launch from the command line (eclipse.exe), and the other one is the one that startup.jar launches. What you're finding is that VM1's classpath, set on the command line, doesn't match VM2's classpath (the one that your RCP app and any other code is written in). So specifying the classpath on the runtime argument doesn't have any effect. That's why when Eclipse goes down, you get a nice dialog box giving you information. That box is from VM1, and that's the one whose classpath you have set. It's also why the Update Manager can restart Eclipse after an update; it returns with a specific error code that signals to VM1 that it should be relaunched afterwards. Frankly, the fact that there are two VMs in operation at this time may result in unexpected/unexplained behaviour. If there were a global variable that was being expected to be set by the JVM when it initialised, then what would be the effect if a second VM was started in the same process? Would it get overwritten? My guess is probably not (though I couldn't be sure). It wouldn't surprise me if this wasn't a bug at all, and in fact you're connecting to VM1 and VM1's thread (that is blocked on the synchronous call to fire up the Eclipse engine) and not VM2 and VM2's thread which has the classloader set. Try running that code and accessing 'org.eclipse.core.launcher.Main' as a class. You won't find that in the Eclipse runtime (I think; but best check by firing up a quick RCP app to ensure), but you will find it in the bootstrap VM. That will easily tell you whether you're looking at the right VM or not, and with it, the answer to all of your questions. In summary; cache the VM that you expect to see, and not one that may be provided, so that you know which one you're looking at. Regarding other comments (E.g. eclipse -classpath foo.jar;startup.jar) -- that is *only* going to change the classpath of VM1, not VM2. You might get better mileage if you put the -classpath in the 'eclipse.ini' (or app.ini) file, or after the -vmargs argument, since that controls the options from VM2. Alex.
just to clarify, eclipse.exe starts only one vm. in fact, it is the other way around. several eclipse.exe's are run. One to launch the vm then the startup code of Eclipes (in java) launches another to run the splash screen.
Need to doc the behavior described by Simon in comment #9.
Created attachment 43315 [details] readme patch This patch adds a section to the readme for this issue.
Releng team, please release the readme to document this limitation. Thanks.
Patch released.
Hello, is there any news regarding this problem? I have third party C-dll's (hardware drivers) which use extensivly callback methods. The vendors will never change their calling of callback methods to work with Eclipse RCP. So there will be a limitation using RCP. The solution "adding a Readme patch" doensn't really solve the problem. Probably I miss the point or perhaps you have additional arguments, I don't know?
