Community
Participate
Working Groups
Here's how to reproduce : 1) Create a new Java project 2) Create in the project New-->Folder and a link to RSE to the 'My Home' 3) I try to open one of the files within the linked folders 4) Eclipse starts "Building workspace", reaches to 50% and get stucked. 3) The Eclipse (3.3) get stucked and saying "Not Responding" I need to kill the Eclipse application I'm using RC3. Yaron
It looks like all subfolders are synchronized when the link is created. It could be an issue if the remote file tree is huge or the connection is slow. Kushal, do you know if this synchronization is an EFS feature ?
Javier -- I'm assigning this to you for now, but its clearly EFS related. Feel free to assign it to DaveM or Martin after you've done the initial work with Kushal. -- Dave
Javier - did your investigations show whether this was JDT's fault or ours?
Javier did you make any investigations that you could share? - I'm marking this Major now since the effect is a major one, to ensure it's prominently listed on the "known issues" page - even though it's most likely a JDT problem.
(In reply to comment #4) > Javier did you make any investigations that you could share? - I'm marking this > Major now since the effect is a major one, to ensure it's prominently listed on > the "known issues" page - even though it's most likely a JDT problem. > I can't reproduce the issue described in the bug. Even synchronizing huge ftp trees works, but it takes a while retrieving the whole list of folders. This action is done in a separate thread and doesn't block the UI (could it be a recent change ?). Also EFS doesn't retrieve the files, just lists them and gets some basic information as size or read-only. So it works for me. Could somebody else check it as well in your workspace using the Europa final release ? What I have seen though is that deleting the link freezes Eclipse, but it should require more investigation.
Yaron, based on comment #5, could you try again with a recent RSE build, e.g. http://download.eclipse.org/dsdp/tm/downloads/drops/I20070724-0735/index.php If still an issue, give us more information: * What kind of connection (ssh, ftp, dstore)? From our point of view, we cannot reproduce the Freeze, it is slow but it works. Note that the issue could also be in JDT.
PS also let us know the absolute path on remote to which the linked folder resolves, and how many files/directories (approximately) are below it. Perhaps you have a cyclic symbolic link below that folder and that was why JDT did not complete. to verify you could try this on the remote: cd $HOME find . -type l -o -type d -ls > all_symlinks.txt
Here's how I reproduce the "Freeze" : (I performed it several times) 1. I have an FTP to which I have access in our LAN. 2. I have 2 folders : - test1: AAA.java, CCC.java, GGG.java - test2: XXX.java Each of the files simply contains the following content: class CLASS_NAME { } 3. I launch Eclipse with its default java perspective 4. I launch RSE view and create new connection to my FTP 5. I open the tree nodes and perform login with my user name and password 6. I create a new Java project 7. I create a linked folder to RSE --> test1 8. I create abother linked folder to RSE --> test2 The links are created under the "src" root folder ! 9. I go to the "Package Explorer" and open (using double clicks) the Java files in the following order : AAA.java, CCC.java, GGG.java BUT... when I double click XXX.java the Eclipse freezes and "Not responding" An example of one of the files path : rse://CHEESE/home/yaron.m/test1/AAA.java
I also have th same problem the scenario is the same. I have Eclipse 3.3 RSE build RSE-SDK-N20070727-0100 Eclipse simple hangs. No console log. No error log. I have tried to access only 1 file. Not many files or folders.
Contd to comment No 9.. Eclipse freezes when I try to open a java file. But it is differnct in case of a jsp file When I have a linked RSE folder containing jsp files. I am able to see the folders & files within them But when I try to open a jsp file I get the following error Unable to create editor ID org.eclipse.jst.jsp.core.jspsource.source: Editor could not be initialized. java.lang.NullPointerException at org.eclipse.wst.sse.core.internal.FileBufferModelManager.createURIResolver(FileBufferModelManager.java:419) at org.eclipse.wst.sse.core.internal.FileBufferModelManager.getModel(FileBufferModelManager.java:656) at org.eclipse.wst.sse.core.internal.model.ModelManagerImpl._getModelFor(ModelManagerImpl.java:393) at org.eclipse.wst.sse.core.internal.model.ModelManagerImpl.getModelForEdit(ModelManagerImpl.java:1020) at org.eclipse.wst.sse.ui.StructuredTextEditor.doSetInput(StructuredTextEditor.java:2113) at org.eclipse.ui.texteditor.AbstractTextEditor$19.run(AbstractTextEditor.java:2994) at org.eclipse.jface.operation.ModalContext.runInCurrentThread(ModalContext.java:369) at org.eclipse.jface.operation.ModalContext.run(ModalContext.java:313) at org.eclipse.jface.window.ApplicationWindow$1.run(ApplicationWindow.java:758) at org.eclipse.swt.custom.BusyIndicator.showWhile(BusyIndicator.java:67) at org.eclipse.jface.window.ApplicationWindow.run(ApplicationWindow.java:755) at org.eclipse.ui.internal.WorkbenchWindow.run(WorkbenchWindow.java:2451) at org.eclipse.ui.texteditor.AbstractTextEditor.internalInit(AbstractTextEditor.java:3012) at org.eclipse.ui.texteditor.AbstractTextEditor.init(AbstractTextEditor.java:3039) at org.eclipse.wst.sse.ui.StructuredTextEditor.init(StructuredTextEditor.java:2615) at org.eclipse.ui.internal.EditorManager.createSite(EditorManager.java:794) at org.eclipse.ui.internal.EditorReference.createPartHelper(EditorReference.java:643) at org.eclipse.ui.internal.EditorReference.createPart(EditorReference.java:426) at org.eclipse.ui.internal.WorkbenchPartReference.getPart(WorkbenchPartReference.java:592) at org.eclipse.ui.internal.PartPane.setVisible(PartPane.java:299) at org.eclipse.ui.internal.presentations.PresentablePart.setVisible(PresentablePart.java:179) at org.eclipse.ui.internal.presentations.util.PresentablePartFolder.select(PresentablePartFolder.java:268) at org.eclipse.ui.internal.presentations.util.LeftToRightTabOrder.select(LeftToRightTabOrder.java:65) at org.eclipse.ui.internal.presentations.util.TabbedStackPresentation.selectPart(TabbedStackPresentation.java:400) at org.eclipse.ui.internal.PartStack.refreshPresentationSelection(PartStack.java:1256) at org.eclipse.ui.internal.PartStack.setSelection(PartStack.java:1209) at org.eclipse.ui.internal.PartStack.showPart(PartStack.java:1604) at org.eclipse.ui.internal.PartStack.add(PartStack.java:499) at org.eclipse.ui.internal.EditorStack.add(EditorStack.java:103) at org.eclipse.ui.internal.PartStack.add(PartStack.java:485) at org.eclipse.ui.internal.EditorStack.add(EditorStack.java:112) at org.eclipse.ui.internal.EditorSashContainer.addEditor(EditorSashContainer.java:63) at org.eclipse.ui.internal.EditorAreaHelper.addToLayout(EditorAreaHelper.java:217) at org.eclipse.ui.internal.EditorAreaHelper.addEditor(EditorAreaHelper.java:207) at org.eclipse.ui.internal.EditorManager.createEditorTab(EditorManager.java:774) at org.eclipse.ui.internal.EditorManager.openEditorFromDescriptor(EditorManager.java:673) at org.eclipse.ui.internal.EditorManager.openEditor(EditorManager.java:634) at org.eclipse.ui.internal.WorkbenchPage.busyOpenEditorBatched(WorkbenchPage.java:2719) at org.eclipse.ui.internal.WorkbenchPage.busyOpenEditor(WorkbenchPage.java:2633) at org.eclipse.ui.internal.WorkbenchPage.access$12(WorkbenchPage.java:2625) at org.eclipse.ui.internal.WorkbenchPage$10.run(WorkbenchPage.java:2577) at org.eclipse.swt.custom.BusyIndicator.showWhile(BusyIndicator.java:67) at org.eclipse.ui.internal.WorkbenchPage.openEditor(WorkbenchPage.java:2572) at org.eclipse.ui.internal.WorkbenchPage.openEditor(WorkbenchPage.java:2556) at org.eclipse.ui.internal.WorkbenchPage.openEditor(WorkbenchPage.java:2547) at org.eclipse.ui.ide.IDE.openEditor(IDE.java:644) at org.eclipse.ui.ide.IDE.openEditor(IDE.java:603) at org.eclipse.jdt.internal.ui.javaeditor.EditorUtility.openInEditor(EditorUtility.java:285) at org.eclipse.jdt.internal.ui.javaeditor.EditorUtility.openInEditor(EditorUtility.java:138) at org.eclipse.jdt.ui.actions.OpenAction.run(OpenAction.java:194) at org.eclipse.jdt.ui.actions.OpenAction.run(OpenAction.java:175) at org.eclipse.jdt.ui.actions.SelectionDispatchAction.dispatchRun(SelectionDispatchAction.java:268) at org.eclipse.jdt.ui.actions.SelectionDispatchAction.run(SelectionDispatchAction.java:244) at org.eclipse.jdt.internal.ui.packageview.PackageExplorerActionGroup.handleOpen(PackageExplorerActionGroup.java:316) at org.eclipse.jdt.internal.ui.packageview.PackageExplorerPart$5.open(PackageExplorerPart.java:613) at org.eclipse.jface.viewers.StructuredViewer$2.run(StructuredViewer.java:820) at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:37) at org.eclipse.core.runtime.Platform.run(Platform.java:857) at org.eclipse.ui.internal.JFaceUtil$1.run(JFaceUtil.java:46) at org.eclipse.jface.util.SafeRunnable.run(SafeRunnable.java:193) at org.eclipse.jface.viewers.StructuredViewer.fireOpen(StructuredViewer.java:818) at org.eclipse.jface.viewers.StructuredViewer.handleOpen(StructuredViewer.java:1079) at org.eclipse.jface.viewers.StructuredViewer$6.handleOpen(StructuredViewer.java:1183) at org.eclipse.jface.util.OpenStrategy.fireOpenEvent(OpenStrategy.java:263) at org.eclipse.jface.util.OpenStrategy.access$2(OpenStrategy.java:257) at org.eclipse.jface.util.OpenStrategy$1.handleEvent(OpenStrategy.java:297) at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:66) at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:938) at org.eclipse.swt.widgets.Display.runDeferredEvents(Display.java:3682) at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3293) at org.eclipse.ui.internal.Workbench.runEventLoop(Workbench.java:2389) at org.eclipse.ui.internal.Workbench.runUI(Workbench.java:2353) at org.eclipse.ui.internal.Workbench.access$4(Workbench.java:2219) at org.eclipse.ui.internal.Workbench$4.run(Workbench.java:466) at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:289) at org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Workbench.java:461) at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.java:149) at org.eclipse.ui.internal.ide.application.IDEApplication.start(IDEApplication.java:106) at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:153) at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:106) at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:76) at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:363) at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:176) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:504) at org.eclipse.equinox.launcher.Main.basicRun(Main.java:443) at org.eclipse.equinox.launcher.Main.run(Main.java:1169)
Looks like the JSP editor simply is not EFS aware, and it will likely fail with any kind of remote resource linked in via EFS. The Eclipse 3.1->3.2 migration guide mentions that since 3.2, it can be possible that resources are no longer local: see section "1. Resources are no longer necessarily in the local file system" on http://help.eclipse.org/help33/topic/org.eclipse.platform.doc.isv/porting/3.2/incompatibilities.html Please file a bug against WST for this -- there are simliar bugs already against CDT (bug #186170) and PDT. Regarding the hang with JDT I'm investigating this right now.
I reproduced the issue today, and it is specific to the FTP service. With SSH, everything works just fine. The problem with FTP is, that EFS operations do downloads through the IFileService#getInputStream() operation. This operation is not protected by the FTPService#_commandMutex. As a consequence, if any other query like listFiles() or getFile() occurs while the download through the stream is still ongoing, a NOOP command is sent to the FTP server while it is still busy performing the stream download. The response of the NOOP command is unexcpected to the download, and consumed by the wrong stream. As a consequence, both the thread which requested the NOOP through getFTPClient() and the other thread which was busy downloading are waiting forever for their proper return codes from the server -- communication is totally broken and all of Eclipse hangs. I guess the reason why it was not reproduced according to comment #5 was that no other operations were performed while JDT was building its index. But following the instructions from comment #8, adding the second linked folder performed a getFile() command for me while JDT was still busy updating its index -- making everything hang. The fix for this should be simple: ALL FTPService operations that run getFTPClient() or perform any commands on the remote must be protected by the _commandMutex. As an enhancement for the future, it may make sense to connect TWO FTP clients: one just for getFile() and list() commands and the other for doing actual downloads, since the Jakarta commons net client is essentially single threaded.
Created attachment 74949 [details] Thread dump showing the hang in FTPService
Created attachment 75023 [details] Patch using mutex Added mutex control for getInputStream and getOutputStream actions. With this patch FTP EFS folders can be created, deleted and files open and edited as usual. I have also realized that copy/paste external files into an EFS folder in the workspace uploads the file correctly to the server, but it can't be opened from the EFS folder as it's marked as non-existing. I have reproduced it with FTP and SSH. I'll investigate it and open a new bug report for it if necessary.
Yes I also noticed that when creating a new class in JDT on an EFS-backed folder, the class will be marked not existing. Please do open a new bug for this.
Applied patch
Created attachment 75211 [details] Additional patch for delete(), upload(), setReadOnly() The previous fix was incomplete. Attached patch also fixes critical sections for delete(), upload(), setReadOnly().
Patch applied, please review: [192610] Fix thread safety for delete(), upload(), setReadOnly() operations FTPService.java
I'm not 100% sure, but I think there may still be issues in the case where an FTP connection is used to open multiple Streams on the remote side. FTPClient#getInputStream(String,String,boolean,monitor) acquires the _commandMutex just for opening the Stream. After that, the Mutex is free again to be used. But at the time the FTPBufferedInputStream#close() method is being called, Commons Net requires an additional command to be sent: client.completePendingCommand(); I am afread that if other threads interfere with the FTP channel after the stream has been opened but before it is being closed, this could again kill the connection because the completePendingCommand() would not be finished correctly and/or would not have the correct context. Javier please test this scenario: * Create FTP Connection * Create a project, with linked folder to RSE EFS / FTP file system * Have four large text files on the remote, in two different directories * Quickly dbl click on the four files in succession in order to open all of them in the editor I think it might kill the session because there are some "cd" commands interspersed with opening the streams and downloading. Other issues could also arise when the EFS-shared remote directory contains a large Java project, such that the Java indexing thread would constantly request many streams to be opened. Again, multiple such streams could be open by multiple threads at the same time; dbl clicking to open files in the editor, while JDT is busy indexing, could kill the connection. We need to really throughly test this before we can really set this fixed. In case issues arise, I'm not 100% sure what's the best solution. Keeping the _commandMutex acquired until the client closes the input stream might be one solution; but it is risky, since some clients could forget closing the stream and then the whole FTP connection is locked. Therefore, it might be better to go and open a fresh empty FTP session for each getInputStream() / getOutputStream() request. Then, the stream is independent of the query channel. See also bug 198636 for a similar discussion for performance improvements.
(In reply to comment #19) > We need to really throughly test this before we can really set this fixed. Best would be writing a JUnit test for the scenario mentioned. This could be pretty simple: * Get FTPService instance from RSE * Create 2 directories, upload 4 files * In 4 different threads, getInputStream() and start downloading but do not close the streams yet * Try performing some list/chdir commands * Close the 4 streams one after another with more list/chdir commands interspersed Having such a unit test would be fabulous, and should be applied to the SSH (Sftp) case as well. In fact, such a unit test would be the most important to protect against regressions with the EFS integration. I filed bug 198661 asking for such a unit test.
Original issue is fixed, committed and released for I20070802-1215 For multiple streams to remote files remaining open for longer time, and potential issues with these, we'll use bug 198661 and bug 198683.
Comment on attachment 75023 [details] Patch using mutex iplog- since Javier was committer since Aug 2006 already: http://dev.eclipse.org/mhonarc/lists/dsdp-tm-dev/msg00375.html
