Community
Participate
Working Groups
Created attachment 130728 [details] The Configuration Details of my normal configuration. In some cases, when I expand or refresh "My Home" or any other folder in "My Home", I get "Empty List" in "My Home" for a "FTP Only"-connection in the tree of the Remote Systems View. Then "Empty List" is the only entry in "My Home". Sometimes "Operation failed. File system input or output error" occurs before the "Empty List" appeares. If I refresh "My Home" there appear only NOOP commands in the console and no LIST command. Thus the remote-folder is not reloaded and the tree still shows "Empty list" in "My Home". A reconnect doesn't change anything. The "Empty list" stays and the only way to get the remote folders is to restart Eclipse. I tested the FTPSClient of Jakarta Commons Net 2.0 with RSE while I recognize this problem. Therefor I checked out the required plugins for ftp connections from the DSDP repository and put the commons net stuff into a jar and changed the FTPClient in org.eclipse.rse.internal.services.files.ftp.FTPService. The problem also appeares with my normal configuration. But with the newer plugins (3.1) the "Empty list" occurs more often. I gess, that the problem arises at org.eclipse.rse.subsystems.files.core.subsystems.RemoteFileSubSystem.internalResolveFilterString(String, IProgressMonitor). At line 664 to 666 parent of type IRemoteFile is checked of owning some entries. This is done by testing parent.hasContents(RemoteFileChildrenContentsType.getInstance(), filter), parent.hasContents(RemoteFolderChildrenContentsType.getInstance(), filter) and parent.hasContents(RemoteChildrenContentsType.getInstance(), filter). If the "Empty List" is in the tree, filter is * and parent._content looks like this {org.eclipse.rse.core.subsystems.RemoteChildrenContentsType@a1d7d9={*=[Lorg.eclipse.rse.internal.subsystems.files.ftp.model.FTPRemoteFile;@9e592c}} where [Lorg.eclipse.rse.internal.subsystems.files.ftp.model.FTPRemoteFile;@9e592c is [], the empy list. Further parent.hasContents(RemoteChildrenContentsType.getInstance(), filter) yields "true" and the other two return "false". That is that there are no folders and there are no files but there are files and folders. Does that make any sence? Anyway, because of the positive test the remote folder is not read out via FTP and the empty list is processed until org.eclipse.rse.internal.ui.view.SystemViewFilterReferenceAdapter detects that the list is empty.
Created attachment 130729 [details] The Configuration Details of my FTPS experiment.
Commons Net 2.0 is now available from Orbit, as per bug 267473. Full IP Due Diligence approval is still pending, but we can use Commons Net 2.0 under the terms of parallel IP for mature projects. Martin, I'm afraid we cannot do much for you until we can reproduce the issue here. What you experience might be a bug in FTPS (unless you can prove that it also happens with FTP or other connection types). I'd also think that since this only happens with a file service that's not part of normal RSE, it cannot be a blocker (normal RSE operation can still be tested, and blockers are defined as blocking any kind of testing). I'm reducing severity to "NORMAL" since I cannot even see major loss of functionality in anything provided by RSE itself -- it's only your own extension experiment that's failing.
Thanks for the link to Orbit but FTPS is not the problem. As I mentioned, the problem appears with my normal configuration too, that is with org.eclipse.rse.services.files.ftp (3.0.1) "RSE FTP Service" and also with org.eclipse.rse.subsystems.files.core (3.1.0) where the RemoteFileSubSystem is located. The problem is, if you get an "Empty List" in a FTP-project you can not refresh, you're out. I gess this is a major loss of functionality if you have to restart eclipse. Here I don't think that the problem is the occurrence of "Empty List" but that one can not refresh if it appears. As I wrote, the RemoteFileSubSystem 'thinks' the tree is still alright and doesn't need to be reloaded. If I comment out the following lines at line 669 in RemoteFileSubSystem the problem vanishes: "if (hasFileAndFolderContents) { // has everything } else" To reproduce the issue you need a not stale "Empty List" of type RemoteChildrenContentsType in your "My Home"-folder or you imagine that in RemoteFileSubSystem at line 669 hasFileAndFolderContents is true, hasFileContents at line 673 is false and at line 678 hasFolderContents is false too. This looks like a contradiction to me. Maybe "Empty List" is a kind of content but not the right one. With "Empty List" hasFileAndFolderContents should be false. It might be inconsitent data from the FTP Service, because "Empty List" is not stale or something like that. But is hasFileAndFolderContents necessary at all?
Hm... so there is two possibilities: (a) FTP runs into a state that should be forbidden. In other words, the file subsystem / framework operates OK but the FTP service fails in a way that it should never fail. If this is the case, we'll need to find the root problem why "empty list" occurs. (b) The "empty list" case is not a forbidden one and the subsystem fails. In this case we'll need to fix the subsystem. Which of the two cases do you think we run into? The other things that worries me is that you say you have to quit and restart Eclipse. Disconnect / reconnect doesn't help, right? I agree that having to restart Eclipse with a feature that's part of stock RSE is a "major" loss of functionality. If you can reproduce the issue with stock FTP, then attach your Eclipse .log file as well as the FTP console log, this may help tracking down the problem (looking for exceptions in the logs).
Created attachment 132430 [details] FTP console log
Right, disconnect / reconnect doesn't help. Now, I know more details. The FTPClient gets null from FTPListParseEngine while trying to connect to the FTP-server and so the socket is null and nothing is read. With wireshark I can see that the FTP-server sends the data but two seconds later it sends "421 timeout". The console looks like attachment 132430 [details], whitout a mention of timeout. The FTPService requests the files from the FTPClient and gets an empty array FTPFile[0]. Because there is an array of files, even if the array is empty, the subsystem treats the array as a regular file list. The list is put in "My Home" as cached content and after this the subsystem realizes that it is an empty list and show the "Empty list"-message in the tree. If you refresh the subsystem it recognizes a not stale entry in "My Home" and decides not to reload. You can reproduce "Empty list", if you set socket to null in FTPClient.initiateListParsing(FTPFileEntryParser parser, String pathname) and comment out the line witch contains _openDataConnection_. After the documentation of FTPClient._openDataConnection_(int command, String arg): "Null is returned if an FTP protocol error is reported at any point during the establishment and initialization of the connection." Otherwise an IOExcetion is thrown. So maybe this is a), because FTPClient should always throw an Exception if the data connection fails. It seems that the FTPClient doesn't realize the response of the server for the passive mode, because it sees 200 when it expects 227. But on the the other hand it might be b) too, because the subsystem should not treat an empty array as a regular file list, especially if it later recognizes it as an empty list. And I guess the subsystem should avoid the state where a tree entry is neither a file nor a folder but a file and a folder, witch leads to no refreshing.
(In reply to comment #6) > It seems that the FTPClient doesn't realize the response of the server for the > passive mode, because it sees 200 when it expects 227. Is your server publicly available so I could try this out? I agree that the main problem seems in FTPClient (we won't ever be able to get any useful data from this server anyways). Without knowing that an exception condition has happened, the subsystem cannot perform any better handling. I'm wondering whether 227 is a valid response to PASV and what it means. Filing a bug against commons net might be in order: http://commons.apache.org/net/issue-tracking.html BTW, I just released RSE I20090420-0900 as the first download including commons net 2.0.
I use a public server of suse for testing: ftp.suse.com. After the RFC 959 for FTP http://tools.ietf.org/html/rfc959 227 is the reply code for "Entering Passive Mode". The server sends the correct 227 but FTPClient gets an older received response, witch is maybe 200 for noop.
I can confirm this bug. Mostly (not always) when i log in to my ftp account, the folder/file list appears to be empty. I am trying to connect in passive mode but the LIST command seems to be completely missing when i try to expand the menu item "Root" in the Remote Systems pane. Console output: 220 80.237.***.*** FTP server ready USER user 331 Password required for user PASS ****** 230 User user logged in SYST 215 UNIX Type: L8 TYPE I 200 Type set to I PWD 257 "/" is the current directory NOOP 200 NOOP command successful NOOP 200 NOOP command successful NOOP 200 NOOP command successful NOOP NOOP 200 NOOP command successful NOOP NOOP 200 NOOP command successful NOOP 200 NOOP command successful NOOP 200 NOOP command successful NOOP 200 NOOP command successful NOOP 200 NOOP command successful NOOP 200 NOOP command successful NOOP 200 NOOP command successful NOOP 200 NOOP command successful NOOP 200 NOOP command successful NOOP 200 NOOP command successful NOOP 200 NOOP command successful CWD / 200 NOOP command successful PASV 200 NOOP command successful Refreshing doesn't help (as described by Martin Liebenberg).
I read again the analysis in comment #6, and I agree that it is an error to use cached data when refresh is requested. Refresh must always discard cached data and thus reload -- this is the main issue we need to solve, and it looks like Martin has described a way for me to reproduce the issue so this should be possible. The other related thing is that we're getting an "empty list" in the first place. this seems to be caused by excessive sending of NOOP commands, so the situation should improve when bug 269171 gets addressed. Other than that, I'm afraid there is not much we can do if the Commons Net FTPClient doesn't inform us about an error condition but sends an empty list instead. We could probably treat directory lists that don't even include the typical "." entry as erroneous, but then there may be some FTP servers which do not send the "." entry so if we go that route it would have to be a configurable option.
Bulk update of target milestones to 3.2
Bulk moving 3.3 deferred items to 3.3.1
