Community
Participate
Working Groups
Javadoc for IFileStore#openInputStream() seems to assume that the EFS Provider implementation is always capable of opening an additional stream -- at least the "reasons include" for exceptions does not talk about the case where no more streams are available. The problem for EFS providers is, that they cannot keep an arbitrary number of stream open. Typical SSH / Sftp servers, for instance, allow only 4 concurrent file download sessions to be open at the same time. For FTP, only a single stream at a time may be supported. Clients may be interested in keeping a stream open for longer. But when a client has used up all available streams, what should the provider do? Is there a minumum number of concurrently open streams that it must support? Should it just throw a CoreException saying "no more streams"? Should it retreat to downloading the entire requested file to a temp location and return a stream to that file? Should there be a timeout after which pending open streams are closed automatically? Or should openStram() requests just be queued, such that the progress for a new openInputStream() would be stalled infinitely until a new handle gets available by closing another stream? I think that Javadoc for IFileStore#openInputStream() IFileStore#openOutputStream() should 1. Make clients aware that the number of concurrently open streams can be very limited, and they are encouraged to close open streams as soon as possible. 2. Define the behavior that providers should expose in case they run out of available streams (supposedly just throw an exception; queueing requests would be another option though more risky I'd guess; perhaps there should be a queuing time of e.g. 20 seconds and throwing the "no more streams" exception afterwards). 3. Potentially define a minimum number of streams that an EFS provider must support in order to be valid. For instance, one could require that the bare mimimum is to support - one open input stream - one open output stream - plus the capability to do childNames() or fetchInfo() concurrently while the two streams are open. For providers like FTP, this would mean that they need to support 3 concurrent connections at the same time. Another option would be to allow providers only support a single stream at any one time, so "well behaving" clients should not assume they can keep multiple streams open. Any comments on that issue?
1 and 2 sound reasonable. For 3, I don't like the idea of creating an arbitrary limit. A very simple client could function on an implementation that only supported a single connection, so it would be overly restrictive to define such a limit - it would prevent certain uses that would otherwise be fine.
Ok, so it looks like to improve the Javadoc for IFileStore#getInputStream() IFileStore#getOutputStream() in the following ways: 1. Tell clients that the number of concurrently open Streams can be limited, the limit can depend on the Provider, and may be as low as only 1; so, the client should close the Stream as soon as it can 2. Specify that a CoreException can be thrown when no more Streams are available 3. Specify that providers _may_ enqueue the get*Stream() request in case no Streams are available, such that they wait until a Stream becomes available or the (maximum timeout?) expires and they fire a "no more streams" CoreException. We may want to discuss what that timeout could be, perhaps it should be free for the provider to choose. At any rate, while the Provider waits for a Stream, it should periodically check the IProgressMonitor so the request can be canceled when it takes too long. Because these are all changes of Javadoc, I changed the Summary to more clearly reflect what we want (previous value was: "Need advice how providers should handle multiple open streams")
Looking at the FileStore#copy() implementation, which is also referenced by FileStore#move(), it seems that the minimum requirement for a proper EFS provider is to support 1 InputStream 1 OutputStream in parallel (or, the provider would need to override the copy() and move() operations such that it works with a single Stream only).
Martin, are you considering providing the patch?
We're past the time for this kind of item for 3.4. Please move milestone.
I think that the EFS chapter in the dev guide should be updated along with javadocs. It's not too late for such bugs, however I will take care of it during 3.5.
Created attachment 134609 [details] Proposal v01
+1. The only change is "has exceeded" should be changed to "has been exceeded".
Thanks John. The update to javadoc released to HEAD.
