Re: [jetty-dev] Reactive Streams

[Simone Bordet <sbordet@xxxxxxxxxxx>]

Greg,

On Fri, May 29, 2015 at 8:49 AM, Greg Wilkins <gregw@xxxxxxxxxxx> wrote:
>
> Simone,
>
> I thought we should move this discussion here as it would be good to get
> community feedback/thoughts on something that might be core to Jetty-10
>
> I'm very interested in what you showed me regarding the Reactive Streams
> proposed for java 9
> http://gee.cs.oswego.edu/dl/jsr166/dist/docs/java/util/concurrent/Flow.html
>
> But the handling of ByteBuffers cannot be done as you have done in your
> prototype because their is a hard rule that the unit of backpressure
> requests is exactly the number of items that can be sent.  Thus saying
> request(4096) would not allow 4096 bytes to be sent, but 4096 ByteBuffers to
> be sent, each of arbitrary size!

But I don't do that, because it would be impossible to know how many
bytes the TCP connection can send.
What I do is request(1), meaning that I demand 1 ByteBuffer, no matter its size.

This is what we already do right now, and what any async write does:
you attempt the write, but have no idea how many bytes will be
written.
If not all of them, you arrange to save the remaining bytes aside, and
retry when you can.

> The guys developing the API did discuss this problem briefly in
> https://github.com/reactive-streams/reactive-streams-jvm/issues/47   But
> then basically gave up as being too hard!

I don't see it this way. It was basically a no-issue, especially with
bytes and sockets.

> So I've reopened the discussion at
> https://github.com/reactive-streams/reactive-streams-jvm/issues/270 and
> http://cs.oswego.edu/pipermail/concurrency-interest/2015-May/014265.html
> (not sure which is the correct forum).
>
> If Flows are going to be the new cool way of processing streams
> asynchronously then I think it would be good to preserve the same semantic
> all the way down to the byte stream... specially as he back flow mechanism
> is essentially the same as the HTTP2 flow control window, so it would be
> good allow applications to influence that window via this API.

And it is already preserving the same semantic.

I am not sure I understand the problem you are having ?

Even in the case where the application writes a 1 GiB byte[], how do
we do it now ?
We attempt the write, write some, store it away, and wait for NIO to
call us back.
The same happens with this API when you do a request(1), meaning 1 ByteBuffer.

Will put my initial work on a GitHub repo, so we can look at the same code.

-- 
Simone Bordet
----
http://cometd.org
http://webtide.com
http://intalio.com
Developer advice, training, services and support
from the Jetty & CometD experts.
Intalio, the modern way to build business applications.