Re: [jetty-dev] Reactive Streams

[Simone Bordet <sbordet@xxxxxxxxxxx>]

Hi,

On Mon, Jun 1, 2015 at 2:48 PM, Greg Wilkins <gregw@xxxxxxxxxxx> wrote:
>
> On 1 June 2015 at 18:21, Simone Bordet <sbordet@xxxxxxxxxxx> wrote:
>>
>> Makes sense ?
>
>
> I think I can see what you are getting at, but I don't think it fits with
> the whole Reactive Stream model.   You are saying that you can make an N to
> M processor work by not fully consuming the items passed to onNext.   This
> allows the onNext to return if their is no demand for more parts/chunks and
> threadlessly wait until there is demand.

Correct.

> But that relies on using ByteBuffers with mutable  position() and limit()
> that record how much as been consumed.  I don't think that it is intention
> of the reactive stream model to have mutable items that can be used as a
> back channel.

Using ByteBuffer is an implementation detail.

> For a start, it would not work with byte[] or String as there
> are no mutable pointers that can communicate how much has been consumed.

That is true, but the APIs give you back that information for bytes,
which is what you will eventually end up to if you work with sockets.

But even imagining a non-socket system that consumes strings, you
already have those Strings allocated in memory.
So you should not worry about memory, and how to backpressure parts of
a huge String, when you have already all of it in memory.

Bottom line for me is this: you can backpressure easily with this
model, but not with a single byte precision. But you can never have
single byte precision, so there is no point in wanting that.
You can easily implement a proxy with proper backpressure, which in my
experience is the toughest use case to get right.

For a replacement API for Servlet 3.1 async I/O I still have some
concern on what I called "utility classes", but I can see the model
much simpler than what Servlet 3.1 async I/O currently offers.

> But you've given me something to think about....
>
> Do you have any comments about the README and code in the
> FragmentingProcessor branch?

Will have a detailed look now.

-- 
Simone Bordet
----
http://cometd.org
http://webtide.com
http://intalio.com
Developer advice, training, services and support
from the Jetty & CometD experts.
Intalio, the modern way to build business applications.