Re: [jetty-dev] Jetty-9 development

[Greg Wilkins <gregw@xxxxxxxxxxx>]

To continue my flow of consciousness brain dump on jetty-9..... 


Today I'm considering refactoring the  dispatching of the EndPoints.

Currently the SelectorManager (as part of the connector) runs an NIO select set and when it selects an endpoint for IO activity it calls the EndPoint#schedule method, which does:
  
 * If there is a thread blocked waiting to read/write and if the endpoint is now readable/writable, then wake up the thread and return.
 * If we are not already dispatched, then dispatch a thread to run SelectChannelEndPoint#handle

SelectChannelEndPoint#handle calls the connection#handle method, which is normally AsyncHttpConnection#handle.  It calls the parser to parse the request, which then calls the handlers to handle the request.   These handlers will read/write input/ouput to/from the request/response in a blocking manner.  If a read/write blocks, then the calling thread(which is the thread dispatched by the EndPoint to call EndPoint#handle) is parked in the EndPoint blockforXxx methods.  When the endpoint becomes readable/writable it's schedule method is again called by the SelectorManager and finds the blocked thread, which it wakes up.   The read/write then continues by calling the parser/generator to frame the input/output as HTTP.   This means that the parser is reentrant the call stack is:


SelectorManager->EndPoint#schedule->dispatch(handle)
EndPoint#handle->Connection#handler->HttpParser#parse->Connection#handlerRequest->Server#service->HanderOrServlet->Response.getInputStream().read()->HttpParser#parse
                                                                                                                                                   ->EndPoint#blockForInput
SelectorManager->EndPoint#schedule->(wakeup blocked thread)



The reentrancy is a little complex, but it does make for very efficient waking up of blocked readers and writers.  However, now with Async servlets and websockets, the reader/writers may be threads other than the dispatched thread, so the parser now has to be reentrant and thread safe, which is tougher and more complex and requires slower locks.     Also with MUX that is needed for SPDY and websocket, the thread that parses the frames won't always be the thread that calls the handlers.


So for Jetty-9, I'm  reconsidering this design to see if we can do something simpler and/or more maintainable, but at least as fast.


I've already discussed how I'm splitting the AsyncHttpConnection into HttpConnection and HttpChannel, so the basic calling stack is going to be something like:

SelectorManager->EndPoint#schedule->dispatch(handle)
Endpoint#handle->HttpConnection#canRead->HttpParser#parse->HttpChannel#setXyz
                                       ->HttpChannel#handleRequest->Server#service->HanderOrServlet->Response.getInputStream().read()->HttpChannel#blockForContent

SelectorManager->EndPoint#schedule->dispatch(handle)
Endpoint#handle->HttpConnection#canRead->HttpParser#parse->HttpChannel#handleContent-> (wakeup blocked thread)


So you can see that the parser is neither reentrant nor multi threaded.  It will always be called from the thread that is dispatched by the selector manager schedule call.   But because of this, we need another dispatch so that 3 threads are now involved rather than 2. This is because previously we could use the selector thread to wake up the blocked thread and it would do the parsing.  But now we can't get the selector thread to do the parsing (too long), so it has to dispatch another thread to do the parsing of the data, which will then wake up the blocked thread by passing the content the the HttpChannel.

So this extra dispatch feels a little expensive... but I don't think it is. it only happens when we have blocked on the network, so we are already looking at 10's or 100's of ms of latency while the network flow control works out when to continue, so the time for an extra dispatch is small.  Also in SPDY and MUX websocket we would have this dispatch anyway, because we have a 1 to n relationship between SpdyConnection and SpdyChannels (for example).

Note also that this puts the blocking logic in the Channel and that will simplify the connections, connectors and endpoints.

Also I think that we can have separate canRead and canWrite callbacks as the handling could be different.    We have to work out which thread(s) are going to be calling the generator and then flush the resulting buffers.  Will it be the threads that call the outputStream.write methods, or another thread dispatched to the connection via canWrite???  or is it the thread that called handleRequest that completes the response after it returns (this is the canRead thread)!?!?!.  Have to think about that one a bit more.

cheers