Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
[jetty-dev] Jetty 9.1 and Servlet 3.1 asynchronous IO



The plan is for Jetty-9.1 to support the Servlet 3.1 API and JSR356 websocket API.

Jan has been working on much of the 3.1 API and Joakim is close with 356 (and we might slip that into a 9.0.x if it can be done without breaking existing APIs).

I'm now working on the servlet 3.1 asynchronous IO layer and thought I'd do some thinking out loud about the design so that if there is anybody out there that is seriously thinking about using this API, they can speak up and start giving us feedback and/or contribute.


Firstly some background about the architecture of Jetty-9.x.   One of the big changes we did for 9 was to split apart the transport and semantic aspects of a protocol.    This is so that we can achieve exactly the same HTTP application semantics over HTTP, SPDY or eventually HTTP/2.0.      To achieve this, the HttpConnection class of Jett-7/8 was split out into a HttpConnection and a HttpChannel class with the typical initiation for plain HTTP connection being:

   EndPoint -> HttpConnection -> HttpChannel -> Handler -> HttpServlet

The HttpConnection deals with all the transport stuff (e.g. HTTP headers, chunking, etc.) and the HttpChannel deals with the servlet API semantic (e.g. request dispatching, AsyncContext etc.).      A good way to think about the difference is to consider where an active thread comes from.   If a Http request arrives on a socket, it is an IO event that comes via the EndPoint and HttpConnection instances before being dispatched into the HttpChannel.   However, if a suspended request is woken up by an AsyncContext#dispatch, then that is something that happens purely at the HttpChannel layer and does not involve a dispatch via the HttpConnection.

This allows us to run the Http semantic over different protocols, eg SSL and/or SPDY:

  EndPoint -> SSLConnection -> SPDYConnection ---n-> HttpChannel -> Handler -> HttpServlet

Note that a SPDYConnection is muxed and can have n HttpChannels!  The servlet request/response lifecycle is handled in HttpChannel, so it works the same regardless of transport. 

We also support upgrade of our HTTP connections to WebSocket, which replaces the Connection so we get:

  EndPoint -> WebSocketConnection --n-> WebSocketSession -> WebSocket

Which allows us to write a really nice and efficient websocket implementation directly to the jetty IO layer. 


Now along comes Servlet 3.1, which has support for both Async IO and upgrade, with at least one use-case being implementing protocols websocket in the webapp rather than in the container.       I'm not entirely sure why you'd want to do this... but then there are other use-case for async IO like handling large uploads and downloads.

Note that async IO is entirely different to async servlets.   Async servlets are suspended until something wakes them up, where that something is usually an event like a web services response arriving, a DB connection being available, a comet message arriving.   Async IO is the ability to have a callback when a write has been completed or there is data available to read.

Servlet 3.1 has added asynchronous IO APIs to the existing ServletInputStream and ServletOutputStream classes - specifically they allow listeners to be added that have onWritePossible(OWP) and onReadPossible(ORP) callbacks.         This essentially creates 2 new entry points to the servlet container, as previously everything was initiated via a Request Dispatch (RD) to a servlet.   To address the issue that most of  the servlet API is not thread safe, the asynchronous IO callbacks must not be concurrent with other entries to the container.   Thus for one "request" there must only be 1 container dispatched thread in OWP, ORP or RD at a time.  We already implement a 1 thread in RD policy in the HttpChannel state, so the logical thing to do is extend this state to also enforce the 1 thread in OWP/ORP/RD policy.

But!  OWP and ORP are IO level events and are meant to be usable for implementing things like websocket, so it is a little strange to me that dispatches to a WebSocket impl will be controlled by a HttpChannel state!   But then on reflection, I think this is OK because such a WebSocket impl would be written to use the ServletInputStream and ServletOutputStream so it would be inextricably linked to HTTP semantics anyway.     Thus a WebSocket impl based on servlet 3.1 would look like:

   EndPoint -> HttpConnection -> HttpChannel -> WebSocketImpl -> WebSocket

ie the HttpConnection and HttpChannel would remain in place providing the ServletInputStream and ServletOutputStream

Having servlet async IO live in HttpChannel also works out for non HTTP async IO use-cases that are not dependent on upgrade - eg for doing a large uploads and downloads.   Code written to the servlet async IO needs to work on SPDY as in:

  EndPoint -> SSLConnection -> SPDYConnection ---n-> HttpChannel -> OWP/ORP

IE the HttpChannel still exists on SPDY and still is the controller of dispatch to RD/OWP/ORP


So that's the way I'm currently thinking/implementing......   please yell if you think I'm missing something or otherwise want to be part of  the "fun"

If you are interested in servlet async IO, then it is well worth reading the export group archives: https://java.net/projects/servlet-spec/lists/jsr340-experts/archive as you can see that even with a voted on public final draft there are plenty of undecided aspect to this.

cheers
























 






--
Greg Wilkins <gregw@xxxxxxxxxxx>
http://www.webtide.com
Developer advice and support from the Jetty & CometD experts.
Intalio, the modern way to build business applications.

Back to the top