Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [jetty-users] Jetty9 websocket design
Thanks for the detailed answer. I have a few questions and comments inline.

On Thu, May 30, 2013 at 9:01 AM, Joakim Erdfelt <joakim@xxxxxxxxxxx> wrote:
That's an awfully wide open topic. ;-)

A quick breakdown of everything from the start (obviously from a Jetty 9.0.x perspective)...

In order to have a websocket, you must first create the universe.
skipping a few obvious steps...

The incoming connection arrives to Jetty as a standard HTTP (plain or SSL) connection, which hits the normal flow of internal Jetty handlers and eventually makes its way to the WebSocketServlet implementation.

The WebSocket Upgrade Handshake:
At this point a few tests are done against the request headers to determine if its a HTTP upgrade, is it a WebSocket upgrade, does the origin match, etc.
If this all checks out then the WebSocketCreator is asked for a websocket instance.  The default creator will create a new instance of the WSE (WebSocket Endpoint) each time, but you can provide your own logic and reuse instances or even go stateless with a single instance for all WSEs if you so desire.
If the WebSocketServlet gets a WSE at this point it proceeds to respond on the HTTP that the upgrade is successful, sending HTTP Response 101 Switching Protocols.

The WebSocket Initialization:
In the setup of the WebSocketConnection, the following is initialized as well..
 * WebSocket Session (this represents the connected session)
 * WebSocket Parser (frame parser)
 * WebSocket Generator (frame generator)
 * EventDriver (this is the common abstraction for talking to a WSE instance using any of the WSE definition techniques)
 * WebSocket Extension Stack (using any of the negotiated extensions during the WebSocket upgrade handshake)

The plumbing from the physical connection to the WSE is done by chaining up IncomingFrames and OutgoingFrames implementations.
It looks sort of like this ...

Inline image 1

Now, internally to Jetty, the WebSocketServlet has notified the HTTP handling that the HTTP connection handling is now a WebSocketConnection.
The WebSocketConnection is setup with the WebSocket Endpoint

The Open WebSocket:
At this point the RFC6455 state transitions from CONNECTING to OPEN.
The local WSE is notified that the Session is now open.
The JIOE (Jetty I/O Endpoint) now does its Async/NIO handling of read and write interests and notifying the WebSocketConnection when the physical connection is capable of doing a read or a write.

Is this a single thread, multiple, or configurable? Meaning the async selector?
 
If a read, it fills up an input buffer and parses what it can.
The Parser notifies its IncomingFrames on each frame encountered, which goes through the extension stack and eventually hits the WSE.
If its a write, the Connection's outgoing queue of frames is used to create a output buffer via the Generator.

At this point, the Thread that is handling the JIOE side is the same thread that Jetty pulled from the ThreadPool for the HTTP handling.
When the WebSocket Connection gets its read/write events it will continue to use that same thread to parse, send the frame through the extension stack, and hit the EventDriver, and eventually the WSE.
When the WSE gets the event for say a Text message, then its actually operating on the same Thread that the JIOE is using.
When a JIOE event to write is detected, the same Thread is used to generate and write to the network.

Memory consumption is not constant with WebSocket, due to the nature of the message based (not streaming) WebSocket protocol.

Buffering? Is this handled per connection or system wide, e.g. a buffer pool?
 

Everything you do is based on a WebSocket message. reading and writing.
Per RFC6455, a WebSocket message has no upper limit in size, only the individual frames do.
A WebSocket message is 0..n frames.  a frame can be 0 to 9,223,372,036,854,775,807 bytes in size (63-bit value btw).

In Jetty, we obviously can't support that with modern hardware (who has 9 exabytes of memory on their computer??)
So we provide a means to limit the per-message size via the WebSocketPolicy.setMaxMessageSize() which will be checked on incoming frames, if in violation it results in a RFC6455 close code 1009 MESSAGE_TOO_LARGE.
Outgoing messages are not checked, as it should be obvious to you if you can handle it or not.

Dealing with Extensions:
The need to support RFC6455 WebSocket extensions means we have a layer between the WSE and the physical connection called the ExtensionStack.
This complicates matters as the extensions can change the framing, the number of frames, etc, making the traditional approach of read/write to the physical connection from the WSE impossible.

Inline image 2

Performance with Jetty 9.0.3:
We've done some simple (really simple) load testing on WebSockets.
On a simple test of 1000 active WebSockets we got an average rate of 200,000 messages a second per socket. 
But we need MUCH more testing here, along with paying attention to memory and CPU, not just I/O.
The cometd project has already embraced Jetty 9's WebSocket and have been testing the performance from their point of view, while they have some concerns with memory (the jetty websocket implementation seems to have frequent, but small, GC behavior), overall they are happy with what they see.

Have you looked into direct memory buffers? They seem ideal for this sort of thing.
 

The future of WebSocket on Jetty 9.
We are close to completion on our implementation of JSR-356 (the javax.websocket standard API).
It will likely show up in Jetty 9.1 (but this isn't fully decided yet)

What's the alternative? 9.0 or 9.2?
 

It brings a few nice things to the Jetty side API as well.
 * The WebSocketPolicy.setMaxMessage() is split up between text and binary now.
 * Support for streaming messages has been added 
   - you can write a message with an OutputStream or Writer now.
   - you can receive InputStream or Readers (one for each message) on your annotated WSE's now
 * The RemoteEndpoint (how you write to a remote WSE) has some guards to notify you if you abuse the message writing in a multi-threaded environment.
   - Note: the RemoteEndpoint operates in a similar way, threading wise, as a OutputStream (in other words, it is not advisable to attempt to write from multiple threads)

Once we get a stable build with JSR-356 out into the hands of our users, we can start to tackle various performance testing and concerns.

Hope this rambling post helped...

Very much, yes. Thanks.
 


--
Joakim Erdfelt <joakim@xxxxxxxxxxx>
Developer advice, services and support

from the Jetty & CometD experts


On Thu, May 30, 2013 at 5:51 AM, Nils Kilden-Pedersen <nilskp@xxxxxxxxx> wrote:
Trying once more. Presumably this is documented somewhere I haven't been able to find.


On Mon, May 27, 2013 at 8:34 AM, Nils Kilden-Pedersen <nilskp@xxxxxxxxx> wrote:
Could someone comment on how Jetty9 handles websockets, from a design perspective? I'm thinking primarily about resource consumption, such as memory usage per connection, number of threads and any (non-) blocking behavior, etc.

Thanks,
Nils


_______________________________________________
jetty-users mailing list
jetty-users@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/jetty-users



_______________________________________________
jetty-users mailing list
jetty-users@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/jetty-users


Back to the top