[jetty-users] Concurrent issue with WebSocket + FragmentExtension

[<xbruggheman.ext@xxxxxxxxxx>]

Hi,

I'm embeddeding Jetty for WebSocket client and server and using the frag extension (set to 1360 bytes to nearly fill an MTU with SSL headers).

I'm sporadically getting the following error which drops the WebSocket connection:

org.eclipse.jetty.websocket.api.ProtocolException: Unexpected BINARY frame, was expecting CONTINUATION
	at org.eclipse.jetty.websocket.common.Parser.parseFrame(Parser.java:383)
	at org.eclipse.jetty.websocket.common.Parser.parse(Parser.java:234)
	at org.eclipse.jetty.websocket.common.io.AbstractWebSocketConnection.read(AbstractWebSocketConnection.java:551)
	at org.eclipse.jetty.websocket.common.io.AbstractWebSocketConnection.onFillable(AbstractWebSocketConnection.java:459)
	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.run(AbstractConnection.java:358)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:596)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:527)
	at java.lang.Thread.run(Unknown Source)

This occurs when one of the threads (A) invokes sess.getRemote().sendBytes(...) with a big message (~20KB) and shortly after another thread (B) does the same with a small message needing no fragmentation (~100 bytes). The binary frame of the small message is sent before all the CONTINUATION frames of the large message have been sent, and the other side detects the issue and drops the connection.

The issue seems to be in the following method: org.eclipse.jetty.websocket.common.extensions.fragment.FragmentExtension#outgoingFrame(...): there is no concurrent mechanism preventing one thread from sending non control frames while another one is busy creating fragmented frames.

Assuming the session is intended to be threadsafe, in my opinion everything past the end of the first IF block handling control frames should be done under a lock relative to the LogicalConnection, in order to ensure that all the fragmented CONTINUATION frames and the FIN frame get added via AbstractExtension#nextOutgoingFrame() before another thread can inject its own frames in nextOutgoing. Making the method synchronized is obviously wrong here since it wouldn't allow the sending of control frames while a fragmented message is being sent, which the spec allows.

I'm reluctant to synchronize all my uses of sendBytes(), so for now i've disabled the frag extension as a workaround since I only used it to improve network throughput.

Am I wrong in assuming the Session is threadsafe ? If not, could you please provide us with a fix ?

Regards,

Xavier BRUGGHEMAN



_________________________________________________________________________________________________________________________

Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
France Telecom - Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, France Telecom - Orange is not liable for messages that have been modified, changed or falsified.
Thank you.