[jetty-users] Hang in HttpOutput.close

[Gordon Jahn <gjahn@xxxxxxxxxxxxxxxxx>]

Hi folks,

We're running an app using embedded Jetty 9.4.43 and have had two hangs in 36 hours after all being well for, well, weeks.

We're wondering if it's a browser change that's caused it (this is fronted by Apache and the Apache logs show client bump from Chrome 94 to 95) but our thread dump (using jstack) shows 193 threads blocked at  org.eclipse.jetty.server.HttpOutput.close(HttpOutput.java:639).  We suspect it's stopped responding to requests because the thread pool is out of available connections.

The stack dump from HttpOutput.close() is as follows:

"qtp2104336222-27891" #27891 prio=5 os_prio=0 tid=0x00007f7d780b4800 nid=0x3ba waiting on condition [0x00007f7d4f235000]
   java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for  <0x0000000581b953d0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at org.eclipse.jetty.util.SharedBlockingCallback$Blocker.block(SharedBlockingCallback.java:241)
at org.eclipse.jetty.server.HttpOutput.close(HttpOutput.java:639)

We're trying to work around, but aren't sure what's happening.  Had a quick look at the 9.4.44 release notes and can see this change https://github.com/eclipse/jetty.project/issues/6562 but not sure it's necessarily going to address this.

Can anyone advise? We're currently trying to workaround (by not calling "close"!).

Thanks in advance,

Gordon Jahn