[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [jetty-users] Jetty completely unresponsive, seems to be caused by ChannelEndPoint.close()

oh, i was sure you were going to say windows :)

I suspect the first thing you should do it update to 7.6.3, there were
a couple of edge case issues addressed can you could be subject to one
of those...if you still get the behavior let us know as it could be
another edge case

jesse

--
jesse mcconnell
jesse.mcconnell@xxxxxxxxx


On Wed, May 2, 2012 at 3:39 PM, Mennonite _ <mennonite@xxxxxxxxxxx> wrote:
> CentOS 5.5 64-bit
>
>
> ________________________________
>
>
>> what operative system?
>
> ________________________________
>
> Jetty version: 7.6.2
> Java version: 1.6.0 U30
>
> Every once in a while, Jetty all but completely stops responding to
> HTTP/HTTPS connections. I got a Java thread dump, and tracing through the
> locks, it looks like there's a single
> org.eclipse.jetty.io.nio.ChannelEndPoint.close() which, via locks, seems to
> block all other close()s, bringing everything to a grinding halt.
>
> There are a ton of threads, all with very similar/identical stack traces,
> waiting for one lock:
> "qtp683347213-21440" prio=10 tid=0x00002aaab57ac000 nid=0x3172 waiting for
> monitor entry [0x0000000042d8f000]
> ÂÂ java.lang.Thread.State: BLOCKED (on object monitor)
> ÂÂÂÂÂÂÂ at java.nio.channels.spi.AbstractSelector.cancel(Unknown Source)
> ÂÂÂÂÂÂÂ - waiting to lock <0x00000006e659db48> (a java.util.HashSet)
> ÂÂÂÂÂÂÂ at java.nio.channels.spi.AbstractSelectionKey.cancel(Unknown Source)
> ÂÂÂÂÂÂÂ - locked <0x00000007398808d0> (a sun.nio.ch.SelectionKeyImpl)
> ÂÂÂÂÂÂÂ at
> org.eclipse.jetty.io.nio.SelectChannelEndPoint.close(SelectChannelEndPoint.java:691)
> ÂÂÂÂÂÂÂ at
> org.eclipse.jetty.io.nio.SslConnection$SslEndPoint.close(SslConnection.java:652)
> ÂÂÂÂÂÂÂ at
> org.eclipse.jetty.server.AsyncHttpConnection.onInputShutdown(AsyncHttpConnection.java:194)
> ÂÂÂÂÂÂÂ at
> org.eclipse.jetty.io.nio.SslConnection.handle(SslConnection.java:210)
> ÂÂÂÂÂÂÂ at
> org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:609)
> ÂÂÂÂÂÂÂ at
> org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:45)
> ÂÂÂÂÂÂÂ at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
> ÂÂÂÂÂÂÂ at
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
> ÂÂÂÂÂÂÂ at java.lang.Thread.run(Unknown Source)
>
> Which is held by this thread, which is waiting for another lock:
> "qtp683347213-90 Selector7" prio=10 tid=0x000000004f5c3000 nid=0x7773
> waiting for monitor entry [0x0000000041bdb000]
> ÂÂ java.lang.Thread.State: BLOCKED (on object monitor)
> ÂÂÂÂÂÂÂ at sun.nio.ch.SocketChannelImpl.kill(Unknown Source)
> ÂÂÂÂÂÂÂ - waiting to lock <0x00000007398806a0> (a java.lang.Object)
> ÂÂÂÂÂÂÂ at sun.nio.ch.EPollSelectorImpl.implDereg(Unknown Source)
> ÂÂÂÂÂÂÂ at sun.nio.ch.SelectorImpl.processDeregisterQueue(Unknown Source)
> ÂÂÂÂÂÂÂ - locked <0x00000006e659db48> (a java.util.HashSet)
> ÂÂÂÂÂÂÂ at sun.nio.ch.EPollSelectorImpl.doSelect(Unknown Source)
> ÂÂÂÂÂÂÂ at sun.nio.ch.SelectorImpl.lockAndDoSelect(Unknown Source)
> ÂÂÂÂÂÂÂ - locked <0x00000006e659e300> (a sun.nio.ch.Util$2)
> ÂÂÂÂÂÂÂ - locked <0x00000006e659e2f0> (a
> java.util.Collections$UnmodifiableSet)
> ÂÂÂÂÂÂÂ - locked <0x00000006e659daf0> (a sun.nio.ch.EPollSelectorImpl)
> ÂÂÂÂÂÂÂ at sun.nio.ch.SelectorImpl.selectNow(Unknown Source)
> ÂÂÂÂÂÂÂ at
> org.eclipse.jetty.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:531)
> ÂÂÂÂÂÂÂ at
> org.eclipse.jetty.io.nio.SelectorManager$1.run(SelectorManager.java:285)
> ÂÂÂÂÂÂÂ at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
> ÂÂÂÂÂÂÂ at
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
> ÂÂÂÂÂÂÂ at java.lang.Thread.run(Unknown Source)
>
> Which is held by this thread:
> "qtp683347213-19755" prio=10 tid=0x000000005005b800 nid=0x7f5e runnable
> [0x00000000440a3000]
> ÂÂ java.lang.Thread.State: RUNNABLE
> ÂÂÂÂÂÂÂ at sun.nio.ch.FileDispatcher.preClose0(Native Method)
> ÂÂÂÂÂÂÂ at sun.nio.ch.SocketDispatcher.preClose(Unknown Source)
> ÂÂÂÂÂÂÂ at sun.nio.ch.SocketChannelImpl.implCloseSelectableChannel(Unknown
> Source)
> ÂÂÂÂÂÂÂ - locked <0x00000007398806a0> (a java.lang.Object)
> ÂÂÂÂÂÂÂ at
> java.nio.channels.spi.AbstractSelectableChannel.implCloseChannel(Unknown
> Source)
> ÂÂÂÂÂÂÂ at java.nio.channels.spi.AbstractInterruptibleChannel.close(Unknown
> Source)
> ÂÂÂÂÂÂÂ - locked <0x000000073987aea0> (a java.lang.Object)
> ÂÂÂÂÂÂÂ at
> org.eclipse.jetty.io.nio.ChannelEndPoint.close(ChannelEndPoint.java:205)
> ÂÂÂÂÂÂÂ at
> org.eclipse.jetty.io.nio.SelectChannelEndPoint.close(SelectChannelEndPoint.java:700)
> ÂÂÂÂÂÂÂ at
> org.eclipse.jetty.io.nio.SslConnection$SslEndPoint.close(SslConnection.java:652)
> ÂÂÂÂÂÂÂ at
> org.eclipse.jetty.server.AsyncHttpConnection.onInputShutdown(AsyncHttpConnection.java:194)
> ÂÂÂÂÂÂÂ at
> org.eclipse.jetty.io.nio.SslConnection.handle(SslConnection.java:210)
> ÂÂÂÂÂÂÂ at
> org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:609)
> ÂÂÂÂÂÂÂ at
> org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:45)
> ÂÂÂÂÂÂÂ at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
> ÂÂÂÂÂÂÂ at
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
> ÂÂÂÂÂÂÂ at java.lang.Thread.run(Unknown Source)
>
>
> This happens very sporadically, and so I can't reproduce it on demand. I'd
> expect that there are certain environmental factors that cause this, but I
> don't know what those are.
>
> Any thoughts about this?
>
> _______________________________________________
> jetty-users mailing list
> jetty-users@xxxxxxxxxxx
> https://dev.eclipse.org/mailman/listinfo/jetty-users
>