Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [jetty-users] Getting a java.util.concurrent.RejectedExecutionException
First, and most important, your thread pool size is far too low for a production machine.

Some napkin math for you to work this out.
maximum thread in thread pool = 16 + ((num of cpu cores) * 4) + (peak concurrent connections)

Next, update your Java VM.
Your JVM is officially expired and out of date.
Would start with using official oracle JVMs, not the OpenJDKs series (at least in the Java 1.6 class).
1.6_29 fixed a number of JIT issues that caused problematic looping that caused a *wide* range of problems.
1.6_30 fixed an SSL regression introduced in u29
1.6_31 was a minor update (mainly for Timezone updates)
1.6_32 contained important networking and garbage collection bug fixes
1.6_33 had many updates, the only one impacting a server would be the Timezone updates.
1.6_34 had a bunch of networking and util/collections bug fixes
1.6_35 emergency patch for CVE-2012-4681 (not server affecting) and more Timezone updates
1.6_37 more timezone updates (and other non-server impacting updates)
1.6_38 networking fixes (for windows) and SSL close issue
1.6_39 more timezone updates
1.6_41 more timezone updates
1.6_43 announced end for Java 6, plus emergency patch for CVE-2013-1493 (not server affecting)
1.6_45 fix for Runtime.exec & RMI bug fixes along with (yup you guessed it) a Timezone update

At least get past update 38.
Also, if you are using SSL in production, it would be highly advised that you upgrade to Java 1.7, as many of the SSL fixes have not been put into Java 1.6. 

--
Joakim Erdfelt <joakim@xxxxxxxxxxx>
Expert advice, services and support from from the Jetty & CometD experts

On Tue, Sep 30, 2014 at 1:44 PM, Jeff Smyth <jds@xxxxxxxxxxxx> wrote:

​The Java Version is OpenJDK 1.6.0_27

The App is running on Heroku and uses Ubuntu 10.04

Jetty is running embedded, and the ExecutorThreadPool is being invoked with the following parameters:

corePoolSize=5

MaximumPoolSize=20

keepAliveTime=5000 ms

The LinkedBlockingQueue that is passed to the ExecutorThreadPool has a capacity of 20.


- Jeff
From: jetty-users-bounces@xxxxxxxxxxx <jetty-users-bounces@xxxxxxxxxxx> on behalf of Joakim Erdfelt <joakim@xxxxxxxxxxx>
Sent: Tuesday, September 30, 2014 4:01 PM
To: JETTY user mailing list
Subject: Re: [jetty-users] Getting a java.util.concurrent.RejectedExecutionException
 
What specific version of Java 1.6? (some versions of Java 1.6 have problems)
On what environment/platform/OS? (some platforms have shown problems)
Also, what are your threadpool settings? (low threadpool settings can cause problems)


--
Joakim Erdfelt <joakim@xxxxxxxxxxx>
Expert advice, services and support from from the Jetty & CometD experts

On Tue, Sep 30, 2014 at 12:46 PM, Jeff Smyth <jds@xxxxxxxxxxxx> wrote:
Jetty Users Group,

I have recently been facing an issue with my Production Server which runs on Jetty. I hope this is the correct location to ask for assistance. The following is the error that is being thrown:

app/web.4: 2014-09-30 03:42:36,794 [pool-2-thread-1 Selector0] WARN org.eclipse.jetty.io.nio - Dispatched Failed! SCEP@5ebab0e9{l(/172.16.96.233:57773)<->r(/172.16.96.234:24302),s=-1,open=true,ishut=false,oshut=false,rb=false,wb=false,w=true,i=1r}-{AsyncHttpConnection@32b4c195,g=HttpGenerator{s=0,h=-1,b=-1,c=-1},p=HttpParser{s=-14,l=0,c=0},r=0} to org.eclipse.jetty.server.nio.SelectChannelConnector$ConnectorSelectorManager@45544e03

app/web.4: 2014-09-30 03:42:36,794 [pool-2-thread-1 Selector0] WARN org.eclipse.jetty.util.thread.ExecutorThreadPool -
app/web.4: java.util.concurrent.RejectedExecutionException
app/web.4:  at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:1992)
app/web.4:  at org.eclipse.jetty.server.nio.SelectChannelConnector$ConnectorSelectorManager.dispatch(SelectChannelConnector.java:300)
app/web.4:  at org.eclipse.jetty.io.nio.SelectChannelEndPoint.dispatch(SelectChannelEndPoint.java:236)
app/web.4:  at org.eclipse.jetty.io.nio.SelectChannelEndPoint.schedule(SelectChannelEndPoint.java:195)
app/web.4:  at org.eclipse.jetty.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:619)
app/web.4:  at org.eclipse.jetty.io.nio.SelectorManager$1.run(SelectorManager.java:290)
app/web.4:  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
app/web.4:  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
app/web.4:  at java.lang.Thread.run(Thread.java:679)

This error spits out over 100 times a second. When the server is brought up, it runs fine for about 4-8 hours until this error starts showing, and the application is soon unresponsive.

The application runs on a Heroku server, it is on Java 1.6, using Jetty Version 7.6.16.v20140903

I also found this (seemingly) related bug: https://jira.codehaus.org/browse/JETTY-1423 , however, since I am using Jetty 1.6 I am not using an affected version.

Any help or insight into what may be causing this issue would be appreciated.

Thanks!


_______________________________________________
jetty-users mailing list
jetty-users@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/jetty-users


_______________________________________________
jetty-users mailing list
jetty-users@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/jetty-users

Back to the top