Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [jetty-users] maxQueued

Hi Joakim

I'm sorry to pile on another email onto the last….

But going back and reading the Jetty7 code. It seems that Jetty's BlockingArrayQueue does offer an upper bound.

From: Chris Berry <cberry@xxxxxxxxxxxx>
Date: Wednesday, November 20, 2013 9:04 AM
To: JETTY user mailing list <jetty-users@xxxxxxxxxxx>
Subject: Re: [jetty-users] maxQueued

Hi Joakim,

Thanks for the reply.
Nice to know about the JMX stuff.

Per this statement;

The wiki about high load doesn't mention the maxQueued configuration for QueuedThreadPool, as that's not related to high load in our minds.
It would be like trying to improve the performance of an F1 racer by changing the length of the spark plug wires.

Actually the Eclipse Wiki does mention it -- indirectly.
AFAICT, The ArrayBlockingQueue bit below is exactly equivalent when specifying maxQueued to the QueuedThreadPool code.

Thread Pool

It is very important to limit the task queue of Jetty. By default, the queue is unbounded! As a result, if under high load in excess of the processing power of the webapp, jetty will keep a lot of requests on the queue. Even after the load has stopped, Jetty will appear to have stopped responding to new requests as it still has lots of requests on the queue to handle.

For a high reliability system, it should reject the excess requests immediately (fail fast) by using a queue with a bounded capability. The capability (maximum queue length) should be calculated according to the "no-response" time tolerable. For example, if the webapp can handle 100 requests per second, and if you can allow it one minute to recover from excessive high load, you can set the queue capability to 60*100=6000. If it is set too low, it will reject requests too soon and can't handle normal load spike.

Below is a sample configuration:

<Configure id="Server" class="org.eclipse.jetty.server.Server">
    <Set name="ThreadPool">
      <New class="org.eclipse.jetty.util.thread.QueuedThreadPool">
        <!-- specify a bounded queue -->
        <Arg>
           <New class="java.util.concurrent.ArrayBlockingQueue">
              <Arg type="int">6000</Arg>
           </New>
      </Arg>
        <Set name="minThreads">10</Set>
        <Set name="maxThreads">200</Set>
        <Set name="detailedDump">false</Set>
      </New>
    </Set></Configure>

And for exactly the reason quoted from the Wiki above, I need a bounded queue.
I'd rather return a 503 and keep the Service humming along nicely than overload it and destroy performance for all. And then, if possible, add more Servers or tune accordingly. 
And I'd rather not run, say, Apache, in front of Jetty to give me a "Listen Backlog" because of the added complexity.

So is there another way I can accomplish this in Jetty7?  (We unfortunately cannot move to Jetty9 yet)
Is the ArrayBlockingQueue truly more heinous than allowing my systems to get overloaded?

The ExecutorThreadPool — while available in Jetty7 — does appear to offer bounded job queueing. 
But it also uses the problematic ArrayBlockingQueue.
So would you recommend that over the QueuedThreadPool, since both appear to have the same weakness?

Any other suggestions are greatly appreciated.

Thanks much,
-- Chris 

BTW: Thanks for the pointers to the Mechanical Sympathy discussion. It looks fascinating.

From: Joakim Erdfelt <joakim@xxxxxxxxxxx>
Reply-To: JETTY user mailing list <jetty-users@xxxxxxxxxxx>
Date: Wednesday, November 20, 2013 7:46 AM
To: JETTY user mailing list <jetty-users@xxxxxxxxxxx>
Subject: Re: [jetty-users] maxQueued

I'll start with the JMX part of the question first.


Looks like the maxQueued attribute is not exposed.
That's a bug, filed an issue against it -> https://bugs.eclipse.org/bugs/show_bug.cgi?id=422137

As for the rest of the question...
We had too many issues with ArrayBlockingQueue, we removed support for maxQueued and the ArrayBlockingQueue from QueuedThreadPool starting in Jetty 9.
It started out as a performance question, then part of our mechanical sympathy efforts.

Eventually, some performance tests were run ...

And we determined that the ArrayBlockingQueue performs some harsh JDK locking that actually hurts performance.
So we removed it as part of our Queue/ThreadPool cleanup

However, we also made Jetty 9 use the built-in java.util.concurrent techniques.
Which allowed us to even expose that entire framework as an option for developers to use as the ThreadPool for jetty.

The wiki about high load doesn't mention the maxQueued configuration for QueuedThreadPool, as that's not related to high load in our minds.
It would be like trying to improve the performance of an F1 racer by changing the length of the spark plug wires.


--
Joakim Erdfelt <joakim@xxxxxxxxxxx>
Expert advice, services and support from from the Jetty & CometD experts


On Tue, Nov 19, 2013 at 6:22 PM, Chris Berry <cberry@xxxxxxxxxxxx> wrote:
Greetings,

I want to use maxQueued in the <server> config.
And I'm pretty certain I have it setup properly in my jetty.xml
(Note: I DO see my other settings confirmed in the JMX MBean)

<Configure id="Server" class="org.eclipse.jetty.server.Server">
    <Set name="ThreadPool">      <New class="org.eclipse.jetty.util.thread.QueuedThreadPool">        <Set name="name">Jetty</Set>
        <Set name="maxQueued"><Property name="jetty.maxQueued" default="-1"/></Set>
        <Set name="minThreads"><Property name="jetty.minThreads" default="10"/></Set>
        <Set name="maxThreads"><Property name="jetty.maxThreads" default="200"/></Set>
        <Set name="detailedDump">false</Set>
      </New>
    </Set>

I read the source code, and as far as I can tell, by defining "maxQueued" I should get an ArrayBlockingQueue created.
But see no way to confirm that I actually got this right.

When I look at JMX, I see no mention of "maxQueued" -- whether I define it or not.

BTW: this doc: http://wiki.eclipse.org/Jetty/Howto/High_Load  makes no mention of "maxQueued".
This seems odd, as "maxQueued" is a more concise method to accomplish this.

But even when I try the method in that doc — by defining the ArrayBlockingQueue directly.
I still see no mention of "maxQueued" in the JMX

So my questions.
  1. Is there something wrong with my XML?
  2. How can I check my work here and confirm that I am in fact using an ArrayBlockingQueue?
  3. Why doesn't the JMX MBean (org.eclipse.jetty.util.thread:type=queuedthreadpool,id=0)  expose "maxQueued"? 
Thanks very much,
-- Chris 

I am using Jetty 7.6.5.v20120716






_______________________________________________
jetty-users mailing list
jetty-users@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/jetty-users



Back to the top