Re: [jetty-users] How to get Jetty to degrade gracefully under load

[Simone Bordet <sbordet@xxxxxxxxxxx>]

Hi,

On Fri, Jun 8, 2012 at 6:21 PM, ccleve <ccleve.tech@xxxxxxxxx> wrote:
> We're seeing an odd problem when load testing our site. Something's probably
> misconfigured, but I don't know where to look.
>
> As we ramp up the number of users with our load testing software, the number
> of requests Jetty can handle tops out at about 100/second. (Our app is doing
> some work to process each request, so the low rate isn't Jetty's fault). As
> more and more requests come in, average page load time creeps up to 6 or 7
> seconds, max load time becomes very high, and then all at once the system
> starts sending 503s. Any new requests get a 503, and the load testing
> software reports that it starts getting about 500 503s per second,
> presumably from earlier requests that were queued up. This continues for
> about a minute, and then the system starts accepting requests again.
>
> So when a lot of users come in, some get serviced, some have to wait a long
> time, and then whole system shuts down for a minute while it recovers.
>
> I'd prefer not to have a catastrophic failure like this. Perhaps the system
> should send 503s earlier so the queue never gets too big. Or maybe it should
> never send a 503, and just make the wait times uniformly longer for all
> users.
>
> We're on Amazon, so we can use monitoring software to detect when page load
> time is > X seconds and spin up more instances.
>
> What parameters can we twiddle to control how Jetty degrades? Or have we
> misconfigured things?

Are you sure that's Jetty sending the 503 and not the application or
some other network component (e.g. Apache) in front of Jetty ?
Are you using the QoSFilter ?

I ask because Jetty does not send, to my knowledge, 503s when the
system is overloaded.

Simon
-- 
www.webtide.com
Developer advice, services and support
from the Jetty & CometD experts.
----
Finally, no matter how good the architecture and design are,
to deliver bug-free software with optimal performance and reliability,
the implementation technique must be flawless.   Victoria Livschitz