Re: [jetty-dev] NIO inefficiency?

[Viktor Szathmary <phraktle@xxxxxxxxx>]

Hi Greg,

On Monday, August 12, 2013 at 1:33 AM, Greg Wilkins wrote:

firstly thanks for starting this thread.   We've put a lot of changes into the IO layer for 9.0.x and even more for 9.1.x, but have not really had a chance yet to dive in with a profiler to see where the fat is (At least not since the 9.0.M0 stage when we solved some mechanical sympathy issues and ensured that we were faster than jetty 8 ).    So it is very timely indeed, as we intend to spend the next few weeks looking for fat to trim.

Indeed, with 9.0.x we have seen a 10-25% speedup in production compared to 8.x (mostly reduced latency, CPU load a bit noisy to see clearly) – I believe this is due to the improved request parsing code. So things are certainly going in the right direction :)

However, I would suggest at this stage, effort is probably best spent on 9.1.x.   I restructured 9.0.x IO with the servlet 3.1 asynchronous IO in mind (hence classes like IteratingCallback), but the refactor was only completed in 9.1 and we have more flexibility to change code there without accidentally breaking the servlet 3.1 APIs.

I guess we will take another look, running with 9.1.x once a release comes out. 

 

Can you briefly describe the kind of load you are sending the server?  lots of pipelined requests? lots of connections? long held connections?

The workload is lots of small GET requests from browser all over the internet (a bunch of params and cookies to parse), the response is a few kb of JSON dynamically generated and gzipped. The servers (EC2 m1.large instances, sitting behind an Elastic Load Balancer) get around 5-600 QPS each. We spend about half a millisecond on average in our application code to generate the response. The rest of our latency is in the stack (jetty/linux/virtualization/network) – a total of around 1.5ms per request as measured by the ELB.

Regards,

  Viktor