firstly thanks for starting this thread. We've put a lot of changes into the IO layer for 9.0.x and even more for 9.1.x, but have not really had a chance yet to dive in with a profiler to see where the fat is (At least not since the 9.0.M0 stage when we solved some mechanical sympathy issues and ensured that we were faster than jetty 8 ). So it is very timely indeed, as we intend to spend the next few weeks looking for fat to trim.
However, I would suggest at this stage, effort is probably best spent on 9.1.x. I restructured 9.0.x IO with the servlet 3.1 asynchronous IO in mind (hence classes like IteratingCallback), but the refactor was only completed in 9.1 and we have more flexibility to change code there without accidentally breaking the servlet 3.1 APIs.
Can you briefly describe the kind of load you are sending the server? lots of pipelined requests? lots of connections? long held connections?
