Re: [jetty-dev] Jetty as High Performant Http Engine - Handle 2Million R

[Greg Wilkins <gregw@xxxxxxxxxxx>]

Firstly 2M requests/second is a pretty high mark to try to get to.  In order to optimise jetty for such extreme loads, you really need to consider the exact hardware, the average request size, the average response size etc..  Tell us more and we may be able to suggest some changes to configuration.     

Note also make sure that your load generation for your benchmark is realistic.  There is a big difference in how a server behaves with a few very busy connections vs many mostly idle connections.  Generally Jetty is configured more for the later case, so testing it with just 20 connections is not the sweet spot.   If your real application will have more connections, then please test with more connections.

Also, when benchmarking, if your client is reporting achieved throughput, then you are not actually measuring achieved throughput.  Rather you are measuring response latency because the client will not send the next request until a prior response is received.  Thus the reported request/sec is a result of the client not sending more requests.

It is far better to have load generator that is capable of sending a specific request rate and then reporting the latency/errors achieved and deciding if that is sufficient quality of service for your application.

Also, this is not jetty specific, but I don't think the way you are converting the input into a string is the best way to do so.  You will create lots of garbage reading line by line.   You'd also be best to use the content length of the request to set the initial capacity of the StringBuffer.

On 22 June 2017 at 21:27, SenthilKumar K <senthilec566@xxxxxxxxx> wrote:

Hello Jetty Experts ,  

     I have been working on the use case where i should create simple http server to serve ~1.5 - 2  Million Requests per Second per Instance ..

Recently i started doing benchmarking different servers and here is the result of Jetty.

Running 1m test @ http://127.0.0.1:8080/

  20 threads and 40 connections

  Thread Stats   Avg      Stdev     Max   +/- Stdev

    Latency     0.95ms  669.89us  17.00ms   96.71%

    Req/Sec     2.20k   170.21     2.95k    73.26%

  Latency Distribution

     50%  849.00us

     75%    0.97ms

     90%    1.14ms

     99%    4.23ms

  2622609 requests in 1.00m, 382.67MB read

Requests/sec:  43637.79

Transfer/sec:      6.37MB

At max i can see ~50K reqs/sec which is less for my use case .. 

Here is the Jetty Server and Handler Code:

Server server = new Server();

        String port = cmd.getOptionValue("p");

        HttpConfiguration httpConfiguration = new HttpConfiguration();

        httpConfiguration.setSendServerVersion(false);

        httpConfiguration.setSendXPoweredBy(false);

        

        ServerConnector httpConnector = new ServerConnector(server, new HttpConnectionFactory(httpConfiguration));

        httpConnector.setPort(Integer.parseInt(port));

        httpConnector.setName("http");

        httpConnector.setIdleTimeout(30000);

        httpConnector.setSoLingerTime(20);

        server.addConnector(httpConnector);

        

        ContextHandler context = new ContextHandler();

        context.setContextPath( "/helloworld" );

        

        JettyHandler requestHandler = new JettyHandler();

        HandlerCollection handlers = new HandlerCollection();

        handlers.addHandler(context);

        handlers.addHandler(requestHandler);

        server.setHandler(handlers);

        server.start();

        server.join();

  public class JettyHandler extends AbstractHandler

  {

  public void handle( final String target, final  Request baseRequest,

      final  HttpServletRequest request, final HttpServletResponse response

      ) 

  throws IOException, ServletException

  {

  response.setContentType("text/html;charset=utf-8");

  response.setStatus(HttpServletResponse.SC_OK);

  baseRequest.setHandled(true);

  final StringBuffer reqBuff = new StringBuffer();

  String line = null;

  BufferedReader reader = null;

  try {

    reader = request.getReader();

    while ((line = reader.readLine()) != null)

      reqBuff.append(line);

  } catch (Exception e) { }

   finally{reader.close();}

  final PostToKafka post2Kafka = new PostToKafka();

  try {

  post2Kafka.write2Kafka(reqBuff.toString());  --> This is really High Performant :)..

  } catch (Exception e) {

  e.printStackTrace();

  }

  response.getWriter().println("SUCCESS");

  }

  }

How to tune Jetty Performance ?

--Senthil

_______________________________________________
jetty-dev mailing list
jetty-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/jetty-dev

--

Greg Wilkins <gregw@xxxxxxxxxxx> CTO http://webtide.com