[jetty-users] Avoid rapid threadpool growth to max when started under lo

[Travis Spencer <travis@xxxxxxxxx>]

Hi, Jetty list!

We are embedding Jetty in our own product, and are currently on
9.4.43.v20210629. By default, the minimum number of Jetty request
threads is set to 8 and the maximum is 500. For some requests, Nashorn
JavaScript procedures are executed.

We recently upgraded from Java 8 to 11 (but were using the G1GC even
on 8). In this new Java version, Nashorn's optimistic typing is
enabled by default. This seems to place a lock that all Jetty request
threads (that use Nashorn) get blocked on. Often, a new Jetty node is
spun up dynamically when the cluster is under heavy load. In such
situations, the node gets bombarded immediately. Because the request
threads get blocked directly as Nashorn performs this type
optimization, the number of request threads grows very quickly to its
max of 500. This causes so much context switching and overhead that
the node takes longer to arrive at a steady state. For some
deployments, this has been fixable by setting the maximum thread count
to 100. This avoided the thrashing on startup (for systems where 100
threads is bearable during upfront optimization of Nashorn types), but
may cause it to be under utilized later when this initial logjam is
resolved.

For this reason, we're wondering if there is a straightforward way to
add a maximum thread count during the initial couple minutes after
starting. Right now, we're not doing anything exhotic with the Jetty
threading or scheduling, but are aware that there's a lot possible.
For this reason, we wanted to ask if someone can point us in the
general direction (or if this kinda thing just isn't
doable/recommended).

--

TIA!

Travis Spencer