| Re: [jetty-users] SSLEngineImpl initHandshaker Internal error |
Thanks so much Greg and Simone! We use the PROXY protocol so the real source IP is actually in the trace (you may remember me as the crazy guy who wanted to write an autodetecting / passthrough PROXY connector!) I will definitely try with a newer Jetty. Since we only see it in production, and we are under a code freeze for the rest of the year, this will have to wait til January. Also another user reported seeing it on 9.3.15 already It seems to come from 'dsl-xx-xx-xx-xx-dyn.prod-infinitum.com.mx.' I'm glad it's not from Russia, we at least have a business presence in Mexico :) Spot check of 2 or 3 more shows that they all seem to come from dynamic IPs, Infinitum MX, Comcast, Verizon. And they all had actual successful requests that look like legit users. So probably not a botnet / attack (phew!) Additionally, they are mostly using swcd (unknown version) CFNetwork/808.2.16 Darwin/16.3.0 (Probably our iOS app) Since that's one of our biggest traffic drivers I'm not 100% sure that's a smoking gun -- I also saw e.g. Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; rv:11.0) like Gecko I took the connection IDs and ran a grep through our logs. I've attached that. egrep '@(1c7c0306|2917e8db|5a23fff6)' logfile | xz Hopefully this is a good way to get relevant information, let me know if you have a better approach. Thanks again guys for validating our use of Jetty with top-notch mailing list support :)
Attachment:
1c7c0306.txt.xz
Description: Binary data
> On Dec 22, 2016, at 2:30 AM, Simone Bordet <sbordet@xxxxxxxxxxx> wrote:
>
> Hi,
>
> On Thu, Dec 22, 2016 at 1:53 AM, Steven Schlansker
> <stevenschlansker@xxxxxxxxx> wrote:
>> These are the tail of Jetty logs for the thread that died:
>>
>>
>> 2016-12-21T23:44:12.816Z DEBUG <> [qtp294008652-58364] o.e.j.util.thread.QueuedThreadPool - queue EPC Prod/Pend/org.eclipse.jetty.io.ManagedSelector$SelectorProducer@6fad21e
>> 2016-12-21T23:44:12.818Z DEBUG <> [qtp294008652-58364] o.e.j.u.t.s.ExecuteProduceConsume - EPC Prod/Pend/org.eclipse.jetty.io.ManagedSelector$SelectorProducer@6fad21e run SelectChannelEndPoint@5a23fff6{/10.30.64.212:16725<->4443,Open,in,OSHUT,FI,-,1838/30000,SslConnection@5b279677}{io=1/0,kio=1,kro=1}:runFillable
>> 2016-12-21T23:44:12.820Z DEBUG <> [qtp294008652-58364] org.eclipse.jetty.io.FillInterest - FillInterest@61f9356f{true,AC.ReadCB@5b279677{SslConnection@5b279677{NEED_UNWRAP,eio=-1/-1,di=-1} -> HttpConnection@1c7c0306[DecryptedEndPoint@2917e8db{/201.114.209.201:61299<->443,Open,in,OSHUT,FI,-,1856
>
> Both the SelectChannelEndPoint@5a23fff6 and DecryptedEndPoint@2917e8db
> say they are OSHUT already.
> Can you track back in your logs where they were OSHUT (i.e.
> shutdownOutput() was called on them) ?
>
> What could have happened is that Jetty decided that the TLS connection
> was not good, sent back a TLS Close message, and then OSHUT it, but
> the client insisted in continuing to talk, and SSLEngine got confused
> by this, eventually throwing the "IllegalStateException: Internal
> error".
>
> --
> Simone Bordet
> ----
> http://cometd.org
> http://webtide.com
> Developer advice, training, services and support
> from the Jetty & CometD experts.
> _______________________________________________
> jetty-users mailing list
> jetty-users@xxxxxxxxxxx
> To change your delivery options, retrieve your password, or unsubscribe from this list, visit
> https://dev.eclipse.org/mailman/listinfo/jetty-users
Attachment:
signature.asc
Description: Message signed with OpenPGP using GPGMail