Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [jetty-dev] Solr unable to upgrade to Jetty 9.3 from Jetty 9.2.

Mark,

This is a change in behaviour in jetty 9.3, but not one that should affect any application.

The problem with consumeAll previously is that it was an unlimited commitment - a client could send chunks forever and consume a thread forever.

The only reason we need to consume all is to maintain a persistent connection.  If we don't consumeAll, we can close that connection and the client needs to start a new one.

So the approach that 9.3 takes is that if the application has not read all it's content (which is normally an error condition), then jetty will make a best effort attempt to read the content, but if it has to block - it gives up and the connection should be closed.

So I don't see how this can break an application - unless the client is trying to do pipelining and not correctly retrying queued requests if the connection closes  OR perhaps jetty is not correctly closing the connection (this I will check shortly).

Can you tell us more about how this is causing a problem for SOLR?     Why is the content not consumed?  Why does it cause a problem if Jetty does not consume it?


cheers










On 8 January 2016 at 03:48, Mark Miller <markrmiller@xxxxxxxxx> wrote:


On Thu, Jan 7, 2016 at 10:52 AM Joakim Erdfelt <joakim@xxxxxxxxxxx> wrote:
This would be news.


Yes, and SOLR-8453.
 
Have you verified that the client sends the correct data via wireshark (or tcpdump) yet?

Yes, it looks correct to me. It's also the same code that has worked against many other Jetty versions.

I'm pretty confident the behavior has changed, and if the whole request stream has not been consumed, if the client pauses too long before finishing the request, the server will reset the connection. Previously it would wait and consume for the client until it was done, or unrelated timeouts were hit. That would leave the connection around for reuse.

- Mark
 

We test regularly with various http clients, under load testing and in real world scenarios (browsers, command line, other http client libs, our http client libs, rest libs, etc...).
While we have an occasional report of a quirk or bug, we investigate those.
Of the ones reported against Jetty 9.3, we have identified a few bugs related to improper handling of multipart data parsing on our side, and fixed those.
Others have either been bugs in the http client usage, or bad http client assumptions / expectations (such as thinking that once a response starts, the request is complete).

AFAIK, there's only 1 outstanding bug, and its a report of strange Request object handling with Handlers under load when the server is on Windows (which we are currently investigating).
 

Joakim Erdfelt / joakim@xxxxxxxxxxx

On Thu, Jan 7, 2016 at 8:12 AM, Mark Miller <markrmiller@xxxxxxxxx> wrote:
After upgrading Solr to Jetty 9.3, we noticed a lot of Solr tests started to randomly fail, and a couple would consistently fail.

After some investigation, I tracked down two issues that I think are involved.

One issue seems to be based on Locale. I think there is a default Locale bug or something, but I still have to dig into that one. More to come.


Specifically, it seems to be an HttpInput#consumeAll change.

The code went from:

T item = getNextContent();
if (item == null)
 _contentState.waitForContent(this);

To:

Content item = nextContent();
if (item == null)
break; // Let's not bother blocking

Is that legal? And even if it is, isn't it at least a pretty big back compat break?

We can hit this case quite easily - and in 9.2 Jetty seems to properly wait on the server until the client is done sending and consumes all it should. In 9.3, if there is even a short pause in the wrong place on the client side, the server resets the connection on the client as it consumeAll bails early now. That messes with the client (broken pipe, connection reset) and prevents connection reuse.

Comments?

- Mark
--

_______________________________________________
jetty-dev mailing list
jetty-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/jetty-dev

_______________________________________________
jetty-dev mailing list
jetty-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/jetty-dev
--

_______________________________________________
jetty-dev mailing list
jetty-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/jetty-dev



--

Back to the top