Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [jetty-users] HttpClient handling of response that doesn't send full response according to Content-Length
Doing a little further testing with this I've realized that the web server does send back the correct number of bytes for both HTTP 1.0 and HTTP 1.1. I've been able to verify this using curl:

HTTP/1.1:
chris@chris-Vostro-3400:~/workspace/Blaze/lib/jetty$ curl -i -H "Accept-Encoding: gzip" https://www.fidelity.com | awk '/Content-Length/{print $0}'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0Content-Length: 20506
100 20506  100 20506    0     0  36016      0 --:--:-- --:--:-- --:--:-- 42810

HTTP/1.0:
chris@chris-Vostro-3400:~/workspace/Blaze/lib/jetty$ curl -i -0 -H "Accept-Encoding: gzip" https://www.fidelity.com | awk '/Content-Length/{print $0}'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0Content-Length: 20506
100 20506  100 20506    0     0  30360      0 --:--:-- --:--:-- --:--:-- 36683

So, there seems to be something funny going on with the Jetty HttpClient when it tries to do HTTP 1.0 communication with this web server.

- Chris

On 12-01-25 03:15 PM, Chris Dumoulin wrote:
It sends the Content-Length if you add "Accept-Encoding: gzip" to the request.

- Chris

On 12-01-25 03:02 PM, Joakim Erdfelt wrote:
Cannot replicate with https://www.fidelity.com/
It has no Content-Length header specified in the HTTP response.

$ curl --dump-header fid.header --output fid.html https://www.fidelity.com/
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 78341    0 78341    0     0   139k      0 --:--:-- --:--:-- --:--:--  193k
$ cat fid.header 
HTTP/1.1 200 OK
Server: FWS/7.0
Content-Type: text/html;charset=UTF-8
Cache-Control: cache, must-revalidate
Expires: Tues, 01 Jan 1980 00:00:00 GMT
Pragma: no-cache
Fsreqid: REQ4f205ed80a04283020001001001baa33
Fscalleeid: fidweb411
Fselapsedtime: 5169
Date: Wed, 25 Jan 2012 19:58:16 GMT
Transfer-Encoding:  chunked
Connection: keep-alive
Connection: Transfer-Encoding
Set-Cookie: HP_VER=C1; path=/; expires=Thu, 26-Jan-2012 19:58:16 GMT
Set-Cookie: v1st=C315B55A171791C5; path=/; expires=Wed, 19 Feb 2020 14:28:00 GMT; domain=.fidelity.com
Set-Cookie: HttpOnly
Set-Cookie: JSESSIONID=B52AFB243713F2FCF39CF77DDE45838F; path=/pf/destination
Set-Cookie: MC=g^Og7t2FWVVtYQGUh1runkqsIDYSAk8gXtgKBCgwIAAQAQAdqjMGBAAAAQAGBU8gXtgAP03; path=/; domain=.fidelity.com; expires=Thu, 24-Jan-2013 19:58:16 GMT
Set-Cookie: v1st=C315B55A171791C5; path=/; domain=.fidelity.com; expires=Wed, 19 Feb 2020 14:28:00 GMT
Set-Cookie: v1st=C315B55A171791C5; path=/; domain=.fidelity.com; expires=Wed, 19 Feb 2020 14:28:00 GMT

Even with HTTP/1.0

$ curl --dump-header fid.header --output fid.html --http1.0 https://www.fidelity.com/
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 78334    0 78334    0     0   118k      0 --:--:-- --:--:-- --:--:--  182k
$ cat fid.header 
HTTP/1.0 200 OK
Server: FWS/7.0
Content-Type: text/html;charset=UTF-8
Cache-Control: cache, must-revalidate
Expires: Tues, 01 Jan 1980 00:00:00 GMT
Pragma: no-cache
Fsreqid: REQ4f205f110a022a3320001b2a0019aa33
Fscalleeid: fidweb221
Fselapsedtime: 7388
Date: Wed, 25 Jan 2012 19:59:13 GMT
Connection: close
Set-Cookie: HP_VER=C1; path=/; expires=Thu, 26-Jan-2012 19:59:13 GMT
Set-Cookie: v1st=87929215BDF1851; path=/; expires=Wed, 19 Feb 2020 14:28:00 GMT; domain=.fidelity.com
Set-Cookie: HttpOnly
Set-Cookie: JSESSIONID=8F7DDEFFE75CA872A9881AFDCA018C2F; path=/pf/destination
Set-Cookie: MC=8xxvSOBpbhzzUtG8x3VeBR7H^sQSAk8gXxEKAiozIAAbKgAbqjMGBAAAAQAGBU8gXxEAP03; path=/; domain=.fidelity.com; expires=Thu, 24-Jan-2013 19:59:13 GMT
Set-Cookie: v1st=87929215BDF1851; path=/; domain=.fidelity.com; expires=Wed, 19 Feb 2020 14:28:00 GMT
Set-Cookie: v1st=87929215BDF1851; path=/; domain=.fidelity.com; expires=Wed, 19 Feb 2020 14:28:00 GMT


--
Joakim Erdfelt

(the people behind jetty and cometd)



On Wed, Jan 25, 2012 at 12:47 PM, Chris Dumoulin <chris@xxxxxxxx> wrote:
I've noticed a difference in behaviour from Jetty HttpClient version 7.2.0 to 7.5.4 when a web server indicates a Content-Length, sends back some data (but not the full amount) and then closes the connection. With version 7.2.0 onException() wouldn't be called and onResponseComplete() would, but in 7.5.4 onException() gets called but not onResponseComplete(). Note that this only seems to happen when using HTTP 1.0 with the HttpClient.

Is this change in behaviour expected? Which is the correct behaviour? I see this in the the HTTP spec ( http://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html):

" When a Content-Length is given in a message where a message-body is allowed, its field value MUST exactly match the number of OCTETs in the message-body. HTTP/1.1 user agents MUST notify the user when an invalid length is received and detected."

However, browsers seem to handle the situation without an error. https://www.fidelity.com is an example of a web page that sets the Content-Length but doesn't send the full amount.

I've attached a test program that I was using to reproduce this.

Thanks,
Chris

_______________________________________________
jetty-users mailing list
jetty-users@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/jetty-users




_______________________________________________
jetty-users mailing list
jetty-users@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/jetty-users

Back to the top