Bug 259977 - RAP applications occasionally render character garbage to the browser
Summary: RAP applications occasionally render character garbage to the browser
Status: RESOLVED FIXED
Alias: None
Product: RAP
Classification: RT
Component: RWT (show other bugs)
Version: 1.0   Edit
Hardware: All All
: P1 critical with 1 vote (vote)
Target Milestone: 1.3 M2   Edit
Assignee: Project Inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords: needinfo
Depends on:
Blocks:
 
Reported: 2009-01-05 12:31 EST by Mark Freiheit CLA
Modified: 2009-09-07 17:09 EDT (History)
6 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Mark Freiheit CLA 2009-01-05 12:31:22 EST
We have discovered a troubling error when running our RAP application when hosted on Tomcat.

It is not trivial to reproduce, but happens eventually with moderate use of our RAP application.  When our testers change perspectives a few times, we see errors in the browser (pasted below).

This error occurs on both Firefox and IE, but seems to be easier to replicate on Firefox.

Our testers intercepted the packets between the browser and Tomcat using Ethereal, and it does not appear to be a gzip error nor an error with the mismatch in request/response mime types.

This error cannot be re-created using the Jetty web server... Only on Tomcat.
Comment 1 Ivan Furnadjiev CLA 2009-02-05 03:43:51 EST
Can you provide a self running snippet that can be used to reproduce the problem?
BTW what is the error? It is missing in the description.
Comment 2 Ivan Furnadjiev CLA 2009-02-06 04:32:51 EST
Did you set an encoding explicitly in the build or leave it to default?
Comment 3 Marcus Lehmann CLA 2009-05-08 04:40:45 EDT
We have the same problem. I think it is the same error like this one:

http://dev.eclipse.org/newslists/news.eclipse.technology.rap/msg04195.html
http://dev.eclipse.org/newslists/news.eclipse.technology.rap/msg04242.html
Comment 4 Marcus Lehmann CLA 2009-05-08 05:34:00 EDT
Shown in Browserwindow:
Could not evaluate javascript response:

SyntaxError: illegal character

&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;&#65533;]o&#65533;0&#65533;&#65533;&#65533;&#65533;&#65533;fR&#65533;&#65533;&#65533;i&#65533;&#65533;&#65533;lI&#65533;%&#65533;&#65533;V&#65533;7E%&#65533;bh&#65533;?CHRE&#65533;a$&#65533;"&#65533;&#65533;&#1551;&#65533;c&#65533;+&#535;&#65533;&#65533;&#65533;m&#65533;W&N&#65533;P&#65533;b&#65533;C/&#65533;q&#65533;^&#65533;&#59887;&#65533;nb&#65533;&#65533;&#65533;&#65533;$t&#65533;_R.&#65533;&#65533;&#65533;(&#65533;.&#65533;&#65533;K-<)~&#65533;&#65533;0&#65533;qt&#65533;&#65533;&#65533;&#1462;&#65533;&#65533;&#65533;|O&#65533;3qCw&#65533;&#65533;&&#65533; &#65533;]&#65533;G?&#65533;v&#65533;&#65533;&#65533;&#531;&#65533;n	&#65533;Y&#65533;[&#65533;&#65533;b&#65533;z&#65533;&#65533;'&#65533;&#65533;&#65533;T&#65533;&#65533;z]Q&#65533;&#65533;*&#41570;^q~&#65533;&#65533;&#65533;b&#65533;&#65533;&#330;&#65533;&#65533;&#65533;&#65533;*&#65533;&#65533;|&#65533;&#65533;y&#65533;&\&#65533;^>&#65533;&#65533;3&#65533;)T&#65533;QAK&#65533;&#65533;/&#65533;&#65533;&#65533;{<&#65533;&#65533;&#65533;&#65533;&&#65533;&#65533;'qz&#65533;&#65533;.4c&#65533;&#65533;&#36942;&#65533;&#65533;o&#65533;0
&#65533;&#65533;8M&#65533;m!&#65533;&#65533;(&#65533;&#65533;)&#65533;&#65533;H&#65533;&#65533;@&#65533;&#65533;&#65533;&#65533;Dt&#65533;=&#65533;&#65533;~&#65533;&#65533;&#65533;<&#65533;&#1929; &#65533;O&#65533;&#65533;P&#65533;&#65533;:&#65533;5B&#65533;&#65533;F&#65533;!y&#65533;O&#65533;&#65533;&#65533;vACd&#65533;}\&#65533;&#65533;c&#65533;&#65533;&#65533;'&#65533;&#65533;&#65533;&#65533;&#65533;`&#65533;H6"#&#65533;C&#1943;t&#65533;e3r&#65533;&#65533;&#65533;
\&#65533;&#65533;Y&#65533;&#65533;&#65533;KA&#65533;BdV&#65533;&#65533;;&#65533;f4,&#65533;
R&#65533;&#65533;&#65533;&#65533;
Z&#65533;&#65533;!2&#65533;i&#65533;r&#65533;
i&#65533;h&#1082;4tE1 &#65533;&#65533;&#1112;6&#65533;AZ&#65533;&#65533;&#65533;&#1080;&#65533;~&#65533;&#65533;Y`B&#65533;dAI1r&#65533;&#65533;&#65533;(b0&#65533;b~&#1887;&#65533;&#65533;&#65533;&#65533;>&#65533;&#65533;0t&#65533;&#65533;&#65533;Y]#0#h&#65533;%&#65533;Ua&#65533;54&#65533;&#65533;
&#65533;.
M&#65533;&#65533;u&#65533;&#65533;&#65533;Y2&#65533;g
i&#65533;&#65533;huiP&#65533;
dOCf=&#65533;&#65533;&#1990;4&#65533;h&#65533;&#65533;	R&#65533; 5LK&#65533;&#65533;&#65533;Y&#65533;&#65533;&#65533;$$r &#65533;&#65533;&#1629;=&#65533;;&#65533;!pf&#65533;3h&#65533;^-Q&#65533;&#65533;z&#65533;4,&#65533;H&#65533;'~&#1667;&#65533;&#65533;h&#65533;&#65533;&#309;+N&#65533;&#65533;"&#65533;&#65533;S&#65533;&#65533;}&#65533;&#65533;tm&#65533;&#65533;,+9K6&#65533;#&#65533;p&#65533;&#65533;&#65533;CGkYL&#65533;&#65533;@&#1384;&#65533;&#65533;g&#65533;1&#65533;A&#65533;&#35373;b&#1517;6q&#65533;&FM_&#65533;&#65533;&#65533;&#65533;Q&#65533;&#65533;\:
u_&#35373;j&#65533;-71V&#65533;&#65533;o&#65533;&#65533;qL&#65533;n&#65533;h&#65533;&#65533;&#65533;e&#65533;f)T&#65533;&#65533;C&#65533;Q5&#65533;&#65533;&#65533;7gq&#65533;&#65533;&#65533;&#1287;&#65533;(&#65533;&#65533;76&#65533;&#65533;&#65533;iD7&#65533;&#65533;&#65533;:&#65533;p&#65533;,s&#65533;&#65533;&#65533;&#65533;W&#65533;&#65533;&#65533;SG&#65533;&#65533;


Firebug Console:

Response Headers
Server:Apache-Coyote/1.1
Transfer-Encoding: chunked
Date: Fri, 08 May 2009 09:02:27 GMT

Request Headers
Host: 127.0.0.1:8080
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.0.10) Gecko/2009042316 Firefox/3.0.10

Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-gb,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
X-Qooxdoo-Response-Type: text/plain
Pragma: no-cache
Cache-Control: no-cache
X-Requested-With: qooxdoo
X-Qooxdoo-Version: 0.7.4 (r16878)
Referer: http://127.0.0.1:8080/demo/start
Content-Length: 201
Cookie: JSESSIONID=BC2532F69BDB8826E2F247DDB4E6B04C
Comment 5 Ivan Furnadjiev CLA 2009-05-08 07:57:17 EDT
Markus, can you provide a self running snippet/project to reproduce the problem?
Comment 6 Marcus Lehmann CLA 2009-05-08 08:31:57 EDT
I tried, but I'm not able to reproduce the error. I'm not able to say do this and that and then the error occurs. So it is difficult to post a snippet.

Maybe the browser expect characters (Javascipt/Html/XML?) but he get some binary data (image?).
Comment 7 Ralf Sternberg CLA 2009-07-21 06:12:27 EDT
This bug has been observed by many people now, still it is hard to reproduce. What we have so far:

* We received a couple of such character cheese dumps as in comment #4 from users, most of them (but not all) contain the unicode replacement character (&#65533;, U+FFFD, ?) over and over. This character is shown when an application cannot render a certain code point.

* The problem affects the initial HTML page, which arrives mangled at the client and is then cached.

Something might go wrong during assembling, zipping, or encoding the initial page. To be sure, a wireshark log of a failed session may be helpful. To get such a log, access the application with a clean browser cache and receive the character garbage instead of the initial page. Since we couldn't reproduce the problem by ourselves so far, we still don't know what exactly is sent from server to client in this case.
Comment 8 Ralf Sternberg CLA 2009-07-30 16:41:04 EDT
After hours of tcp dump analysis, we know a bit more about the problem:
* The client receives a response that is gzip-compressed, but the header "Content-Encoding: gzip" is missing.
* Since the "Content-Type: html" header is set, the client tries to render the gzip stream as html which leads to the garbage.
* It seems that this always happens in conjunction with chunked transfer encoding.

The gzip compression is done by RAP, not Tomcat.

I'm not sure under which circumstances Tomcat chooses chunked transfer encoding, but the reason might be a delayed output from the servlet. It is not clear whether Tomcat drops the Content-Encoding header when using chunked transfer encoding or if RAP fails to set the header in some weird cases which in turn also cause Tomcat to use chunked mode.

As a workaround, it should help to turn off the gzip compression by RAP by adding the vm parameter -Dcompression=false. It should be possible to enable compression in Tomcat instead.
Comment 9 Stefan Röck CLA 2009-07-31 03:16:18 EDT
Wouldn't it make sense to turn off RAP-side compression be default? I would rather see this as a task for the web container task than for RAP.
Comment 10 Markus Knauer CLA 2009-07-31 09:19:31 EDT
First tests with

[1] JVM parameter
    -Dcompression=false

and 

[2] Tomcat's server.xml

    <Connector port="8080"
        protocol="HTTP/1.1"
        proxyPort="80"
        compression="on"

are looking really good. They result in a HTML header such as

  HTTP/1.1 200 OK
  Server: Apache-Coyote/1.1
  Cache-Control: max-age=0, no-cache, must-revalidate
  Transfer-Encoding: chunked
  Content-Type: text/html
  Content-Encoding: gzip

and I am quite confident that the browsers are cabable of decoding this combination of content-type and content-encoding. Without the two changes above the "Content-Encoding" was missing in some cases.
Comment 11 Ralf Sternberg CLA 2009-08-05 04:53:37 EDT
(In reply to comment #9)
> Wouldn't it make sense to turn off RAP-side compression be default? I would
> rather see this as a task for the web container task than for RAP.

I agree, opened bug 285669 for this.
Comment 12 Rüdiger Herrmann CLA 2009-09-07 17:09:12 EDT
As bug 285669 is resolved now, I consider this bug fixed as well.