Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [jetty-dev] Merging small fix for HTTP 1.1 parsing error?

BTW, the docs you linked are for Jetty 6.  (not Jetty 7/8/9)

Simply moving it from src/main/java to src/test/java only means that the class would move from being a runtime class to being a testing class.

They would still be available on maven central, but through the <classifier>tests</classifier> (which exists currently)

Example maven artifact reference:
http://git.eclipse.org/c/jetty/org.eclipse.jetty.project.git/tree/tests/test-integration/pom.xml?h=jetty-9.2.x#n120

    <dependency>
      <groupId>org.eclipse.jetty</groupId>
      <artifactId>jetty-http</artifactId>
      <version>${jetty.version}</version>
      <classifier>tests</classifier>
      <scope>test</scope>
    </dependency>

Its not meant to be a runtime class for production capacity and use.
Only for testing.  These don't even use the connection layer framework (which with Jetty 9.3 and http/2 will become even more important)

Now, onto the core of the problem.
The original bug is filed under the assumption that HTTP/1.1 parsing is bad/broken.

The bug is in the assumption that you use a protocol representation with HttpTester to test with.

When SPDY was introduced, it was quickly determined that the HTTP world does not revolve around HTTP the protocol, but rather HTTP the semantic.  With SPDY and now HTTP/2 you have headers, and request parameters, and uris, but not the HTTP protocol.  The HttpTester was changed to behave in a protocol neutral, but HTTP semantic way.
You create a Request object, give it to the tester and let the jetty internals process it / dispatch it / get the response state / headers / content, and return it to you, as an Response object.

Opened up 2 bugs around this:

https://bugs.eclipse.org/446952 - fixing the documentation and implementation for HttpTester and ServletTester
https://bugs.eclipse.org/446944 - the tests classifier artifact location for the HttpTester and ServletTester

If you have protocol specific things to test, using HttpTester and ServletTester are not the correct way to do that.  Especially if you are wanting to test content encoding or behavior (such as compressed, gzip, deflate, or chunked encoding)

Finally, here's a real test of the actual in-use HTTP/1.1 protocol, showing that there is no HTTP/1.1 protocol parsing bug.

package jetty.example;

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.OutputStream;
import java.io.Reader;
import java.net.InetAddress;
import java.net.InetSocketAddress;
import java.net.Socket;
import java.net.SocketAddress;
import java.nio.charset.StandardCharsets;

import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import org.eclipse.jetty.server.Server;
import org.eclipse.jetty.servlet.ServletContextHandler;
import org.junit.AfterClass;
import org.junit.BeforeClass;
import org.junit.Test;

public class Bug440823Test
{
    @SuppressWarnings("serial")
    public static class HelloServlet extends HttpServlet
    {
        @Override
        protected void doGet(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException
        {
            resp.setContentType("text/plain");
            resp.getWriter().println("Hello Test");
        }
    }
    
    private static Server server;
    
    @BeforeClass
    public static void startServer() throws Exception
    {
        server = new Server(8080);
        ServletContextHandler contexts = new ServletContextHandler(ServletContextHandler.NO_SESSIONS);
        contexts.addServlet(HelloServlet.class,"/*");
        server.setHandler(contexts);

        server.start();
    }
    
    @AfterClass
    public static void stopServer() throws Exception
    {
        server.stop();
    }
    
    @Test
    public void testRequest() throws Throwable
    {
        String request =
                "GET / HTTP/1.1\r\n" + 
                "Host: localhost\r\n" + 
                "Header1: value1\r\n" + 
                "Connection: close\r\n" + 
                "Accept-Encoding: gzip, deflated\r\n" + 
                "Accept: unknown\r\n" + 
                "\r\n";
        
        InetAddress destAddr = InetAddress.getByName("localhost");
        int port = 8080;

        SocketAddress endpoint = new InetSocketAddress(destAddr,port);

        try (Socket socket = new Socket())
        {
            socket.connect(endpoint);

            try (OutputStream out = socket.getOutputStream();
                    InputStream in = socket.getInputStream();
                    Reader reader = new InputStreamReader(in);
                    BufferedReader buf = new BufferedReader(reader))
            {
                // Send GET Request
                System.err.print(request);
                System.err.printf("Request: %,d bytes%n",request.length());
                out.write(request.getBytes(StandardCharsets.ISO_8859_1));

                // Get Response
                String line = null;
                while ((line = buf.readLine()) != null)
                {
                    System.err.printf("[response]: %s%n",line);
                }
                System.err.println("[done]");
            }
        }

    }
}




--
Joakim Erdfelt <joakim@xxxxxxxxxxx>
Expert advice, services and support from from the Jetty & CometD experts

On Mon, Oct 13, 2014 at 9:50 AM, Chris Heisterkamp <cheister@xxxxxxxxxxxx> wrote:
So if I understand you correctly HttpTester.parseRequest is being removed from the jetty-http API?  I assume other projects use it for testing as well since it used to be the recommended way to test http parsing http://docs.codehaus.org/display/JETTY/ServletTester

Now if we want to use the HttpParser for testing then we need to setup our own RequestHandler and build an HttpParser and use that?  Basically re-implement HttpTest.parseRequest but use a RequestHandler that works?  



On Mon, Oct 13, 2014 at 9:14 AM, Joakim Erdfelt <joakim@xxxxxxxxxxx> wrote:
For starters, there is nothing in Jetty, or Jetty testing, that actually uses the HttpTester.parseRequest() method.
The HttpTester request side is setup manually (not in raw form then parsed), and is focused on request processing and response validation.
In fact, that entire class is even in the wrong place, it shouldn't be in src/main/java. (We just moved it to src/test/java a few minutes ago)
The HttpTester.Request object isn't even setup as a proper HttpParser.RequestHandler (never was)
The HttpTester.Request object extends from Message, which is really just designed for request generation (not parsing).


--
Joakim Erdfelt <joakim@xxxxxxxxxxx>
Expert advice, services and support from from the Jetty & CometD experts

On Mon, Oct 13, 2014 at 12:30 AM, Chris Heisterkamp <cheister@xxxxxxxxxxxx> wrote:
Joakim, thank you for taking a look at the bug. 

Can you elaborate on how it was invalid?

The bug happens when you use HttpTester.parseRequest to parse the response.  The test you have added uses 

HttpParser.RequestHandler handler  = new Handler();
HttpParser parser= new HttpParser(r);
parser.parseNext(buffer);

which uses the HttpParserTest.Handler as the RequestHandler passed to the HttpParser.   This test case works, with \n or \r\n ending the lines

HttpTester.parseRequest method uses

Request r=new Request();
HttpParser parser =new HttpParser(r);
parser.parseNext(request);

which fails because HttpTester.Request is used as the RequestHandler. 

On Sat, Oct 11, 2014 at 10:56 PM, Joakim Erdfelt <joakim@xxxxxxxxxxx> wrote:
Test case was invalid, it wasn't testing what you thought it was.

Committed fixed testcase, no RuntimeError present with correct tests.
This is on branch jetty-9.2.x (aka jetty 9.2.4-SNAPSHOT)

Closing bug as invalid.

--
Joakim Erdfelt <joakim@xxxxxxxxxxx>
Expert advice, services and support from from the Jetty & CometD experts

On Sat, Oct 11, 2014 at 8:35 PM, Chris Heisterkamp <cheister@xxxxxxxxxxxx> wrote:
I'm wondering if someone can tell me how I could get a patch to the HttpTester merged?   There is a small bug in the HttpTester when it parses an HTTP/1.1 message that has a straightforward test case and small fix.   Bug 440823 https://bugs.eclipse.org/bugs/show_bug.cgi?id=440823

It has been sitting without activity for two months and is keeping use from updating past Jetty 9.1.4.  Does it need to be updated with more information or is this just the typical response time for patches?

Thanks,
    Chris

_______________________________________________
jetty-dev mailing list
jetty-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/jetty-dev


_______________________________________________
jetty-dev mailing list
jetty-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/jetty-dev


_______________________________________________
jetty-dev mailing list
jetty-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/jetty-dev


_______________________________________________
jetty-dev mailing list
jetty-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/jetty-dev


_______________________________________________
jetty-dev mailing list
jetty-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/jetty-dev


Back to the top