[jetty-users] Making SSL renegotiation work in Jetty 9

[Juan Calero <j.calero@xxxxxxxxxxxxx>]

FInally I had some time to work again on the issue, and I've managed to 
make it work (kind of)

I've created a custom version of ClientCertAuthenticator which triggers 
a SSL Handshake if the request doesn't have a X509Certificate chain.

I added this in validateRequest:


            if (certs == null) {

                LOG.info("Trying SSL client cert renegotiation");

                for (EndPoint endpoint : 
HttpChannel.getCurrentHttpChannel().getConnector().getConnectedEndPoints())
                {

                    if (endpoint.getConnection() instanceof 
SslConnection) {
                        SslConnection sslConnection = 
(SslConnection)endpoint.getConnection();
                        SSLEngine engine = sslConnection.getSSLEngine();
                        engine.setWantClientAuth(true);

                        //if (doEndpointHandshake(engine, 
sslConnection.getEndPoint())) {    // MODO 1
                        if (doManualHandshake(engine, 
sslConnection.getEndPoint())) {    // MODO 2
                            // Correct Handshake, but client can send 
no certificate
                            // (We used  want, not need), so underlying 
handlers can do the real authentication
                            // (or show a custom access error)
                            try {
                                certs = 
(X509Certificate[])engine.getSession().getPeerCertificates();
request.setAttribute("javax.servlet.request.X509Certificate", certs);
                            }
                            catch(SSLPeerUnverifiedException e) {
                                LOG.warn("Renegotiation worked, but no 
cert presented");
                            }

                            break;
                        }
                    }
                }
            }

I tried two handshake modes. First, using the DecryptedEndpoint fill and 
flush stuff. It manages all the handshaking theoretically, but I can't 
make it work.
The problem seems to be in the way browsers manage certificate 
renegotiation. I'm not completeley sure of this, but when they have to 
show the user a dialog for certificate selection,
they close the current connection, and when the selection is made, they 
make another attempt, in this case with the certificate data ready to be 
transferred.


    private boolean doEndpointHandshake(SSLEngine engine, EndPoint 
endpoint) throws IOException, ClosedHandshakeException {

        engine.beginHandshake();

        SslConnection sslConnection = 
(SslConnection)endpoint.getConnection();
        // int appBufferSize = 
engine.getSession().getApplicationBufferSize();
        return 
sslConnection.getDecryptedEndPoint().flush(ByteBuffer.allocate(0));

    }


So finally I ended up implementing another method, doing manually all 
the handshaking through the SSLEngine. Men, that's not easy!
If the socket gets closed, I "suicide" myself throug a runtime 
exception, to avoid trying to write an error response, and in the next 
call from the browsers it works!

The problem is that the implementation works sometimes, but it is buggy, 
mainly because I don't really understand all the underlying technology
    - SSL/TLS Protocol and its subtleties
    - Jetty IO model (NIO and all the filling/flushing), with all the 
callbacks, etc.
    - How browsers behave in the renegotiation (Looks like they close 
the first connection and then try again)


So, I would highly appreciate some help on the matter, some pointers or 
directions on how to make it more reliable, or at least some advice 
about if I should mess with this or not...

Thanks


private boolean doManualHandshake(SSLEngine engine, EndPoint endpoint) 
throws IOException, ClosedHandshakeException {

        if (!(endpoint instanceof ChannelEndPoint))
            return false;
        ChannelEndPoint channelEndPoint = (ChannelEndPoint)endpoint;

        ByteChannel socketChannel = channelEndPoint.getChannel();

        // WARN I create some buffers for net data. I asumme Jetty 
buffers doesn't have anything pending
        SSLSession session = engine.getSession();
        ByteBuffer myNetData = 
ByteBuffer.allocate(session.getPacketBufferSize());
        ByteBuffer peerNetData = 
ByteBuffer.allocate(session.getPacketBufferSize());
        // Fin ojo


        // Create byte buffers to use for holding application data.
        // In handshake shouldn't be any app data reading or writing
        int appBufferSize = engine.getSession().getApplicationBufferSize();
        ByteBuffer myAppData = ByteBuffer.allocate(appBufferSize);
        ByteBuffer peerAppData = ByteBuffer.allocate(appBufferSize);

        // Begin handshake
        try {

            if (endpoint.isInputShutdown() || 
!engine.getSession().isValid()) {
                LOG.info("Handshake impossible. Endpoint shutdown");
                return false;
            }

            engine.beginHandshake();
            SSLEngineResult.HandshakeStatus hs = 
engine.getHandshakeStatus();

            // Process handshaking message
            while (hs != SSLEngineResult.HandshakeStatus.FINISHED &&
                hs != SSLEngineResult.HandshakeStatus.NOT_HANDSHAKING) {

                if (hs == HandshakeStatus.NEED_UNWRAP)
                {
                     // Receive handshaking data from peer
                    int readBytes = socketChannel.read(peerNetData);
                    if (readBytes < 0) {
                        throw new ClosedHandshakeException();
                    }

                    // Process incoming handshaking data
                    peerNetData.flip();
                    SSLEngineResult res = engine.unwrap(peerNetData, 
peerAppData);
                    peerNetData.compact();
                    hs = res.getHandshakeStatus();

                    if (res.getStatus() == Status.BUFFER_UNDERFLOW) {
                        // peerNetData.compact();
                        // I assume we'll loop again with NEED_UNWRAP 
and data get's read again
                    }
                    else if (res.getStatus() == Status.BUFFER_OVERFLOW) {
                        // peerAppData no debería rellenarse con nada
                    }
                    else if (res.getStatus() == Status.CLOSED) {
                        throw new ClosedHandshakeException();
                    }
                }
                else if (hs == HandshakeStatus.NEED_WRAP) {
                     // Empty the local network packet buffer.
                    myNetData.clear();

                    // Generate handshaking data
                    SSLEngineResult res = engine.wrap(myAppData, 
myNetData);
                    hs = res.getHandshakeStatus();

                    if (res.getStatus() == Status.OK) {
                         myNetData.flip();

                        // Send the handshaking data to peer
                        while (myNetData.hasRemaining()) {
                            if (socketChannel.write(myNetData) < 0) {
                                throw new ClosedHandshakeException();
                            }
                        }
                    }
                    else if (res.getStatus() == Status.BUFFER_UNDERFLOW) {
                        // No real reading from myAppData in handshake, 
this shouldn't happen
                    }
                    else if (res.getStatus() == Status.BUFFER_OVERFLOW) {
                        // Enlarge myNetData buffer size?
                    }
                    else if (res.getStatus() == Status.CLOSED) {
                        throw new ClosedHandshakeException();
                    }
                }
                else if (hs == HandshakeStatus.NEED_TASK) {
                    Runnable task;
                    while ((task=engine.getDelegatedTask()) != null) {
                        // new Thread(task).start();
                        // Por ahora en el mismo hilo y bloqueante
                        task.run();
                    }
                    hs = engine.getHandshakeStatus();
                }
            }

            return true;

        } catch (ClosedHandshakeException e) {
            LOG.warn("Error during renegotiation handshake");
            engine.closeInbound();
            endpoint.close();
            throw e;
//            return false;
        }

    }