Bug 478655 - [Mars] https:// update sites requiring SNI cannot be used
Summary: [Mars] https:// update sites requiring SNI cannot be used
Status: REOPENED
Alias: None
Product: ECF
Classification: RT
Component: ecf.providers (show other bugs)
Version: unspecified   Edit
Hardware: PC Linux
: P3 normal (vote)
Target Milestone: ---   Edit
Assignee: ecf.core-inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2015-09-29 12:22 EDT by Antonio Garcia-Dominguez CLA
Modified: 2017-05-08 14:21 EDT (History)
2 users (show)

See Also:


Attachments
Use SSLSocketImpl#setHost to enable SNI with the old HttpClient API (3.69 KB, patch)
2015-09-29 12:24 EDT, Antonio Garcia-Dominguez CLA
no flags Details | Diff
Use SSLSocketImpl#setHost to enable SNI with the old HttpClient API (also changes HttpClientRetrieveFileTransferFactory) (8.13 KB, patch)
2015-10-04 13:34 EDT, Antonio Garcia-Dominguez CLA
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Antonio Garcia-Dominguez CLA 2015-09-29 12:22:58 EDT
We have a target platform definition that mentions an https:// update site that requires Server Name Indication. Firefox and Chrome can fetch the files just fine, with no warnings, but Eclipse Mars under Oracle Java 8 reports the following exception:

!SUBENTRY 1 org.eclipse.equinox.p2.transport.ecf 4 1002 2015-09-29 17:05:24.738
!MESSAGE Unable to read repository at https://www.cs.york.ac.uk/mondo/downloads/hawk/content.xml.
!STACK 0
javax.net.ssl.SSLException: hostname in certificate didn't match: <www.cs.york.ac.uk> != <www-users.cs.york.ac.uk> OR <www-users.cs.york.ac.uk>
	at org.apache.http.conn.ssl.AbstractVerifier.verify(AbstractVerifier.java:238)
	at org.apache.http.conn.ssl.BrowserCompatHostnameVerifier.verify(BrowserCompatHostnameVerifier.java:54)
	at org.apache.http.conn.ssl.AbstractVerifier.verify(AbstractVerifier.java:159)
	at org.apache.http.conn.ssl.AbstractVerifier.verify(AbstractVerifier.java:140)
	at org.apache.http.conn.ssl.SSLSocketFactory.verifyHostname(SSLSocketFactory.java:561)
	at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:536)
	at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:403)
	at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:177)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:304)
	at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:611)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:446)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:863)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.eclipse.ecf.provider.filetransfer.httpclient4.HttpClientFileSystemBrowser.runRequest(HttpClientFileSystemBrowser.java:260)
	at org.eclipse.ecf.provider.filetransfer.browse.AbstractFileSystemBrowser$DirectoryJob.run(AbstractFileSystemBrowser.java:69)
	at org.eclipse.core.internal.jobs.Worker.run(Worker.java:55)

From what I've seen, it appears to be related to this bug in Apache HttpClient, which was supposedly resolved in 4.3.2:

https://issues.apache.org/jira/browse/HTTPCLIENT-1119

Eclipse Mars has HttpClient 4.3.6, which includes the fix mentioned in "apache_httpclient_4.2.x_sni.patch", but I've tried placing a debugger right at the point of the fix that was finally committed (in SSLConnectionSocketFactory#createSocket) and it's not hit when I attempt to browse through the update site.

It appears that since HttpClientBrowseFileTransferFactory uses the deprecated DefaultHttpClient API, it does not go through the SNI fix. I have applied the attached patch, which is based on the other patch mentioned in HTTPCLIENT-1119 (the October 2012 one), and it works for me now. It also appears to pass all filetransfer.httpclient4 tests.
Comment 1 Antonio Garcia-Dominguez CLA 2015-09-29 12:24:02 EDT
Created attachment 256924 [details]
Use SSLSocketImpl#setHost to enable SNI with the old HttpClient API
Comment 2 Scott Lewis CLA 2015-09-29 12:46:27 EDT
(In reply to Antonio Garcia-Dominguez from comment #1)
> Created attachment 256924 [details]
> Use SSLSocketImpl#setHost to enable SNI with the old HttpClient API

Hi Antonio.

Thanks for the investigation and patch.  It generally looks fine, but a couple of things:

1) You provide a patch for HttpClientBrowseFileTransferFactory.   I think we will also need a similar (exactly the same change?) for HttpClientRetrieveFileTransferFactory.   Could you make similar additions for that class also and apply as patch?

2) It's been my intention to move to the new/non-deprecated API for the DefaultHttpClient creation, but we just haven't had the resources to do so.  Would you like to jointly work on that instead?  We would probably need to enlist the assistance of the folks on bug 471325 who recently contributed another patch to work around the deprecated API.

Please let me know what you think about these.

Thanks,

Scott
Comment 3 Antonio Garcia-Dominguez CLA 2015-10-04 13:21:04 EDT
Sorry for the delay, but we've been busy with other things at work.

(In reply to Scott Lewis from comment #2)
> (In reply to Antonio Garcia-Dominguez from comment #1)
> > Created attachment 256924 [details]
> > Use SSLSocketImpl#setHost to enable SNI with the old HttpClient API
> 
> 1) You provide a patch for HttpClientBrowseFileTransferFactory.   I think we
> will also need a similar (exactly the same change?) for
> HttpClientRetrieveFileTransferFactory.   Could you make similar additions
> for that class also and apply as patch?

Sure! I'll do those in a moment and provide a new patch.

> 2) It's been my intention to move to the new/non-deprecated API for the
> DefaultHttpClient creation, but we just haven't had the resources to do so. 
> Would you like to jointly work on that instead?  We would probably need to
> enlist the assistance of the folks on bug 471325 who recently contributed
> another patch to work around the deprecated API.

I am sorry to say I do not have enough time for that at the moment, due to other engagements. October is looking really busy. Perhaps for mid- or late November?
Comment 4 Antonio Garcia-Dominguez CLA 2015-10-04 13:34:38 EDT
Created attachment 257022 [details]
Use SSLSocketImpl#setHost to enable SNI with the old HttpClient API (also changes HttpClientRetrieveFileTransferFactory)

Here is a new patch that extracts the customized DefaultHttpClient to a new class and uses it from both HttpClientRetrieveFileTransferFactory and HttpClientBrowseFileTransferFactory.
Comment 5 Scott Lewis CLA 2015-10-06 13:15:45 EDT
Thanks Antonio for the patch.

I've applied the patch, tested locally, and pushed to master:

http://git.eclipse.org/c/ecf/org.eclipse.ecf.git/commit/?id=0f564abd3db01c32ef168a8399d81c5ced1029e6

I've also built and tested on ECF's builder, and the httpclient4 repo is here:

http://build.ecf-project.org/repo/C-HEAD-filetransfer.httpclient4.feature/lastSuccessful/archive/site.p2/

Antonio I would appreciate it if you would either get the changed source and build, or install the binaries from the above repo and test in your environment.   Once you've verified that the change works in your environment I'll resolve this bug and open a new one for updating to non-deprecated API.  I'll also look into when this fix can get into the Mars and Neon streams.

Thanks again.
Comment 6 Antonio Garcia-Dominguez CLA 2015-10-07 05:16:15 EDT
(In reply to Scott Lewis from comment #5)
> Thanks Antonio for the patch.

You are welcome :-).

> Antonio I would appreciate it if you would either get the changed source and
> build, or install the binaries from the above repo and test in your
> environment.   Once you've verified that the change works in your
> environment I'll resolve this bug and open a new one for updating to
> non-deprecated API.  I'll also look into when this fix can get into the Mars
> and Neon streams.

Thank you for your help. I have installed your updated binaries and my target platform definition can resolve from scratch on a new workspace, so I think it works fine.
Comment 7 Scott Lewis CLA 2015-10-07 10:28:55 EDT
(In reply to Antonio Garcia-Dominguez from comment #6)
> (In reply to Scott Lewis from comment #5)
> > Thanks Antonio for the patch.
> 
> You are welcome :-).
> 
> > Antonio I would appreciate it if you would either get the changed source and
> > build, or install the binaries from the above repo and test in your
> > environment.   Once you've verified that the change works in your
> > environment I'll resolve this bug and open a new one for updating to
> > non-deprecated API.  I'll also look into when this fix can get into the Mars
> > and Neon streams.
> 
> Thank you for your help. I have installed your updated binaries and my
> target platform definition can resolve from scratch on a new workspace, so I
> think it works fine.

Ok terrific.   Resolving as fixed.
Comment 8 Scott Lewis CLA 2017-03-12 12:08:14 EDT
Hi Antonio.

It seems that this fix is now causing troubles wrt Java 9 migration.  Please see bug 513332.

Antonio I would appreciate some consultation on this.  Would this java9 problem and the failing work around go away if ECF was to move to the non-deprecated APIs?   If so, would you be able willing to provide some guiadance and testing with this?

Thanks.
Comment 9 Antonio Garcia-Dominguez CLA 2017-03-15 16:45:08 EDT
Sorry, I wasn't able to look into this until now. According to your comment 4, you have pushed some code to work around this. Is there some binary I could use to verify that it still works wih your fix?
Comment 10 Antonio Garcia-Dominguez CLA 2017-03-15 16:47:22 EDT
(In reply to Antonio Garcia-Dominguez from comment #9)
> Sorry, I wasn't able to look into this until now. According to your comment
> 4, you have pushed some code to work around this. Is there some binary I
> could use to verify that it still works wih your fix?

To be clear, it's comment 4 on bug 513332.
Comment 11 Scott Lewis CLA 2017-03-15 22:23:35 EDT
(In reply to Antonio Garcia-Dominguez from comment #9)
> Sorry, I wasn't able to look into this until now. According to your comment
> 4, you have pushed some code to work around this. Is there some binary I
> could use to verify that it still works wih your fix?

There is a binary with this workaround/fix here:

http://download.eclipse.org/rt/ecf/snapshot/site.p2