Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [cross-project-issues-dev] Proxy Testing Tool?

For everyone's info, there's been a long discussion of what appear to be HttpComponents-based proxy difficulties on this bug

https://bugs.eclipse.org/bugs/show_bug.cgi?id=422665

ECF (used for p2) has the ability to disable the default HttpComponents-based provider and use an JRE URLConnection-based provider, which under some proxy use cases and configurations appears to work:

https://bugs.eclipse.org/bugs/show_bug.cgi?id=422665#c137

This comment references the documentation to disable the HttpComponents-based provider.

https://wiki.eclipse.org/Disabling_Apache_Httpclient

I'm not sure what Code Recommenders and the Automated Error Reporting uses of ECF/p2, but if it uses what p2 uses then it also will inherit the multi-provider support described above.

From the ECF point of view, if anyone is able/willing to do proxy testing, and suggest, implement, and test changes or other workarounds to ECF, then contributions are welcome. I can't speak or act for the Apache HttpComponents project, however.

Scott


On 6/9/2015 8:30 AM, Andreas Sewe wrote:
Hi all,

a bit more info.

To analyze the proxy issues plaguing Code Recommenders and the Automated
Error Reporting, we have included a network communication test job in
earlier RCs that just pinged a stats URI at download.eclipse.org using
both Apache HttpComponents or ECF (through p2's
RepositoryTransport.getLastModified(..)).

If anyone is interested, here's the code [1]. You can view the collected
data at [2] using a "partial file name" of
"/stats/recommenders/network-communication-test/".

This yields results like the following:

/stats/recommenders/network-communication-test/apache/java-1.8.0_45/Windows+7-6.1/Native-none/ 	746
/stats/recommenders/network-communication-test/p2/java-1.8.0_45/Windows+7-6.1/Native-none/ 	737
The stats URLs encode the platform, Java version, whether Eclipse was
configured with "Native", "Manual" or "Direct" proxy settings (General >
Network Communication) and which authentication method (if any) was
mandated by the "Proxy-Authenticate" header.

In the above, you can see that a Windows 7 configuration using the
"Native" provider and no authentication method ("none") pinged 746 over
Apache HttpComponents and 737 over p2. This *may* mean that p2 had 6
times trouble communicating where plain HttpComponents had not.

Most of the time, the situation is the other way around, however. Here's
a "diff" of the results, in case anyone can spot a pattern.

In these configuration, we don't see Apache at all:

java-1.7.0/Windows+7-6.1/Native-NTLM;Basic/	p2	4
java-1.7.0_60/Windows+7-6.1/Native-unknown/	p2	1
java-1.7.0_21/Windows+7-6.1/Manual-NEGOTIATE;NTLM;BASIC/	p2	1
java-1.7.0_67/Windows+7-6.1/Manual-NTLM;BASIC/	p2	1
java-1.7.0_71/Windows+7-6.1/Manual-Basic/	p2	3
java-1.7.0_72/Windows+8.1-6.3/Native-none/	p2	1
java-1.7.0_79/Windows+7-6.1/Manual-Basic/	p2	6
java-1.7.0_75/Windows+XP-5.1/Manual-NEGOTIATE;NTLM;BASIC/	p2	1
java-1.7.0_80/Windows+7-6.1/Manual-NTLM;BASIC/	p2	3
java-1.7.0_79/Windows+7-6.1/Native-unknown/	p2	1
java-1.7.0_79/Windows+8.1-6.3/Native-unknown/	p2	1
java-1.7.0_80/Windows+8-6.2/Native-unknown/	p2	1
java-1.8.0_05/Windows+7-6.1/Manual-BASIC/	p2	3
java-1.8.0_05/Windows+7-6.1/Manual-Negotiate;NTLM;Basic/	p2	1
java-1.8.0_31/Mac+OS+X-10.10.3/Direct-unknown/	p2	1
java-1.8.0_31/Windows+7-6.1/Manual-NTLM;Basic/	p2	4
java-1.8.0_31/Windows+7-6.1/Manual-Negotiate;Kerberos;NTLM;Basic/	p2	2
java-1.8.0_31/Windows+7-6.1/Native-Negotiate;Kerberos;NTLM/	p2	1
java-1.8.0_45/Mac+OS+X-10.10.3/Direct-unknown/	p2	3
java-1.8.0_45/Windows+7-6.1/Manual-Basic/	p2	7
java-1.8.0_45/Windows+7-6.1/Manual-Negotiate;Basic/	p2	2
Here, we don't see p2 at all:

java-1.7.0_51/Linux-3.13.0-24-generic/Native-unknown/	apache	1
java-1.8.0_31/Linux-3.5.0-54-generic/Native-none/	apache	1
java-1.8.0_31/Windows+7-6.1/Native-unknown/	apache	1
In these configurations, we see small differences (which *may* be
explained by a cache somewhere):

java-1.7.0_11/Windows+7-6.1/Native-none/	apache	7
java-1.7.0_11/Windows+7-6.1/Native-none/	p2	8
java-1.7.0_15/Mac+OS+X-10.10.3/Manual-none/	apache	5
java-1.7.0_15/Mac+OS+X-10.10.3/Manual-none/	p2	4
java-1.7.0_45/Linux-3.13.0-53-generic/Native-none/	apache	5
java-1.7.0_45/Linux-3.13.0-53-generic/Native-none/	p2	4
java-1.7.0_51/Linux-3.13.0-24-generic/Native-none/	apache	22
java-1.7.0_51/Linux-3.13.0-24-generic/Native-none/	p2	21
java-1.7.0_75/Linux-2.6.32-504.16.2.el6.x86_64/Native-none/	apache	5
java-1.7.0_75/Linux-2.6.32-504.16.2.el6.x86_64/Native-none/	p2	4
java-1.7.0_79/Windows+7-6.1/Native-none/	apache	71
java-1.7.0_79/Windows+7-6.1/Native-none/	p2	70
java-1.8.0_05/Windows+8-6.2/Native-none/	apache	3
java-1.8.0_05/Windows+8-6.2/Native-none/	p2	2
java-1.8.0_20/Windows+7-6.1/Native-unknown/	apache	3
java-1.8.0_20/Windows+7-6.1/Native-unknown/	p2	2
java-1.8.0_20/Windows+8-6.2/Native-none/	apache	7
java-1.8.0_20/Windows+8-6.2/Native-none/	p2	6
java-1.8.0_25/Windows+7-6.1/Native-none/	apache	78
java-1.8.0_25/Windows+7-6.1/Native-none/	p2	77
java-1.8.0_31/Windows+7-6.1/Native-none/	apache	127
java-1.8.0_31/Windows+7-6.1/Native-none/	p2	128
java-1.8.0_40/Mac+OS+X-10.9.5/Direct-none/	apache	11
java-1.8.0_40/Mac+OS+X-10.9.5/Direct-none/	p2	10
java-1.8.0_40/Windows+7-6.1/Manual-BASIC/	apache	1
java-1.8.0_40/Windows+7-6.1/Manual-BASIC/	p2	4
java-1.8.0_40/Windows+7-6.1/Native-none/	apache	131
java-1.8.0_40/Windows+7-6.1/Native-none/	p2	129
java-1.8.0_40/Windows+7-6.1/Native-unknown/	apache	1
java-1.8.0_40/Windows+7-6.1/Native-unknown/	p2	2
java-1.8.0_45/Linux-3.16.0-38-generic/Native-none/	apache	19
java-1.8.0_45/Linux-3.16.0-38-generic/Native-none/	p2	18
java-1.8.0_45/Linux-3.2.0-4-amd64/Native-none/	apache	3
java-1.8.0_45/Linux-3.2.0-4-amd64/Native-none/	p2	4
java-1.8.0_45/Mac+OS+X-10.10.3/Direct-none/	apache	150
java-1.8.0_45/Mac+OS+X-10.10.3/Direct-none/	p2	149
java-1.8.0_45/Windows+7-6.1/Manual-NTLM;BASIC/	p2	9
java-1.8.0_45/Windows+7-6.1/Manual-NTLM;Basic/	apache	6
java-1.8.0_45/Windows+7-6.1/Native-none/	apache	746
java-1.8.0_45/Windows+7-6.1/Native-none/	p2	737
java-1.8.0_45/Windows+7-6.1/Native-unknown/	apache	7
java-1.8.0_45/Windows+7-6.1/Native-unknown/	p2	9
java-1.8.0_45/Windows+8-6.2/Native-none/	apache	304
java-1.8.0_45/Windows+8-6.2/Native-none/	p2	301
java-1.8.0_45/Windows+8-6.2/Native-unknown/	apache	4
java-1.8.0_45/Windows+8-6.2/Native-unknown/	p2	5
As you can see, plain Apache HttpComponents still has issues with both
NTLM and SPNEGO proxy authentication, although at least for NTLM there
are a few proxies out there for which HttpComponents can authenticate
with NTLM (but never SPNEGO).

Moreover, there a few configurations (notably,
"java-1.7.0_79/Windows+7-6.1/Manual-Basic/"), for which HttpComponents
fails even with straight-forward settings: "Manual" proxy provider and
"Basic" authentication, i.e., username and password. I am not quite sure
what to make of this other than "you use the HttpComponents API wrong!"

Any insights are greatly appreciated.

Andreas

[1]
<https://git.eclipse.org/c/recommenders/org.eclipse.recommenders.git/tree/plugins/org.eclipse.recommenders.rcp/src/org/eclipse/recommenders/internal/rcp/NetworkCommunicationTestJob.java?id=b5f416514f1cbc37bb9558a442f10f6d757377f3>
[2] <https://dev.eclipse.org/committers/committertools/stats.php>




Back to the top