Community
Participate
Working Groups
Created attachment 144522 [details] Shot from the proxy dialog Build ID: I20090611-1540 Steps To Reproduce: 1. live behind a firewall 2. open a XML file 3. a proxy dialog appears with no information where to connect to and no ability to remember the password More information: Appears one or two times when opening a XML file on a fresh started eclipse, regardless which platform (Win, Linux, etc.). I attached a screenshot from Linux but the Windows one looks similar. Appears some times when invoking the Content-Assist in Java Editors (but this may be caused by another plug-in). This is a dialog requiring to enter my proxy credentials. However this is not obvious (see attached shot). I think this dialog comes from the java runtime not eclipse. This dialog appears since at least from Eclipse 3.4 (I think it was there in 3.3 too). I'm writing plug-ins on my own, so I know in which cases this dialog appears: This may be caused from a URLConnection which does not properly use the platform proxy settings. A possible fix would be: private void configureProxy(final URL url, final HttpURLConnection conn) { final Activator plugin = Activator.getDefault(); final IProxyService proxyService = plugin.getProxyService(); final IProxyData[] proxyDatas = proxyService.getProxyDataForHost(url.getHost()); if (proxyDatas.length > 0) { final IProxyData proxyData = proxyDatas[0]; if (proxyData.isRequiresAuthentication()) { final String user = proxyData.getUserId(); final String password = proxyData.getPassword(); final String value = "Basic " + Base64.encode((user + ":" + password).getBytes()); //$NON-NLS-1$ //$NON-NLS-2$ conn.setRequestProperty("Proxy-Authorization", value); //$NON-NLS-1$ } } } So, please have a look at your URLConnections and please check if the proxy settings a correcly set in the connection. Thank you.
I'm not sure where to route this one to, but since mentions "opening an XML file" will start there. The Platform, though, actually owns the proxy dialogs and settings if it turns our related to something they are doing. If we are using a "raw" URL connection, I suspect we always have been and are in many places? I wonder why this hasn't shown up before? Maybe some "order of execution" effect?
This effect shows up in our company a long time. It's not only me. I have waited for the 3.5 release and hoped this one would be fixed, but it's still there. I did a little debugging (behind a proxy): - closed all running eclipse instances (just to be sure no java process is running) - opened eclipse - created a new java project (maybe a simple project will be sufficient as well) - created a new file: "test.xml" - typed some tags - typed a DOCTYPE statement in the first line -> the dialog pops up
Created attachment 144801 [details] Screenshot from Debugger when dialog appears Added screenshot from stacktrace while debugging.
Just trying to clarify: are the proxy settings in the platform properly configured? Did you go to Window->Preferences->General->Network Connections and entered the required proxy information (host, authentication info)?
Yes, the proxy settings are properly configured. I can install plug-ins/extensions and do updates through the proxy and there the user/password authentication is working.
Please have also a look at bug 287145 and bug 215320 which is the same/similar problem at another product/component.
Peter, have you ever seen something similar while you worked on the platform proxy support? My expectation here is that the HTTPUrlConnection used in this scenario will use by default the platform (JVM) configured proxy and authentication settings and that the user is not prompted again for the proxy/authentication information.
My understanding is that the userid and password specified on the Eclipse preference page will set JVM system properties so that HTTP connections will use the specified userid and password automatically. If the JVM gets an authentication error opening the connection it calls a registered authenticator to pop up the dialog to get authentication info from the user and it attempts to open the connection again. I believe Eclipse registers the authenticator that is used to pop up the dialog you are seeing. Therefore, if someone could reproduce this problem in a purely Eclipse environment this defect could be routed to Eclipse to fix.
Thank you Peter. Daniel, as per Peter's comment, the stack trace seems to say that the HttpURLConnection code is trying to connect, fails for some reason, and then, logically, defers to the platform registered authenticator, which prompts the user to provide the credentials. If this is consistently reproducible, could you please debug again and note what is the parameter value when org.eclipse.wst.internet.cache.internal.Cache.cacheResource(String) is called? You could then write a simple plug-in with an action and in the action use code similar to the one used by the WTP Internet Cache. Something like: public testURL(String uri) { URLConnection conn = null; InputStream is = null; try { URL url = new URL(uri); conn = url.openConnection(); conn.connect(); // Determine if this resource can be cached. if(conn.getUseCaches()) { is = conn.getInputStream(); .... You get the idea. If you run it on the base platform with the same URI the cache is trying to get at and see the same behaviour, then WTP is out of the picture. This would help narrow down the source of the problem.
Debugging the DOCTYPE validation will call the Cache.getResource(String) method with this URL: http://somehost.org/somedtd.dtd I typed in this DOCTYPE statement: <!DOCTYPE Root PUBLIC "Public Identifier" "http://somehost.org/somedtd.dtd" > I think this looks right. Then I made a little plug-in with an action with the code suggestion. When invoking the action, the proxy dialog appears. So this looks like a general problem. I will attach the screen shots from the debugger. The blacked-out values are the host names of the proxy and my machine. The proxy is some sort of NTLM chained proxy. According to Peter, the dialog is a eclipse platform dialog. So I would suggest to add a check-box to remember the credentials. The preference page "Network Connections" should be enhanced to show the remembered/additional proxy entry. Another approach would be to send each NTLM proxy the (already) specified credentials (as my code above). And to fall back to a dialog if this fails. In my case, the credentials are always the same.
Created attachment 145395 [details] Debugger Variables View of Current Proxy Credentials
Created attachment 145396 [details] Debugger Variables View of Responses
Hi Daniel, Thank you for taking the time to debug this further. If you read through the original bug 154100 used to move the WTP proxy settings to the Platform, you'll find a few comments discussing NTLM authentication. One of them points here here http://java.sun.com/j2se/1.5.0/docs/guide/net/properties.html, specifically the http.auth.ntlm.domain system property. If I read this right, it seems that there's a way to encode the domain as part of the user id. Could you please give this approach a try? Additional information about the Platform proxy settings design are here: http://wiki.eclipse.org/index.php/ProxySupport.
I always have to prefix my username with the domain. Otherwise I would not get through the proxy at all (this applies to FireFox and other programs as well). So in the preferences I specified as the user name: <domain>\<username>. So there is no need to set the system property as I use option 2 as described here http://java.sun.com/j2se/1.5.0/docs/guide/net/properties.html. I read http://wiki.eclipse.org/index.php/ProxySupport. There is no special treatment of NTLM proxies or NTLM chained proxies in particular. I suspect, that there may be more than one proxy asking for authentication. I know that I am behind a "chained" proxy. Whatever "chained" means, I am not a proxy specialist. I wrote that before, see above. From FireFox, I know, that I am sometimes asked by some other proxy for authentication. But since the dialog does not show anything after "Connect to:" and I am not able to remember my credentials. I do not know which proxy asked. Maybe the cache can only hold one proxy for each target URL and if the proxy switches, I have to enter my credentials again. But from my experience wit FireFox, the proxy switches not very often. However I always get a proxy dialog when entering a doctype in a XML file (on a freshly started eclipse). Sometimes I get two dialogs (another URLConnection). If I enter wrong credentials, I get one more for each wrong authentication. After two wrong credentials per URLConnection it wont ask me any more. As from this bugs description, my code will get through the proxy without popping up the proxy password dialog.
Szymon, do you know if this prompt is coming from the Platform proxy support?
Maybe one of you can have a look at jCIFS which can (among many other things) do a correct NTLM authentication which passes all chained proxies in my company. http://jcifs.samba.org/
*** Bug 299232 has been marked as a duplicate of this bug. ***
The similar issue appears also for me. During reproduction I encountered also bug 215320 (Javadoc contents retrieval doesn't support proxy connections). The question about proxy settings appears only once in the platform and when I type correct username/password, it does not appear again in any other place until restart. In other words: 1. Starting Eclipse 2. Using content assist in the Eclipse. I am asked about user/password, I am answering with the correct one. 3. Trying to validate xml file - the proxy settings menu does not appear 4. Running my own plugin using internet connection - proxy settings menu does not appear When I swap actions from point 2 and 3, I am asked only during the validation. Daniel, could you confirm the same in your case or we have different bugs? In my issue, the bug 286631 and bug 215320 seem to have the same cause. When I start the platform (I20100312-1448), I am getting such errors in spite of the fact I had set these values in connection preferences before: !ENTRY org.eclipse.core.net 1 0 2010-04-19 13:21:03.781 !MESSAGE System property http.proxyHost is not set but should be localhost. !ENTRY org.eclipse.core.net 1 0 2010-04-19 13:21:03.796 !MESSAGE System property http.proxyPort is not set but should be 3128. !ENTRY org.eclipse.core.net 1 0 2010-04-19 13:21:03.796 !MESSAGE System property https.proxyHost is not set but should be localhost. !ENTRY org.eclipse.core.net 1 0 2010-04-19 13:21:03.796 !MESSAGE System property https.proxyPort is not set but should be 3128. !ENTRY org.eclipse.core.net 1 0 2010-04-19 13:21:03.796 !MESSAGE System property socksProxyHost is not set but should be localhost. !ENTRY org.eclipse.core.net 1 0 2010-04-19 13:21:03.812 !MESSAGE System property socksProxyPort is not set but should be 3128. !ENTRY org.eclipse.core.net 1 0 2010-04-19 13:21:03.812 !MESSAGE System property https.proxyHost is not set but should be localhost. !ENTRY org.eclipse.core.net 1 0 2010-04-19 13:21:03.812 !MESSAGE System property https.proxyPort is not set but should be 3128. !ENTRY org.eclipse.core.net 1 0 2010-04-19 13:21:03.828 !MESSAGE System property socksProxyHost is not set but should be localhost. !ENTRY org.eclipse.core.net 1 0 2010-04-19 13:21:03.828 !MESSAGE System property socksProxyPort is not set but should be 3128. !ENTRY org.eclipse.core.net 1 0 2010-04-19 13:21:03.828 !MESSAGE System property socksProxyHost is not set but should be localhost. !ENTRY org.eclipse.core.net 1 0 2010-04-19 13:21:03.843 !MESSAGE System property socksProxyPort is not set but should be 3128.
Krzysztof, I can confirm that we have the same bug. Your reproduction seems to be right. If I once entered my correct credentials for the XML problem I won't be asked on the Content Assist problem and vice versa. I think this is because the URLConnection is not connected/synchronized with the platform proxy settings. I can remember, that I have seen the same messages in the error log. But for the quick test for you now, they didn't appear. Since these are just warnings, I ignored them.
Created attachment 165735 [details] patch in progress I am working on the patch. At current state, the patch prevents for displaying proxy dialog window (Needs correct indentation and plugin dependencies versions yet). Keith or Valentin: could you review the patch and provide any feedback?
Krzysztof, thank you for the patch. As per our chat, one question that pops to mind is: do we have to change all our uses of URL or URLConnection to configure them with the platform proxy settings? Michael, Szymon, any chance you could re-read through this and let us know what you think?
Valentin, My patch resolves the communication problem only during fetching schema. The patch resolves only this bug. In general we should change all our uses to use with the platform proxy settings, but I think, it is another bug.
(In reply to comment #19) > I think this is because the URLConnection is not connected/synchronized with > the platform proxy settings. Platform make user proxy settings available via API calls (IProxyService) and via system properites. On most VMs system properties containing user credentials (like http.proxyUser, http.proxyPassword) are simply not consumed by Sun's HTTP client. That's why you see the prompt. On the other hand some proxy API consumers use provided credentials. That's the case of p2 which uses ECF and Apache HTTP client as a default. That's why after restarting Eclipse browsing remote update sites works (with Apache's client at the bottom) but opening a content assists (Sun's client) results in a prompt. Seems like the only way to use provided credentials with Sun's HTTP client is what Krzysztof proposed in a patch. Similar fixes should be prepared for other code opening connections though.
Created attachment 166316 [details] Snap (In reply to comment #0) > 3. a proxy dialog appears with no information where to connect to and no > ability to remember the password > Information shown depends on a proxy server. Here's a snap from my machine with proxy server on localhost. By remembering the password you mean populating the dialog with the values user entered in the previous session? Or bypassing the dialog and simply trying these credentials?
Pawel, (In reply to comment #24) > By remembering the password you mean populating the dialog with the values user > entered in the previous session? Or bypassing the dialog and simply trying > these credentials? I would like at least that the previously entered values be already filled in (remembered from a previous dialog). Maybe a checkbox should appear to choose if the values should be remembered. It would be better/the best if the dialog does not appear at all and try to use the settings from the platforms preferences. From the users point of view, it is not clear that there are two differnt kinds of URL connection implementations are used. As a user I would like to configure the proxy credentials only once and that's it. It is somewhat confusing to get a proxy password dialog even though it is configured in the platforms preferences. The fact, that in my dialog the messages where to connect remains empty may be from that M$ ISA proxy which is used in my company. Maybe the title of the dialog could be changed to "Proxy Password Required" instead of just "Password Required" to give the user a hint what's the password for.
(In reply to comment #25) > I would like at least that the previously entered values be already filled in > (remembered from a previous dialog). Maybe a checkbox should appear to choose > if the values should be remembered. > > It would be better/the best if the dialog does not appear at all and try to use > the settings from the platforms preferences. > > From the users point of view, it is not clear that there are two differnt kinds > of URL connection implementations are used. These cases should be fixed one after another. Krzysztof's fix solves the issue for XML documents. Separate bugs like bug 215320 should be opened to track other problematic code. > The fact, that in my dialog the messages where to connect remains empty may be > from that M$ ISA proxy which is used in my company. Maybe the title of the > dialog could be changed to "Proxy Password Required" instead of just "Password > Required" to give the user a hint what's the password for. If the authenticator can be used only to authenticate to proxy servers then that's reasonable. This has to checked though.
Daniel, what VM do you use? Do your colleagues on Windows have the same problem? I know that Sun's client can support NTLM authentication so on windows domain accounts the problem shouldn't appear.
(In reply to comment #27) > Daniel, what VM do you use? Do your colleagues on Windows have the same > problem? I know that Sun's client can support NTLM authentication so on windows > domain accounts the problem shouldn't appear. On Windows I use Suns 32-bit VM. My colleagues have the same problem. They also use Sun VMs. We have Windows configured to locally login (without beeing in a windows domain). In order to go through the proxy into the internet we have to use another username/password (this time within a domain). On Linux (Ubuntu) I use OpenJDK 64-Bit IcedTea6. In order to get through the proxy I have to use the same user/password from above within the domain.
(In reply to comment #28) > We have Windows configured to locally login (without beeing in a > windows domain). In order to go through the proxy into the internet we have to > use another username/password (this time within a domain). Makes sense. I've tested once that on domain accounts the problem does not appear, see bug 287558, comment 10.
Thank you, folks. Superficially, I'm not all that excited about repeating this type of code pattern (interacting with the proxy service) every time we have a need to use a URL connection. Perhaps a better approach would be for a lower level component to provide either a factory for proxied URL connections or something like getProxiedInputStream. The component could be a common WTP component or lower in the platform. That seems to me like a better, more encapsulated solution.
Created attachment 166929 [details] Connection helper in platform Pawel, what you think about such additional service? Valentin, did you mean something like that?
I talked to Pawel and he found that a it could be a potential enhancement in the next release. Since proxy API is Team I've opened bug 311701 to track the issue on their side. Valentin, we need this patch to 3.6 and backport it to previous versions of Eclipse. what do you think about my initial patch in this case?
Hi Krzysztof, as you know, WTP 3.2 is in shutdown mode (RC1). Could you please describe why is this a pressing issue that must be addressed in WTP 3.2? As I explained here https://bugs.eclipse.org/bugs/show_bug.cgi?id=299232#c4, to avoid internet access during XML validation or editing it is recommended to contribute all needed XML grammars (DTDs, XSDs) to the WTP XML Catalog.
Created attachment 168627 [details] Alternate platform level patch I kept trying to better understand what's going on here and it's become clear to me that there's no getting away from having to set the Proxy-Authorization header when running through a proxy. As I said above, setting the request header each time we use a URL connection smells so I looked for a better place to do this. The patch in comment 31 is one possibility but it seemed somehow still too high level. Digging deeper, it appears to me the org.eclipse.ui.internal.net.auth.NetAuthenticator could be a such better place to inject the proxy authorization info. The authenticator is used for both proxy and server authentication but there's a way to find out exactly who's asking for credentials: we can call getRequestorType() and check for RequestorType.PROXY. Once we know it's really a proxy asking for info, we can ask the Platform's proxy service to tell us the configured user id and password and we can either bypass the dialog or, as Daniel sugested, pre-populate it with the values from the proxy preferences. I'm attaching a tentative patch which seems to get the things going. I'm sure there's more to do, for example to figure out what should happen when the credentials change in the preferences, etc. Thoughts?
(In reply to comment #34) > Thoughts? There is bug 312228 to track authentication improvements in core.net. Please set dependency on the bug as appropriate.
Created attachment 168817 [details] Stack traces Here are some more details about the open editor vs validator scenarios, more precisely about the single vs. double prompt. I'm attaching the stack traces I collected while testing these scenarios. - In the open editor case, there are two threads that compete to cache the grammar: the content model thread and the as you type validator. Any one of these results in the first prompt. The second prompt is caused by the as you type validator. It seems to me that the cache should be synchronized to have all these treads not step onto each other. - In the manual validation case, there's only one thread. For all these tests, I've started from scratch - new platform instance, and the platform cache empty. There are actually at least three uses of URL or URLConnection in these code paths, and there may be more. Chris' patch only addresses one. That's why I'd rather see the fix pushed down into the NetAuthenticator, even as partial fix until the platform authentication story is being addressed (probably no sooner than the next major release). Regarding the warnings from the proxy code, I think that's a bug. Here's the stack trace: ProxyType.verifySystemPropertyEquals(String, String) line: 308 ProxyType.verifyDataMatchesHttpSystemProperties(ProxyData, boolean) line: 244 ProxyType.verifyDataMatchesSystemProperties(ProxyData) line: 226 ProxyType.createProxyData(String, Preferences, int) line: 154 ProxyType.getProxyData(int) line: 137 ProxyManager.getProxyData() line: 176 ProxySelector.getProxyData(String) line: 89 ProxyType.getProxyData(String) line: 426 ProxyType.updateHttpSystemProperties() line: 382 ProxyType.updateSystemProperties(IProxyData) line: 211 ProxyType.initialize() line: 509 ProxyManager.initialize() line: 282 Activator.start(BundleContext) line: 179 .... getProxyData(int) is called with VERIFY_EQUAL during the initialization code, while trying to update the system properties. Of course that fails, and the warnings are printed in the log.
Thank you all for your input, here are my conclusions. Short version: I'm thinking to mark this as a duplicate of bug 312228 and propose the patch in comment 34 (or an improved version) as the temporary solution for Eclipse 3.6 / WTP 3.2. Long version: - the problem here is relatively minor: the dialog prompt comes up once per Platform session. Once the resource (DTD, XSD, etc) is cached, the XML tools (editor, content model, validator) will find it in the cache and use it from there. Further, instead of relying on the cache, the best practice is to contribute all grammars to the XML catalog (WTP does this for quite a few standard grammars). - the underlying cause is really a limitation in the platform network authentication layer: there's no talk between the NetAuthenticator and the proxy service but there could be - the patch proposed by Chris fixes the problem too high up the stack in WTP and only for basic proxy authentication. It introduces a few extra dependencies and only fixes one use of the URL and related classes. Other uses, like the one in the internet cache plug-in are not addressed. - we'll need to further investigate and if needed fix the perceived synchronization issues in the WTP internet cache plug-in - the log warnings likely deserve a separate bug against core.net While we wait for the optimal long term solution tracked through bug 312228 I propose that the Platform team considers the patch I contributed in comment 34 as a short term fix for Eclipse 3.6. This would address not only the XML tools issue but likely also the javadoc issue described in bug 215320.
After discussion with Szymon, I agree that fixing this bug may not be a wise move given possible impact if anything goes wrong (Eclipse is currently in RC stage). Releasing the fix into 3.7 and 3.6.1 seems to be much better choice. However, an accepted patch would be welcome sooner.
(In reply to comment #37) > I'm thinking to mark this as a duplicate of bug 312228 and propose the patch in > comment 34 (or an improved version) as the temporary solution for Eclipse 3.6 / > WTP 3.2. > Done. It's relatively low consumer impact and high risk of the fix though (it can cause users are blacklisted silently). Because of that bug 312228 will be fixed in 3.7. *** This bug has been marked as a duplicate of bug 312228 ***
*** Bug 326887 has been marked as a duplicate of this bug. ***
*** Bug 345525 has been marked as a duplicate of this bug. ***
