Community
Participate
Working Groups
Created attachment 152110 [details] dialog I'm being pummeled with I am getting 16 back-to-back dialogs asking for SOCKS authentication when I bring up the update manager (see attached screenshot). For some reason the password isn't taking, but that's irrelevant. Point is, whether I put in the wrong userid/password, or I hit cancel, I get asked 15 more times...even if I hit cancel in the dialog. After about the eight dialog, I was convinced that I was caught in an infinite loop and that my only recourse was to hard-kill the eclipse process via the Task Manager. But now I know better because I've dug into the issue and I know there's a finite number of times I'll be asked. Still, it might as well be infinite, because no sane user is going to keep hitting cancel when there seems to be no end in sight. How many update sites do I have? Only two. That's the scary thing. If I had 10, I would be hit with 80 dialogs. Obviously, this is an unacceptable situation. Why is the user being prompted eight time per update site? a. The first problem is that CacheManager.createCache() initiates two http requests. One for the jar, and when that fails, one for the xml. There is logic to prevent a second iteration when there is a timeout, but what if the failure is a (java.net.SocketException: SOCKS : authentication failed), as in my case. There is no logic to prevent a retry in that case, and it seems to me there should be. BTW, here are the two calls, each which result in an http request: -> lastModifiedRemote = getTransport().getLastModified(jarLocation, submonitor.newChild(1)); -> lastModifiedRemote = getTransport().getLastModified(xmlLocation, submonitor.newChild(1)); b. for each attempt in (a), there will actually be four attempts, and that's because by default an http request has a retry count of 3. To prevent that, HttpClientFileSystemBrowser.runRequest() could override the retry behavior as follows HttpMethodParams params = new HttpMethodParams(); params.setParameter(HttpMethodParams.RETRY_HANDLER, new DefaultHttpMethodRetryHandler() { public boolean retryMethod( final HttpMethod method, final IOException exception, int executionCount) { return false; } } ); headMethod.setParams(params);
Created attachment 152115 [details] dialog I'm being pummeled with
what version are you running?
I'm seeing identical behavior in both 3.6M3 and 3.5.1. I troubleshooted the problem with 3.6M3.
I think this is for ECF.
Adding Henrich Kraemer. I'm not yet certain whether this behavior interacts with the p2 ui handling of file transfer cancellation, but it seems possible.
(In reply to comment #5) > Adding Henrich Kraemer. > > I'm not yet certain whether this behavior interacts with the p2 ui handling of > file transfer cancellation, but it seems possible. There is no file transfer to cancel out of at that point. The code is merely trying to establish a connection to the update site. The user is prompted for his SOCKS user/password, and if he puts the wrong password or wants to cancel out altogether, he's going to be prompted 8X the number of update sites.
(In reply to comment #6) > (In reply to comment #5) > > Adding Henrich Kraemer. > > > > I'm not yet certain whether this behavior interacts with the p2 ui handling of > > file transfer cancellation, but it seems possible. > > There is no file transfer to cancel out of at that point. The code is merely > trying to establish a connection to the update site. The user is prompted for > his SOCKS user/password, and if he puts the wrong password or wants to cancel > out altogether, he's going to be prompted 8X the number of update sites. When I said 'file transfer cancellation', all I meant is the cancellation for the authentication step as you describe above. This authentication is part of the ECF file transfer API (even though it's not the actual file transfer as you point out).
If you have a stack trace handy to post that would help speedup investigation. Thanks Henrich
(In reply to comment #8) > If you have a stack trace handy to post that would help speedup investigation. > > Thanks Henrich The situation is rather complex. A call stack won't get you very far. I can do better, though: Reproducibility steps: *** Requires access to a SOCKS server. *** 1. Launch eclipse 2. Ensure you have at least one non-local update site configured in Windows > Preferences > General > Install/Update > Available Software Sites 3. In Windows > Preferences > General > Network Connections, set Active Provider to Manual and configure the proxy server description. Don't include the authentication information 4. Go to Help > Install New Software... 5. Authentication dialog seen in attached screenshot comes up. 6. Hit cancel Dialog comes back up again and will do so seven more times. If you have two update sites, it will comes up 15 more times, etc. I am also preparing a patch that has alleviated the problem for me.
Created attachment 152175 [details] Solution that alleviates the problem for me
(In reply to comment #10) > Created an attachment (id=152175) [details] > Solution that alleviates the problem for me Switching off HttpClient retries in general might have unwanted effects for general robustness. I would like to see whether this can be done more selectively. At the moment I am having unrelated trouble setting me up for reproducing this. Thanks for the patch and repro steps.
(In reply to comment #9) .. > Reproducibility steps: > *** Requires access to a SOCKS server. *** ... > 4. Go to Help > Install New Software... > 5. Authentication dialog seen in attached screenshot comes up. > 6. Hit cancel > Dialog comes back up again and will do so seven more times. If you have two > update sites, it will comes up 15 more times, etc. > > I am also preparing a patch that has alleviated the problem for me. I do not see the authentication dialog you are seeing. This is with Eclipse SDK 3.5.1. Thoughts?
(In reply to comment #12) > (In reply to comment #9) > .. > > Reproducibility steps: > > *** Requires access to a SOCKS server. *** > ... > > 4. Go to Help > Install New Software... > > 5. Authentication dialog seen in attached screenshot comes up. > > 6. Hit cancel > > Dialog comes back up again and will do so seven more times. If you have two > > update sites, it will comes up 15 more times, etc. > > > > I am also preparing a patch that has alleviated the problem for me. > > I do not see the authentication dialog you are seeing. This is with Eclipse SDK > 3.5.1. > > Thoughts? 3.5.1 and 3.6(M3). Strange. Try the following: 1. Only define a SOCKS proxy (no http, or https) 2. Make sure the definition is a 'manual' one (rather than 'native') 3. In the definition (in the global prefs), make sure you do not include the authentication info. If after all that, you still do not get an authentication dialog when you go to Help > Install New Software..., we have quite a mystery.
> Strange. Try the following: > 1. Only define a SOCKS proxy (no http, or https) > .. Yes this makes a difference! After clearing manual credential data for http and https I can observe the issue. With a BP on java.net.Authenticator.setDefault(Authenticator) one can see, that the default java.net.Authenticator (a static variable) is set initially to org.eclipse.ui.internal.net.auth.NetAuthenticator when org.eclipse.ui.internal.ide.application.IDEWorkbenchAdvisor.postStartup() executes: java.net.Authenticator.setDefault(java.net.Authenticator) line: 117 org.eclipse.core.internal.net.ProxyManager.registerAuthenticator() line: 384 org.eclipse.core.internal.net.ProxyManager.initialize() line: 283 org.eclipse.core.internal.net.Activator.start(org.osgi.framework.BundleContext) line: 179 ... org.eclipse.ui.internal.ide.application.IDEWorkbenchAdvisor.activateProxyService() line: 258 org.eclipse.ui.internal.ide.application.IDEWorkbenchAdvisor.postStartup() line: 238 I had already set the proxy preferences in a previous session. So next I selecting menu item Help|Install New Software .. And observed that the socks proxy is filtered out because the type is not "SOCKS". The type is derived from the URL scheme which is "http" for "http://download.eclipse.org/eclipse/updates/3.5/compositeContent.jar". This happens in: org.eclipse.core.internal.net.ProxyManager.getProxyDataForHost(java.lang.String, java.lang.String) line: 374 org.eclipse.core.internal.net.ProxyManager.select(java.net.URI) line: 560 org.eclipse.ecf.provider.filetransfer.httpclient.HttpClientFileSystemBrowser(org.eclipse.ecf.provider.filetransfer.browse.AbstractFileSystemBrowser).setupProxies() line: 245 org.eclipse.ecf.provider.filetransfer.httpclient.HttpClientFileSystemBrowser.runRequest() line: 166 org.eclipse.ecf.provider.filetransfer.browse.AbstractFileSystemBrowser$DirectoryJob.run(org.eclipse.core.runtime.IProgressMonitor) line: 71 As a consequence no proxy server is selected as far as ECF is concerned. If a proxy was selected it seems that ultimately this would change the default authenticator to be set org.eclipse.ecf.provider.filetransfer.util.JREProxyHelper.setupProxy(Proxy). It appears that this would be problematic as there can only be one Authenticator in the system. But the JRE URLConnection based ECF transfer providers seem to rely on this nonetheless. So perhaps this is the proper way it should work. I assume however that unsetting the global net authenticator will have bad consequences. Is someone has better information, please let me know! The code as written progresses to perform the browse operation resulting in the attached stack trace. As the NetAuthenticator does not have a cancel return value it return null as credentials. The SocksSocketImpl code at some point uses the system property 'user.name' and sends this without a password to the socks proxy serve, which fails the servers password check of course. As a consequence it throws: throw new SocketException("SOCKS : authentication failed"); This is where the fact the user pressed cancel gets lost! HttpClient currently repeats the request for 3 times as described. My best guess would be that ECF should have not filtered out the socks proxy. In addition it should probably register its own authenticator which which is used only for the current ECF requests, possibly using thread local variables to make that decision. If an authenticate call is made from a different context the original authenticator would be used. However there is no API to get the current Authenticator. This seems like quite a bit of work however. Perhaps someone on the proxy API team has a suggestion on how this could be properly implemented?
Created attachment 152361 [details] Stack trace as described
(In reply to comment #14) > My best guess would be that ECF should have not filtered out the socks proxy. I agree. In fact, if you look at my fix for 295030, you'll see that what I do is use the SOCKS proxy regardless of the URL scheme if a SOCKS proxy has been configured. So, really, what you are observing here is why SOCKS support is just outright broken (295030) > In addition it should probably register its own authenticator which which is > used only for the current ECF requests, possibly using thread local variables > to make that decision. > If an authenticate call is made from a different context the original > authenticator would be used. However there is no API to get the current > Authenticator. > This seems like quite a bit of work however. Yeah. Mucking with the system-wide *default* authenticator seems like trouble. But I have to think there's probably a way to provide a custom authenticator for a particular operation. I'll dig into this some more and see if that makes sense.
Henrich, I've dug into this some more and I'm not sure how providing a custom authenticator would help solve the problem. It's not only java.net.Authenticator that lacks the concept of the user wanting to cancel the operation; SocksSocketImpl is equally oblivious. In other words, SocksSocketImpl calls into the authenticator by invoking Authenticator.requestPasswordAuthentication(). That simply returns a java.net.PasswordAuthentication object, which has no concept of a 'cancel'. Thus, you can register a custom authenticator that is able to detect and wants to inform its caller that the operation was canceled, but there's nothing in the caller (java.net.SocksSocketImpl#authenticate()) that will allow it do so do. So it seems to me the only solution lies somewhere in the ability to provide a custom retry handler to the apache layer so that it doesn't keep prompting. This is what my solution does, but as you noted, it may not be targeted enough.
(In reply to comment #17) The eclipse authenticator could remember that user pressed cancel and provide API or provide a listener interface which provides this information. This would allow ECF to determine to react to cancel properly. Unless the proxy team suggests a different solution for handling cancellation I would think that there should be an RFE for this. In the absence of this I think a focused switching off retries would give some relief and not cause harm. However it is not a full solution as the ECF caller will not be aware of this an may retry itself. The problem occurs even if one does provide the correct SOCKS credentials in the proxy dialogs (see comment 9, step 3)
(In reply to comment #18) > (In reply to comment #17) > The eclipse authenticator could remember that user pressed cancel and provide > API or provide a listener interface which provides this information. > This would allow ECF to determine to react to cancel properly. Hm. That sounds rather messy. Not sure there is an ideal solution here, but to be honest, that one sounds complicated and fragile to me. > The problem occurs even if one does provide the correct SOCKS credentials in > the proxy dialogs (see comment 9, step 3) Again, that's bugzilla 295030. I've submitted a patch for that.
Created attachment 152430 [details] Patch filtering out retries of SOCKS authentication errors The attached patch mitigates the issue, but does not solve it. The patch acts on the particular message of the SocketException which is certainly not API. There is a chance that the bug would reemerge if the JRE implementation changed these strings. This appears quite unlikely to me however.
Filtering on the exception string occurred to me, too, but it seemed far to fragile (way too much of a hack). Even if the strings don't change, you need to consider localization. Can we assume the exception string will always be in English?
Also note that the retry handler only halves the number of dialogs thrown at the user. The other part of my proposed solution is to not try getting the update-site jar if getting the xml encountered a SocketException. This sort of logic already existed for timeout. I believe it should be extended for SocketExceptions as well (which would include the SOCKS authentication failure)
One approach that we have experimented with in Mylyn is to use thread local to pass request specific information down to a global callback. We store a request object and progress monitor before calling into a library which can then be retrieved when the callback is invoked and the socket can be closed for instance if the request is cancelled. For the password prompt, wouldn't it make sense to only ever show one dialog at a time and block all other requests until the dialog has been dismissed?
(In reply to comment #23) > For the password prompt, wouldn't it make sense to only ever show one dialog at > a time and block all other requests until the dialog has been dismissed? In this scenario, only dialog *is* ever shown at a time. The "pummeling" happens sequentially.
In this scenario, only ONE dialog *is* ever shown at a time
(In reply to comment #21) > Filtering on the exception string occurred to me, too, but it seemed far to > fragile (way too much of a hack). Even if the strings don't change, you need to > consider localization. Can we assume the exception string will always be in > English? Hi John, When SocksSocketImpl throws SocketExceptions the strings are not localized. So it appears to me that one can rely on these messages being in English. This matches my experience with similar cases in another code base. Still it is a hack. A solution not requiring this would probably be better. I would recommend the patch only if another solution does not present itself or nobody can be found to go through the effort of implementing that.
(In reply to comment #23) > One approach that we have experimented with in Mylyn is to use thread local to > pass request specific information down to a global callback. We store a request > object and progress monitor before calling into a library which can then be > retrieved when the callback is invoked and the socket can be closed for > instance if the request is cancelled. Using thread locals can only solve the issue, but I think it will require changes in the proxy API.
(In reply to comment #26) > (In reply to comment #21) > > Filtering on the exception string occurred to me, too, but it seemed far to > > fragile (way too much of a hack). Even if the strings don't change, you need to > > consider localization. Can we assume the exception string will always be in > > English? > > Hi John, > > When SocksSocketImpl throws SocketExceptions the strings are not localized. So > it appears to me that one can rely on these messages being in English. This > matches my experience with similar cases in another code base. > > Still it is a hack. A solution not requiring this would probably be better. I > would recommend the patch only if another solution does not present itself or > nobody can be found to go through the effort of implementing that. Right. I saw that in the SocksSocketImpl. I'm just thinking that a future or different version of the Java runtime might localize such strings (they should be localized). But anyway, yes. I guess as a last resort, that will have to do. Having to hit cancel on tens of dialogs is certainly much less acceptable :-/
(In reply to comment #27) > (In reply to comment #23) > > One approach that we have experimented with in Mylyn is to use thread local to > > pass request specific information down to a global callback. We store a request > > object and progress monitor before calling into a library which can then be > > retrieved when the callback is invoked and the socket can be closed for > > instance if the request is cancelled. > > Using thread locals can only solve the issue, but I think it will require > changes in the proxy API. Henrich and John: Szymon and Pawel work on/with the proxy API and were recently added to this bug at my request. Would you please summarize (for them) the issues here with the proxy API and socks support?...and for all of us what you see as the options for addressing John's difficulties? Thanks.
(In reply to comment #29) > (In reply to comment #27) > > (In reply to comment #23) > > > One approach that we have experimented with in Mylyn is to use thread local to > > > pass request specific information down to a global callback. We store a request > > > object and progress monitor before calling into a library which can then be > > > retrieved when the callback is invoked and the socket can be closed for > > > instance if the request is cancelled. > > > > Using thread locals can only solve the issue, but I think it will require > > changes in the proxy API. > > Henrich and John: Szymon and Pawel work on/with the proxy API and were > recently added to this bug at my request. Would you please summarize (for > them) the issues here with the proxy API and socks support?...and for all of us > what you see as the options for addressing John's difficulties? Thanks. Well, there's a lot of detail, but in brief 1. SOCKS support simply doesn't work when used by ECF. But that's an ECF bug, not a proxy API one. That's a different bug 295030. 2. the user is hit with a ton of authentication dialogs when trying to cancel out of an ECF request when a SOCKS server is defined in Eclipse. The problem is ultimately a limitation in the java.net layer--it makes no distinction between a failed authentication attempt and a canceled one. Henrich thinks this may be solvable with a Proxy API change. Please refer to the long series of posts for the missing detail, or ask questions if you need additional insight.
My primary suggestions are in comment 18. This is based on the analysis in comment 14. The relevant portion regarding why I think the proxy team should weigh in here is >.. As the NetAuthenticator does not have a cancel return > value it return null as credentials. .. > This is where the fact the user pressed cancel gets lost! I think only the net authenticator can provide an API to get to the fact that user presses cancel. But perhaps the proxy API team sees a different way to implement this. Another possible mechanism to implemented this is suggested by Steffen in comment 23.
(In reply to comment #31) > My primary suggestions are in comment 18. > This is based on the analysis in comment 14. The relevant portion regarding why > I think the proxy team should weigh in here is > > >.. As the NetAuthenticator does not have a cancel return > > value it return null as credentials. .. > > This is where the fact the user pressed cancel gets lost! > > I think only the net authenticator can provide an API to get to the fact that > user presses cancel. > But perhaps the proxy API team sees a different way to implement this. NetAuthenticator does provide a return value for 'Cancel', it's simply a null value. The problem is that java.net doesn't handle this value appropriately. > Another possible mechanism to implemented this is suggested by Steffen in > comment 23. What about throwing a specialized runtime exception from NetAuthenticator? I've checked it and it propagates down to HttpClientRetrieveFileTransfer which is ECF.
> What about throwing a specialized runtime exception from NetAuthenticator? I've > checked it and it propagates down to HttpClientRetrieveFileTransfer which is > ECF. That sounds feasible from an ECF perspective. Would core.net plugin be upversioned so that clients can adjust? Alternatively I would think one could enable this behavior using some other API perhaps utilizing thread locals.
The version has already been bumped in 3.6 cycle.
This is a serious problem and discussions seem to have stalled again...
(In reply to comment #35) > This is a serious problem and discussions seem to have stalled again... Would someone that is familiar with what's going on (perhaps Pawel and/or Henrich ) please summarize where we are with this issue and bug 295030...so that next steps can be identified and accomplished? It's not at all clear to me what is left to do to address this and/or bug 295030 in time for Helios, and if it's not clear to me it's probably not clear to others as well.
(In reply to comment #36) > Would someone that is familiar with what's going on (perhaps Pawel and/or > Henrich ) Pawel, I was expecting you or someone on the core.net proxy service team to provide a cancel mechanism that ECF could leverage. Are you waiting for additional input? - Henrich
Pawel will be available on Monday, then he will comment on it.
Henrich, is the solution from comment 32 the way to go? If yes I'll implement the changes in the core.net project.
(In reply to comment #39) > Henrich, is the solution from comment 32 the way to go? If yes I'll implement > the changes in the core.net project. It appears this solution should work.
This issue seems to have stalled again.
Created attachment 163407 [details] core.ui.net_v01 A patch making OperationCanceledException is thrown after user cancels the dialog. While it makes easy for ECF to provide a fix it impacts other code leveraging the authenticator. I'll investigate using thread local variables, like Henrich suggested. Hope the same result can be achieved not impacting other clients.
Created attachment 163412 [details] core.ui.net_v02 Henrich, this is a solution with thread local variable. I haven't tested it though. Since it doesn't break clients I prefer this one.
(In reply to comment #43) > Henrich, this is a solution with thread local variable. I haven't tested it > though. Since it doesn't break clients I prefer this one. I am not sure how I would access the static method from within ECF. Perhaps it can be made available by the core net plugin perhaps via some addition to IProxyService?
(In reply to comment #44) You have to add the package to Import-package in the manifest file. The package is internal but is being exported from the bundle. Since API is frozen and it's a plain hack we are not likely to publish it as an API, at least not in 3.6. For now I can comment the code so it's not changed by accident.
(In reply to comment #45) > (In reply to comment #44) > > You have to add the package to Import-package in the manifest file. The package > is internal but is being exported from the bundle. > > Since API is frozen and it's a plain hack we are not likely to publish it as an > API, at least not in 3.6. For now I can comment the code so it's not changed by > accident. Take note of https://bugs.eclipse.org/bugs/show_bug.cgi?id=295030#c61 In particular the insight that "In the current state of the eclipse net authenticator (org.eclipse.ui.internal.net.auth.NetAuthenticator) is not usable as it prompts users repeatedly even if users provided the correct credentials to the prompt dialog or prior in the preference page." Therefore the plan at the moment is to have ECF set its own Authenticator which means that both proposed patches will not be needed. The linked comment also mentions that the net API should probably evolve to handle the central Authenticator and manage the credentials.
*** Bug 248787 has been marked as a duplicate of this bug. ***
I believe this got addressed in 3.6 as changes got done in p2 to serialize pwd prompting. if the pb still occurs please reopen or open a new bug. Thx.
