Bug 422220 - VerifyTests that use Util#getFreePort() occasionally fail with "Address already in use at java.net.PlainSocketImpl.socketBind"
Summary: VerifyTests that use Util#getFreePort() occasionally fail with "Address alrea...
Status: VERIFIED FIXED
Alias: None
Product: JDT
Classification: Eclipse Project
Component: Core (show other bugs)
Version: 4.4   Edit
Hardware: PC Linux
: P2 normal (vote)
Target Milestone: 4.7 M7   Edit
Assignee: Stephan Herrmann CLA
QA Contact:
URL: http://download.eclipse.org/eclipse/d...
Whiteboard:
Keywords:
: 442450 446261 469074 511148 (view as bug list)
Depends on:
Blocks:
 
Reported: 2013-11-21 03:56 EST by shankha banerjee CLA
Modified: 2017-05-10 02:48 EDT (History)
7 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description shankha banerjee CLA 2013-11-21 03:56:12 EST
BUILD ID: N20131120-2000

http://download.eclipse.org/eclipse/downloads/drops4/N20131120-2000/testresults/html/org.eclipse.jdt.core.tests.compiler_linux.gtk.x86_6.0.html


Unexpected error running resulting class file for testFallThrough.java: --[START]-- Exception in thread "main" java.net.BindException: Address already in use at java.net.PlainSocketImpl.socketBind(Native Method) at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376) at java.net.ServerSocket.bind(ServerSocket.java:376) at java.net.ServerSocket.<init>(ServerSocket.java:237) at java.net.ServerSocket.<init>(ServerSocket.java:128) at org.eclipse.jdt.core.tests.util.VerifyTests.run(VerifyTests.java:201) at org.eclipse.jdt.core.tests.util.VerifyTests.main(VerifyTests.java:198) ---[END]---

junit.framework.AssertionFailedError: Unexpected error running resulting class file for testFallThrough.java:
--[START]--
Exception in thread "main" java.net.BindException: Address already in use
at java.net.PlainSocketImpl.socketBind(Native Method)
at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376)
at java.net.ServerSocket.bind(ServerSocket.java:376)
at java.net.ServerSocket.<init>(ServerSocket.java:237)
at java.net.ServerSocket.<init>(ServerSocket.java:128)
at org.eclipse.jdt.core.tests.util.VerifyTests.run(VerifyTests.java:201)
at org.eclipse.jdt.core.tests.util.VerifyTests.main(VerifyTests.java:198)
---[END]---

at org.eclipse.jdt.core.tests.compiler.regression.AbstractRegressionTest.runTest(AbstractRegressionTest.java:2344)
at org.eclipse.jdt.core.tests.compiler.regression.AbstractRegressionTest.runTest(AbstractRegressionTest.java:2120)
at org.eclipse.jdt.core.tests.compiler.regression.AbstractRegressionTest.runConformTest(AbstractRegressionTest.java:1191)
at org.eclipse.jdt.core.tests.compiler.regression.SwitchTest.testFallThrough2(SwitchTest.java:1792)
at org.eclipse.jdt.core.tests.util.CompilerTestSetup.run(CompilerTestSetup.java:55)
at org.eclipse.test.EclipseTestRunner.run(EclipseTestRunner.java:657)
at org.eclipse.test.EclipseTestRunner.run(EclipseTestRunner.java:310)
at org.eclipse.test.CoreTestApplication.runTests(CoreTestApplication.java:36)
at org.eclipse.test.CoreTestApplication.run(CoreTestApplication.java:32)
at org.eclipse.equinox.internal.app.EclipseAppContainer.callMethodWithException(EclipseAppContainer.java:587)
at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:198)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:109)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:80)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:372)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:226)
at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:636)
at org.eclipse.equinox.launcher.Main.basicRun(Main.java:591)
at org.eclipse.equinox.launcher.Main.run(Main.java:1450)
at org.eclipse.equinox.launcher.Main.main(Main.java:1426)
at org.eclipse.core.launcher.Main.main(Main.java:34)
Comment 1 Dani Megert CLA 2013-11-21 09:46:13 EST
See also bug 414653.
Comment 2 Markus Keller CLA 2014-10-28 11:35:12 EDT
*** Bug 442450 has been marked as a duplicate of this bug. ***
Comment 3 Markus Keller CLA 2014-10-28 11:35:24 EDT
*** Bug 446261 has been marked as a duplicate of this bug. ***
Comment 4 Markus Keller CLA 2014-10-28 11:54:20 EDT
This is a fundamental design problem in all tests that rely on org.eclipse.jdt.core.tests.util.Util#getFreePort(). That method returns a port number that *was* free when the call returned, but there's no guarantee that it stays free until you actually start another ServerSocket with that port.

A decent OS tries to avoid using the same temporary port number in quick succession, but that's not always easy to achieve and is not guaranteed.


There are two possible ways to fix this (neither is simple):

a) Change all socket servers to clients and all clients to servers. That way, you can open a new ServerSocket(0) and keep it open until the test is done.

b) Don't pass a server port number as command line argument, but generate it at the place where the ServerSocket is opened, and then somehow pass the chosen port back to the test. Either via System.out/err, or via another "long-running" socket that gets opened on the test vm and to which clients can connect. That way, you effectively pass open client port numbers (in both directions), but not temporary server port numbers.
Comment 5 Markus Keller CLA 2014-10-28 11:59:10 EDT
FYI: JUnit launch configurations have a similar problem, see bug 322266.
Comment 6 Stephan Herrmann CLA 2015-06-02 03:44:14 EDT
*** Bug 469074 has been marked as a duplicate of this bug. ***
Comment 7 Dani Megert CLA 2015-06-04 03:49:40 EDT
AnnotationTest.test190 failed with same problem (also Linux).

We should get rid of those random test failures. They cause distractions.
Comment 8 Jay Arthanareeswaran CLA 2015-06-04 04:17:52 EDT
(In reply to Dani Megert from comment #7)
> AnnotationTest.test190 failed with same problem (also Linux).
> 
> We should get rid of those random test failures. They cause distractions.

The problem is with the test infrastructure, which means this could theoretically happen with any test using it. We should try out the options Markus suggested during 4.6.
Comment 9 Manoj N Palat CLA 2015-08-18 23:56:42 EDT
(In reply to Jay Arthanareeswaran from comment #8)
> (In reply to Dani Megert from comment #7)
> > AnnotationTest.test190 failed with same problem (also Linux).
> > 
> > We should get rid of those random test failures. They cause distractions.
> 
> The problem is with the test infrastructure, which means this could
> theoretically happen with any test using it. We should try out the options
> Markus suggested during 4.6.

Jay: are we targeting this for 4.5.1?
Comment 10 Jay Arthanareeswaran CLA 2015-08-19 01:12:32 EDT
(In reply to Manoj Palat from comment #9)
> Jay: are we targeting this for 4.5.1?

Sorry, couldn't spend time on this. Moving out of 4.5.1 plan. Fortunately for us, failures haven't surfaced for a while now.
Comment 11 Jay Arthanareeswaran CLA 2016-04-05 04:34:28 EDT
No progress yet and unlikely to get time during 4.6. Moving out.
Comment 12 Stephan Herrmann CLA 2017-01-27 18:15:33 EST
*** Bug 511148 has been marked as a duplicate of this bug. ***
Comment 13 Stephan Herrmann CLA 2017-02-09 15:54:10 EST
In my understanding this bug renders Gerrit fairly useless these days.

Jay, should I give it a try? I'd rather fix this than busy myself clicking retrigger on gerrit and spend the rest of the day waiting for test results :-X
Comment 14 Eclipse Genie CLA 2017-02-09 18:04:45 EST
New Gerrit change created: https://git.eclipse.org/r/90785
Comment 15 Stephan Herrmann CLA 2017-02-09 18:10:22 EST
(In reply to Eclipse Genie from comment #14)
> New Gerrit change created: https://git.eclipse.org/r/90785

I gave a quick try to Markus's idea (a):

(In reply to Markus Keller from comment #4)
> a) Change all socket servers to clients and all clients to servers. That
> way, you can open a new ServerSocket(0) and keep it open until the test is
> done.


Implementing this was easier than I thought.
@Markus, when you said it's not simple, did you have any specific problems in mind?

I only tried it for VerifyTests, none of the other 4 clients of Util.getFreePort()
Comment 16 Dani Megert CLA 2017-02-10 11:47:21 EST
(In reply to Stephan Herrmann from comment #15)
> @Markus, when you said it's not simple, did you have any specific problems
> in mind?

FYI: Markus is on vacation. Back February 20.
Comment 17 Stephan Herrmann CLA 2017-03-06 19:13:39 EST
(In reply to Dani Megert from comment #16)
> (In reply to Stephan Herrmann from comment #15)
> > @Markus, when you said it's not simple, did you have any specific problems
> > in mind?
> 
> FYI: Markus is on vacation. Back February 20.

I think it's past Feb 20 :)

so ...

(In reply to Stephan Herrmann from comment #15)
> (In reply to Eclipse Genie from comment #14)
> > New Gerrit change created: https://git.eclipse.org/r/90785
> 
> I gave a quick try to Markus's idea (a):
> 
> (In reply to Markus Keller from comment #4)
> > a) Change all socket servers to clients and all clients to servers. That
> > way, you can open a new ServerSocket(0) and keep it open until the test is
> > done.
> 
> 
> Implementing this was easier than I thought.
> @Markus, when you said it's not simple, did you have any specific problems
> in mind?
> 
> I only tried it for VerifyTests, none of the other 4 clients of
> Util.getFreePort()


@Markus, do you recall why you said "neither is simple"?

The change in https://git.eclipse.org/r/90785 doesn't look scary to me. Did I miss something?
Comment 18 Markus Keller CLA 2017-03-06 19:59:32 EST
(In reply to Stephan Herrmann from comment #15)
> > a) Change all socket servers to clients and all clients to servers.
> 
> Implementing this was easier than I thought.
> @Markus, when you said it's not simple, did you have any specific problems
> in mind?

None that I would remember. Usually, you only see the real difficulties when you try to implement a simple idea. Looks like this time, it was the other way 'round.
Comment 19 Markus Keller CLA 2017-03-06 20:07:26 EST
I probably remembered that bug 322266 was not easy to fix. However, the difficulty in that bug is not the client <-> server exchange, but an API boundary that makes it difficult to pass a ServerSocket instead of port number.

(In reply to Stephan Herrmann from comment #17)
> The change in https://git.eclipse.org/r/90785 doesn't look scary to me. Did
> I miss something?

Doesn't scare me either. Go for it, if it works! (I didn't test.)
Comment 21 Stephan Herrmann CLA 2017-04-04 11:25:34 EDT
(In reply to Eclipse Genie from comment #20)
> Gerrit change https://git.eclipse.org/r/90785 was merged to [master].
> Commit:
> http://git.eclipse.org/c/jdt/eclipse.jdt.core.git/commit/?id=f77df400e3009bbbdecb497b75bcd116844d089e
> 

Released for 4.7 M7: swapped server/client for most connections, except for those  relating to "-Xrunjdwp:.." etc.
Comment 22 Sasikanth Bharadwaj CLA 2017-05-10 02:48:46 EDT
Verified for 4.7 M7 using I20170508-2000 build