Comment 1 Markus Keller 2014-06-04 08:10:52 EDT

We already got test failures in TypeHierarchyTests earlier, see e.g. bug 365180.

The problem is that some tests create resources they don't delete again. I can reliably reproduce such failures using

public static Test suite() {
	return buildModelTestSuite(TypeHierarchyTests.class, ALPHABETICAL_SORT);
}

Comment 2 Jay Arthanareeswaran 2014-06-04 08:54:47 EDT

(In reply to Markus Keller from comment #1)
> We already got test failures in TypeHierarchyTests earlier, see e.g. bug
> 365180.
> 
> The problem is that some tests create resources they don't delete again. I
> can reliably reproduce such failures using
> 
> public static Test suite() {
> 	return buildModelTestSuite(TypeHierarchyTests.class, ALPHABETICAL_SORT);
> }

Indeed! Thanks Markus! I guess at this point the right thing to do is to force the order by adding individual tests to the suite like we do in ClasspathTests and other suites.

Comment 3 Jay Arthanareeswaran 2014-06-04 09:05:46 EDT

Created attachment 243923 [details]
Fix to hardcode the order of tests

This is a test only change. I suppose I can release this already? Or any approval required?

Comment 4 Dani Megert 2014-06-04 09:08:26 EDT

(In reply to Jayaprakash Arthanareeswaran from comment #3)
> Created attachment 243923 [details]
> Fix to hardcode the order of tests
> 
> This is a test only change. I suppose I can release this already? Or any
> approval required?

Can't you use org.eclipse.test.OrderedTestSuite?

Comment 5 Jay Arthanareeswaran 2014-06-04 09:14:26 EDT

(In reply to Dani Megert from comment #4)
> Can't you use org.eclipse.test.OrderedTestSuite?

I could've. But I just checked and figured that one also requires the test methods to be specified. Looks like not much different.

Comment 6 Dani Megert 2014-06-04 09:19:29 EDT

(In reply to Jayaprakash Arthanareeswaran from comment #5)
> (In reply to Dani Megert from comment #4)
> > Can't you use org.eclipse.test.OrderedTestSuite?
> 
> I could've. But I just checked and figured that one also requires the test
> methods to be specified.

No, see org.eclipse.test.OrderedTestSuite.OrderedTestSuite(Class).

Comment 7 Jay Arthanareeswaran 2014-06-04 10:03:53 EDT

(In reply to Dani Megert from comment #6)
> No, see org.eclipse.test.OrderedTestSuite.OrderedTestSuite(Class).

The org.eclipse.test.performance in my workspace was begging for a pull with hundred of incoming commits :)

After the pull I see the constructor. But looks like the TypeHierarchyTests is wired bit differently - it extends SuiteOfTestCases which extends org.eclipse.jdt.core.tests.junit.extension.TestCase.

I can workaround this by creating setUp() and tearDown() to call the existing  setupSuite() and tearDownSuite() respectively. But for some reason I am finding the tests (each taking about 2 sec.) taking a lot of time. Obviously something's not right. But I guess we can take this up at a later time can't we. For now, I will go with the patch from comment #3.

Comment 8 Jay Arthanareeswaran 2014-06-04 11:13:12 EDT

I have released the test here:

http://git.eclipse.org/c/jdt/eclipse.jdt.core.git/commit/?id=b76e53a6fc7ba1b5a583615524c0957038a52ce1

with a note so we can get rid of the manual addition of test cases. Either we should use OrderedTestSuite or even better make sure the order of execution doesn't matter - i.e. tests should clean up what they create.

Comment 9 Dani Megert 2014-06-05 07:12:31 EDT

Test passed in I20140604-2000.

Comment 10 Markus Keller 2014-06-13 12:12:17 EDT

*** Bug 365180 has been marked as a duplicate of this bug. ***