Bug 474304 - support JUnit 4 style performance tests
Summary: support JUnit 4 style performance tests
Status: REOPENED
Alias: None
Product: Platform
Classification: Eclipse Project
Component: Runtime (show other bugs)
Version: 4.5   Edit
Hardware: PC Linux
: P3 enhancement (vote)
Target Milestone: ---   Edit
Assignee: platform-runtime-inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2015-08-05 06:07 EDT by Lars Vogel CLA
Modified: 2019-12-01 15:47 EST (History)
4 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Lars Vogel CLA 2015-08-05 06:07:17 EDT
I'm sorry but I'm still not familiar with the setup of the performance tests. In Platform UI runtime I have ContextPerformanceTest.java. 

I would like to migrate that test to JUni 4. Is that supported by our performance test infrastructure? If yes, what do I need to do?
Comment 1 David Williams CLA 2016-04-11 14:33:37 EDT
We run them just like the JUnit tests, so shouldn't matter (as far as I know)  but the "performance" flag turned on (and performance capturing bundles/fragments installed). 

If you already have performance tests, just make sure it's part of the suite that's named in the test.xml file. 

The "performance flag" is what causes our framework to execute the target in your test.xml that looks similar to the following (this example is from 
org.eclipse.ui.tests.performance)

  <target name="performance" depends="init,performance-suite,jface-performance-suite,cleanup">
    <ant target="collect" antfile="${library-file}" dir="${eclipse-home}">
      <property name="includes" value="org*.xml"/>
      <property name="output-file" value="${plugin-name}.xml"/>
    </ant>
  </target>

If the code is not yet "instrumented" at all, you will need to read 
https://wiki.eclipse.org/Performance/Automated_Tests 
and decide how you want to instrument it. (I'd start off minimally :) 

HTH
Comment 2 Lars Vogel CLA 2016-04-11 14:37:34 EDT
Thanks David.
Comment 3 Thomas Watson CLA 2016-04-11 15:11:48 EDT
I don't think this is a releng bug.  Or perhaps I misunderstand what Lars asked.

Eventually we will have to remove the usage of junit.framework junit 3 type packages (https://wiki.eclipse.org/JUnit4Migration)

But the PerformanceTestRunner which gets used by (all of?) the performance tests takes the junit 3 API junit.framework.TestCase in its run methods:

org.eclipse.core.tests.harness.PerformanceTestRunner.run(TestCase, String, int, int)
Comment 4 Thomas Watson CLA 2016-04-11 15:13:26 EDT
There are similar issues with our session test harness code:

org.eclipse.core.tests.session.SessionTestSuite
org.eclipse.core.tests.session.SessionTestRunner
Comment 5 David Williams CLA 2016-04-11 16:23:36 EDT
Thanks for the clarification and education, Tom. I've been confused about this before. :) (Since "TestCase" is part of JUnit 4 -- I need things spelled out for me -- at least in areas I that I do not follow closely) . 

So, I do think it is a valid enhancement request since it could be interpreted as meaning provide support for "the new 4.0 (only) JUnit-Style of tests" (if that is the right wording .. if not, please correct the title) And we'll carry forward as that sort of request).
Comment 6 Markus Keller CLA 2016-04-15 13:15:23 EDT
(In reply to Thomas Watson from comment #3)
> Eventually we will have to remove the usage of junit.framework junit 3 type
> packages (https://wiki.eclipse.org/JUnit4Migration)

No, that was false information. Sopot has not given any evidence why JUnit 3 tests wouldn't run in JUnit 5.

JUnit 4 includes JUnit 3, so even if JUnit 5 wouldn't include direct support for JUnit 3, those tests could still be run using the JUnit 4 runner. There's no need to migrate any tests.
Comment 7 Lars Vogel CLA 2016-04-16 01:55:33 EDT
(In reply to Markus Keller from comment #6)
> There's no need to migrate any tests.

Except to take advantage of any new functionality in JUnit and to attract contributors, which typically want to use latest and greatest. 

I had once one student working for me and after I showed him our platform UI JUnit3 tests, he said "these are not test, the annotations are missing". That was my trigger to start migrating the platform UI tests.