Bug 391023 - [JUnit] Support UI extensions for custom runners
Summary: [JUnit] Support UI extensions for custom runners
Status: ASSIGNED
Alias: None
Product: JDT
Classification: Eclipse Project
Component: UI (show other bugs)
Version: 4.3   Edit
Hardware: All All
: P3 enhancement with 2 votes (vote)
Target Milestone: ---   Edit
Assignee: JDT-UI-Inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords: helpwanted
Depends on:
Blocks:
 
Reported: 2012-10-03 10:50 EDT by Moritz Eysholdt CLA
Modified: 2019-05-01 03:52 EDT (History)
3 users (show)

See Also:


Attachments
runner_ui_extensionpoint_v01 (56.93 KB, application/zip)
2012-10-03 10:50 EDT, Moritz Eysholdt CLA
no flags Details
Example Screenshot for the theories runner (136.56 KB, image/png)
2012-10-03 10:54 EDT, Moritz Eysholdt CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Moritz Eysholdt CLA 2012-10-03 10:50:36 EDT
Created attachment 221840 [details]
runner_ui_extensionpoint_v01

I'm proposing to contribute an extension point which allows to customize the JUnit View for specific JUnit runner implementations. 

JUnit has this great concept of custom runners which are responsible for executing a set of test cases. This brings a lot of flexibility: The runner can load test data (from files, Java classes, etc.) and decide what a "test case" actually is (A Java method, a text snippet, etc.) and how often a test case should be invoked with which test data. 

However, the current JUnit view has some limitations:
- The test's label in the tree view must begin with the Java method's name
- Navigating to Java methods (double click on the tree item or context menu) only works when the Java method has no parameters.
- Navigation to test data (for example in *.csv files) is not possible.
- Visualization of test failures is restricted to stack traces and the diff dialog. 

The extensions point I'm proposing allows to register a handler class for a specific JUnit runner. This handler class may:
- specify the label of the test (-case/-suite) as shown in the tree view
- add items to the context menu for tree view items
- handle double-clicks on tree view items. 

This is an example for the "Theories" runner form JUnit itself (see also bug 200403):


-------- A JUnit test using theories ----------
@RunWith(Theories.class)
public class TheorieTest {

	@DataPoint
	public static String a = "a";

	@DataPoint
	public static String b = "bb";

	@DataPoint
	public static String c = "ccc";

	@Theory
	public void stringTest(String x, String y) {
		Assert.assertTrue(x.length() >= 1);
		System.out.println(x + " " + y);
	}
}
---------------------------------------------


---------- Example extension ----------------
<extension point="org.eclipse.jdt.junit.runners.junitRunnerUI">
  <junitRunnerUIHandler
    handlerClass="de.itemis.junit.theories.ui.TheoriesRunnerUIHandler"
    runnerClass="org.junit.experimental.theories.Theories">
  </junitRunnerUIHandler>
</extension>
----------------------------------------------


----------- Extension implementation ---------
public class TheoriesRunnerUIHandler implements IRunnerUIHandler {

  @Override
  public boolean contextMenuAboutToShow(ViewPart part, ITestElement element, IMenuManager menu) {
    if (element instanceof TestCaseElement) {
      menu.add(new OpenTestTheoryAction((TestRunnerViewPart) part, (TestCaseElement) element));
      return true;
    }
    return false;
  }

  @Override
  public boolean handleDoubleClick(ViewPart part, ITestElement element) {
    if (element instanceof TestCaseElement) {
      Action action = new OpenTestTheoryAction((TestRunnerViewPart) part, (TestCaseElement) element);
      if (action.isEnabled()) {
        action.run();
        return true;
      }
    }
    return false;
  }

  @Override
  public String getSimpleLabel(ViewPart part, ITestElement element) {
    if (element instanceof ITestCaseElement)
      return ((ITestCaseElement) element).getTestMethodName() + " theory";
    if (element instanceof ITestSuiteElement)
      return ((ITestSuiteElement) element).getSuiteTypeName();
    return "unknown";
  }

}
--------------------------------------------


The current implementation (see attached or at https://github.com/meysholdt/eclipse_jdt_junit_runners) is quite hacky (base on Java reflection). 

I'm posting this to see if an extension point like this is thinkable, to get feedback, and to gather with others interested in this feature.
Comment 1 Moritz Eysholdt CLA 2012-10-03 10:54:30 EDT
Created attachment 221842 [details]
Example Screenshot for the theories runner
Comment 2 Moritz Eysholdt CLA 2012-10-03 10:58:59 EDT
This extension would make it possible to solve the use cases of the following bugzillas via external plugins:

Bug 343935 - [JUnit] JUnit test case with customized Runner, can't locate the method when it contains parameters after running
Bug 200403 - [JUnit] Navigation support for JUnit theories
Bug 338770 - [JUnit] Custom Runners cannot communicate file information
Comment 3 Markus Keller CLA 2013-09-08 12:00:43 EDT
I agree JUnit 4 tests with custom runners need more flexibility, and I agree with the set of requested features.

We can consider such a contribution, but as you said, it would need quite some cleanup:

- Its should be possible to write a UI extension without resorting to reflection and without accessing internal APIs. Implementations should use ITestElement and its subinterfaces. If more access is necessary (e.g. to open an element in an editor), then this needs to be added as a proper API.

- The Eclipse SDK usually doesn't ship extension points / APIs it doesn't consume, so we should ship at least one working implementation that doesn't rely on internal jdt.junit classes (e.g. a cleaned-up version of the UI for Theories).
Comment 4 Moritz Eysholdt CLA 2013-09-09 07:53:13 EDT
hi Markus,

thank you for the feedback. This is still relevant for me, since Xpect depends on it. I'd be willing to work on a contribution once I find the time for it.

See:
http://www.xpect-tests.org
http://blog.moritz.eysholdt.de/2013/09/introduction-to-xpect.html
http://www.eclipsecon.org/europe2013/executable-specifications-xtext-languages
Comment 5 Dani Megert CLA 2013-09-09 07:59:30 EDT
(In reply to Moritz Eysholdt from comment #4)
> hi Markus,
> 
> thank you for the feedback. This is still relevant for me, since Xpect
> depends on it. I'd be willing to work on a contribution once I find the time
> for it.
> 

Let us know when you start.