Bug 559160 - JUnit 5 tests are executed from other projects
Summary: JUnit 5 tests are executed from other projects
Status: NEW
Alias: None
Product: JDT
Classification: Eclipse Project
Component: UI (show other bugs)
Version: 4.14   Edit
Hardware: Macintosh Mac OS X
: P3 normal (vote)
Target Milestone: ---   Edit
Assignee: JDT-UI-Inbox CLA
QA Contact:
URL:
Whiteboard: stalebug
Keywords:
Depends on:
Blocks:
 
Reported: 2020-01-14 08:58 EST by Sam Brannen CLA
Modified: 2022-10-29 16:44 EDT (History)
4 users (show)

See Also:


Attachments
Example projects (3.45 KB, application/zip)
2020-01-14 08:58 EST, Sam Brannen CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Sam Brannen CLA 2020-01-14 08:58:06 EST
Created attachment 281489 [details]
Example projects

Actual Behavior:

Given two Java projects that both use JUnit 5 (see example projects in the attached Archive.zip), if project 2 depends on project 1 (via a classpath entry), then tests from project 1 and project 2 are executed when selecting "Run as / JUnit test" on the test folder in project 2.

For example, if you right-click on the test folder in project 1 and select "Run as / JUnit test", you will see one failure in Project1Tests. Whereas, if you do the same thing in project 2 you will see two failures, one in Project1Tests and one in Project2Tests.

I believe, however, that the above undesired behavior only occurs if project 1 contains tests that reside in the same package as tests that reside in project 2.


Expected Behavior:

Executing all tests in the test folder in project 2 results in only test classes from project 2 being executed.


Assumptions:

I assume that Eclipse is selecting packages from the test folder in project 2 and passing those to the JUnit Platform (e.g., via a PackageSelector), which in turn performs classpath scanning within each of those packages and their subpackages. Thus, the problem is likely that the test folder from project 1 is on the classpath used to launch the tests for project 2, which should not be the case.

Ideally, the classpath used to launch tests for a given project should never include test source folders from other projects, or there should be a mechanism for opting out of this behavior.

Note that setting the "without_test_code" classpath attribute to "true" in the classpath entry from project 2 to project 1 does not appear to have any effect.
Comment 1 Marc Philipp CLA 2020-01-14 12:01:10 EST
If Eclipse does indeed use `PackageSselectors` which I haven't checked, it should instead use `ClasspathRootSelectors` via `DiscoverySelectors.selectClasspathRoots()` and pass the test outputs folders of the selected project.
Comment 2 Eclipse Genie CLA 2022-10-29 16:44:24 EDT
This bug hasn't had any activity in quite some time. Maybe the problem got resolved, was a duplicate of something else, or became less pressing for some reason - or maybe it's still relevant but just hasn't been looked at yet.

If you have further information on the current state of the bug, please add it. The information can be, for example, that the problem still occurs, that you still want the feature, that more information is needed, or that the bug is (for whatever reason) no longer relevant.

--
The automated Eclipse Genie.