Bug 278844 - [JUnit] Separate UI from non-UI code
Summary: [JUnit] Separate UI from non-UI code
Status: VERIFIED FIXED
Alias: None
Product: JDT
Classification: Eclipse Project
Component: UI (show other bugs)
Version: 3.5   Edit
Hardware: All All
: P2 enhancement (vote)
Target Milestone: 3.6 M2   Edit
Assignee: Achim Demelt CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks: 263547 243293 278845
  Show dependency tree
 
Reported: 2009-06-02 14:32 EDT by Achim Demelt CLA
Modified: 2009-09-21 14:37 EDT (History)
14 users (show)

See Also:
daniel_megert: pmc_approved+
markus.kell.r: review+


Attachments
Patch, first draft (551.73 KB, patch)
2009-06-17 10:15 EDT, Achim Demelt CLA
no flags Details | Diff
Now uses IStatusHandler to display message dialog (8.31 KB, patch)
2009-06-17 12:02 EDT, Achim Demelt CLA
no flags Details | Diff
Updated patch (540.32 KB, patch)
2009-07-10 07:46 EDT, Achim Demelt CLA
no flags Details | Diff
Updated patch (540.41 KB, patch)
2009-07-26 13:55 EDT, Achim Demelt CLA
no flags Details | Diff
Updated patch (541.11 KB, patch)
2009-08-11 02:35 EDT, Achim Demelt CLA
markus.kell.r: iplog+
Details | Diff
Patch 6 (477.67 KB, patch)
2009-08-20 12:54 EDT, Markus Keller CLA
no flags Details | Diff
Fixes for concurrency issues (16.01 KB, patch)
2009-08-26 13:26 EDT, Achim Demelt CLA
markus.kell.r: iplog+
Details | Diff
Final Patch (498.26 KB, patch)
2009-09-02 10:43 EDT, Markus Keller CLA
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Achim Demelt CLA 2009-06-02 14:32:53 EDT
Currently, the org.eclipse.jdt.junit bundle contains both UI and non-UI code. This prevents using the JUnit launcher code in a headless context and is basically a blocker for a decent JUnit integration with Buckminster. See this discussion in the buckminster-dev newsgroup: http://dev.eclipse.org/mhonarc/lists/buckminster-dev/msg00759.html
Comment 1 Dani Megert CLA 2009-06-03 02:40:40 EDT
The headless part is org.junit ;-)

org.eclipse.jdt.junit is designed to be the UI part inside Eclipse. What's the code you need from it?
Comment 2 Achim Demelt CLA 2009-06-03 03:32:31 EDT
It's the JUnit launching stuff in JUnitLaunchConfigurationDelegate (and everything that goes with it). This could be used in a headless workbench. The UI code in jdt.junit, however, prevents this use case.

The alternative would be to basically duplicate the launching code within the Buckminster project. While this _may_ be feasible for JDT's JUnit integration, duplicating PDE's JUnit stuff is not a good idea. Since the 3.6 cycle is about to begin, now seemed a good opportunity to raise this issue.

I'd be willing to help out with this one. Hints and advice appreciated :-)
Comment 3 Dani Megert CLA 2009-06-03 03:35:08 EDT
Creating a separate plug-in for just one class is really overkill.
Comment 4 Thomas Hallgren CLA 2009-06-03 03:38:36 EDT
I think the problem is much more complex then that. Creating a separate plugin in JDT for this is just one part of the solution. It is a prerequisite for making a lot more things in PDE headless.

Please reconsider the WONTFIX status on this one.
Comment 5 Dani Megert CLA 2009-06-03 03:41:10 EDT
If PDE has so much more code and it's justified to make a separate pde core plug-in then they can simply duplicate that class in there.
Comment 6 Thomas Hallgren CLA 2009-06-03 03:44:33 EDT
I've had my fair share of problems with duplicated code in Eclipse lately so I must say, that is a solution that doesn't sound very appealing.

If a new bundle is overkill (although I don't really understand the motivation behind that), then perhaps the class could be moved to one of the bundles that doesn't have any UI dependencies?

I would really like to use this for other purposes then PDE b.t.w. so PDE is not the only place that would need a copy.
Comment 7 Achim Demelt CLA 2009-06-03 05:09:18 EDT
The JUnitLaunchConfigurationDelegate alone is not self-sufficient. It depends on a whole bunch of other classes. Basically, most of the contents of the following packages would be required in a separate bundle:

org.eclipse.jdt.internal.junit
org.eclipse.jdt.internal.junit.launcher
org.eclipse.jdt.internal.junit.model
org.eclipse.jdt.internal.junit.util
org.eclipse.jdt.junit
org.eclipse.jdt.junit.launcher
org.eclipse.jdt.junit.model

This is just my first glance. I guess we'd need to move some more classes, but the above packages should cover the bulk of it.

So it's not just one class that would go into a separate bundle. Dani, I'd be happy if you gave this issue some second thoughts.
Comment 8 Dani Megert CLA 2009-06-03 05:17:14 EDT
>The JUnitLaunchConfigurationDelegate alone is not self-sufficient
Yes, of course ;-)

Is this request just to get rid of the UI dependency or is the stuff not running because it breaks? If so, we can eventually fix this. If it's just to reduce the dependency/size for test plug-ins then it's really not a path we want to follow.
Comment 9 Thomas Hallgren CLA 2009-06-03 05:39:30 EDT
Dani, this is a request to remove the UI dependencies but the reasons for separating the code goes way beyond testing. It affects the full assemble, build, test, deploy circle. Our tool is specialized to support this in an unmaintained mode without any involvement from the UI.

The current situation will require us to create our own private copy of very large parts of the JDT and PDE in order to enable JUnit testing so we are indeed very interested in this separation. The alternative is to have our product triple in size and complexity introducing a large amount of bundles that will never be used.

I'm not sure why you say that this is 'really not a path we want to follow' and I hope it's open for discussion. For us, that path is very essential and the separation is indeed already maintained in very large parts of the SDK today. So why not here?

Comment 10 Dani Megert CLA 2009-06-03 05:53:53 EDT
Again: the issue is just size, right? Is that a real problem or just an 	aesthetic one? If so, wouldn't you be able to write a much smaller and simpler launcher from scratch?

>I hope it's open for discussion. 
Sure, as you can see ;-)

>the separation is indeed already maintained in very large parts of the SDK today.
>So why not here?
jdt.junit and pde.junit were explicitly designed and written for the UI and has not been designed to be(come) API. Of course this can be done afterwards but it will need investment/resources to do this.

If this is really important to you I suggest to write a patch that
1) introduces org.eclipse.jdt.junit.launching
2) updates org.eclipse.jdt.junit
which we can review and then continue based on that. Are you willing to do this?
Comment 11 Achim Demelt CLA 2009-06-03 06:04:54 EDT
(In reply to comment #10)
> Again: the issue is just size, right? Is that a real problem or just an        
> aesthetic one? If so, wouldn't you be able to write a much smaller and
> simpler launcher from scratch?

No, it's not just size, but the UI dependencies it entails. The headless and unattended "assemble, build, test, deploy" cycle that Thomas mentioned simply has no UI. With the current jdt.junit code _expecting_ to run in the UI, it's just impossible to include it in that scenario.

> >So why not here?
> jdt.junit and pde.junit were explicitly designed and written for the UI and has
> not been designed to be(come) API. Of course this can be done afterwards but it
> will need investment/resources to do this.

Some parts _are_ API, and my hope is that these parts are sufficient for consumption by Buckminster. As far as I can see, a separation of the launcher code would not introduce or open up any additional API.

> 
> If this is really important to you I suggest to write a patch that
> 1) introduces org.eclipse.jdt.junit.launching
> 2) updates org.eclipse.jdt.junit
> which we can review and then continue based on that. Are you willing to do
> this?
> 

Yes, I'd like to give it a try.
Comment 12 Dani Megert CLA 2009-06-03 06:13:00 EDT
>With the current jdt.junit code _expecting_ to run in the UI, it's
>just impossible to include it in that scenario.
That's what I previously meant: we've changed several parts of the jdt.ui code so that it can run in headless mode. If that's not the case for jdt.junit, then that could probably be fixed so that you could include it in your headless scenario.

>Some parts _are_ API, and my hope is that these parts are sufficient
Ah, OK. That's good news if you don't need more API.

Reopening as someone is going to work on this.
Comment 13 Dani Megert CLA 2009-06-03 06:14:11 EDT
Achim, I'm going to give this bug to you for now, as you want to work on it.
Comment 14 Thomas Hallgren CLA 2009-06-03 06:51:47 EDT
This is really good news. Achim, let me know if there's anything I can do to assist you in this effort.
Comment 15 Dann Martens CLA 2009-06-04 05:42:17 EDT
Congratulations to all parties involved to get things moving around these issues!
Comment 16 Achim Demelt CLA 2009-06-17 09:31:27 EDT
Good news: I succeeded in separating the core code from the UI code. No additional API was introduced. In fact, even the internal packages remain mostly unchanged for a consumer of org.eclipse.jdt.junit. :-)

Now we're at a point where we need to discuss some technical details. I'm not sure if attaching a patch at this stage is a good idea, since the bulk of the moved classes obscure the actual changes in the code. The refactored code is in an SVN repo at Google Code: http://code.google.com/p/jdt-junit/source/checkout. The original jdt.junit code is tagged "original" in the repo. If that's not convenient, I can also attach a patch to this bug.

Now, here's what I did: I basically moved everything from o.e.jdt.junit that does not require a UI into a new bundle o.e.jdt.junit.core. This includes code as well as some declarations from the plugin.xml. I chose the name "core" instead of "launcher" since this bundle contains more than the launcher itself. For example, the whole "model" packages (both public and internal) were moved to o.e.jdt.junit.core. The old o.e.jdt.junit bundle requires and re-exports the o.e.jdt.junit.core bundle. This way, the o.e.jdt.junit API remains unchanged, while the headless stuff in o.e.jdt.junit.core is completely self-sufficient and functional. There are, however, some technical details that I would like to point out:

1) Refactoring stuff: Everything concerning jdt.junit's refactoring support is left in the o.e.jdt.junit bundle. This includes the junitLaunchConfigs extension point and everything that goes with it. Technically I guess it would be possible to move this over to the core bundle as well, but IMHO refactoring support for headless JUnit execution isn't really something anyone would want to have...

2) Extension Points: I moved the internal_testKinds extension point over to the core bundle and removed it from the o.e.jdt.junit bundle. It's internal, so I guess that's ok. The testRunListeners extension point is public, so I left the original extension point in place, made it deprecated, and created a new copy in the o.e.jdt.junit.core plug-in. The core plug-in reads the extension registry for both extension IDs, org.eclipse.jdt.junit.testRunListeners and org.eclipse.jdt.junit.core.testRunListeners.

3) Preferences: All preferences are now stored in the scope of o.e.jdt.junit.core. I changed the use of the JFace preferences API to the o.e.core.runtime.preferences API where appropriate. The JunitPreferenceInitializer now not only sets the prefs in the default scope, but also migrates the instance scope preferences from o.e.jdt.junit to o.e.jdt.junit.core. Note that the actual preference _keys_ remain unchanged.

4) Messages: For now, I've only copied the NLS messages from o.e.jdt.junit to o.e.jdt.junit.core. In other words, both bundles contain the same (duplicate) set of messages. I chose to postpone this task until all other issues are resolved.

And now for some more problematic questions:

5) The JUnitLaunchConfigurationDelegate used to open a message dialog in case of an error. This is obviously no longer possible, now that the launch takes place in headless mode. We need to judge and discuss the implications of this change.

6) Launch listener to open JUnit view: The JUnitModel registers a (debug) ILaunchListener in order to be notified of JUnit launch executions. Its main purpose is to create a new TestRunSession and open the socket connection to the test runner. In the "old" JUnitModel from o.e.jdt.junit, this listener also opened the JUnit view. This is no longer possible with the new JUnitModel being in the headless core bundle.

In the new implementation, the JUnitPlugin (from o.e.jdt.junit) now registers a TestRunSessionLister with the JUnitModel. This listener then opens the JUnit view when a new test run session is started. This implies subtle changes in behavior:

* TestRunSessionListener notifications are no longer sent in the UI thread.
Listeners must therefore call Display.asyncExec() on their own. There were some comments regarding this in the code, but now that the JUnitModel and the listeners have no access to the UI, sending events in the UI thread is no longer an option (which, IMHO, was never a good idea anyway). Since all of this is internal API, I'd say changing this behavior is acceptable.
* The old implementation was strictly linear: First the JUnit view part was created, then test run session events were sent. Since the JUnit view is a TestRunSessionListener, it could update itself and show the currently running test. In the new implementation, the JUnit view is created as a _result_ of the event. The view itself therefore no longer receives the event and does not show the test run.
* To overcome this problem, the JUnit view now _always_ initializes itself with the youngest test run in its createPartControl() method.  This results in the same behavior as with the old implementation whenever the JUnit view is opened automatically as a consequence of starting a JUnit launch. If the view is opened manually, though, the youngest test run is also shown, which was not the case in the old implementation.

That's it for now, I think. I'm looking forward to a lively discussion :-)
Comment 17 Dani Megert CLA 2009-06-17 09:44:07 EDT
Given you have the code it doesn't hurt to attach it here. That's much simpler for people to fetch it as patch than to go to a non-eclipse source.
Comment 18 Darin Wright CLA 2009-06-17 09:46:03 EDT
(In reply to comment #16)

> 5) The JUnitLaunchConfigurationDelegate used to open a message dialog in case
> of an error. This is obviously no longer possible, now that the launch takes
> place in headless mode. We need to judge and discuss the implications of this
> change.

We've handled similar "prompting" issues in the debug platform by use of org.eclipse.debug.core.IStatusHandler's. Basically, you can register a handler for a specific status and delegate to the handler when it happens. This allows a handler to be registered by a UI plug-in to handle a status from a core plug-in and display a dialog. For example, the debug platform does this to prompt for launching in the face of compilation errors.
Comment 19 Achim Demelt CLA 2009-06-17 10:15:28 EDT
Created attachment 139428 [details]
Patch, first draft

The patch was created from the Google SVN repo. It can be applied over the jdt.junit code checked out from CVS. For some reason, the two deleted "model" packages are not included in the patch, so you will have to delete these two packages manually after applying the patch.
Comment 20 Achim Demelt CLA 2009-06-17 10:21:43 EDT
> 
> We've handled similar "prompting" issues in the debug platform by use of
> org.eclipse.debug.core.IStatusHandler's. Basically, you can register a handler
> for a specific status and delegate to the handler when it happens. This allows
> a handler to be registered by a UI plug-in to handle a status from a core
> plug-in and display a dialog. For example, the debug platform does this to
> prompt for launching in the face of compilation errors.
> 

Darin, thanks for the hint. I'll look into this.
Comment 21 Achim Demelt CLA 2009-06-17 12:02:57 EDT
Created attachment 139445 [details]
Now uses IStatusHandler to display message dialog

Note: This patch must be applied after the first one.

Errors are now simply thrown as CoreExceptions. I registered a status handler for those four codes that used to be displayed in a message box. The DebugUI plug-in now calls this handler, which now shows the message dialog. If no handler is registered (e.g. in headless mode), the errors are logged as usual.
Comment 22 Markus Keller CLA 2009-06-18 07:17:10 EDT
Thanks a lot Achim, that sounds very good. I quickly glanced at the patch, and that also looks fine (but I didn't try it so far).

Just a question to the JunitPreferenceInitializer: It looks like the migration code runs on every restart. Doesn't that mean that the settings in the core plug-in will be overwritten with the old settings from the UI plug-in all the time?

Since 3.6 builds will only start in July, I cannot commit it right now. And since I'll be away for most of the next 5 weeks, it unfortunately also won't happen right after the builds start. But I'll review and commit it asap after my return.
Comment 23 Achim Demelt CLA 2009-06-18 07:32:28 EDT
> Just a question to the JunitPreferenceInitializer: It looks like the migration
> code runs on every restart. Doesn't that mean that the settings in the core
> plug-in will be overwritten with the old settings from the UI plug-in all the
> time?

Yes, you're right. I'll fix that. Will it suffice to simply delete all keys from the old UI plug-in instance scope prefs after the migration? Or is there a more elegant solution?

> 
> Since 3.6 builds will only start in July, I cannot commit it right now. And
> since I'll be away for most of the next 5 weeks, it unfortunately also won't
> happen right after the builds start. But I'll review and commit it asap after
> my return.
> 

Great! I guess this last statement makes a lot of people very happy :-) Enjoy your vacation---I assume it's not 5 weeks of business travel...
Comment 24 Markus Keller CLA 2009-06-18 08:39:29 EDT
> Will it suffice to simply delete all keys
> from the old UI plug-in instance scope prefs after the migration?

Yep, that's the easiest solution, given that the preference keys are not API and these are only workspace prefs (no project-specific settings).

> Enjoy your vacation---I assume it's not 5 weeks of business travel...

Thanks, I'll definitely enjoy the vacation part (but not the 3 wasted weeks of military service :-/ ).
Comment 25 Achim Demelt CLA 2009-07-10 07:46:12 EDT
Created attachment 141279 [details]
Updated patch

This is an updated version of the complete patch. It includes everything from the previous patches, plus the following:

* The old preference settings are now deleted after copying them into the new scope.
* The message files have been split in to UI and non-UI entries.
* Recent check-ins from CVS have been merged into the junit.core bundle.
Comment 26 Achim Demelt CLA 2009-07-10 07:54:24 EDT
With the last patch, I think that the separation of jdt.junit and jdt.junit.core is done. Markus, I'm assigning back to you to perform a review.
Comment 27 Dani Megert CLA 2009-07-10 09:45:30 EDT
It's better to have the bug assigned to the one who fixes it.

It might take some weeks until Markus can review due to Military service and holiday.
Comment 28 Achim Demelt CLA 2009-07-26 13:55:46 EDT
Created attachment 142606 [details]
Updated patch

Updated patch to reflect recent commits for the 3.6 stream. A little reminder: after applying the patch you'll have to delete the two "model" packages from the original junit bundle.
Comment 29 Achim Demelt CLA 2009-08-11 02:35:07 EDT
Created attachment 144003 [details]
Updated patch

Once again, I updated the patch to include the latest changes from CVS.

Since M1 is out now, do you think you can apply the patch in the next couple of days? The problem is that I always have to check for changes in the CVS code and manually merge these changes into the refactored code. This is very tedious, not to mention error-prone, because many classes have been moved and I have to apply half of the changes to jdt.junit and the other half to jdt.junit.core.

So I'd really appreciate if I wouldn't have to do this too many times more ;-) Markus, if you want to you can Skype me (ademelt) when reviewing the patch.
Comment 30 Markus Keller CLA 2009-08-12 06:10:35 EDT
(In reply to comment #29)
Sorry about that. I had this bug on my list but never found the time to do the full review, and I didn't think of the conflicts that other little fixes caused.

Thanks for creating an updated patch. I'll review it this afternoon or tomorrow.
Comment 31 Markus Keller CLA 2009-08-12 13:24:16 EDT
I need a PMC approval for creating the new plug-in org.eclipse.jdt.junit.core.
Comment 32 Achim Demelt CLA 2009-08-20 05:57:19 EDT
Markus, how is the review going? If you have any questions about specific parts of the refactoring, don't hesitate to ask me.
Comment 33 Markus Keller CLA 2009-08-20 12:54:26 EDT
Created attachment 145148 [details]
Patch 6

Sorry, I've been caught up by concurrency issues I couldn't resolve so far.

I've created the project org.eclipse.jdt.junit.core in CVS at /cvsroot/eclipse and (on the server) copied over the history of the files we want to move from the original location. If you now check out that project from HEAD and then apply my patch, it's easy to see the changes w.r.t. the current state in HEAD.

Notable fixes I already made in this patch:
- org.eclipse.jdt.junit.core has to start its life with version 3.6, see bug 265699 for explanations
- fixed compile errors in tests (in the org.eclipse.jdt.ui.tests project)
- added new plug-in to JDT feature
- moved declarations of extension points testRunListeners and internal_testKinds to org.eclipse.jdt.junit.core, thereby keeping their fully qualified names. This saves us from all hassles like deprecating and having multiple extension points.
- org.eclipse.jdt.junit.core/plugin.xml: id of content type must be explicitly qualified with the old plug-in id so that it stays the same

Real code changes (not just due to renaming or split up) that look good:
JunitPreferenceInitializer, JUnitLaunchConfigurationDelegate, LaunchErrorStatusHandler

Real changes that need a closer look:
JUnitModel, JUnitPlugin, TestRunnerViewPart

When I run the JUnitJUnitTests, I constantly get a failure in org.eclipse.jdt.junit.tests.TestRunSessionSerializationTests3.testFailingSuite(). I have not yet a clear understanding why this happens now. A reason could be that a '$' in a nested class name is converted to a '.' when test results are exported or swapped out to disk, and swapping now happens at another point in time.

But when I started debugging with breakpoints, I also got other test failures that were not caused by time outs[1] but really some missing data. Looking closer at the TestRunSession, I found that at least swapIn() and swapOut() need to be synchronized since they can now be called concurrently. Also access to JUnitModel.fTestRunSessions probably needs to be synchronized.

If you could have a look at these threading issues and resolve the test failure, that would be great!


[1] To debug the tests, you need to increase TIMEOUT in AbstractTestRunSessionSerializationTests
Comment 34 Achim Demelt CLA 2009-08-21 08:21:10 EDT
It's been fun hunting down a '$' that should have been a '.' ;-) I cannot present a final fix at this point, but I'm pretty sure I understand _why_ the test fails. Here's the story:

1) We're starting the test run. This can lead to two things
 a) The JUnit view (TestRunnerViewPart) is not yet open. It will then be forced to open by the JUnitPlugin. It registers a TestRunSessionListener and fakes a sessionAdded() event with the youngest test run session.
 b) The JUnit view is already open. In this case, its TestRunSessionListener gets the regular sessionAdded() event.
 
2) We should always have 1 a) in our failing test case, but it actually doesn't matter. In both cases, the listener will execute setActiveTestRunSession() in the display thread.

3) setActiveTestRunSession() registers a TestSessionListener for the new session.

4) Now the test run has finished and tries to swapOut() the test run session. However, the TestRunnerViewPart's TestSessionListener is still registered and thus prevents the session from swapping out.

Unfortunately, it's the swapping out that replaces the '$' with the '.' in the test class name. If swapping out never happens, comparing the in-memory version (containing '$') with the serialized version (containing '.') leads to the test failure.

I actually don't know why the test consistently _didn't_ fail in the old implementation. This can only happen if the test run has finished before the UI has had a chance to register the TestSessionListener.

Anyway, with that knowledge I attempted to fix that problem. I made two changes:

1) TestRunnerViewPart.setActiveTestRunSession() now only registers the TestSessionListener if the test run session is actually running.
2) The TestSessionListener now deregisters itself when the session has finished, stopped, or ended.

Both changes now ensure that the listener no longer blocks swapping out the session and the test case is consistently green now :-)

Unfortunately, this leads to TestRunSerializationTests3 and 4 failing two test cases which obviously expect that the session has _not_ been swapped out. Any ideas here?
Comment 35 Markus Keller CLA 2009-08-25 05:53:03 EDT
> I made two changes: [..]

Could you please attach these here (maybe together with other changes for the items below)?

> Unfortunately, this leads to TestRunSerializationTests3 and 4 failing two test
> cases which obviously expect that the session has _not_ been swapped out. Any
> ideas here?

If it makes sense in those scenarios not to swap out the session, then we can also adapt the tests to the new reality.


(In reply to comment #33)
> Looking
> closer at the TestRunSession, I found that at least swapIn() and swapOut() need
> to be synchronized since they can now be called concurrently. Also access to
> JUnitModel.fTestRunSessions probably needs to be synchronized.

Did you already have a look at these problems? Before, everything was implicitly synchronized since all the relevant stuff happened in the UI thread, but now, we have to explicitly synchronize. If we don't, it may work for simple scenarios, but cause weird bugs when the user switches between history entries and runs tests in parallel.
Comment 36 Achim Demelt CLA 2009-08-25 06:18:52 EDT
> Could you please attach these here (maybe together with other changes for the
> items below)?
> 

I tried to attach the changes with my last post, but it seems I was too stupid to create a decent patch. What I actually wanted to send were the actual changes I made. But I found no way how to do that, because after applying _your_ patch, basically all files in my workspace are changed. I can only create a patch against CVS HEAD, which is way too big to be useful. Do you know how I can create a patch that contains _my_changes vs. _your_ changes?

> > Unfortunately, this leads to TestRunSerializationTests3 and 4 failing two test
> > cases which obviously expect that the session has _not_ been swapped out. Any
> > ideas here?
> 
> If it makes sense in those scenarios not to swap out the session, then we can
> also adapt the tests to the new reality.
> 

Adapting the test cases sounds reasonable to me, too. After all, we can give no guarantee to a client whether a session has ever been swapped out or not. Since swapping out and in results in a different in-memory representation of the session, a client must be able to cope with both versions.

> 
> (In reply to comment #33)
> > Looking
> > closer at the TestRunSession, I found that at least swapIn() and swapOut()
> need
> > to be synchronized since they can now be called concurrently. Also access to
> > JUnitModel.fTestRunSessions probably needs to be synchronized.
> 
> Did you already have a look at these problems? Before, everything was implicitly
> synchronized since all the relevant stuff happened in the UI thread, but now, we
> have to explicitly synchronize. If we don't, it may work for simple scenarios,
> but cause weird bugs when the user switches between history entries and runs
> tests in parallel.

I'll look into that tomorrow.
Comment 37 Markus Keller CLA 2009-08-25 10:13:25 EDT
> Do you know how I can create a patch that contains _my_changes vs. _your_ changes?

I don't know how this could be done with CVS. There are a few open requests in that area in the Eclipse compare framework (e.g. bug 71374). But I guess the easiest is to just put the changed files into a ZIP.
If you send a new complete patch, I can check out the affected projects a second time in the same workspace, then apply the patch to it, and then compare it to my version.
Comment 38 Achim Demelt CLA 2009-08-26 13:26:08 EDT
Created attachment 145697 [details]
Fixes for concurrency issues

That was tough. Here's what I found out:

1) We can no longer give guarantees about when exactly things are happening when the JUnit model stuff no longer runs in the UI thread. The listeners themselves do Display.asyncExec(), and this is pretty much unpredictable.

2) The test cases rely on the fact that the test run session has been swapped out and swapped back in again. Only then will the comparison between the in-memory TestRunSession and the de-serialized expected result be ok.

3) The TestRunnerViewPart$TestSessionListener prevents the TestRunSession from being swapped out. It's a matter of luck whether that listener is still there (or not yet, or no longer) at the time the comparison is done. I've seen the same tests go from red to green to red again.

4) And yes, swapping in and out may happen concurrently. I'm not sure if that really wasn't the case in the original implementation, but now we definitely have that problem.

Based on that, I did the following:

1) The TestRunnerViewPart is now very lenient with the Listener thing. When it receives a new TestRunSession, it only attaches a listener if that session is currently starting or is still running. In other words, it will _not_ attach a listener if the session has already stopped by the time the UI gets a chance to update itself with that session. One exception, though: If the session is kept alive, we'll still have to attach that listener in order to be notified of reruns.

2) The TestRunnerViewPart$TestSessionListener now deregisters itself when the session has ended, stopped, or was terminated. Again, if the session is kept alive, we don't deregister.

3) The swapIn(), swapOut(), getTestRoot(), and addTestSessionListener() methods in TestRunSession are now synchronized.

4) Even with these changes, it's still very often the case that a session has not been swapped out. I can elaborate on why that's the case, but that would be two more paragraphs of text... I therefore changed the AbstractTestRunSessionSerializationTests.runExportImport() method to explicitly swap out the session before comparing it to the expected result.

This way, we have a predictable outcome of the tests. Which are all green now on my machine.

I attached a patch against CVS HEAD for the three files. This way, the patch itself is rather small and readable, and you can apply it easily. You'll just have to revert-to-base these three files before you do it. That's:

* TestRunnerViewPart in org.eclipse.jdt.junit
* TestRunSession in org.eclipse.jdt.junit.core
* AbstractTestRunSessionSerializationTests in org.eclipse.jdt.ui.tests
Comment 39 Achim Demelt CLA 2009-08-27 10:39:12 EDT
I found one more (hopefully last) quirk. Here's how to reproduce it:

1) Create a workspace with a project containing a test case. Do some System.out.println() to see that the test really runs. Create a JUnit launch config for that and run it.

2) Close the JUnit view (that's important). Then close and restart Eclipse.

3) Run the JUnit launch config from the launch drop-down menu.

=> Test test runs (see output on the console), but the JUnit view is not opened.

In fact, this shows how well the separation of UI and launching works :-) Simply launching a JUnit test does not activate the jdt.junit (UI) bundle. Without activation, the TestRunSessionListener is not registered and the UI is not notified of a newly launched JUnit test.

What would be the best solution for this? Use the deprecated listener extension point?
Comment 40 Markus Keller CLA 2009-09-02 10:43:21 EDT
Created attachment 146284 [details]
Final Patch

OK, this is done and released to HEAD. Thanks a lot Achim for your patches and investigations. Could you please verify that this fits your needs? It will show up in nightly builds and in upcoming I-builds.

I'll probably rename the internal packages in o.e.jdt.junit.core to org.eclipse.jdt.internal.junit.core[.*], but I didn't want to do this now to avoid even more changes.

(In reply to comment #39)
I added support for contributing subclasses of TestRunListener to the testRunListeners extension point and then used that to load o.e.jdt.junit and open the view when a test is started.

Other small fixes I applied:
- TestRunSession: removed System.out.println
- Fixed log entries for 3 unused strings
- Synchronized access to JUnitModel.fTestRunSessions. While doing that, I also fixed bug 139697.
Comment 41 Markus Keller CLA 2009-09-02 10:44:18 EDT
Done.
Comment 42 Markus Keller CLA 2009-09-02 10:57:08 EDT
Comment on attachment 144003 [details]
Updated patch

I have to set the iplog+ flag on this patch, since it contains IP-worthy changes in 6 files (JunitPreferenceInitializer, JUnitLaunchConfigurationDelegate, LaunchErrorStatusHandler, JUnitModel, JUnitPlugin, TestRunnerViewPart).

All the rest is just splitting up existing code and moving it into different packages and into the new project.

Relevant changes were < 250 lines.
Comment 43 Kim Moir CLA 2009-09-02 11:52:02 EDT
Opened bug 288375 against ant.ui so they can update dependencies due to this change.  Otherwise, the build will fail due to unresolved dependencies.
Comment 44 Markus Keller CLA 2009-09-02 14:35:51 EDT
> Opened bug 288375 against ant.ui so they can update dependencies due to this
> change.  Otherwise, the build will fail due to unresolved dependencies.

That was a red herring. o.e.jdt.junit has already been bumped to 3.6.0 before this change. But I didn't update our map file promptly, so the releng tools didn't check out the new plug-in during a test build, and the API that has been moved to o.e.jdt.junit.core was no longer available.
Comment 45 Achim Demelt CLA 2009-09-03 03:16:13 EDT
> OK, this is done and released to HEAD. Thanks a lot Achim for your patches and
> investigations. Could you please verify that this fits your needs? It will show
> up in nightly builds and in upcoming I-builds.
> 

Markus, thanks for taking the time to review and accept this huge contribution. I'd also like to express my gratitude on behalf of the greater Buckminster community, which will soon benefit from headless JUnit testing. This would not have been possible without these changes.

I'll verify the final results ASAP.
Comment 46 Achim Demelt CLA 2009-09-10 03:09:50 EDT
Headless JUnit launching works like a charm :-) Marking this as verified. Thanks again!