Community
Participate
Working Groups
The launching of SWTBot tests fails because the SWTBot test launcher is not compatible with Eclipse 4.x : see Bug 363334. So I disabled the launching of all the SWTBot unit tests until the problem is solved.
I tried to use a different testing application in revision 1126: "-application org.eclipse.pde.junit.runtime.uitestapplication" But it didn't work so I reverted it. Maybe one of these would work: org.eclipse.swtbot.eclipse.core.swtbottestapplication org.eclipse.test.coretestapplication org.eclipse.test.uitestapplication I found this article with more information about headless test launching: http://www.eclipse.org/articles/Article-PDE-Automation/automation.html
I tried the other applications that are defined in "org.eclipse.test". It didn't work with the previous version of "org.eclipse.test" that was used for the previous releases : I got an error that no tests were found in the test suite. So I understood that it didn't support Junit 4. So I tried to get a more recent version of "org.eclipse.test". I thought it moved to git along with the rest of the platform this year, so I tried to clone the latest version from git: git clone git://dev.eclipse.org/org.eclipse.test/org.eclipse.test.git It worked on my PC, But I couldn't fetch it from Buckminster or from the shell script on Hudson or on the command line from build.eclipse.org. I got either: "Cloning into org.eclipse.test... fatal: The remote end hung up unexpectedly" or from Buckminster: "org.eclipse.jgit.api.errors.JGitInternalException: Invalid remote: origin" I tried a bunch of configurations of the Buckminster git connector, but I couldn't get past this error. Then I remembered the webmaster recommended to use the local file system instead of the remote protocols. So I tried to find "org.eclipse.test" under /gitroot. But it doesn't seem to be there. So I realized that "org.eclipse.test" was only on a Git mirror, but the "real" project was still on CVS. So I switched back to CVS, and it works with the trunk version of "org.eclipse.test", which is on: :pserver:anonymous@dev.eclipse.org:/cvsroot/eclipse,org.eclipse.test With it, I could finally make the SWTBot tests start under Juno with the application "org.eclipse.test.uitestapplication", which is defined in "org.eclipse.test". Now, the SWTBot tests start, but not much more : judging from the ton of errors in the log and the screenshots taken during the SWTBot tests, nothing seems to open in the UI. It looks like the events triggered by SWTBot are not correct for the new workbench model. For example, I see a bunch of errors like this: Exception while dispatching event org.osgi.service.event.Event [topic=org/eclipse/e4/ui/model/ui/UIElement/visible/SET] to handler org.eclipse.e4.ui.services.internal.events.UIEventHandler@16fb17e5 java.lang.IllegalArgumentException: Argument not valid
I ran the SWTBot tests locally on an Eclipse 4.2M4, and the tests don't block as on Hudson (but 16 tests are still failing). But I noticed that the UI looks broken (overlapping editors and views, huge empty space, cropped views, ...) when the tests maximize the editor or shell. So I disabled the maximizations in revision 1144. I also added a debugging screenshot in TableEditabilityTest#beforeTests where the tests stayed stuck on Hudson.
I now close all views in addition to editors and shells before each test, and I added a timeout on each test so that one blocking test shouldn't block the whole build. I also enabled SWTBot log4j logging, by following the instructions on: http://wiki.eclipse.org/SWTBot/FAQ#How_do_I_configure_log4j_to_turn_on_logging_for_SWTBot.3F
The SWTBot tests block the build, and prevent it from finishing correctly, even with the timeouts I set on each test. I tried to install a Linux VM (OpenSuse 12.1 in VirtualBox) in order to reproduce the problem locally, but the Eclipse segfaults in SWT code for Gtk at the beginning of the tests. This is clearly not the same problem as what happens on Hudson, since the build would stop with an error if the Eclipse instance had crashed. Then I tried to obtain a stackdump at the moment the Hudson tests were stuck. Since I can't run jstack directly on Hudson because I don't have ssh access to the Hudson instances, I wrote a thread that is started from the SWTBot tests and dumps the whole stack state every minute (cf SWTBotUtils#startDebugDumpingIfEnabled). With this, I discovered that the tests were running in the UI thread. That's why they are blocking: SWTBot tests must not be started from the UI thread. So now we know that both applications "coretestapplication" and "uitestapplication" in "org.eclipse.test" start the tests in the UI thread. So then I tried "uitestapplication" in "org.eclipse.pde.junit.runtime", which appears to run tests *not* in the UI thread (since its implementing class inherits from NonUIThreadTestApplication). But it doesn't produce XML test result files directly. It needs a port to connect a listener that collects test results. I am in the process of setting it up. see: http://www.eclipse.org/articles/article.php?file=Article-PDEJUnitAntAutomation/index.html
I successfully started "org.eclipse.pde.junit.runtime.uitestapplication" and the test collector from pde.test.utils as explained in the article. But unfortunately it does not find the EMF Facet tests because JUnit 4 is not supported in "org.eclipse.pde.junit.runtime", so it's back to square one... We have decided to stop trying to run the SWTBot tests on Eclipse 4.2. I will make a separate build for EMF Facet on Eclipse 3.8 that will run the SWTBot tests.
I noticed that Eclipse was using this parameter when launching the tests: "-testLoaderClass org.eclipse.jdt.internal.junit4.runner.JUnit4TestLoader" It might be worth to try to see if it works with the "org.eclipse.pde.junit.runtime.uitestapplication" launcher.
The tests seem to block when taking screenshots, so I have disabled the SWTBot screenshots temporarily to see if it's the only thing preventing the tests from running correctly.
I thought the application org.eclipse.pde.junit.runtime.uitestapplication was running in a separate thread, because it is implemented by org.eclipse.pde.internal.junit.runtime.UITestApplication, which inherits from org.eclipse.pde.internal.junit.runtime.NonUIThreadTestApplication. But apparently this is not the case: UITestApplication overloads runApp to pass false as "runTestsInSeparateThread". So I am now running the tests with the application "nonuithreadtestapplication". But this application runs "coretestapplication" by default, so it cannot run UI tests. So I added this parameter "-testApplication org.eclipse.ui.ide.workbench" (see NonUIThreadTestApplication#getApplicationToRun) to make it able to run the tests that access the workbench. try with this configuration: -application org.eclipse.pde.junit.runtime.nonuithreadtestapplication -testApplication org.eclipse.ui.ide.workbench -loaderpluginname org.eclipse.jdt.junit4.runtime -testLoaderClass org.eclipse.jdt.internal.junit4.runner.JUnit4TestLoader
With the modifications mentioned above, the SWTBot tests now run on Hudson. So I'm marking this bug as resolved.
This bug can be closed.
