Community
Participate
Working Groups
Now that we can compile the UITestSuites in 4.2. We need to examine the tests and group them into 1 of 4 categories 1) valid and pass 2) valid and fail 3) invalid and pass 4) invalid and fail The goal is to delete category 3 and 4 tests completely. Category 1 is great :-) Category 2 we need to record, try and fix over the next 2 days, and then somehow disable without losing.
Start working through the test Suites specified in org.eclipse.ui.tests.UiTestSuite 1) Pick a suite, and then comment on the bug that you are working on it. 2) Run just that suite. 3) open a bug on the suite, and depend on it from this one. Start with how many fail and a quick analysis (one problem, multiple problems, are the tests valid, etc). We want to get rid of invalid tests that pass or fail. We want to make sure any tests left are making valid changes and examining useful state. PW
Dean's already started on org.eclipse.ui.tests.dialogs.UIAutomatedSuite.UIAutomatedSuite() I'd avoid org.eclipse.ui.tests.api.ApiTestSuite.ApiTestSuite() for the moment. PW
I'll look into properties and preferences.
I've started looking at org.eclipse.ui.tests.menus.MenusTestSuite PW
Trying to sort out org.eclipse.ui.tests.multieditor.MultiEditorTestSuite.
QuickAccess test suite passes
Here's the breakout of the test suites we need to look at and create bugs for. Eric: org.eclipse.ui.tests.dnd.DragTestSuite.DragTestSuite() org.eclipse.ui.tests.zoom.ZoomTestSuite.ZoomTestSuite() org.eclipse.ui.tests.presentations.PresentationsTestSuite.PresentationsTestSuite() org.eclipse.ui.tests.internal.InternalTestSuite.InternalTestSuite() org.eclipse.ui.tests.themes.ThemesTestSuite.ThemesTestSuite() org.eclipse.ui.tests.intro.IntroTestSuite.IntroTestSuite() org.eclipse.ui.tests.dynamicplugins.DynamicPluginsTestSuite.DynamicPluginsTestSuite() Remy: org.eclipse.ui.tests.multieditor.MultiEditorTestSuite.MultiEditorTestSuite() org.eclipse.ui.tests.concurrency.ConcurrencyTestSuite.ConcurrencyTestSuite() org.eclipse.ui.tests.statushandlers.StatusHandlingTestSuite.StatusHandlingTestSuite() org.eclipse.ui.tests.systeminplaceeditor.OpenSystemInPlaceEditorTest org.eclipse.ui.tests.multipageeditor.MultiPageEditorTestSuite.MultiPageEditorTestSuite() org.eclipse.ui.tests.navigator.NavigatorTestSuite.NavigatorTestSuite() org.eclipse.ui.tests.leaks.LeakTests Paul: org.eclipse.ui.tests.menus.MenusTestSuite.MenusTestSuite() org.eclipse.ui.tests.commands.CommandsTestSuite.CommandsTestSuite() org.eclipse.ui.tests.progress.ProgressTestSuite.ProgressTestSuite() org.eclipse.ui.tests.services.ServicesTestSuite.ServicesTestSuite() org.eclipse.ui.tests.contexts.ContextsTestSuite.ContextsTestSuite() org.eclipse.ui.tests.keys.KeysTestSuite.KeysTestSuite() org.eclipse.ui.tests.filteredtree.FilteredTreeTests org.eclipse.ui.tests.api.StartupTest Oleg: org.eclipse.ui.tests.propertysheet.PropertySheetTestSuite.PropertySheetTestSuite() org.eclipse.ui.tests.preferences.PreferencesTestSuite.PreferencesTestSuite() org.eclipse.ui.tests.adaptable.AdaptableTestSuite.AdaptableTestSuite() org.eclipse.ui.tests.activities.ActivitiesTestSuite.ActivitiesTestSuite() org.eclipse.ui.tests.decorators.DecoratorsTestSuite.DecoratorsTestSuite() org.eclipse.ui.tests.datatransfer.DataTransferTestSuite.DataTransferTestSuite() org.eclipse.ui.tests.fieldassist.FieldAssistTestSuite.FieldAssistTestSuite() Dean: org.eclipse.ui.tests.dialogs.UIAutomatedSuite.UIAutomatedSuite() org.eclipse.ui.tests.api.ApiTestSuite.ApiTestSuite() org.eclipse.ui.parts.tests.PartsReferencesTestSuite org.eclipse.ui.tests.quickaccess.QuickAccessTestSuite.QuickAccessTestSuite() org.eclipse.ui.tests.encoding.EncodingTestSuite.EncodingTestSuite() org.eclipse.ui.tests.operations.OperationsTestSuite.OperationsTestSuite()
(In reply to comment #7) > org.eclipse.ui.tests.progress.ProgressTestSuite.ProgressTestSuite() Runs fine. PW
PartsReferenceTestSuite seems to pass if you close the windows manually. But needs to have Bug 366449 fixed before it can be considered to work properly. It will have to be removed from the automated tests until that defect is fixed.
EncodingTestSuite() is fine.
org.eclipse.ui.tests.operations.OperationsTestSuite.OperationsTestSuite() Runs fine. Did some spot checks and the tests seem valid.
Opened bug 366451 for the leak test failures.
(In reply to comment #7) > org.eclipse.ui.tests.filteredtree.FilteredTreeTests These run fine. PW
APITestSuite (345 tests)runs to completion but with: 50 Failures 8 errors Many in IWorkbenchPageTest
(In reply to comment #7) The following test suites pass: > org.eclipse.ui.tests.adaptable.AdaptableTestSuite.AdaptableTestSuite() > org.eclipse.ui.tests.datatransfer.DataTransferTestSuite.DataTransferTestSuite() > org.eclipse.ui.tests.fieldassist.FieldAssistTestSuite.FieldAssistTestSuite()
(In reply to comment #7) > org.eclipse.ui.tests.preferences.PreferencesTestSuite.PreferencesTestSuite() Those tests pass. There is a missing piece in the overall styling story: if we use CSS for styling, why we still use "old" way to define fonts? I am not sure how many of those are still used, but some surely are (such as text editor fonts).
Opened bug 366625 for the failure in MultiPageEditorTestSuite.
General results... org.eclipse.ui.tests.dnd.DragTestSuite.DragTestSuite() - DOA: DragOperations has been changed to throw an exception saying 'we have work to do' - Bug 366630 org.eclipse.ui.tests.zoom.ZoomTestSuite.ZoomTestSuite() - (there are bogus tests here since we don't unzoom as much as the old zoom handling did) - We're about 60% success here - Bug 366631 org.eclipse.ui.tests.presentations.PresentationsTestSuite.PresentationsTestSuite() - DOA we don't use this mechanism. Interesting, this has been 'updated' to use the model...maybe it's just we're checking against the wrond CTF ? Doesn't really matter, we should remove these IMO org.eclipse.ui.tests.internal.InternalTestSuite.InternalTestSuite() - 8 of 23 tests have failures, others are all green TrimLayoutTest: - DOA (Remy's fixes *may* help here) EditorActionBarsTest: 2 failures (fo 4) likely due to our not having a CoolBar ActionSetExpressionTest: All fail with "unable to find menu manager" Bug41931Test: - Fails with "Pre-test order is not correct"...page.getSortedParts() wrong ? - Bug 366632 ResourceMappingMarkersTest: - Fails with a CCE between ErrorPart and a show view of "org.eclipse.ui.tests.markerTests" - Bug 366633 WorkbenchWindowSubordinateSourcesTests: - All fail TextHandlerTest: - testEditableText fails on assertTrue of 'view.getCutAction().isEnabled()' StickyViewManagerTest: - All fail trying to set up by opening a secondary view 'page.showView("org.eclipse.ui.tests.api.MockViewPartMultSticky", "secondary", IWorkbenchPage.VIEW_ACTIVATE);' org.eclipse.ui.tests.themes.ThemesTestSuite.ThemesTestSuite() All Green....lol org.eclipse.ui.tests.intro.IntroTestSuite.IntroTestSuite() - 3 failures out of 14...1 of these seem bogus since they expect that the intro can be closed in one perspective and not in some other. - Bug 366634 org.eclipse.ui.tests.dynamicplugins.DynamicPluginsTestSuite.DynamicPluginsTestSuite() - 18 failures, 3 Errors of 76 tests, mostly around commands - Bug 366635
(In reply to comment #7) > org.eclipse.ui.tests.concurrency.ConcurrencyTestSuite.ConcurrencyTestSuite() > org.eclipse.ui.tests.statushandlers.StatusHandlingTestSuite.StatusHandlingTestSuite() > org.eclipse.ui.tests.systeminplaceeditor.OpenSystemInPlaceEditorTest > org.eclipse.ui.tests.navigator.NavigatorTestSuite.NavigatorTestSuite() These four are green.
In the last arch call we decided to get back to green builds and decided to disable the failing UI tests to achieve this. We can keep this bug to track the failures. Paul, I'm giving you the bug - feel free to delegate.
I've disabled the failing build tests via Bug 398733 PW
(In reply to comment #21) > I've disabled the failing build tests via Bug 398733 > > PW There are still 18 remaining failed tests in N20130122-2000: http://download.eclipse.org/eclipse/downloads/drops4/N20130122-2000/testresults/html/org.eclipse.ui.tests_macosx.cocoa.x86_5.0.html
We have multiple "aggregator" bugs for test failures or inactive tests. Lets combine them (Bug 485167). *** This bug has been marked as a duplicate of bug 485167 ***
