Bug 408748 - [test] equinox.p2.tests.discovery and equinox.p2.tests.ui are not run during tests
Summary: [test] equinox.p2.tests.discovery and equinox.p2.tests.ui are not run during ...
Status: VERIFIED FIXED
Alias: None
Product: Platform
Classification: Eclipse Project
Component: Releng (show other bugs)
Version: 4.3   Edit
Hardware: PC Mac OS X
: P3 normal (vote)
Target Milestone: 4.3 RC3   Edit
Assignee: David Williams CLA
QA Contact:
URL:
Whiteboard:
Keywords: test
: 403602 (view as bug list)
Depends on:
Blocks:
 
Reported: 2013-05-22 16:03 EDT by Thomas Watson CLA
Modified: 2013-05-29 03:11 EDT (History)
5 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Thomas Watson CLA 2013-05-22 16:03:06 EDT
Looks like these two test suites have not run since M5a which was the last milestone before migrating to CBI:

equinox.p2.tests.discovery
equinox.p2.tests.ui

I assume this is just some configuration issue that prevented these suites from being included in the test pass.
Comment 1 David Williams CLA 2013-05-22 16:47:22 EDT
The latter has this comment: 

    <!--
       disable this tests for mac, for now, since always DNSs. See
       https://bugs.eclipse.org/bugs/show_bug.cgi?id=390392
    -->
    <target
        name="equinoxp2ui"
        depends="setJVMProperties, checkOS"
        unless="isMac">
        <property
            name="jvm"
            value="${J2SE-5.0}" />
        <!--only run test if J2SE-5.0 property  set-->
        <condition property="skip.test">
            <not>
                <isset property="J2SE-5.0" />
            </not>
        </condition>
        <runTests testPlugin="org.eclipse.equinox.p2.tests.ui" />
    </target>

for former says it "depends on" 1.5 JRE ... which, seem unliky to be true, any longer. Can I just remove these "restrictions and conditions"?
Comment 2 Pascal Rapicault CLA 2013-05-23 13:56:09 EDT
The tests are probably still failing on Mac so I don't know if we can bypass that.
As for the required VM, I don't see why this is needed as I've been running these tests on a 1.7 VM and that the p2 core tests (which the UI tests depend on) require at least a 1.6 VM.
Comment 3 David Williams CLA 2013-05-23 21:25:56 EDT
I think unit test are fair game, even in RC3? Right? 
So, at a minimum, I'll remove the 1.5 condition. (My guess it that's a left over from the days when tests were ran with 1.4 by default!?) 

I will also try run with mac re-enabled. Since it was reported they "always timed out", there have been LOTS of changes in EF Infrastructure, AND the way we set up our tests, to use whatever proxies etc., they happen to have in use. If Mac tests "time out" even once, I'll disable them for mac again until that can be investigated.
Comment 4 Pascal Rapicault CLA 2013-05-23 21:26:49 EDT
Thanks David!
Comment 5 David Williams CLA 2013-05-25 12:43:36 EDT
This commit removes the "Java 5" requirement. 

http://git.eclipse.org/c/platform/eclipse.platform.releng.aggregator.git/commit/?id=d91dcc9c802e792d9f321dc38344a4177041fa70


This commit removes the "not mac" requirement. This second one is the one that might have to be reverted, if in fact we still see constant time-outs on Mac test machine .... until that can be investigated and fixed. 

http://git.eclipse.org/c/platform/eclipse.platform.releng.aggregator.git/commit/?id=76d02d3ee02e58cf650c9ab13695053bd84889f5
Comment 6 David Williams CLA 2013-05-25 23:57:28 EDT
This didn't seem suffice. 

I do see one I "missed", 

<runTests testPlugin="org.eclipse.equinox.p2.tests" />

But also need to sort through lists, and make sure still called. 

If they are, then may be more subtle "setup" issues?
Comment 7 David Williams CLA 2013-05-26 00:27:32 EDT
This commit handles the remaining cases mentioned. 

http://git.eclipse.org/c/platform/eclipse.platform.releng.aggregator.git/commit/?id=5c1f9b8f072ebe377043833ac76c5ea51ed4dfaa

But, not sure how that one unit test not running can prevent others from running, unless there is some implicit "setup type dependency" where they must be ran in order, or something". 

The other factor is at least some of these tests depend on several "property files", such as 

    <target
        name="installExtraPlugins"
        if="extraIU">
        <loadproperties srcfile="equinoxp2tests.properties" />

and 

       <property
            name="currentBuildRepo"
            value="${org.eclipse.equinox.p2.tests.current.build.repo}" />


May have to dig through logs to find how some of these values are set at test time. (But, I will say, if this gets too complicated, I'll work with p2 team to "move down" some of this logic/setup to their own tests suites. As far as I know, there is no reason the "test infrastructure" needs to do it". As far as I know).
Comment 8 David Williams CLA 2013-05-26 21:03:05 EDT
From the build at 
http://download.eclipse.org/eclipse/downloads/drops4/I20130526-0500/

it seems most recent change was sufficient to get 

org.eclipse.equinox.p2.tests

to run on the Mac! 

But ... still no 

 equinox.p2.tests.discovery or equinox.p2.tests.ui

From studying the log, and the bundles themselves, my hypothesis is those test bundles are "jarred", instead of taking the directory shape. 

And, ... I've read some places that Tycho does not pay attention to feature settings on whether to unpack or not, it goes entirely by the 
Eclipse-BundleShape: dir
in the MANIFEST.MF file. 

I've not looked exhaustively, but seems most our test bundles do have 
Eclipse-BundleShape: dir

Pascal, can you you add this directive to those two bundles? 

The 'org.eclipse.equinox.p2.tests' does have one.
Comment 9 Pascal Rapicault CLA 2013-05-26 21:10:08 EDT
> Pascal, can you you add this directive to those two bundles? 
   Done http://git.eclipse.org/c/equinox/rt.equinox.p2.git/commit/?id=0ca9c7abe52ea7df9d6ccf69cabf08d0870284cb
Comment 10 David Williams CLA 2013-05-26 22:10:30 EDT
Just to document it, here's the reference that talks about the limitation: 

http://wiki.eclipse.org/Tycho/Packaging_Types#eclipse-feature

I don't see any bug open to support "unpack" in feature.xml ... doubt they ever would ... since features do not "drive" the build ... but, guess they could tweak the p2 metadata as they go. At any rate, I opened Tycho bug 409097.
Comment 11 David Williams CLA 2013-05-26 22:22:04 EDT
I'm pretty confident all these changes will fix this bug. 

The reason is I have done a comprehensive search of one of the full console log, and most of the time, while "finding" the plug to test, there are two echos, one based on name* and another based on name*/test.xml. 

For example, 

[echo] trying to find org.eclipse.equinox.security.tests_*
[echo] trying to find org.eclipse.equinox.security.tests_1.0.100.v20130327-1442\test.xml

And only two showed "blank" looking for test.xml:

[echo] trying to find org.eclipse.equinox.p2.tests.ui_*
[echo] trying to find  

[echo] trying to find org.eclipse.equinox.p2.tests.discovery_*
[echo] trying to find  

We won't know until the 10/27 8 PM build, but will mark fixed for now. 
(And, guess there's always a chance we'll have to put back in the "mac check", if we get hit the 2 hour time limit for any of those added back in)
Comment 12 David Williams CLA 2013-05-27 18:07:36 EDT
Verified in I20130527-0800 build. All three suites ran and produced reasonable looking results. (p2 team would have to check tests/logs themselves to confirm no hidden issues, but on the surface they are "back in the tests" and "running".
Comment 13 Pascal Rapicault CLA 2013-05-27 21:52:11 EDT
I confirm that all the tests are now being executed. Thanks for your time on this David.
Comment 14 Pascal Rapicault CLA 2013-05-27 21:53:03 EDT
*** Bug 403602 has been marked as a duplicate of this bug. ***