| Re: [platform-releng-dev] Test environment / isolation? |
Hi Stephan,
For the official build we use the SDK build built during our build process. So it will contain all the components(excluding test components and their dependencies).
During the test for a particular component
1. Install test bundles for that component including dependencies(these are picked up from the Automated tests bundle available in our download page)
2. Run the tests defined in test.xml
3. Remove test bundles
This process is repeated for all the components. During the test we have full SDK and test bundles for the component under test(and dependencies) are installed.
Hope this helps
Thanks
Sravan
-----Original Message-----
From: Stephan Herrmann [mailto:stephan.herrmann@xxxxxxxxx]
Sent: Sunday, February 05, 2017 10:59 PM
To: Eclipse platform release engineering list. <platform-releng-dev@xxxxxxxxxxx>
Subject: [platform-releng-dev] Test environment / isolation?
Hi,
In the past we have seen some ways in which mere presence of, e.g., org.eclipse.jdt.launching could influence JDT/Core tests.
Today I reproduced a case, where presence of o.e.jdt.debug.tests causes exceptions in a JDT/Core test (didn't actually fail the test, but still looks brittle).
This lets me ask: what level of test isolation can we assume for official builds? Are tests for each component run in an environment that only contains this component and its dependencies, or are all components of the SDK installed into one big thing and then all tests run on top of that?
FYI, the recent failure involved the following chain of events:
- jdt.core:
Test JavaProjectTests.testBug462756() is run in headless mode
- jdt.launching:
JREPreferenceModifyListener.preApply() is triggered
- jdt.debug.tests:
VMInstallTestsLibraryLocationResolver.getSourcePath() is called
- the latter tries to DebugUIPlugin.start()
which fails because the workbench has not been started (since running headlessly)
Is it OK to assume that such interaction will not occur during builds, or should we make our tests robust against such?
thanks,
Stephan
_______________________________________________
platform-releng-dev mailing list
platform-releng-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit https://dev.eclipse.org/mailman/listinfo/platform-releng-dev