Re: [platform-ui-dev] Running individual performance tests

[Michael Van Meekeren <Michael_Van_Meekeren@xxxxxxxxxx>]

Stefan,

I think you're talking abou tests that
test the system vs tests that test a particular feature or unit.  What
Kim is intending for the current tests was the fact that you have minimal
tests where the a particular editor (for example) exercises a single feature
in the UI.  This enables us to see that if the test goes bad (slow
etc...) it is likely due to this one thing.  Havaing many of these
on different aspects of the UI allows us to separate the components and
quickly identify where problems occur.  There likely will be interaction
issues etc.. that don't get caught in this case and that should be tested
(I think this is your point) and at the same time the more complex a test
becomes the more difficult it is to see smaller variations in individual
parts or whether multiple problems introduced are contributing to a poor
result etc...

/michael

Stefan Xenos/Ottawa/IBM@IBMCA
Sent by: platform-ui-dev-admin@xxxxxxxxxxx

09/16/2004 02:27 AM

Please respond to
platform-ui-dev

To	platform-ui-dev@xxxxxxxxxxx
cc
Subject	Re: [platform-ui-dev] Running individual performance tests

Kim, 


Nice job with the UI performance framework. I added several new performance
tests tonight, and it was really easy to figure out your pattern. 

(BTW, your code is really clean, so if my checkin tonight doesn't fit in
with your plans, feel free to roll it back or let me know what needs fixing).


I ran into one stumbling block, though: there doesn't seem to be a good
place to put the tests I want to write. To benchmark perspective switching
properly, we need two perspectives with lots of actions and context and
an active editor that is similarly complex. Our tests currently use minimalist
views, editors, and perspectives, so they're giving the artificially optimistic
results for part activation and perspective switching (I got 170ms for
perspectives). To fix this, I added a test that switches between the debug
and java perspectives with a java editor open. This started to give more
realistic numbers (980ms)... but it seems to go against the grain of the
ui test suites, which avoid making any reference to jdt plugins. My current
hack only references them by ID and disables the test if they can't be
found, but there's got to be a better way. 

I'm having a similar problem with cheatsheets. I've added some nice infrastructure
for benchmarking layouts, but a lot of the layouts that we'd like to test
are contributed by downstream plugins. For example, the cheatsheet tests
would really benefit from some layout benchmarks, but they don't reference
org.eclipse.ui.tests, so they can't use my infrastructure (or yours).

Is there already a good place for this stuff, or do we need to create some
new test plugins? 

  - Stefan

Kim Horne/Ottawa/IBM@IBMCA
Sent by: platform-ui-dev-admin@xxxxxxxxxxx

09/13/2004 02:09 PM

Please respond to
platform-ui-dev

To	platform-ui-dev@xxxxxxxxxxx
cc
Subject	[platform-ui-dev] Running individual performance tests

The UI performance tests currently rely on the fact that they're run after
being wrapped with a UIPerformanceTestSetup TestSetup object.  As
such, attempting to run any given set of tests outside of the main performance
test suite will result in errors (missing resources, primarily).  In
order to run individual test classes I've checked in org.eclipse.ui.tests.performance.PerformanceTestRunner.
 This class will wrapper the class specified by the "org.eclipse.ui.performance.test"
system property with the appropriate test setup so that it may be run seperate
from the entire test suite.  I'm going ot investigate making this
easier (read: transparant) in the future.