Re: [platform-ui-dev] Running individual performance tests

[Nick Edgar <Nick_Edgar@xxxxxxxxxx>]

I think we should structure the performance tests so that each component 
tests its own layer of the cake, but does so under different loads.
So, we should not reference JDT or even cheatsheets, but we can construct 
mock parts that are more heavyweight if needed.  Kim's framework to allow 
different variations should enable this.
I would expect JDT to bench the cost of switching to the Java perspectives 
from a simpler one like Resource (and back), opening JDT views and 
editors, etc.

Nick




Stefan Xenos/Ottawa/IBM@IBMCA 
Sent by: platform-ui-dev-admin@xxxxxxxxxxx
09/16/2004 02:27 AM
Please respond to
platform-ui-dev


To
platform-ui-dev@xxxxxxxxxxx
cc

Subject
Re: [platform-ui-dev] Running individual performance tests







Kim, 


Nice job with the UI performance framework. I added several new 
performance tests tonight, and it was really easy to figure out your 
pattern. 

(BTW, your code is really clean, so if my checkin tonight doesn't fit in 
with your plans, feel free to roll it back or let me know what needs 
fixing). 


I ran into one stumbling block, though: there doesn't seem to be a good 
place to put the tests I want to write. To benchmark perspective switching 
properly, we need two perspectives with lots of actions and context and an 
active editor that is similarly complex. Our tests currently use 
minimalist views, editors, and perspectives, so they're giving the 
artificially optimistic results for part activation and perspective 
switching (I got 170ms for perspectives). To fix this, I added a test that 
switches between the debug and java perspectives with a java editor open. 
This started to give more realistic numbers (980ms)... but it seems to go 
against the grain of the ui test suites, which avoid making any reference 
to jdt plugins. My current hack only references them by ID and disables 
the test if they can't be found, but there's got to be a better way. 

I'm having a similar problem with cheatsheets. I've added some nice 
infrastructure for benchmarking layouts, but a lot of the layouts that 
we'd like to test are contributed by downstream plugins. For example, the 
cheatsheet tests would really benefit from some layout benchmarks, but 
they don't reference org.eclipse.ui.tests, so they can't use my 
infrastructure (or yours). 

Is there already a good place for this stuff, or do we need to create some 
new test plugins? 

  - Stefan 




Kim Horne/Ottawa/IBM@IBMCA 
Sent by: platform-ui-dev-admin@xxxxxxxxxxx 
09/13/2004 02:09 PM 

Please respond to
platform-ui-dev


To
platform-ui-dev@xxxxxxxxxxx 
cc

Subject
[platform-ui-dev] Running individual performance tests









The UI performance tests currently rely on the fact that they're run after 
being wrapped with a UIPerformanceTestSetup TestSetup object.  As such, 
attempting to run any given set of tests outside of the main performance 
test suite will result in errors (missing resources, primarily).  In order 
to run individual test classes I've checked in 
org.eclipse.ui.tests.performance.PerformanceTestRunner.  This class will 
wrapper the class specified by the "org.eclipse.ui.performance.test" 
system property with the appropriate test setup so that it may be run 
seperate from the entire test suite.  I'm going ot investigate making this 
easier (read: transparant) in the future.