Re: [platform-ui-dev] Running individual performance tests

[Stefan Xenos <Stefan_Xenos@xxxxxxxxxx>]

Agreed. 

Unfortunately, the biggest bottleneck
for the java-to-debug-perspective-switch is currently in the UI (in the
commands framework)... and until we have a chance to write an equally heavyweight
editor and pair of perspectives, we have no other test coverage for this
bottleneck. I guess we should try to push the infrastructure for "switch
perspective test" up to the common tests plugin and the actual test
case down to the JDT plugin.

Kim also made an interesting suggestion:
we could try to take a snapshot of the 3.0.1 JDT plugins and include them
with our tests. That would let us test part activation and switching perspectives
with "real" views/editors/perspectives, but it would mean that
subsequent changes to JDT won't impact our performance tests.

  - Stefan

Nick Edgar/Ottawa/IBM@IBMCA
Sent by: platform-ui-dev-admin@xxxxxxxxxxx

09/16/2004 10:52 AM

Please respond to
platform-ui-dev

To	platform-ui-dev@xxxxxxxxxxx
cc
Subject	Re: [platform-ui-dev] Running individual performance tests

I think we should structure the performance tests
so that each component 
tests its own layer of the cake, but does so under different loads.
So, we should not reference JDT or even cheatsheets, but we can construct
mock parts that are more heavyweight if needed.  Kim's framework to
allow 
different variations should enable this.
I would expect JDT to bench the cost of switching to the Java perspectives
from a simpler one like Resource (and back), opening JDT views and 
editors, etc.

Nick




Stefan Xenos/Ottawa/IBM@IBMCA 
Sent by: platform-ui-dev-admin@xxxxxxxxxxx
09/16/2004 02:27 AM
Please respond to
platform-ui-dev


To
platform-ui-dev@xxxxxxxxxxx
cc

Subject
Re: [platform-ui-dev] Running individual performance tests







Kim, 


Nice job with the UI performance framework. I added several new 
performance tests tonight, and it was really easy to figure out your 
pattern. 

(BTW, your code is really clean, so if my checkin tonight doesn't fit in
with your plans, feel free to roll it back or let me know what needs 
fixing). 


I ran into one stumbling block, though: there doesn't seem to be a good
place to put the tests I want to write. To benchmark perspective switching
properly, we need two perspectives with lots of actions and context and
an 
active editor that is similarly complex. Our tests currently use 
minimalist views, editors, and perspectives, so they're giving the 
artificially optimistic results for part activation and perspective 
switching (I got 170ms for perspectives). To fix this, I added a test that
switches between the debug and java perspectives with a java editor open.
This started to give more realistic numbers (980ms)... but it seems to
go 
against the grain of the ui test suites, which avoid making any reference
to jdt plugins. My current hack only references them by ID and disables
the test if they can't be found, but there's got to be a better way. 

I'm having a similar problem with cheatsheets. I've added some nice 
infrastructure for benchmarking layouts, but a lot of the layouts that
we'd like to test are contributed by downstream plugins. For example, the
cheatsheet tests would really benefit from some layout benchmarks, but
they don't reference org.eclipse.ui.tests, so they can't use my 
infrastructure (or yours). 

Is there already a good place for this stuff, or do we need to create some
new test plugins? 

  - Stefan 




Kim Horne/Ottawa/IBM@IBMCA 
Sent by: platform-ui-dev-admin@xxxxxxxxxxx 
09/13/2004 02:09 PM 

Please respond to
platform-ui-dev


To
platform-ui-dev@xxxxxxxxxxx 
cc

Subject
[platform-ui-dev] Running individual performance tests









The UI performance tests currently rely on the fact that they're run after
being wrapped with a UIPerformanceTestSetup TestSetup object.  As
such, 
attempting to run any given set of tests outside of the main performance
test suite will result in errors (missing resources, primarily).  In
order 
to run individual test classes I've checked in 
org.eclipse.ui.tests.performance.PerformanceTestRunner.  This class
will 
wrapper the class specified by the "org.eclipse.ui.performance.test"
system property with the appropriate test setup so that it may be run 
seperate from the entire test suite.  I'm going ot investigate making
this 
easier (read: transparant) in the future. 

_______________________________________________
platform-ui-dev mailing list
platform-ui-dev@xxxxxxxxxxx
http://dev.eclipse.org/mailman/listinfo/platform-ui-dev