Bug 163531 - add performance testing to overnight build
Summary: add performance testing to overnight build
Status: NEW
Alias: None
Product: CDT
Classification: Tools
Component: cdt-core (show other bugs)
Version: 4.0   Edit
Hardware: PC Windows 2000
: P3 normal (vote)
Target Milestone: ---   Edit
Assignee: Project Inbox CLA
QA Contact: Jonah Graham CLA
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2006-11-06 08:19 EST by Andrew Ferguson CLA
Modified: 2020-09-04 15:27 EDT (History)
2 users (show)

See Also:


Attachments
adds the necessary parts to the releng and test plugins to enable the performance test framework (14.49 KB, patch)
2006-12-06 13:01 EST, Andrew Ferguson CLA
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Andrew Ferguson CLA 2006-11-06 08:19:54 EST
the org.eclipse.test.performance plug-in allows unit tests to persist resource usage to a database, and to make assertions about resource usage. It also includes a report generation facility for generating graphs of performance over time.

I'd like to enable this in the cdt overnight build, and aim to start work on this soon
Comment 1 Andrew Ferguson CLA 2006-11-27 15:14:31 EST
I've got this working on my local linux box, the one remaining issue seems to be that when the report generation utility is launched, it thinks its running on (linux,x86,motif) rather than (linux,x86,gtk) and fails to resolve its bundle dependencies
Comment 2 Andrew Ferguson CLA 2006-12-06 13:01:15 EST
Created attachment 55148 [details]
adds the necessary parts to the releng and test plugins to enable the performance test framework

this patch adds the necessary parts to the releng and test plugins to enable the performance test framework

see http://www.eclipsecon.org/2005/presentations/EclipseCon2005_13.2ContinuousPerformance.pdf
for details of the framework

Its not quite in the form I tested it in as I had to adapt the map files to use the COPY fetch factory rather than go via CVS

the build.sh script will now fail unless eclipseDist and eclipse.perf.dbloc are both set. This will not stop people running the build locally but they would need to specify a dummy location to write performance data to (e.g. /var/tmp/cdtperfDB)

the patch includes a DemoPerformanceTest that shows how this could be used

the two questions left are
 (1) the performance test results are generated as php files - are these supported in the current location? for example, the server supports them here
http://download.eclipse.org/eclipse/downloads/drops/R-3.2.1-200609210945/performance/performance.php
 (2) what do we use a baseline for performance comparison? the presentation above suggests branching a known good build, and back-porting performance tests to this. I'm not sure how this would relate to CDT 4.0..