Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [cross-project-issues-dev] Scalability tests and execution time measurements on Hudson

Some time ago we set up a slave called hudson-perf1-tests.  This slave had a dedicated CPU core and only one thread executor, and in our own tests, over a 24h period of mixed activity levels, our performance results have been consistent. 

https://hudson.eclipse.org/hudson/computer/hudson-perf1-tests/

We (webmasters) have not been able to confirm if the slave produces consistent performance indicators when run in Hudson, however, so please, feel free to try it out and report back.

Please see bug 333594 for more background.

Denis


On 04/11/2011 08:22 AM, fgiquel@xxxxxxxxxxxxxxxx wrote:
Hi,

the simultaneous release page, http://eclipse.org/indigo/planning/EclipseSimultaneousRelease.php, describes the following requirement :

"Performance. Project should have measurable performance criteria that are regularly tested against. Projects should devote at least one milestone to performance and scalability improvements."

We have some non reg scalability tests dealing with memory usage for some of our components (MoDisco project) and we are thinking about additional tests for execution time non regression. We are asking ourselves how a relevant time measurement can be done on continuous integration on Hudson ... Hudson CPU availability is inconstant depending on use made from the different eclipse project teams, so one "basic" execution time measurement (using System.currentTimeMillis()) has no meaning.


Has anyone met this kind of problem ? Does the eclipse architecture council have some recommendations about that (did not find in http://wiki.eclipse.org/Architecture_Council/Top_Ten_Project_Development_Practices) ?


Fabien Giquel.

Back to the top