Bug 343814 - Performance results are not getting generated
Summary: Performance results are not getting generated
Status: RESOLVED FIXED
Alias: None
Product: Platform
Classification: Eclipse Project
Component: Releng (show other bugs)
Version: 3.7   Edit
Hardware: PC Windows XP
: P3 normal (vote)
Target Milestone: ---   Edit
Assignee: Kim Moir CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-04-26 06:15 EDT by Satyam Kandula CLA
Modified: 2011-08-26 13:41 EDT (History)
4 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Satyam Kandula CLA 2011-04-26 06:15:51 EDT
Performance results are not generated for I20110424-2000 and N20110423-2000

The tests have run and the results are available in the database. It is just that the results have not been generated. 

I could find the console logs for the linux boxes but not for the windows boxes. 
http://download.eclipse.org/eclipse/downloads/drops/I20110424-2000/performance/consolelogs/linux.gtk.perf2_consolelog.txt

According to the log, the run has failed at the end trying to find the sources for the coverage reports. The performance tests shouldn't be running with coverage tool. There has been changes made for this, but I don't know why this is still happening. 

The performance tests shouldn't be running for windows machine and hence this shouldn't be impacted but I don't see the console logs as well as the generated results.
Comment 1 Kim Moir CLA 2011-04-26 09:17:58 EDT
The performance results are now available for I20110424-2000.  As is the log for the second windows performance machine. For some reason, the tests are completing, but not giving up their network connection.  I had to kill the rsh
connection manually so the build could proceed to a state where the results could be generating. As for the tests failing generating the jacoco report, I have reopened bug 342785.
Comment 2 Satyam Kandula CLA 2011-05-03 04:43:31 EDT
(In reply to comment #1)
> The performance results are now available for I20110424-2000.  As is the log
> for the second windows performance machine. For some reason, the tests are
> completing, but not giving up their network connection.  I had to kill the rsh
> connection manually so the build could proceed to a state where the results
> could be generating. As for the tests failing generating the jacoco report, I
> have reopened bug 342785.
Kim, 
Thanks for taking care of the results for I20110424-2000. I do see the performance results for I20110428-0848 and not for N20110430-2000. Do you think the problem is happening some times and it is good some times?
Comment 3 Kim Moir CLA 2011-05-03 15:38:06 EDT
The rsh connections to the test machines keep hanging and preventing the performance results from being generated.  The results for http://download.eclipse.rog/eclipse/downloads/drops/N20110430-2000/performance/performance.php

are now available
Comment 4 Deepak Azad CLA 2011-05-05 08:11:52 EDT
The results are missing for I20110504-0800 :(
Comment 5 Kim Moir CLA 2011-05-05 09:19:46 EDT
They are being generated now. Again, the rsh connections to the windows machines stay open after the tests have completed, delaying the results generation. I'm still trying to find the root cause of the problem.
Comment 6 Kim Moir CLA 2011-05-10 09:39:43 EDT
Talked to our sysadmin and there haven't been any changes to the machines lately. He also said that there aren't any issues with long running network connections on the machines. Since I'm at a loss at how to solve this, I'll implement something like this to kill rsh processes that are over a day old.

ps -eo uid,pid,etime,cmd | egrep '^ *507' | grep rsh | egrep ' ([1-9]+-)?([0-9]{2}:?){3}' | awk '{print $2}'| xargs -I{} kill {}
Comment 8 Kim Moir CLA 2011-05-11 10:09:54 EDT
I added 
30 6 * * *  ps -eo uid,pid,etime,cmd | egrep '^ *507' | grep rsh | egrep '([1-9]+-)?([0-9]{2}:?){3}' | awk '{print $2}'| xargs kill -9

to the crontab to kill defunct rsh processes that are delaying the performance results generation.  

Also, the results for I20110510-0800 are now being generated.  Hopefully tomorrow they will be available more quickly.
Comment 9 Kim Moir CLA 2011-08-26 13:41:14 EDT
I think this can be closed.