Re: [platform-releng-dev] First ever maintenance stream performance test

[David M Williams <david_williams@xxxxxxxxxx>]

John's advice is well timed. 

As people have been "pointing out
issues" I have tried "re-rerunning" the analysis program,
and at this exact moment, the "results displayed" appear to be
even more incorrect than they were :( 

I'm fairly sure I am beginning to understand
the issues, which I am tracking them in 
https://bugs.eclipse.org/bugs/show_bug.cgi?id=455035
and hope to get it sorted out by the
end of today (Friday, 12/12). 

Once I do this for "maintenance",
will likely re-run the analysis part of the scripts on other builds (such
as, the milestone). 

In this case, I won't apologize for
the churn :) ... as it is simply part of what must happen to understand
what was done before, and what the results should be, and much of that
will require participation of component leads and committers to verify
their data. 

Thanks for the comments/reviews so far!






From:      
 John Arthorne <John_Arthorne@xxxxxxxxxx>
To:      
 "Eclipse platform
release engineering list." <platform-releng-dev@xxxxxxxxxxx>,

Date:      
 12/11/2014 03:33 PM
Subject:    
   Re: [platform-releng-dev]
First ever maintenance stream performance        tests
are displayed ....
Sent by:    
   platform-releng-dev-bounces@xxxxxxxxxxx

---

Great work! Thanks to David and everyone
else who helped with getting these tests going again. There is a lot of
community interest in seeing these tests again, so we definitely want to
make the effort to ensure the results are accurate and useful, so they
don't give a misleading impression of our performance.

John 




From:        David
M Williams <david_williams@xxxxxxxxxx> 
To:        "Eclipse
platform release engineering list." <platform-releng-dev@xxxxxxxxxxx>
Date:        12/11/2014
02:15 PM 
Subject:        [platform-releng-dev]
First ever maintenance stream performance        tests
are displayed .... 
Sent by:        platform-releng-dev-bounces@xxxxxxxxxxx

---

Well, the resurrected performance tests, I should say. (first in a long
time, in other words) ... though, we've had the Mars stream performance
tests stumbling a long for a while now. 

And so far, only the "short set" have ran in maintenance, not
the long running ones. I will have to restart those by hand, and will wait
until the Milestone ones are done, so will likely be Monday before those
are ready for viewing.   

http://download.eclipse.org/eclipse/downloads/drops4/M20141210-0900/performance/performance.php

But, a) I was pretty happy it worked at all :)  and b) wanted to call
attention to "umbrella
bug" 454921 which is
where I will track bugs related to fixing or improving current "display
or workflow" of performance tests. 

I would appreciate especially component owners digging though a few performance
results (Luna and Mars) and if you see anything wrong, or things that could
be improved, if there is not already a bug for it, to open one, and mark
it blocking that umbrella bug. That will help me prioritize the remaining
work. Not to mention, if you have failing performance tests, to "comment
them out" until they can be fixed. (Especially if they fail in both
streams, especially on "current builds" and "baselines"
since those presumably would need some effort to debug and fix? Though,
maybe not in all cases (api framework tests for example, are failing  because
max heap size is set to a value smaller than minimum heap size? I'll open
bug and recommendations for that later (if, not obvious how to fix)  ...
have to run off this afternoon. 

Thanks, _______________________________________________
platform-releng-dev mailing list
platform-releng-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe
from this list, visit
https://dev.eclipse.org/mailman/listinfo/platform-releng-dev
_______________________________________________
platform-releng-dev mailing list
platform-releng-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe
from this list, visit
https://dev.eclipse.org/mailman/listinfo/platform-releng-dev