[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
[
List Home]
Re: [cross-project-issues-dev] hipp9 overloaded
|
Hi
On the 'ocl' hipp, I have had to increase the timeout on a 12 minute job
from 60 minutes to 180 to get it to run. This at 6:00 GMT on a Sunday!
While observing the progress interactively, it is my impression that the
JUnit tests are not dramatically slower, but anything involving repos or
GIT, presumably the network, crawls.
Regards
Ed Willink
On 05/02/2017 14:24, Stephan Herrmann wrote:
Could this be a general issue?
The factor of 4 looks similar to what I see in hipp10 (Object Teams).
Some figures:
Two individual test suites A and B as part of the Object Teams build
over time
(from some builds which I happened to keep):
up-to 2016-12-08:
suite A: around 30 min.
suite B: around 30 min.
2017-01-18:
suite A at 47 min.
suite B presumably timed out after 120 min (details no longer
available)
2017-01-22:
suite A at 44 min.
suite B completed at 110 min
2017-01-27:
suite A at 60 min.
suite B presumably timed out after 120 min (details no longer
available)
2017-02-04:
suite A timed out at 120 min
suite B died at 50 min (not sure if this could possibly be a time
out)
Several intermediate builds timed out, providing no useful test results.
To check if this could be a regression in the software being tested, I
inspected logs from
local test runs over the given period, which, however, show no
performance regression.
When I looked up the various hipps on this hipp10, I couldn't see much
info about
build history, in fact only the platform hipp has data in the build
history view.
The same suite A is actually part of JDT/Core so we can do some
comparison of the
Object Teams HIPP with eclipse.jdt.core-Gerrit on Platform HIPP (same
machine hipp10!).
On the latter HIPP we observe alternating time outs but also test runs
completing
normally at the typical 30 min.
In this situation I have to run each build several times until I
happen to get
complete test results. Of course these unnecessary test runs aggravate
the situation...
Short term work around: stop using hipp for testing, test only locally
(saves much time)
use hipp only for building / publishing - which will also take some
load from that machine.
Maybe we should all do this to retain good response times at least for
the actual
production builds?
best,
Stephan
On 02/02/2017 08:08 AM, Markus Knauer wrote:
I see similar problems on hipp8. Its reported load is between 1 and 2
at the moment, sometimes less, i.e. not very high, but the EPP
package builds are extremely slow, much slower than usual. A few
months ago in October these builds used to take 1:20h [1], now we
are close to 6 hours [2] which is 4 times slower for the same build.
In both cases, builds were running on a day during Simultaneous
Release week which rules out this as a reason for the increased
duration.
Maybe its worth to look at this host, too, as the increased build
time doesn't seem to be caused by a high load of the CPU.
And I highly appreciate your efforts to distribute CPU resources in a
better way!
Much thanks,
Markus
[1] https://hudson.eclipse.org/packaging/job/neon.epp-tycho-build/448/
[2] https://hudson.eclipse.org/packaging/job/oxygen.epp-tycho-build/183/
On 2 February 2017 at 01:22, Frederic Gurr <frederic.gurr@xxxxxxxxxxx
<mailto:frederic.gurr@xxxxxxxxxxx>> wrote:
Hi,
Unfortunately hipp9 has been experiencing a near-constant high load
during work hours. Due to a few projects hogging resources other
projects are severely constrained and have problems running their
builds
in a timely manner.
To fix this, I have taken the following near term steps:
- Papyrus HIPP
- disabled execution of concurrent builds for
"Papyrus-Gerrit-Oxygen"
- limited the number of executors to 2
- EMF Compare HIPP
- limited the number of executors to 2
I will also move the EMF Compare HIPP to a different hipp machine
tomorrow at 9am EST. Downtime will be less than one hour.
We will also investigate how CPU usage can be limited to ensure a
fair
distribution of resources.
Regards,
Fred
_______________________________________________
cross-project-issues-dev mailing list
cross-project-issues-dev@xxxxxxxxxxx
<mailto:cross-project-issues-dev@xxxxxxxxxxx>
To change your delivery options, retrieve your password, or
unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev
<https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev>
_______________________________________________
cross-project-issues-dev mailing list
cross-project-issues-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or
unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev
_______________________________________________
cross-project-issues-dev mailing list
cross-project-issues-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or
unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev
---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus