Re: [cross-project-issues-dev] Disk usage report for HIPP/JIPPs

[Jonah Graham <jonah@xxxxxxxxxxxxxxxx>]

Hello folks,

For the last little while I have been working on reducing the disk
usage of CDT in drips and drabs, and until now I couldn't figure out
where the disk space was going as Jenkins disk usage[1] did not
account for much of the space that the report below showed. I thought
I would share this with you as it may help other projects.

It turns out lots of our disk space is being used by workspaces for
concurrent jobs. These are named workspace@2, workspace@3 etc.
Unfortunately, the Jenkins web interface provides no access to these
additional workspaces AFAICT. This makes it difficult to follow the
recommendations[2].

This is made even worse by the upgrade from Hudson to Jenkins as they
use different naming standards. Hudson calls them workspace_2,
workspace_3, etc. On CDT there was 22GB being occupied in these
workspaces and these directories are neither visible nor utilised by
Jenkins.

I have removed all the excess from CDT, cutting down our disk usage by
32GB with this step alone.

You can see the build job[3] that I used to do the cleanup. The key
cleanup lines being:
find /jobs/genie.PROJECTNAMEHERE -name workspace_\* -print0 | xargs -0 rm -rf
find /jobs/genie.PROJECTNAMEHERE -name workspace@\* -print0 | xargs -0 rm -rf

I hope you find this useful in your attempts to help projects reduce
unneeded disk space usage. This should also keep CDT off the "naughty"
list for a while :-)

[1] https://ci.eclipse.org/cdt/plugin/disk-usage/
[2] https://wiki.eclipse.org/CI_best_practices#Clean_workspace
[3] https://ci.eclipse.org/cdt/view/Releng/job/jonah-releng-temp/21/console

Jonah
~~~
Jonah Graham
Kichwa Coders Ltd.
www.kichwacoders.com


On 27 February 2018 at 18:30,  <genie@xxxxxxxxxxx> wrote:
> Compiled 2018-02-27-13:30:02
>
> Projects exceeding 80G of disk space on HIPP/JIPPs:
> ===================================================
> 807G /jobs/genie.packaging
> 221G /jobs/genie.openj9
> 174G /jobs/genie.capella
> 148G /jobs/genie.kitalpha
> 141G /jobs/genie.webtools
> 133G /jobs/genie.platform
> 122G /jobs/genie.xtext
> 114G /jobs/genie.papyrus
> 102G /jobs/genie.ice
> 96G /jobs/genie.app4mc
> 92G /jobs/genie.osee
> 85G /jobs/genie.cdt
> 80G /jobs/genie.ecp
> ===================================================
>
> If you find your project's name in the above list, please check
> your Hudson/Jenkins configuration and try to lower the disk usage.
>
> You can find more information about CI best practices and disk usage
> here: https://wiki.eclipse.org/CI_best_practices#Disk_usage.
> _______________________________________________
> cross-project-issues-dev mailing list
> cross-project-issues-dev@xxxxxxxxxxx
> To change your delivery options, retrieve your password, or unsubscribe from this list, visit
> https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev