Re: [cross-project-issues-dev] Disk usage report for HIPP/JIPPs

[Frederic Gurr <frederic.gurr@xxxxxxxxxxxxxxxxxxxxxx>]

Hi,

Thanks Jonah for sharing the knowledge!
I will add this to the best practices document and run a check for such
stale workspace directories.

Regards,

Fred


-- 
Frederic Gurr

Release Engineer
Eclipse Foundation

On 27.02.2018 21:07, Jonah Graham wrote:
> Hello folks,
> 
> For the last little while I have been working on reducing the disk
> usage of CDT in drips and drabs, and until now I couldn't figure out
> where the disk space was going as Jenkins disk usage[1] did not
> account for much of the space that the report below showed. I thought
> I would share this with you as it may help other projects.
> 
> It turns out lots of our disk space is being used by workspaces for
> concurrent jobs. These are named workspace@2, workspace@3 etc.
> Unfortunately, the Jenkins web interface provides no access to these
> additional workspaces AFAICT. This makes it difficult to follow the
> recommendations[2].
> 
> This is made even worse by the upgrade from Hudson to Jenkins as they
> use different naming standards. Hudson calls them workspace_2,
> workspace_3, etc. On CDT there was 22GB being occupied in these
> workspaces and these directories are neither visible nor utilised by
> Jenkins.
> 
> I have removed all the excess from CDT, cutting down our disk usage by
> 32GB with this step alone.
> 
> You can see the build job[3] that I used to do the cleanup. The key
> cleanup lines being:
> find /jobs/genie.PROJECTNAMEHERE -name workspace_\* -print0 | xargs -0 rm -rf
> find /jobs/genie.PROJECTNAMEHERE -name workspace@\* -print0 | xargs -0 rm -rf
> 
> I hope you find this useful in your attempts to help projects reduce
> unneeded disk space usage. This should also keep CDT off the "naughty"
> list for a while :-)
> 
> [1] https://ci.eclipse.org/cdt/plugin/disk-usage/
> [2] https://wiki.eclipse.org/CI_best_practices#Clean_workspace
> [3] https://ci.eclipse.org/cdt/view/Releng/job/jonah-releng-temp/21/console
> 
> Jonah
> ~~~
> Jonah Graham
> Kichwa Coders Ltd.
> www.kichwacoders.com
> 
> 
> On 27 February 2018 at 18:30,  <genie@xxxxxxxxxxx> wrote:
>> Compiled 2018-02-27-13:30:02
>>
>> Projects exceeding 80G of disk space on HIPP/JIPPs:
>> ===================================================
>> 807G /jobs/genie.packaging
>> 221G /jobs/genie.openj9
>> 174G /jobs/genie.capella
>> 148G /jobs/genie.kitalpha
>> 141G /jobs/genie.webtools
>> 133G /jobs/genie.platform
>> 122G /jobs/genie.xtext
>> 114G /jobs/genie.papyrus
>> 102G /jobs/genie.ice
>> 96G /jobs/genie.app4mc
>> 92G /jobs/genie.osee
>> 85G /jobs/genie.cdt
>> 80G /jobs/genie.ecp
>> ===================================================
>>
>> If you find your project's name in the above list, please check
>> your Hudson/Jenkins configuration and try to lower the disk usage.
>>
>> You can find more information about CI best practices and disk usage
>> here: https://wiki.eclipse.org/CI_best_practices#Disk_usage.
>> _______________________________________________
>> cross-project-issues-dev mailing list
>> cross-project-issues-dev@xxxxxxxxxxx
>> To change your delivery options, retrieve your password, or unsubscribe from this list, visit
>> https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev
> _______________________________________________
> cross-project-issues-dev mailing list
> cross-project-issues-dev@xxxxxxxxxxx
> To change your delivery options, retrieve your password, or unsubscribe from this list, visit
> https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev
>