WATcloud Home Directory Quota

[ben-z]

Hi @WATonomous/watcloud-compute-cluster-users,

To enhance the stability and performance of the WATcloud compute cluster, we are implementing a new quota system for the shared filesystem (Ceph) that backs each user's home directory. This quota is in addition to the existing node-local disk quotas that apply to docker and scratch storage on general-use machines.

TLDR

The default quota for each user's home directory ($HOME) will be set to 20GiB. Please review your current storage usage, ensure your usage stays within the quota, and adjust your workloads accordingly prior to September 30, 2024. Please see the documentation for the latest information.

Why a New Quota?

The primary goal of this new quota is to make storage usage more predictable across the cluster. We've recently experienced incidents where Ceph became almost completely full. By setting limits on the amount of data each user can store in the shared filesystem, we can avoid sudden spikes in disk usage that could affect system performance and stability.

Default Per-User Quota

The default quota for each user's home directory will be set to 20GiB.

Transition Period

From now until September 30, 2024, we are in a transition period. During this time, the quota for each user is set to

$$ \max(\text{current}\textunderscore\text{usage} + 10\ \text{GiB},\ 20\ \text{GiB}) $$

This is to ensure that all users have adequate time to adjust to the new quota system without disruption to their work. After September 30, 2024, quotas will be set to the default (20GiB).

How to Check and Manage Your Quota

Details on how to check your current disk usage and quota limits, as well as instructions to request for quota increases are provided in the documentation.

Action Required

Please review your current storage usage and adjust your workloads accordingly. Should you have any concerns or if your workload requires more storage than allocated, you can request a quota increase as described in the documentation.

Thank you for your cooperation and understanding!

☁️

[ben-z]

Just a reminder that there are 10 days left in the transition period. Please review your home directory disk usage and adjust your workloads accordingly. You can see your current usage by running:

getfattr -n ceph.dir.rbytes "$HOME"
getfattr -n ceph.dir.rfiles "$HOME"

More information on quotas can be found in the docs.

[ben-z]

Quick reminder that the home directory quota will come into effect in 2 days. The default limit is 20 GiB (21,474,836,480 bytes). You can check your current usage (in bytes) by running this command on a general-use or SLURM node:

getfattr -n ceph.dir.rbytes "$HOME"

To learn more about quotas in the WATcloud cluster, please refer to https://cloud.watonomous.ca/docs/compute-cluster/quotas

[ben-z]

The home directory quota is now active in the WATcloud compute cluster. You can check your quota using:

getfattr -n ceph.quota "$HOME"

and your current usage using:

getfattr -n ceph.dir.rbytes "$HOME"
getfattr -n ceph.dir.rfiles "$HOME"

For additional details, please refer to the documentation.