Storage Quotas¶

Disk quotas are used to monitor and limit storage consumption on the cluster. Quotas are typically enforced per user and sometimes per project, depending on the filesystem.

Two limits are usually enforced:

Storage usage – total amount of disk space used
File count (inodes) – total number of files

Storage consumption is measured in allocated blocks and number of files.

Determine Quotas¶

Users can determine their quota usage by running the following command:

quota -s

User quota on /home

Disk quotas for user <user> (uid <user_id>):
Filesystem   space   quota   limit   grace   files   quota   limit   grace
store-nfs-ib:/storage/home
              507G   1000G   1024G           2196k*   2048k   2098k

An asterisk (*) indicates the user has exceeded the quota, and writes to /home/ are not allowed.

Command du -sh can be used to view the size of the respective folders:

du -sh /home/<user>/<dir>

Alternatively, users can use the dust command, which prints a tree structure of the directories. The depth of the tree can be specified with the -d parameter.

Output of dust command

login01:~$ dust /home/<user>/<dir>
  640G ─┬ .
  523G  ├─┬ ADMIN
  227G  │ ├─┬ qe
  226G  │ │ └─┬ benchmarks-master
  209G  │ │   └─┬ GRIR443
   59G  │ │     ├── GRIR.wfc1
   29G  │ │     ├── GRIR.wfc2
   14G  │ │     └── GRIR.wfc4
  128G  │ ├─┬ cp2k
  128G  │ │ └─┬ FIST
   69G  │ │   ├── nodes
   55G  │ │   └── nodes_new
   71G  │ ├─┬ python
   71G  │ │ └─┬ miniconda3
   36G  │ │   ├── envs
   34G  │ │   └── pkgs
   60G  │ └── orca
   68G  ├─┬ MD
   67G  │ └── Softbark_wood
   22G  └─┬ Docking
   16G    └── COVID19

Quotas by Filesystems¶

Quota limits differ depending on the storage location.

Home Directory¶

Home directories /home/<user>/ are intended for source code, configuration files, and small datasets:

Devana

Property	Limit
Storage quota	1 TB
Hard limit	1.024 TB
Number of files	2,048,000 files

Project Storage¶

Project storage /project/<project_id> is shared across the HPC clusters and is intended for shared data within a research project.

Project Quotas

Project quotas vary depending on the submitted project and apply to both clusters.

Scratch Storage¶

Scratch storage /scratch/<project_id> is intended for temporary data generated during computations.

Scratch Quotas

Since scratch storage should contain only temporary files, and important results should be moved to project or long-term storage, this storage is currently not subject to quotas on either cluster.

Exceeded Quotas¶

If you exceed your storage quotas, you have several options:

Remove unnecessary files and directories to free up space.
Compress and archive large files to optimize storage. Since write permissions in /home may be suspended, you can first move data to /scratch, compress it, and then store it elsewhere.
- Compress files:
```
tar czvf /scratch/<project_id>/data_backup.tgz /home/<user>/<Directories-files-to-tar-and-zip>

cd /scratch/<project_id>
tar xzvf data_backup.tgz # Verify the content
rm /home/<user>/<Directories-files-to-tar-and-zip>
```
- Move the backup to long-term storage:
```
cp /scratch/<project_id>/data_backup.tgz /project/<project_id>/data_backup.tgz
rm /scratch/<project_id>/data_backup.tgz
```
Temporary Solution

This solution is only temporary. Users should promptly download important files and store them securely.
Request Additional Storage: If more space is needed, the Principal Investigator (PI) can submit a justification for increased storage allocation.

By proactively managing storage and keeping data organized, users can ensure smooth access to resources without exceeding quotas.

Created by: Marek Štekláč, Marek Štekláč