Skip to content

Storage Quotas

Disk quotas are used to monitor and limit storage consumption on the cluster. Quotas are typically enforced per user and sometimes per project, depending on the filesystem.

Two limits are usually enforced:

  • Storage usage – total amount of disk space used
  • File count (inodes) – total number of files

Storage consumption is measured in allocated blocks and number of files.

Determine Quotas

Users can determine their quota usage by running the following command:

quota -s

User quota on /home

Disk quotas for user <user> (uid <user_id>):
Filesystem   space   quota   limit   grace   files   quota   limit   grace
store-nfs-ib:/storage/home
              507G   1000G   1024G           2196k*   2048k   2098k
An asterisk(*) indicates the user has exceeded its quota of storage, and writes to /home/ are not allowed.

Command du -sh can be used to view the size of the respective folders:

du -sh /home/<user>/<dir>

Alternatively, users can use dust command, which prints out the tree structure of the directories, with the largest files in them. Depth of the tree can be specified with -d parameter.

Output of dust command 
login01:~$ dust /home/<user>/<dir>
  640G ─┬ .
  523G  ├─┬ ADMIN
  227G  │ ├─┬ qe
  226G  │ │ └─┬ benchmarks-master
  209G  │ │   └─┬ GRIR443
   59G  │ │     ├── GRIR.wfc1
   29G  │ │     ├── GRIR.wfc2
   14G  │ │     └── GRIR.wfc4
  128G  │ ├─┬ cp2k
  128G  │ │ └─┬ FIST
   69G  │ │   ├── nodes
   55G  │ │   └── nodes_new
   71G  │ ├─┬ python
   71G  │ │ └─┬ miniconda3
   36G  │ │   ├── envs
   34G  │ │   └── pkgs
   60G  │ └── orca
   68G  ├─┬ MD
   67G  │ └── Softbark_wood
   22G  └─┬ Docking
   16G    └── COVID19

Quotas by Filesystems

Quota limits differ depending on the storage location.

Home Directory

Home directories are intended for source code, configuration files, and small datasets.

Path:

/home/<user>

Quotas:

Resource Limit
Storage quota 1 TB
Hard limit 1.024 TB
File count 1,024,000 files

Content will be added.

Project Storage

Project storage is shared among HPC clusters and is intended for shared data within a research project.

Path:

/projects/<project>

Quotas:

Content will be added.

Scratch Storage

Scratch space is intended for temporary data produced during computations.

Path:

/scratch/<user>

Content will be added.

Content will be added.

Warning

Scratch storage should only contain temporary files. Important results should be moved to project or long-term storage.

Exceeded Quotas

If you exceed your storage quotas, you have several options:

  1. Remove unnecessary files and directories fo free up space.

  2. Compress and archive large files to optimize storage. Since write permissions in /home may be suspended, you can first move data to /scratch, compress it, and then store it elsewhere.

    • Compress files: 

      tar czvf /scratch/<user>/data_backup.tgz /home/<user>/<Directories-files-to-tar-and-zip>

cd /scratch/<user>
tar xzvf data_backup.tgz # Verify the content
rm /home/<user>/<Directories-files-to-tar-and-zip>

    • Move the backup to long-term storage: 

      cp /scratch/<user>/data_backup.tgz /projects/<user>/data_backup.tgz
rm /scratch/<user>/data_backup.tgz

    Temporary Solution

    This solution is only temporary. Users should promptly download important files and store them securely.

  3. Request Additional Storage: If more space is needed, the Principal Investigator (PI) can submit a justification for increased storage allocation.

By proactively managing storage and keeping data organized, users can ensure smooth access to resources without exceeding quotas.

Created by: Andrej Sec