By Chris Lepore |

 

Using the Institutional Cluster (IC), including information about central storage, backups, and transfers.

Currently, central storage for the Institutional Cluster (IC) is provided by IBM's Spectrum scale file system (GPFS), a high-performance, clustered filesystem that supports full POSIX filesystem semantics.

If you have used our previous cluster, your files from 'nano' (including /home, /work, and /gscr) are now stored at /hpcgpfs01/cfn/. Please copy those files to your current home directory.

Home directories

User home directories are under /sdcc/u (/gpfshome01/u).

  • User home directories have a quota of 40GB.
  • Home directories are backed up every day once and we keep 30 days of previous backups.
  • Home directories are not meant to be used as high-performance storage.

Central scratch storage space (Temporary)

Users can create directories in "/hpcgpfs01/scratch/" for temporary storage space.

  • Files not accessed 30 days in "/hpcgpfs01/scratch" will be automatically deleted.
  • There is a quota of 2TB enforced on each user.

Persistent central storage space (Permanent)

All requests for persistent storage space must come from your liaisons.

File transfers - Globus online.

File transfers to/from SDCC home directories and /hpcgpfs01 filesystem should be done through Globus online.

See the below link for more information on Globus online.

https://www.globus.org/how-it-works

The public globus endpoint is "SDCC". (UUID - 12782fb1-a599-4f18-b0fb-2e849681e214).

You can activate the endpoint using your SDCC Kerberos credentials, use only your username without the Kerberos realm. Transfers can be done through the globus.org site or through the CLI.

The endpoint can be activated for a maximum of 3 days.

Quotas in GPFS

GPFS filesystem support three types of quotas:

  • User
  • Group
  • Fileset

A fileset in GPFS is like a volume, for example: "/hpcgpfs01/x" directory could be a fileset.

You can have all three types of quotas enforced for a file. For example, you can have user "X" have a quota of 1TB in the fileset /hpcgpfs01/x, which has a fileset quota of 50TB.

You may check your quota using these commands:

quota -s  # For home directories

/usr/lpp/mmfs/bin/mmlsquota --block-size auto hpcgpfs01  #For hpcgpfs01 file system

Performance considerations

As /hpcgpfs01/ is optimized for mixed workloads, you should see a good performance if you are doing streaming sequential IO. Each compute node has a 4GB cache. To achieve good random read performance, try to fit your workload in the 4GB cache. The IC host compute nodes have a 2TB scratch space, which is hosted on a spinning drive and mounted at "/scratch". If you copy your files locally before doing heavy analysis, you should see better performance.