Yen File Storage

File Storage

You have several options for where to store your research files (data sets, programs, output files, and so forth). This guide will help you decide which storage location is best for your situation.

ZFS Directories

The GSB has nearly 1 PB of high-performance storage available from the yen servers under the path /zfs. Currently all yen user directories (~/) reside on ZFS. In addition to user home directories, Stanford GSB faculty (and in some cases students, see below), may request additional project space on ZFS.

GSB Faculty

All GSB faculty may request project space on ZFS and DARC will set up a corresponding Stanford workgroup that you may use to add and remove collaborators for your project. Quotas are set for faculty ZFS directories based on expected space requirements. However, we understand that initial space estimates may be insufficient and thus we can grant storage quota increases upon request later. But kindly do your part and be a good steward of the commons by

  • Archiving projects or infrequently accessed files when you can (see here)
  • Deleting intermediate files that you no longer need to use

GSB Students

While ZFS is primarily a resource for Stanford GSB faculty, under certain conditions Stanford GSB graduate students may also be granted project space in ZFS. If you feel you are in need of student project space on ZFS (as distinct from a faculty led project) please contact us at gsb_darcresearch@stanford.edu. The default size for student project space is 200 GB but let us know if you need more space for your student project.

Backups

Files on ZFS are backed up as “snapshots” and can be restored manually by any user. Please see the page How Do I Recover ZFS Files for instructions on recovering files. We also maintain an off-site disaster recovery solution for both ZFS and home directories.

Yen Home Directories

Home directories on the Yen servers are also stored on our storage system. Your home directory has a quota of 50 GB.

When you log into the Yen servers you will automatically land in your home directory, which is located at /home/users/{SUNet-id} or with the shortcut ~. You are able to access your former AFS home directory by following the afs-home symlink inside your home directory.

Local Disk

On each Yen machine, there is local scratch space mounted at /scratch. All yen users are free to make use of this space. Much like a hard drive on your laptop, this can be accessed only from that single Yen machine.

AFS Volumes

You have a personal AFS volume that is named according to your SUNetID. For example If your SUNetID is johndoe13, then then the path to your AFS directory is: /afs/ir/users/j/o/johndoe13. The two individual letters are the first two letters of the SUNetID.

You may have access to other AFS volumes set up for specific projects, or other people may give you access to a specific directory in their AFS volume. To access other AFS volumes, you need to know what the path is. For example, the path might be something like /afs/ir/data/gsb/nameofyourdirectory.

How to access an AFS volume

There are two options for transferring files to and from AFS:

  1. From your desktop using OpenAFS, a free download available from Stanford. This software will mount your AFS directory so that you can access it using an Explorer (Windows) or Finder (Mac) window as you do with other files.

  2. Through a web interface: https://afs.stanford.edu/. When you go to this url, it will take you to your home directory. To go to a different directory, click the Change button at the top of the page under Current AFS Directory Path.

How to create an AFS volume

If you are working with a faculty member on a project that uses AFS, chances are that person already has an AFS directory created for that project. Just ask the faculty member what the path to the directory is, and to grant you permissions to use it.

Size Limitations

As of this writing, AFS volumes at Stanford can be as large as 256 GB. However, it is possible to chain multiple volumes together in one Linux directory using symbolic links.

Backups

All AFS directories are backed up nightly. Any file or directory that existed for at least 24 hours before it was deleted, can be restored by submitting a HelpSU request.