7. Yen File Storage

File Storage

You have several options for where to store your research files (data sets, programs, output files, and so forth). This guide will help you decide which storage location is best for your situation.

IFS Directories

The GSB now has about 500 TB of high-performance storage available from the yen servers under the path /ifs (Isilon File System). Currently all yen user directories (~/) reside on IFS. In addition to user home directories Stanford GSB faculty (and in some cases students, see below), may request additional project space on IFS.

GSB Faculty

All GSB faculty may request project space on IFS, and DARC will setup a corresponding Stanford workgroup that you may use to add and remove collaborators for your project. Currently there are no quotas for faculty IFS directories. However, since it is costly for the GSB to expand capacity there, this policy might change in the future if users consume that space too quickly. Kindly do your part and be a good steward of the commons:

  • zip up files when you can
  • remove intermediate files that you no longer need to use regularly

GSB Students

While IFS is primarily a resource for Stanford faculty, under certain conditions Stanford graduate students may be granted workspace in IFS. If you feel you are in need of project space on IFS please contact us at gsb_darcresearch@stanford.edu.

All student workspaces are hard limited to 200 GB, and are marked as priority for purging, should disk space on IFS ever become an issue. Disk purging will always be preceded by adequate notification!

Backups

Files on IFS are backed up as “snapshots” and can be restored manually by any user. Please see the page How Do I Recover IFS Files for instructions on recovering files. There is currently an off-site disaster recovery solution implemented as well for both IFS and home directories.

Yen Home Directories

Home directories on the Yen servers are also stored on our Isilon storage system. Your home directory has a quota of 25 GB.

When you log into the Yen servers you will automatically land in your home directory, which is located at /home/users/{SUNet-id} or with the shortcut ~. You are able to access your former AFS home directory by following the afs-home symlink inside your home directory.

Local Disk

On each Yen machine, there is local scratch space mounted at /scratch. All yen users are free to make use of this space. Much like a hard drive on your laptop, this can be accessed only from that single Yen machine.

AFS Volumes

You have a personal AFS volume that is named according to your SUNetID. For example if your SUNetID is johndoe13, then the path to your AFS directory is: /afs/ir/users/j/o/johndoe13. The two individual letters are the first two letters of the SUNetID.

You may have access to other AFS volumes set up for specific projects, or other people may give you access to a specific directory in their AFS volume. To access other AFS volumes, you need to know what the path to them is. For example, the path might be something like /afs/ir/data/gsb/nameofyourdirectory.

How to access an AFS volume

There are two options for transferring files to and from AFS:

  1. From your desktop using OpenAFS, a free download available from Stanford. This software will mount your AFS directory so that you can access it using an Explorer (Windows) or Finder (Mac) window as you do with other files.

  2. Through a web interface: https://afs.stanford.edu/. When you go to this url, it will take you to your home directory. To go to a different directory, click the Change button at the top of the page under Current AFS Directory Path.

How to create an AFS volume

If you are working with a faculty member on a project that uses AFS, chances are that person already has an AFS directory created for that project. Just ask the faculty member what the path to the directory is, and to grant you permissions to use it.

Size Limitations

As of this writing, AFS volumes at Stanford can be as large as 256 GB. However, it is possible to chain multiple volumes together in one Linux directory using symbolic links.

Backups

All AFS directories are backed up nightly. Any file or directory that existed for at least 24 hours before it was deleted, can be restored by submitting a HelpSU request.