Archive Data to Google Drive
Archiving your data to Google Drive (GDrive) using RClone is a free, fast, and secure way to free up on-premises storage. Google Drive can be used with high-risk data following Stanford’s Minimum Security Standards. Our ZFS storage solution is finite, measuring roughly one-half petabyte of space, which means that space is limited to researchers in perpetuity.
Configuring RClone for Google Drive
You must take the following steps to configure Rclone for your Stanford Google Drive account. These configuration steps only need to be taken once, and then you can use your new “remote” as a target for copying data.
- Log in using your Stanford credentials. Login instructions can be found here.
module load rclone && rclone configThe following menu will appear.
nto create a new remote connection to google drive. When prompted, give it a name; in this example, I’ll use
- The next screen will display all the possible connections with the version of RClone installed on the machine. This menu changes between versions, so be sure to select the number corresponding to the “Google Drive.”
- When prompted for the next two options, leave them blank and hit return/enter.
Google Application Client Id - leave blank normally.
Google Application Client Secret - leave blank normally.
- Choose “N” on this prompt “Say N if you are working on a remote or headless machine or Y didn’t work.” Since the Yen cluster is headless, we must be given a unique URL to validate the connection.
- On the next prompt, RClone provides you with a unique URL that you need to paste into a browser. After logging in with your Stanford credentials, Google Drive will give a code to paste back into the terminal window.
- Finally, to finish the configuration in the terminal window setup, click “Y” and “enter” to complete the process if you believe everything is properly setup. In the last prompt, hit “q” to quit. You have setup RClone successfully on your Google Drive.
RClone Common Operations
Initiaitaing a current version of RClone
** OR **
moudle load rclone
Note: this needs to be run once per server per terminal session. After you have initiated
module load rclone, you will only need to call
rclone as normally from the command line.
To list connection points
Create a remote folder on Google Drive using Rclone. Note this will make the folder within your Google Drive base folder.
rclone mkdir smancusoGoogleDrive:GoogleDriveFolderName
To upload contents of a directory to Google Drive using copy and Rclone.
rclone copy /Path/To/Folder/ smancusoGoogleDrive:GoogleDriveFolderName/
List contents of remote folder on Google Drive
rclone ls smancusoGoogleDrive:GoogleDriveFolderName
Download from remote Google Drive.
rclone copy smancusoGoogleDrive:GoogleDriveFolderName /Path/To/Local/Download/Folder