Compute Questions

Software Questions

Data Questions

How Do I Transfer Data between Yens and AWS?

If you need to use AWS services from the command line, you will need programmatic access to AWS by way of keys. These will be generated for you by the DARC team, and should be thought of as a username and password. Anyone with these keys will have access to the resources they grant, so we try to limit the privileges a set of keys has, as well as where they will be effective.

Your Credentials

Your credentials will have two parts, an Access Key and a Secret Key, like the following:

AWS Access Key ID: AKIAI44QH8DHBEXAMPLE
AWS Secret Access Key: je7MtGbClwBF/2Zp9Utk/h3yCo8nvbEXAMPLEKEY

To use them, you will need to follow the setup instructions for the AWS Command Line Interface (CLI) or Boto.

An example

If we provided keys to you, such as your.creds, you can use them in the CLI by setting up a new profile. Note that you may have multiple profiles to access different resources in AWS, which can be selected from the command line using the --profile option. I can edit my AWS configuration to reflect two different settings, a default set of credentials and one for a new profile, called athenacopy.

~/.aws/config

[default]
region = us-west-2

[profile athenacopy]
region = us-west-2

~/.aws/credentials

[default]
aws_access_key_id = AK****
aws_secret_access_key = xb*********************

[athenacopy]
aws_access_key_id = AK****
aws_secret_access_key = fB*********************

I can then list the S3 buckets that are available to me with my athenacopy profile as follows:

aws s3 ls --profile athenacopy

You can read more about the s3 API on the AWS CLI documentation.