6. Introduction to Linux Command Line
If you are new to using a Linux command line, this overview will get you started. The command line lets you do things without the graphical interface that you might be used to doing by double clicking with your mouse via a graphical interface such as a File Browser on your laptop.
Some examples are navigating the file system, creating new files, opening files, unzipping compressed files, and seeing the content of folders (called “directories” on a Linux file system), creating new directories, moving files around and a whole slew of other useful things.
A shell is a program that takes input from the keyboard typed by a user on the command line and gives them to the operating system (in our case
Ubuntu 18.04.5) to execute. There are many shells such as
When you login to the Yens via
ssh command you are in the interactive login shell.
See what shell you are running:
When I run the above command, I see:
which means my login shell is bash but you might see another shell by default.
Before we explore common and useful Linux commands, let’s talk about the file system on the Yens.
The Linux file system has a tree structure with the so called root directory (
/) being, well, the root, of the tree structure.
The absolulte paths mean that you specify the full path to a directory or a file starting with the root directory.
Every user on the Yens has a home directory. This is where you are when you login to the system. Check the absolute path with:
This will print your working directory (where
<SUNetID> is your SUNet ID):
To see this schematically, here is a visualization of the home directory on the file system:
The squares with
... in them indicate more directories that are not shown in the graph.
The path to your home directory is stored in
$HOME environment variable. To see it, run:
echo command prints out the environment variable
$HOME which stores the path to your home directory
<SUNetID> is your SUNet ID):
The home directory is not for storing large files or outputting large files while working on a project. It is a good place to store small files like scripts and text files. Your home directory storage space is capped at 25 G.
To see how much space you have used in your home directory, run:
You should see your home directory usage:
/home/users/<SUNetID>: currently using X% (XG) of 25G available
XG will be actual percent used and gigabytes used, respectively.
If you are a GSB researcher that is interested in starting a new project on the Yens, please complete and submit DARC’s new project request form. This form allows you to estimate disk usage, and specify any collaborators that should be added to the shared access list. IFS project access is granted by workgroups.
The project directories live on IFS and currently do not have space quotas. However, we ask that you be responsible and delete what you no longer need, intermediate files, etc because the sytem is currently 90% full.
Schematically, we can visualize the path to the project directory as follows:
The absolute path to your project space is:
<your-project-dir> is the name of your project directory (created for you by the DARC team after the request for a new project space form is filled out).
Common Linux Commands
Navigating around the file system
Let’s say we want to navigate to our project directory. To “change directories”, use
Every directory has a link to its parent directory. To navigate from a directory one level up, use:
To go two levels up, use:
If you want to return to the directory where you previously were, use:
To come back to your home directory, type
cd without any options:
Another shortcut for home is
~. Again, you can nagivate to home from anywhere on the file system with:
Or you can use the environment variable
$HOME to return to the home directory:
cd command by navigating to different places on the file system then returning home by using the above commands.
Creating new files / directories
To create a new directory inside your home directory:
cd mkdir test
cd first, you will return to your home directory. The
mkdir command makes a new empty subdirectory called
“test” inside your home directory.
To create an empty text file, you can use the
Listing directory contents
To list what files are inside current working directory, use:
You should see
test.txt file and
test subdirectory listed if you ran this from your home directory.
It is useful to list files in reverse chronological order with more details than
ls gives you. To list files in “long format”, use:
You should see:
total 4.5K drwxr-xr-x 2 nrapstin operator 0 Nov 17 09:01 test -rw-r--r-- 1 nrapstin operator 0 Nov 17 09:01 test.txt
The long format lists details about the directories and files. The first line before the long listing lists a total sum for all the file sizes (4.5K in this example).
The long format in
- The first letter indicates if the object is a direcotry (
d), a regular file (
-), a symbolic link (
l) or other type of file.
- The next three letters list permissions for the user who owns that file or directory:
rwxfor read, write and execute permissions. For example, the
testsubdirectory has user permissions as
rwxwhich means the user who owns that file listed in the third column has full permissions to read, write and execute this directory.
- The next three letters list group permissions: whether the group members have read, write or execute permissions (
rwx) for the file or directory. For example, the
testsubdirectory has group permissions as
r-xwhich means the group members can read and execute in that directory but they are not allowed to write there (only the owner user has write permissions).
- The last three letters list others permissions: whether all other users have read, write or execute permissions.
For example, the
testsubdirectory has other permissions as
r-x(read and execute).
- After the permissions the next column lists the number of hard links. So, the text file
test.txthas one hard link and the directory
testhas two links inside (two subdirectories inside
testdirectory). Empty directories create 2 subdirectories inside upon creation of the directory - one for itself
.and one for its parent
- The third and fourth columns list the user and group who own the directory or file. You should see your
SUNetIDand your group.
- The next column lists file size in bytes. You should see 0 bytes for empty files and directories.
- For each object the time of last modification is listed. Finally, the last columns lists the file or directory name.
All Linux commands have manual pages that could be accessed with
The manual explains the command and lists any optional arguments. Press
Space bar to scroll through the manual and
q to exit.
help is a
bash command so if you are not using bash as your shell, it will not work for you.
You can also use
info to view text-only information about the Linux command (with hyperlinks):
You can copy and move files around the file system (where you have access permissions).
You can copy a file with
cp command. For example, let’s make a copy of the
test.txt file inside the
cp test.txt test ls -ltr test
When we list files inside
test directory with
ls -ltr test, the newly copied file
test.txt is listed there.
Note that if you already have a file called
test.txt inside a directory where you are copying the file with the same name,
the file will be overwritten (with no warning).
We can also move files and directories with
mv test.txt test ls -ltr
We have moved the file
You can match several files to copy or move with the wild character (
*) which means match anything.
For example, if you have several text files all ending in
.txt format then we can move all of them together into the test direcotry with:
mv *.txt test
Similarly, you can move a directory within another directory:
mv my_dir /path-to-move-to
This works for
cp as well but
cp requires a recursive flag
r to copy directories:
cp -r my_dir /path-to-copy-to
Deleting files / directories
To delete a file or directories, use
that will remove
text.txt file inside
If you want to remove a directory, supply a recursive flag
-r. You can also match names to remove all files that match
a certain criteria with
Downloading files from the web
Sometimes you want to download a file from the web. Instead of locally downloading then transferring the file to the Yens,
we can download the file directly to the Yen. First, navigate where you want the file to be downloaded with
wget to download the file. Usually you have to copy the link to the file/folder on the web that you are trying to
For example, let’s say we want to install
Rmpi R package from source. We would navigate to the CRAN website and copy the web
link to the package source file. Then, run
which will download the package source tarball to your Yen directory. If everything worked, you should see:
--2021-03-09 17:54:27-- https://cran.r-project.org/src/contrib/Rmpi_0.6-9.1.tar.gz Resolving cran.r-project.org (cran.r-project.org)... 18.104.22.168 Connecting to cran.r-project.org (cran.r-project.org)|22.214.171.124|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 105893 (103K) [application/x-gzip] Saving to: ‘Rmpi_0.6-9.1.tar.gz’ Rmpi_0.6-9.1.tar.gz 100%[===============================================>] 103.41K 209KB/s in 0.5s 2021-03-09 17:54:28 (209 KB/s) - ‘Rmpi_0.6-9.1.tar.gz’ saved [105893/105893]
When you list the files in the current directory you will see
Compressing and uncompressing files / directories
When you download the files or transfer them, the files could be compressed with
zip and will have
Transferring compressed files is recommended for faster transfer speeds.
To unzip the folder, run:
tar is a Unix utility for collecting files together into one archive file (commonly called a tarball).
The name tar comes from “tape archive” and was used to archive a series of file objects together (as one collection or archive).
To untar the tarball (
x to extract,
v for verbose and
f is the name of the tarball), run:
tar -xvf my_tar.tar
Commonly, the tarball may also be compressed with gunzip (having an extension
.tgz) or bzip2 (with
Then to untar, add the unzip option as well (
-z). Let’s practice with the downloaded Rmpi tarball:
tar -zxvf Rmpi_0.6-9.1.tar.gz
Uncompressing the tarball creates a new direcotry
Rmpi with the source code for this R package.
Finally, let’s clean up the home directory by removing all the files and directories we made in this tutorial.
-f option will not prompt for confirmation and will force delete all of the files, so use it with caution.
This is a good thing to do when you are working on real projects too. Keep your working space tidy!
rm *.gz rm -rf Rmpi test
If you want to learn more about Linux command line, Software Carpentry is an excellent resource.