# How Do I Check My Resource Usage on the Yens?

Monitoring your resource usage is important to be a good citizen of the GSB Research Computing community, but also to be able to effectively estimate your needs on scheduled systems.

These tools can help you characterize your usage:

- **`htop`** allows you to get an overview of the activity on the whole node that you are logged into.
- **`userload`** is a custom tool that allows you to monitor CPU and RAM usage in a simplified view.
- **`topbyuser`** is another custom tool that lists out the active users on your current host and the amount of resources they are currently using.

Stay Within User Limits

We have [User Limits](/_policies/user_limits) to illustrate responsible use of our shared resources that our users are expected to follow. These guidelines ensure that the Yen cluster can remain accessible and performant.

## Monitoring Compute with `htop`

One easy method of getting a quick snapshot of your CPU and memory usage is via the `htop` command line tool. Running `htop` shows usage graphs and a process list that is sortable by user, top CPU, top RAM, and other metrics. Please use this tool liberally to monitor your resource usage, especially if you are running multiprocessing code on shared systems for the first time.

You can toggle to a threaded view of processes by pressing `T`. Below is a sample of `htop` output in threaded mode.

Jobs May be Automatically Killed

Note that jobs that exceed user limits may be terminated automatically to preserve the integrity of the system.

## User Limits and Multiprocessing

Per our [User Limits](/_policies/user_limits), CPU usage should always be limited based on system size. When working with implicitly or explicitly *multiprocessed* code, care must be taken to ensure your code does not use every available processor. Please refer to our [Parallel Processing in R](/_user_guide/parallel_processing_r/) and [Parallel Processing in Python](/_user_guide/parallel_processing_python/) articles for information about how to limit resource consumption when using parallel packages in those languages.

Be sure to monitor your processes, particularly when using a new package, to verify that you are using the expected number of cores.

## Monitoring Disk Usage

Personal home directories have an 80 GB quota, while faculty [project directories](/_user_guide/storage) are much larger.

Do NOT Exceed Your Home Directory Quota!

If you exceed your home directory quota, you cannot access Jupyter or perform many basic system tasks.

Disk storage is a finite resource so to allow us to continue to provide large project spaces, please always be aware of your disk footprint. This includes removing intermediate and/or temp files whenever possible, and avoiding storing multiple copies of the same data set. See the [Storage Solutions](/_user_guide/storage) page for more information about file storage options.

Disk quotas on the Yen servers can be reviewed by using the `gsbquota` command. It produces output like this:

Terminal Command

```
gsbquota
```

Terminal Output

```
/home/users/$USER:
  using 39% (31GiB) of soft quota 80GiB
  hard quota: 300GiB
  time until lockout: no block scheduled
```

You can also check the size of your project space by passing in a full path to your project space to `gsbquota` command:

Terminal Command

```
gsbquota /zfs/projects/students/<my-project-dir>/
```

Terminal Output

```
/zfs/projects/students/<my-project-dir>/: currently using 39% (78G) of 200G available
```

Unsure what's taking up space? The `gsbbrowser` command scans your home directory and provides a visual representation of your directories and files and their associated sizes. You can also provide a path to a project directory to scan that directory:

Terminal Command

```
gsbbrowser /zfs/projects/students/<my-project-dir>/
```

## Example: Monitoring an R Script

Here we present an example in R, to illustrate monitoring code running on multiple cores.

To monitor the resource usage while running a program, we will need *three* terminal windows that are all connected to the **same** Yen server.

Start by `ssh`'ing into any of the interactive yens in the first terminal window:

Terminal 1 Command

```
ssh yen.stanford.edu
```

Check what `yen` node you are connected to in the first terminal:

Terminal 1 Command

```
hostname
```

Terminal 1 Output Example

```
yen3
```

Then `ssh` to the **same** `yen` in the second and third terminal windows. So if you are on `yen3`, you would open two new terminals and `ssh` to `yen3` in both so you can monitor your resources when you start running the R program on `yen3`.

Terminal 2,3 Command

```
ssh yen3.stanford.edu
```

Create an `investment-npv-parallel.R` file below and save it in your desired directory.

We Explicitly Limit the Cores in the Code!

Note that we set **8 cores** for use with the `registerDoParallel` function.

```
# In the context of economics and finance, Net Present Value (NPV) is used to assess
# the profitability of investment projects or business decisions.
# This code performs a Monte Carlo simulation of Net Present Value (NPV) with 500,000 trials in parallel,
# utilizing multiple CPU cores. It randomizes input parameters for each trial, calculates the NPV,
# and stores the results for analysis.

# load necessary libraries
library(foreach)
library(doParallel)

options(warn=-1)

# set the number of cores here
ncore <- 8

# register parallel backend to limit threads to the value specified in ncore variable
registerDoParallel(ncore)

# define function for NPV calculation
npv_calculation <- function(cashflows, discount_rate) {
  # inputs: cashflows (a vector of cash flows over time) and discount_rate (the discount rate).
  npv <- sum(cashflows / (1 + discount_rate)^(0:length(cashflows)))
  return(npv)
}

# number of trials
num_trials <- 500000

# measure the execution time of the Monte Carlo simulation
system.time({
  # use the foreach package to loop through the specified number of trials (num_trials) in parallel
  # within each parallel task, random values for input parameters (cash flows and discount rate) are generated for each trial
  # these random input values represent different possible scenarios
  results <- foreach(i = 1:num_trials, .combine = rbind) %dopar% {
    # randomly generate input values for each trial
    cashflows <- runif(10000, min = -100, max = 100)  # random cash flow vector over 10,000 time periods.
    # these cash flows can represent costs (e.g., initial investment) and benefits (e.g., revenue or savings) associated with the project
    discount_rate <- runif(1, min = 0.05, max = 0.15)  # random discount rate at which future cash flows are discounted

    # calculate NPV for the trial
    npv <- npv_calculation(cashflows, discount_rate)
  }
})
cat("Parallel NPV Calculation (using", ncore, "cores):
")
# print summary statistics for NPV and plot a histogram of the results
# positive NPV indicates that the project is expected to generate a profit (the benefits outweigh the costs),
# making it an economically sound decision. If the NPV is negative, it suggests that the project may not be financially viable.
summary(results)
hist(results, main = 'NPV distribution')
```

Once we have three terminal windows connected to the same `yen`, we are ready to run the script and monitor its resource consumption. In one of the terminals, load the R module:

Terminal 1 Command

```
ml R
```

Install the Necessary Packages

The above script relies on two R packages - `foreach` and `doParallel`. If you have not previously installed them for the version of R that you have loaded, you can install them by running the following command in the R console: `install.packages(c("foreach", "doParallel"))`. See [this page](/_user_guide/r/#installing-r-packages) for details on how to manage R packages on the Yen cluster.

In the same terminal where you loaded the R module, run the `investment-npv-parallel.R` program:

Terminal 1 Command

```
Rscript investment-npv-parallel.R
```

Once the program is running, monitor your usage with `userload` command in the second window:

Terminal 2 Command

```
watch -n 1 userload
```

While the program is running, you should see about 8 CPU cores being utilized in `userload` output. Note that the `watch` command allows you to see the real time CPU cores count every second.

Run `htop -u $USER` in the third window, where `$USER` is your SUNet ID:

Terminal 3 Command

```
htop -u $USER
```

While the program is running, you should see 8 R processes running in the `htop` output because we specified 8 cores in our R program.

In summary, effective monitoring of your resource usage on the Yen cluster ensures that you stay within system limits, maintain performance, and contribute to a fair and efficient computing environment for all users. By leveraging tools like `htop`, `userload`, and `gsbquota`, you can proactively manage your resource consumption.