Monitoring Directory Size and Growth Rate in Real-Time Using Bash Commands

Monitoring the size of a directory and its growth rate can be crucial for managing disk space, tracking data processing jobs, or debugging storage issues. This article presents two Bash command-line methods to monitor the current directory’s size and calculate its growth rate in real-time.

---

Method 1: Using watch with a Temporary File

Command Overview

watch -n 1 'prev_file=/tmp/prev_size_time;
current_size=$(du -sb . | cut -f1);
current_time=$(date +%s);
if [ -f "$prev_file" ]; then
    read prev_size prev_time < "$prev_file";
    delta_size=$((current_size - prev_size));
    delta_time=$((current_time - prev_time));
    if [ $delta_time -gt 0 ]; then
        rate=$(echo "scale=2; $delta_size / $delta_time / 1048576" | bc);
        echo "Size: $(du -sh . | cut -f1), Growth Rate: $rate MiB/sec";
    else
        echo "Size: $(du -sh . | cut -f1), Growth Rate: Calculating...";
    fi;
else
    echo "Size: $(du -sh . | cut -f1), Growth Rate: Calculating...";
fi;
echo "$current_size $current_time" > "$prev_file";'

How It Works

• watch -n 1: Executes the specified command every 1 second.
• prev_file=/tmp/prev_size_time: Defines a temporary file to store previous size and time.
• current_size=$(du -sb . | cut -f1): Calculates the current directory size in bytes.
• current_time=$(date +%s): Gets the current timestamp in seconds since the Unix epoch.
• Checking for Previous Data:
• If the temporary file exists, it reads the previous size and time.
• Calculates the difference in size (delta_size) and time (delta_time).
• Calculating Growth Rate:
• If delta_time is greater than zero, calculates the growth rate in MiB/sec using bc.
• Displays the current size and growth rate.
• First Run:
• If no previous data exists, it displays the current size and mentions that it’s calculating the growth rate.
• Updating Temporary File:
• Saves the current size and time to the temporary file for the next iteration.

Cleanup

After you finish monitoring, remove the temporary file to clean up:

rm -f /tmp/prev_size_time

Pros and Cons

• Pros:
• Simple and straightforward.
• Uses standard Unix utilities.
• Cons:
• Creates a temporary file, which may not be ideal in all environments.
• Output formatting is basic.

---

Method 2: Optimized Real-Time Monitoring Loop

Command Overview

while true; do
    clear
    DIR="${1:-.}"
    CS=$(du -sb "$DIR" | cut -f1)
    CT=$(date +%s)
    SD=$((CS - ${PS:-CS}))
    TD=$((CT - ${PT:-CT}))
    R=$([ $TD -gt 0 ] && echo "scale=2; $SD / $TD" | bc || echo 0)
    echo "Dir: $DIR | Time: $(date '+%Y-%m-%d %H:%M:%S') | Size: $(numfmt --to=iec-i --suffix=B $CS) | Rate: $(numfmt --to=iec-i --suffix=B/s $R)"
    PS=$CS
    PT=$CT
    sleep 1
done

One Liner Version

while true; do clear; DIR="${1:-.}"; CS=$(du -sb "$DIR" | cut -f1); CT=$(date +%s); SD=$((CS - ${PS:-CS})); TD=$((CT - ${PT:-CT})); R=$([ $TD -gt 0 ] && echo "scale=2; $SD / $TD" | bc || echo 0); echo "Dir: $DIR | Time: $(date '+%Y-%m-%d %H:%M:%S') | Size: $(numfmt --to=iec-i --suffix=B $CS) | Rate: $(numfmt --to=iec-i --suffix=B/s $R)"; PS=$CS; PT=$CT; sleep 1; done

How It Works

• Infinite Loop: while true; do ... done runs the commands inside indefinitely.
• clear: Clears the terminal screen for better readability.
• Directory Variable:
• DIR="${1:-.}": Sets the directory to monitor. Defaults to the current directory (.) if no argument is provided.
• Current Size and Time:
• CS=$(du -sb "$DIR" | cut -f1): Gets the current size in bytes.
• CT=$(date +%s): Gets the current timestamp.
• Size and Time Difference:
• SD=$((CS - ${PS:-CS})): Calculates the size difference since the last check.
• TD=$((CT - ${PT:-CT})): Calculates the time difference.
• Growth Rate Calculation:
• R=$([ $TD -gt 0 ] && echo "scale=2; $SD / $TD" | bc || echo 0): Calculates the growth rate in bytes per second using bc. If TD is zero, sets rate to 0.
• Display Output:
• echo: Prints the directory, current time, size, and growth rate.
• numfmt: Formats numbers with human-readable units (e.g., KiB, MiB).
• Update Previous Size and Time:
• PS=$CS: Stores the current size for the next iteration.
• PT=$CT: Stores the current time for the next iteration.
• Sleep:
• sleep 1: Pauses the loop for 1 second before the next iteration.

Advantages

• No Temporary Files: Does not create any temporary files.
• Parameterizable: Allows specifying a directory as an argument.
• Example: ./script.sh /path/to/directory
• Enhanced Formatting: Uses numfmt for better readability of sizes and rates.
• Clear Output: Refreshes the terminal display each second for real-time monitoring.

Example Output

Dir: . | Time: 2024-11-09 13:45:30 | Size: 1.5MiB | Rate: 50KiB/s

---

Comparing the Two Methods

Similarities

• Both methods calculate the directory size and growth rate.
• Use du -sb to get the size in bytes.
• Use date +%s to get the timestamp.

Differences

• Temporary File vs. Variables:
• The first method uses a temporary file to store previous size and time.
• The second method uses shell variables (PS and PT).
• Output Formatting:
• The second method uses numfmt for human-readable sizes and rates.
• The first method provides basic output without advanced formatting.
• Flexibility:
• The second method allows monitoring any directory by passing it as an argument.
• Efficiency:
• The second method avoids file I/O by not using a temporary file.
• Clearing the screen provides a cleaner, real-time display.

---

Conclusion

Monitoring directory size and growth rate can be essential for system administration and development tasks. The two methods presented offer ways to achieve this using Bash commands. The second method is generally more efficient and flexible, making it preferable for most use cases.

Tips

• Permissions: Ensure you have the necessary permissions to read the directory and write to any files if needed.
• Dependencies: The second method uses numfmt, which is available in GNU coreutils.
• Customization: Feel free to adjust the sleep interval or output formatting to suit your needs.

---

Note: Always test scripts in a safe environment before deploying them in a production setting.