Create Bash Script

#!/bin/bash
set -euo pipefail
echo "hello world"

It all starts with a Shebang

The #!/bin/bash at the beginning of the script is called a "shebang" and is essential.

There are two shells to pick from

• The Bourne Shell (#!/bin/sh)
• The Bourne Again Shell (#!/bin/bash)

If you're scripting for modern systems /bin/bash is preferable. It includes a wider range of features including:

• Arrays
• Advanced string manipulation
• Improved process substitution syntax
• Brace expansion
• Sophisticated control flow constructs

So when should you use the Bourne Shell /bin/sh? Only when you have no other choice.

Strict Mode Options

set -euo pipefail helps you write more reliable scripts and for the curious, here is what those options do:

• set -e (errexit): The script will exit immediately if any simple command within it exits with a non-zero status (signaling an error). This helps prevent errors from silently propagating.
• set -u (nounset): The script will fail if you try to use an undeclared variable. This helps to avoid typos and catch potential mistakes.
• set -o pipefail: If any command within a pipeline fails (returns a non-zero status), the entire pipeline's exit status will also be non-zero. This means errors are caught as they occur within a pipeline, rather than just checking the final command in the chain.

Positional Arguments

#!/bin/bash
set -euo pipefail
echo "First arg: $1"
echo "Number of arguments: $#"

These are arguments passed directly after the script name on the command line.

./your_script.sh arg1 arg2

They are accessed using special variables $n, where n is a number starting from 1.

• $1: First argument
• $2: Second argument
• $#: Number of arguments passed

Looping over arguments

for arg in "$@"
do
  echo "Processing argument: $arg"
done

Advanced Arguments

Special argument variables

• $@ - Represents all arguments passed as a single string (useful for passing arguments to another command).
• $* - Similar to $@ but treats arguments with spaces as a single argument (be cautious with spaces).

Flags

Flags are an even more flexible and power way to pass arguments

while getopts ":hf:v" opt; do
  case $opt in
    h) echo "This script helps you process files";;
    f) file="$OPTARG";;  # Assign argument after -f to 'file' variable
    v) verbose=true;;  # Set a flag for verbose mode
    \?) echo "Invalid option: -$OPTARG" >&2; exit 1;;
  esac
done

Important Note:

The colon (: ) at the beginning of getopts ":hf:v" is essential:

• An option preceded by a colon (f:) indicates that it requires an additional argument (e.g., the filename after -f).
• Options without colons are treated as simple flags (e.g., -v).
• Let me know if you'd like more examples or scenarios on how to call the script!

Using the Flags

Displaying Help (-h)

yourscript.sh -h

Output: Prints the help message: "This script helps you process files"

Specifying a File (-f)

yourscript.sh -f input.txt

Effect: Assigns the value "input.txt" to the file variable within the script, allowing you to work with that file.

Verbose Mode (-v)

yourscript.sh -v 

Effect: Sets the verbose flag within the script. The script can then use this flag to output additional info or debugging messages.

Combining Options

You can combine short options that don't require additional arguments:

yourscript.sh -hvf input.txt

This is equivalent to yourscript.sh -h -v -f input.txt and would:

Display the help message Set the 'verbose' flag Assign the value "input.txt" to the file variable.

Source scripts

You have other bash scripts and want to include it in the one you are writing.

First get the path of your current script

SCRIPT_PATH="$( dirname "${BASH_SOURCE[0]}" )"

Then source the other script

source "$SCRIPT_PATH/other_script.sh"

Want to quiet any output from the other script?

source "../other_script.sh" > /dev/null 2>&1

There you go!

Environment Variables

Passing a environment variable to a script is easy

MY_VARIABLE=true ./the_script.sh

Getting an environment variable in the script is even easier

echo "$MY_VARIABLE"

Test: has a value

myvar=${MY_VARIABLE:-}
if [ ! -z "$myvar" ]; then
  echo "MY_VARIABLE is set"
fi

Test: has no value

myvar=${MY_VARIABLE:-}
if [ -z "$myvar" ]; then
  echo "MY_VARIABLE is not set"
fi

Default value

myvar=${MY_VARIABLE:-true}

OS and Architecture

If your script needs to know which OS the system has or which CPU architecture (x86 or arm64)

OS=$(uname -s)
ARCH=$(uname -m)

Capture that info and test for it

if [[ ${ARCH} == 'arm64' ]]; then
  echo "detected arm or apple silicon"
fi
if [[ ${OS} != Linux ]]; then
  echo "you are on linux"
fi
if [[ ${UNAME} == Darwin ]]; then
  echo "nice macbook"
fi

I use this for all sorts of conditional commands

Functions

Making a function is easy

my_function() {
  echo "hi"
}

Passing arguments to it is easy

my_function() {
  arg1=${1:-}
  arg2=${2:-}
}

Return a value with echo

my_function() {
  arg1=${1:-}
  arg2=${2:-}
  echo "$arg1 $arg2"
}

Calling the function

my_function "hello" "world"

Saving the result to a variable

result=$(my_function "hello" "world")

Parallel Processes

Want to run commands in parallel to get them done faster

• run the commands in the background
• log their output in a helpful way
• show when they command and if they were successful

#!/bin/bash
set -euo pipefail

LOG_FILEPATHS_ARRAY=()
PIDS_ARRAY=()

process_pids () {

  log_prefix=${1:-}

  echo ""
  echo " waiting"
  echo "     👀 tail -f /tmp/$log_prefix.*.output"
  echo ""

  for pid in "${PIDS_ARRAY[@]}"
  do
    werr=0
    wait $pid || werr=$?
    if [ "$werr" != 0 ] && [ "$werr" != 127 ] ; then
      echo " ℹ failed - ${LOG_FILEPATHS_ARRAY[$pid]}"
    else
      echo " ✔ ${LOG_FILEPATHS_ARRAY[$pid]}"
    fi
  done
}

run_command() {
  log_prefix=${1:-}
  cmd_name=${2:-}
  cmd=${3:-}

  # log file
  log_filepath="/tmp/$log_prefix.$cmd_name.output"
  echo " ⌛ start $cmd_name"

  # log header
  echo "" >> $log_filepath
  echo "start $cmd_name..." > $log_filepath
  echo $cmd >> $log_filepath
  echo "" >> $log_filepath

  # go!
  $cmd >> $log_filepath 2>&1 &

  # save
  pid=$!
  PIDS_ARRAY+=($pid)
  LOG_FILEPATHS_ARRAY[$pid]=$log_filepath
}

Now you can execute long running commands and watch them as they finish

log_prefix="do.something"
run_command $log_prefix "cmd1" "echo 'cmd1 done'"
run_command $log_prefix "cmd2" "echo 'cmd2 done'"
process_pids

and you'll get output like this

 ⌛ start cmd1
 ⌛ start cmd2

 waiting
     👀 tail -f /tmp/.*.output

 ✔ /tmp/do.something.cmd1.output
 ℹ failed - /tmp/do.something.cmd2.output

The tail command allows you to watch all the processes execute in real time. When a command succeeds you see a "✔" and when one fails "ℹ failed".