A long-running script that receives SIGTERM must clean up its resources before exiting, and the trap is the bash mechanism for handling the signal. Without the trap, the script dies with its temp files, its locks and its background processes, and the resources accumulate until the server runs out. The trap intercepts the signal, runs the cleanup and exits gracefully. This article covers the signal types, the trap patterns, and the cleanup that the production scripts need.
The signal types and the ones the script should handle
The signals that a script should handle are SIGINT (Ctrl+C), SIGTERM (the kill command) and SIGEXIT (the script exit). The trap installs a handler for each: trap cleanup EXIT, trap handle_sigterm SIGTERM. The SIGKILL signal cannot be caught, and the script that must survive SIGKILL needs external protection.
The trap syntax and the cleanup handler
The trap command has two forms: trap function SIGNAL and trap commands SIGNAL. The function form is the cleaner pattern:
cleanup() {
rm -rf $TMPDIR
kill $CHILD_PID 2>/dev/null
exit 0
}
trap cleanup SIGINT SIGTERM EXIT
The handler runs for all three signals, the cleanup removes the resources and the script exits cleanly.
The file locking with flock
The file lock that the trap releases is the flock mechanism: the script opens the lock file, acquires the lock with flock, and the trap releases the lock on exit. The lock prevents concurrent runs of the same script, and the lock is released by the trap on any exit path.
The child process cleanup
The script that spawns background processes must clean them up on exit, and the trap that kills the child processes uses the negative PID: kill -- -$$, which sends the signal to the entire process group. The process group is the set of processes that the script spawned, and the group kill prevents the orphaned children that survive the parent's exit.
The complete pattern
The complete pattern combines the options, the trap, the lock and the cleanup into the production script template that the error handling article established, and the signal handling is the part that makes the template complete for the long-running scripts.
The trap is the mechanism that the bash script uses for the signal handling, and the mechanism is the tool that the production scripts need for the cleanup, the graceful shutdown and the resource management. The trap that the script installs is the handler that the kernel calls when the signal arrives, and the handler is the function that the script defines for the cleanup.
The signal types and their origins
The signals that the script receives come from multiple sources: SIGINT from Ctrl+C, SIGTERM from the kill command and from systemd, SIGHUP from the terminal closing and from the configuration reload, and SIGEXIT from the script's own exit. Each signal has a default action: SIGINT terminates, SIGTERM terminates, SIGHUP terminates for non-shell processes, and SIGEXIT fires on every exit including normal, error and signal exits.
The trap for each signal serves a different purpose: the SIGINT trap handles the operator's Ctrl+C, the SIGTERM trap handles the systemd stop and the docker stop, the SIGHUP trap handles the config reload, and the SIGEXIT trap runs the cleanup on every exit path. The combination of all four traps covers every exit scenario, and the scenario coverage is the production requirement.
The trap for the background process management
The script that spawns background processes must manage them on exit, and the trap that kills the children uses the process group:
trap 'kill -- -$$ 2>/dev/null' EXIT
The negative PID sends the signal to the entire process group, and the process group includes every child that the script spawned. The kill that the trap sends is the cleanup that prevents the orphaned children, the children that survive the parent's exit and that the parent cannot clean up after the exit.
The wait command that the trap issues before the kill gives the children a grace period to finish: the trap sends SIGTERM, waits N seconds with the sleep or the timeout, and sends SIGKILL if the children have not exited. The graceful shutdown sequence is the one that the systemd service management expects, and the sequence is the one that the docker stop command's timeout models.
The trap for the file lock release
The file lock that the script acquires with flock is the lock that prevents concurrent runs, and the lock is released when the file descriptor is closed, which happens automatically on exit. The trap for the lock release is a safety net rather than a necessity, because the fd close on exit releases the lock, but the trap that explicitly releases the lock makes the intent clear.
The lock's purpose is the mutual exclusion that prevents two instances of the same script from running simultaneously, the conflict that would corrupt the data or duplicate the work. The flock command acquires the lock on the fd, and the lock is held for the duration of the script, the mutual exclusion that the cron job needs to prevent the overlapping runs.
The error handling and the trap interaction
The set -e and the trap interact in a specific way: the set -e causes the script to exit on the first error, and the EXIT trap fires on that exit, which means the cleanup runs on the error path as well as on the success path. The combination of set -e and the EXIT trap is the pattern that guarantees the cleanup on every exit, and the guarantee is the production requirement.
The trap's return value affects the script's exit code, and the trap that returns nonzero changes the script's exit code, the detail that the error handling cares about. The trap that returns zero preserves the original exit code, and the preservation is the behavior that the production script wants: the cleanup does not change the script's outcome.
The complete signal handling template
The template that every production bash script should use for the signal handling:
#!/bin/bash
set -euo pipefail
RUNNING=true
CHILD_PIDS=()
cleanup() {
RUNNING=false
for pid in ${CHILD_PIDS[@]}; do
kill $pid 2>/dev/null
done
wait
rm -rf $TMPDIR
}
trap cleanup EXIT SIGINT SIGTERM
TMPDIR=$(mktemp -d)
long_running_task &
CHILD_PIDS+=($!)
while $RUNNING; do
sleep 1
done
The template tracks the child PIDs, the cleanup kills them on exit, the RUNNING flag controls the main loop, and the signals set the flag to false, which triggers the graceful shutdown through the normal loop exit rather than through the signal kill. The pattern is the one that the production scripts use for the long-running operations, and the pattern is the one that the systemd service management expects.
The closing summary for the trap discipline
The trap is the mechanism that the production bash script needs for the cleanup, the graceful shutdown and the resource management, and the mechanism is the tool that the script configures once and that the script relies on for every exit path. The discipline that the trap enforces is the one that the script's authors design: the cleanup that removes the temporary files, the signal handling that stops the children gracefully, and the error handling that logs the exit reason. The script that has all three is the script that the production environment can trust.
The trap mechanism in bash is the kernel's signal delivery mechanism surfaced to the script, and the kernel delivers the signal by interrupting the current operation and calling the handler. The handler runs to completion before the interrupted operation resumes, which means the handler can safely clean up resources that the main script is using, and the safety is the reason the handler must be fast: a slow handler delays the signal delivery and the main script's resumption.
The trap for the EXIT signal is special in bash: it fires on every exit path, including the normal exit, the set -e error exit, the explicit exit and the signal-triggered exit. The EXIT trap is the guaranteed cleanup that runs regardless of how the script exits, and the guarantee is the property that the production script needs for the resource cleanup.
The trap for the temporary file management
The temporary files that the script creates are the resources that the EXIT trap cleans up, and the cleanup is the difference between the script that leaves its mess behind and the script that cleans up. The temporary directory pattern:
TMPDIR=$(mktemp -d)
trap 'rm -rf $TMPDIR' EXIT
The mktemp creates a unique directory, the trap removes it on exit, and the pattern is the one that the bash scripts use for the temporary file management. The mktemp command creates the directory with mode 700, the permissions that prevent other users from reading the temporary files, and the security is the property that the shared systems need.
The trap for the lock file management
The lock file that prevents concurrent execution is the resource that the trap releases:
LOCKFILE=/var/run/myscript.lock
if ! mkdir $LOCKFILE 2>/dev/null; then
echo "already running" >&2
exit 1
fi
trap 'rm -rf $LOCKFILE' EXIT
The mkdir is the atomic lock, the directory creation that either succeeds or fails, and the atomicity is the property that makes mkdir a reliable locking mechanism. The trap removes the lock directory on exit, and the removal is the cleanup that allows the next run to acquire the lock.
The trap for the background process cleanup
The background processes that the script spawns are the processes that the trap must clean up, and the cleanup uses the process group kill:
BG_PID=""
start_bg() {
long_running_command &
BG_PID=$!
}
cleanup_bg() {
[ -n "$BG_PID" ] && kill $BG_PID 2>/dev/null
}
trap cleanup_bg EXIT
start_bg
The cleanup_bg function kills the background process on exit, and the kill prevents the orphaned process that continues running after the script has exited. The orphaned process is the resource leak that the process table accumulates, and the cleanup prevents the leak.
The trap ordering and the multiple trap handlers
The multiple traps for the same signal override each other: the last trap replaces the previous one, and the override is the behavior that the script must account for. The solution is the single handler that calls the multiple cleanup functions:
cleanup_files() { rm -rf $TMPDIR; }
cleanup_lock() { rm -f $LOCKFILE; }
cleanup_children() { kill $BG_PID 2>/dev/null; }
cleanup_all() {
cleanup_files
cleanup_lock
cleanup_children
}
trap cleanup_all EXIT
The single handler calls the individual cleanup functions, and the modularity makes the cleanup maintainable: each function handles one type of resource, and the single trap calls them all.
The trap and the subshell interaction
The trap that the script sets is inherited by the subshells, and the inherited trap fires in the subshell's context, which means the cleanup runs in the subshell as well. The inherited trap is usually undesirable, because the cleanup should run once at the script level rather than at every subshell exit, and the fix is the trap reset in the subshell: the subshell that resets the trap with trap - EXIT prevents the inherited handler from firing.
The set -e and the trap interaction is the detail that the error handling cares about: the set -e causes the script to exit on the first error, the EXIT trap fires on that exit, and the cleanup runs on the error path as well as on the success path. The combination is the guarantee that the cleanup runs on every exit, and the guarantee is the production requirement.
The debugging of the trap behavior
The trap behavior is debugged with the set -x flag, the shell trace that shows every command including the trap handler commands, and the trace is the diagnostic that reveals the trap's behavior: whether the trap fires, what the handler does and what the handler's exit code is. The set -x in the handler function is the debug that the handler's logic needs, and the debug is the tool that identifies the cleanup that is not cleaning up.
The closing summary for the trap in production scripts
The trap is the mechanism that the production bash script needs for the cleanup, the graceful shutdown and the resource management, and the mechanism is the tool that the script configures once and that the script relies on for every exit path. The discipline that the trap enforces is the one that the script's authors design: the cleanup that removes the temporary files, the signal handling that stops the children gracefully, and the error handling that logs the exit reason. The script that has all three is the script that the production environment can trust.