The find command locates files and the xargs command processes them in bulk, and the combination is the pipeline that every server administrator uses for mass operations. The article covers the real differences in performance between -exec and xargs, the handling of filenames with spaces and special characters, and the patterns that make the pipeline safe on production systems.
The find command and its primary options
find walks the filesystem tree and evaluates expressions on each file. The basic invocation:
find /var/log -name '*.log' -mtime +30
finds log files older than thirty days. The primary options are -name for the filename pattern, -type for the file type, -mtime for the modification time, -size for the file size, -user and -group for ownership, and -perm for permissions. The options combine with logical operators: -a for and, -o for or, and the parentheses that group expressions. The escaped parentheses are required on the shell command line.
The -exec option and the shell per file
The -exec option runs a command for each file found:
find /tmp -name '*.tmp' -exec rm {} \;
The {} placeholder is replaced by the filename, and the semicolon terminates the command. The -exec spawns one process per file, which is the performance cost on large file sets: a thousand files produce a thousand rm invocations. The + terminator batches the files into as few invocations as the command-line length allows:
find /tmp -name '*.tmp' -exec rm {} +
which is the same operation with one rm invocation for all files, the performance difference that matters on the filesystems with a hundred thousand files.
The xargs pipeline and the batching
The xargs command reads items from stdin and invokes a command with the items as arguments, and the batching is the performance advantage over -exec:
find /tmp -name '*.tmp' | xargs rm
The pipeline passes the filenames through a pipe, and xargs invokes rm with as many filenames as fit on the command line, repeating as needed. The performance on large file sets is better than -exec per file because the process creation cost is amortized across the batch.
The filenames with spaces and the null separator
Filenames with spaces are the classic problem in the find and xargs pipeline, because the space is both the filename character and the argument separator. The solution is the null byte separator: find prints filenames separated by null bytes with -print0, and xargs reads null-separated filenames with -0:
find /tmp -name '*.tmp' -print0 | xargs -0 rm
The null byte is the only character that cannot appear in a filename, which makes it the safe separator. Every find and xargs pipeline that touches user-controlled directories should use -print0 and -0, because the filename with a space or a newline that the pipeline does not handle is the filename that the rm deletes the wrong file for.
The performance comparison on large file sets
The performance comparison on a directory with a hundred thousand files shows the difference. The -exec with semicolon spawns a hundred thousand processes, each with the fork and exec overhead, and the time is measurable in minutes. The -exec with plus batches into a handful of processes, and the time drops to seconds. The xargs with -0 and a reasonable batch size matches the -exec plus performance, and the choice between -exec plus and xargs is a style preference rather than a performance one.
The safety patterns for destructive operations
The pipeline that deletes files needs the safety patterns: the -print that shows what find would delete before the rm deletes it, the -name pattern that narrows the scope, and the confirmation that the operator runs before the destructive command. The pipeline that renames files uses the -exec sh -c pattern for the complex transformations, and the shell within the -exec handles the filename manipulation that the simple commands cannot.
The find expression logic and the option ordering
find evaluates the expression from left to right, and the implicit AND between options means every option must match for the file to be returned. The -o operator creates an OR, the ! operator creates a NOT, and the parentheses group the expressions. The option ordering matters because find evaluates as it walks: the -maxdepth option before the filters prunes the tree early, the -name filter before the -exec avoids running the command on non-matching files. The expression that combines -name and -type:
find /var -name '*.log' -a -type f -a -size +100M
finds files that are both named .log and larger than 100 MB, and the -a between each is implicit without being written. The explicit -o creates the union: find -name '.log' -o -name '.tmp' finds files matching either pattern.
The time-based filters and the file age calculations
The time filters measure in days by default: -mtime 0 means modified today, -mtime +7 means modified more than seven days ago, -mtime -1 means modified within the last 24 hours. The -mmin option measures in minutes for the finer granularity: -mmin -30 means modified in the last thirty minutes. The -newer option compares against another file: -newer /var/log/rotate-marker finds files newer than the marker, the pattern that the backup rotation uses to identify files changed since the last backup.
The combination of the time filter with the -delete action is the cleanup that the cron jobs run:
find /tmp -mtime +7 -delete
which deletes every file in /tmp that has not been modified in seven days, the cleanup that prevents the tmpfs from filling with stale files. The -delete action is the safest deletion because it is part of find, not a separate command, and the find evaluation order guarantees that only the files matching all the filters are deleted.
The xargs batching and the parallel execution
xargs batches the input items into commands, and the -n flag controls the batch size: -n 1 runs the command once per item, -n 10 groups ten items per invocation. The -P flag controls the parallelism: -P 4 runs four xargs commands simultaneously, the parallelism that the GNU Parallel article covers in detail and that xargs provides in its basic form. The combination of the null separator and the batching and the parallelism:
find /data -name '*.raw' -print0 | xargs -0 -P 4 -n 10 convert
which converts the raw files in batches of ten with four parallel workers, the pipeline that saturates the CPU on the conversion host. The performance on the hundred-thousand-file set is the benchmark that determines the batch size and the parallelism, and the benchmark is the iostat and the vmstat that the monitoring provides during the conversion.
The practical examples from production
The find and xargs scenarios from production are the recipes that the runbook collects. The disk cleanup that finds and deletes files older than the retention: find /var/tmp -type f -mtime +30 -delete. The permission fix that changes ownership on the files created by the wrong user: find /srv/app -user root -exec chown myapp:myapp {} +. The log archive that compresses the old logs: find /var/log/app -name '*.log' -mtime +1 -exec gzip {} +.
Each recipe is a one-liner that the operator runs on the production server, and the one-liner is the tool that answers the question in seconds. The find and xargs pipeline is the tool that the script uses for the bulk operations, and the pipeline is the one that the cron schedule runs on the retention schedule the compliance policy requires.
The integration with the backup and monitoring tools
The find command that the backup scripts use finds the files changed since the last backup, the -newer option that the incremental backup uses, and the find output that the rsync consumes is the file list that the backup covers. The monitoring that tracks the descriptor counts, the disk usage and the file counts uses find for the filesystem inventory, and the inventory is the data that the capacity planning needs.
The integration with the security tools: the find -perm -4000 finds the setuid binaries, the find -perm -002 finds the world-writable files, and the security audit that Lynis runs internally uses find for the same checks. The find and xargs pipeline is the tool that the security hardening uses to apply the permissions across the fleet, and the pipeline is the same one that the logrotate and the backup scripts use for the file operations.
The closing observation about the find and xargs pipeline is that it is the most fundamental text-and-file tool combination in the Unix toolbox, the combination that predates every specialized tool and that the specialized tools still use internally. The find that locates the files and the xargs that processes them in bulk are the tools that every script in this series uses for the file operations, and the pipeline is the tool that the production server has available without any additional installation.
The find -printf option and the custom output formats
The -printf option controls the output format with the format string that the C printf function defines, and the format specifiers name the file properties: %p for the path, %s for the size, %T@ for the modification timestamp, %u for the user, %g for the group. The custom output that the scripts parse:
find /var/log -name '*.log' -printf '%s %p\n' | sort -rn | head -10
which prints the size and the path of the ten largest log files, the output that the disk investigation reads. The -printf option is a GNU find extension, not available in the POSIX find, and the portability consideration applies to the scripts that run on multiple platforms. The -ls option prints the equivalent of ls -dils for each file, the verbose format that includes the inode, the permissions and the timestamps, and the format that the manual inspection reads.
The symlink handling with -type l and the -follow option
The symlink is a file type that find handles specially, and the -type l option finds symlinks while the -follow option follows the symlink to its target. The find that does not follow symlinks walks the real tree, the one where symlinks are entries rather than the targets they point to, and the find that follows symlinks walks the target tree, which can loop for the circular symlinks. The -xtype option tests the type of the file the symlink points to, the test that distinguishes the symlink to a file from the symlink to a directory.
The broken symlinks are the ones whose targets do not exist, and the find -xtype l finds them, the cleanup that the periodic maintenance runs. The symlink that points to a file on an unmounted filesystem appears broken until the filesystem is mounted, and the broken symlink that appears and disappears is the NFS mount cycling, the pattern that the monitoring distinguishes from the genuinely broken symlink.
The find and xargs pipeline is the tool that every production server has available without any additional installation, and the pipeline is the one that the scripts in this series use for the bulk file operations: the backup scripts use find to identify the files to back up, the security scripts use find to identify the permission violations, and the cleanup scripts use find to identify the files past their retention. The tool is the foundation, and the foundation is the one that the specialized tools build on.
The find and xargs pipeline is the fundamental file operation tool that every script in this series uses, and the mastery of the pipeline is the mastery of the bulk file operations that the production server requires.
The find and xargs pipeline is also the tool that the cron jobs use for the periodic maintenance: the cleanup of the old files, the compression of the old logs and the archiving of the old data, and the pipeline that the cron schedule runs is the one that the retention policy requires. The find command that the cron runs is the same command that the operator runs manually, and the consistency between the manual and the automated operations is the discipline that the production environment demands.
The performance that the pipeline achieves on the large file sets is the reason that the -exec with the plus terminator and the xargs with the -0 flag exist: the two mechanisms that batch the file arguments are the optimizations that reduce the process creation cost from per-file to per-batch, and the optimization is the difference between minutes and seconds on the filesystems with a hundred thousand files. The pipeline that uses the batching is the pipeline that the production environment needs for the bulk operations, and the bulk operations are the ones that the backup, the cleanup and the migration scripts run on the fleet's file systems.