An application that writes a million lines per hour fills a gigabyte of log space in minutes, and the log rotation that manages the growth is not a cosmetic exercise but a storage survival mechanism. Logrotate is the tool that every distribution ships for this purpose, and its configuration for a high-volume application differs from the stock defaults in every parameter: the frequency, the compression timing, the copytruncate mode that handles applications that cannot reopen their log files, and the postrotate scripts that signal the application. This article builds the configuration for the high-volume case.
The parameters and their meanings in the high-volume context
The stock logrotate configuration rotates weekly with four rotations kept, and the high-volume application needs every parameter adjusted. The configuration in /etc/logrotate.d/myapp:
/var/log/myapp/app.log {
daily
rotate 14
compress
delaycompress
missingok
notifempty
maxsize 500M
postrotate
systemctl reload myapp
endscript
}
The daily frequency because a weekly rotation of a million-lines-per-hour log produces a file that the compression cannot handle. The rotate 14 keeps two weeks of history. The compress flag turns on gzip compression, and the delaycompress flag postpones compression by one cycle, the detail that matters for the application that might still hold the previous log file open. The missingok tolerates the absence of the log file, the notifempty skips the rotation of an empty file, and the maxsize 500M forces a rotation when the file exceeds the size, regardless of the schedule.
The copytruncate tradeoff
The copytruncate mode is the alternative to the postrotate signal, and the tradeoff between them is the central decision. The postrotate approach: logrotate renames the log file, sends a signal to the application, and the application opens a new file. The copytruncate approach: logrotate copies the file, truncates the original to zero, and the application continues writing to the same inode.
The copytruncate mode is the only option for applications that cannot reopen their log files, but the tradeoff is data loss: between the copy and the truncate, the application may write additional lines that are lost. The postrotate approach has no data loss but requires the application to cooperate with the signal.
The compression strategy
The compression reduces the archive from gigabytes to megabytes. The default gzip is the safe choice, and the zstd compression is the modern alternative, faster and with better ratios, available through the compresscmd directive:
compresscmd /usr/bin/zstd
compressoptions -3
compressext .zst
The delaycompress flag postpones compression by one rotation cycle, and the reason is the application that might still hold the previous log file open: compressing it while still open produces a stale archive.
The scripts and the application integration
The postrotate and prerotate scripts are the integration points between logrotate and the application. The postrotate script that signals the application: SIGUSR1 for the applications that implement it, systemctl reload for the services that support it. The prerotate script runs before the rotation and is the place for the database checkpoint and the application-level log flush.
The monitoring and the checklist
The rotation's health is monitored by three signals: the age of the newest archive, the size of the active log file, and the disk space. The active log file that grows past maxsize between rotations means the rotation is not firing. The archive directory that fills is the retention policy not keeping up.
The checklist: daily rotation with the retention depth that compliance requires. Maxsize that forces rotation before the file becomes unmanageable. The postrotate signal or the copytruncate mode, chosen by the application's capability. Compression with the delaycompress. The scripts that integrate with the application. And the monitoring that watches the log size and the archive age, the two signals that confirm the rotation is working rather than merely configured.
The rotation mechanics and the file descriptor problem
The log rotation mechanics differ fundamentally between the two modes, and the mechanics are why the choice matters. The postrotate rename-and-signal approach works because Unix file descriptors are tied to inodes rather than to names: the application holds a file descriptor for the original inode, logrotate renames the directory entry to app.log.1, the inode does not change, and the application keeps writing to the same inode through the same descriptor. The signal causes the application to close the old descriptor and open a new file with the original name app.log, which is a new inode with a fresh file, and the rotation is complete.
The copytruncate approach works because the file is not renamed: logrotate copies the content to app.log.1, then truncates app.log to zero bytes with the truncate syscall. The application's file descriptor still points to the same inode, but the file size is now zero, and the next write goes to offset zero, overwriting nothing because the file is empty. The write continues in the same file, and the rotation is complete without the application knowing it happened.
The file descriptor detail is where the copytruncate data loss lives: the application's next write after the truncate goes to offset zero, but the kernel buffers the write, and between the copy and the truncate there is a window where the application has already buffered data that will be lost when the truncate happens. The window is microseconds on a modern system, but at a million lines per hour the microseconds contain lines, and the loss is real.
The dateext option and the archive naming
The dateext option names the rotated archives by date rather than by sequence number, app.log-20261014 instead of app.log.1, and the naming convention is the difference between a rotation that the human reads and one that the human counts. The dated names are self-describing: the archive from October 14 is named app.log-20261014, and the question of which archive contains which date is answered by the filename rather than by the modification time.
The dateext option composes with the retention: rotate 14 with dateext keeps fourteen dated archives, and the deletion of the oldest is based on the date in the filename rather than on the sequence number. The dateformat option controls the date format, dateformat -%Y%m%d-%s for the hour-level rotation where the date alone is insufficient, and the format string follows the strftime conventions. The dated names are the archive naming that the backup tools and the human operators both understand, and they are the recommended option for any logrotate configuration that a human will ever need to browse.
The su and create directives for permissions
The su directive controls the user and group that logrotate runs as for the rotation, and the create directive controls the permissions of the new log file. The application that runs as a non-root user needs the log directory writable by that user, and the create directive with the mode, owner and group:
su myapp myapp
create 0640 myapp myapp
The su directive is necessary when the log directory is not writable by root, or when the rotation must run as the application's user to preserve ownership. The create directive ensures that the new log file has the correct permissions from the moment it is created, because the application that opens a log file with wrong permissions either fails or logs to a file that nobody can read.
The state file and the rotation ordering
Logrotate maintains a state file, /var/lib/logrotate/status, that tracks when each log was last rotated, and the state file is the reason the rotation happens on schedule rather than on every invocation. The state file is also the source of the ordering that the multiple log files in one configuration follow: the rotation of all files matching the pattern happens in the same logrotate invocation, and the ordering within the invocation is alphabetical by file path.
The state file is also the debugging tool when the rotation does not fire: the status file shows the last rotation date for each log, and the comparison against the schedule explains whether the rotation was skipped because of the schedule, because of the size test, or because of an error. The logrotate debug flag, -d, runs the rotation in dry-run mode, printing what it would do without doing it, the diagnostic that confirms the configuration before the production change.
The integration with the journal and the central collector
The application that logs through journald does not need logrotate, because journald manages its own rotation and retention through the vacuum commands and the SystemMaxUse setting. The logrotate is for the applications that write to their own files, the ones that predate the journal, the ones that write structured output to a dedicated log path, and the ones whose log volume justifies a separate file rather than a journal entry.
The central collector from the rsyslog article receives the log entries that the application writes, and the logrotate on the application's own file is the local retention that complements the central copy: the local file is rotated after fourteen days, the central copy is retained for ninety, and the two retention periods serve different compliance requirements. The application that writes to both its own file and to the journal, through the standard output and the sd_journal API, gets the best of both: the local file for the fast access and the journal for the structured query.
The complete checklist for production logrotate
The production logrotate configuration is the one that the team has tested, monitored and rehearsed, and the checklist that gets there is the working order. Measure the log volume first: how many lines per hour, how many bytes per day, at what peak. Set the frequency and maxsize from the measurements. Choose the postrotate signal or copytruncate based on the application's capability. Configure the compression with the delaycompress that handles the open-file race. Set the su and create directives for the permissions. Add the monitoring on the log size and the archive age. Test with the logrotate -d dry run. And rehearse the restore from the rotated archives, because the rotation that produces archives nobody can read is a rotation that has failed at its actual purpose, which is preserving the log data for the moment when it is needed.
The logrotate configuration for the high-volume application is the one that the team has tested with the real log volume, monitored with the real alert thresholds, and rehearsed with the real restore. The rotation that produces archives the team can read, at intervals the disk can sustain, with compression the storage can afford, is the logrotate configuration that works, and the configuration above is the template that gets there.
The logrotate configuration is the one that the monitoring confirms is working: the active log file stays under maxsize, the archives appear on schedule, the compression reduces the archive sizes to the expected range, and the disk usage stays within the budget. The monitoring signals are the proof that the configuration is real rather than theoretical, and the proof is what the quarterly review reads.
The application whose logs are managed by logrotate, whose archives are compressed by zstd and whose retention is enforced by the rotate directive, is the application whose log volume is a managed resource rather than a growing liability, and the difference between the two is the difference between the server that runs for years and the server that fills its disk in a week. The logrotate configuration is the tool that makes the difference, and the configuration above is the template that gets there.