The local journal is the diagnostic source, and the central log server is the audit and correlation layer that survives the source's failure. Rsyslog is the transport between them: it reads the journal, applies filtering rules, and forwards the entries to the central collector over an encrypted channel, with disk buffering that holds the entries when the collector is unreachable. This article builds the full forwarding setup: the TLS certificates and the rsyslog configuration on both sides, the buffering that survives network partitions, the filtering that sends only the entries the collector needs, and the monitoring that keeps the transport visible.
The TLS setup, certificates on both sides
Rsyslog's TLS forwarding uses X.509 certificates for both authentication and encryption, and the certificate setup is the prerequisite that the rest of the configuration depends on. The central server holds the server certificate, and each forwarding client holds a client certificate signed by the same CA. The CA's self-signed certificate is the trust anchor on both sides:
Server side $DefaultNetstreamDriverCAFile /etc/rsyslog/tls/ca.crt $DefaultNetstreamDriverCertFile /etc/rsyslog/tls/server.crt $DefaultNetstreamDriverKeyFile /etc/rsyslog/tls/server.key
Client side $DefaultNetstreamDriverCAFile /etc/rsyslog/tls/ca.crt $DefaultNetstreamDriverCertFile /etc/rsyslog/tls/client.crt $DefaultNetstreamDriverKeyFile /etc/rsyslog/tls/client.key
The CA-signed certificates prove identity in both directions: the client verifies the server's certificate, and the server verifies the client's, the mutual TLS that prevents both man-in-the-middle attacks and unauthorized senders. The certificates are generated by the site's CA, the same CA that the SSH certificate authority article described, and the certificate management follows the same rotation discipline.
The server-side configuration, the TLS listener
The central server's rsyslog configuration opens a TLS listener on port 6514, the standard rsyslog TLS port, and the listener receives entries from every client in the fleet:
module(load="imtcp")
input(type="imtcp" port="6514" StreamDriver.Name="gtls"
StreamDriver.Mode="1" StreamDriver.Authmode="x509/name")
$AllowedSender tls, *.example.com
The StreamDriver.Mode 1 is TLS-only, the Authmode x509/name requires a valid certificate with a matching name, and the AllowedSender restricts the connections to the fleet's domain. The received entries are stored in the central server's local files, or forwarded to the analysis system, or both, the collector being the hub that the fleet's log architecture radiates from.
The client-side configuration, forwarding with TLS
The client's rsyslog configuration reads from the journal and forwards to the central server over TLS:
module(load="imjournal")
input(type="imjournal" StateFile="/var/lib/rsyslog/imjournal.state")
action(type="omfwd" target="logs.example.com" port="6514"
protocol="tcp" StreamDriver.Name="gtls" StreamDriver.Mode="1"
StreamDriver.Authmode="x509/name"
queue.type="LinkedList" queue.size="100000"
queue.filename="fwdqueue" queue.maxDiskSpace="1g"
action.resumeRetryCount="-1")
The imjournal module reads from the journal, the omfwd action forwards to the collector, and the queue configuration is the buffering that survives the collector's absence. The queue.type LinkedList is the in-memory queue, queue.filename enables disk-assisted mode when the memory queue fills, queue.maxDiskSpace bounds the disk queue, and action.resumeRetryCount -1 means retry forever, the configuration that loses nothing when the network is down and catches up when it returns.
The filtering that sends only what the collector needs
The forwarding without filtering sends every entry to the central server, which is correct for the compliance tier and wasteful for the general tier. The filtering rules select the entries that the collector needs, and the rules use rsyslog's property-based syntax:
if $programname == "sshd" then action(type="omfwd" ...)
if $syslogseverity <= 4 then action(type="omfwd" ...)
The first rule forwards only the SSH entries, the second forwards only priority 4 and below, warnings and more severe. The rules compose, and the configuration's filtering section is the policy statement: what the central collector receives, what stays local, and the line between the two is the same line that the audit and compliance requirements draw.
The buffering and the failure scenario
The buffering is the property that separates a reliable forwarding setup from a lossy one, and the test that proves the buffering works is the network interruption: stop the central server, generate log entries on the client, watch the queue fill, restart the server, and confirm the entries arrive. The test exercises the disk-assisted queue, the retry mechanism and the reconnection, and the test is the rehearsal that proves the transport loses nothing.
The buffering's disk usage is the resource to monitor, because a collector that is down for days fills the client's disk queue, and the queue.maxDiskSpace is the ceiling that prevents the queue from consuming the client's storage. The queue's usage is visible in rsyslog's statistics, the impstats module that reports queue sizes and drop counts, and the monitoring of the queue depth is the early warning that the collector is unreachable.
The checklist and the closing summary
The checklist for the forwarding setup is the article in working order. Generate the CA and the certificates, distribute them to the server and clients. Configure the server's TLS listener with the AllowedSender restriction. Configure the client's forwarding with the TLS parameters, the queue for buffering and the retry policy. Set the filtering rules that select what the collector receives. Test the buffering by stopping the collector and confirming the catch-up. Monitor the queue depth and the transport's health. The result is a log architecture where the local journal is the immediate source, the central collector is the audit and correlation layer, and the transport between them is encrypted, buffered and monitored, the three properties that separate a log architecture from a log aspiration.
The logrotate on the central server
The central server's log files grow at the rate of the entire fleet's output, and the logrotate configuration on the collector is the discipline that keeps the storage from filling. The daily rotation with ninety days of retention and compression, the sharedscripts that sends one HUP per rotation, and the missingok that tolerates the empty days. The ninety-day retention is the compliance policy's number, and the compressed archives on the central server are the long-term audit trail.
Performance tuning and the queue sizing
The forwarding pipeline's throughput is determined by the queue sizing, the TLS session management and the network bandwidth. The queue sizing formula: the fleet produces N entries per second, the average entry is B bytes, and the queue must hold S seconds of entries during a network outage. A fleet of twenty servers producing a hundred entries per second with an average entry of five hundred bytes needs a queue of five hundred kilobytes per second of outage, and the one-gigabyte ceiling holds thirty-four minutes of full outage.
The TLS session management is the performance detail that matters on high-throughput links: the TLS handshake per connection is expensive, and the connection reuse that the TCP protocol enables keeps the handshake cost at one per client rather than one per entry.
The comparison with other forwarding tools
The forwarding landscape has several tools, and the choice among them is a decision about the integration surface. Rsyslog is the traditional choice, the tool that reads from the journal through imjournal and that every distribution ships. Fluentd and Fluent Bit are the modern alternatives, the tools that speak the log-collector language of the container world. Vector is the newest entry, the Rust-based tool whose performance profile is the strongest. The choice follows the fleet's architecture: rsyslog on the servers that already run it, Fluent Bit on the container fleet, Vector on the high-throughput tier.
The security considerations and the closing checklist
The forwarding pipeline's security extends beyond the TLS transport: the certificate revocation, the log integrity on the collector, and the access control on the log files. The checklist: generate the CA and certificates, configure the server's TLS listener, configure each client's forwarding with TLS and buffering, test the buffering by stopping the collector, test the mutual TLS, configure the logrotate, monitor the queue depth. The result is a log architecture where the fleet's entries reach the central collector encrypted, buffered and monitored.
The forwarding pipeline's health monitoring is the impstats module that rsyslog provides: module(load="impstats" interval="60" severity="7" log.syslog="off" log.file="/var/log/rsyslog-stats.log") adds statistics reporting every sixty seconds, the queue depth, the number of submitted and failed entries, and the TLS session state. The statistics feed the monitoring system the same way the Prometheus exporters do, and the alert on queue depth above threshold is the early warning that the collector is unreachable or the network is degraded.
The central server's disk is the resource that the fleet's log volume consumes, and the sizing follows the same arithmetic as the queue sizing: twenty servers at a hundred entries per second at five hundred bytes per entry is one megabyte per second of aggregate log traffic, which is roughly eighty-six gigabytes per day uncompressed, and the compression that logrotate applies reduces the archive to a fraction of the raw size. The storage sizing is the disk budget calculation that the infrastructure review performs, and the compressed archives on the central server are the artifact that the compliance audit reads.
The closing observation about the forwarding architecture is that it completes the log story that the previous articles started: the journal collects locally, the JSON fields make the entries structured, the rsyslog forwarding transports them to the center, and the Loki article in this series provides the query interface that the central store enables. The pipeline from the application's stdout to the dashboard query is the complete log architecture, and the forwarding transport is the middle layer that every other layer depends on.
The rsyslog TLS forwarding setup is the transport layer that connects the local journal to the central analysis, and the setup's three components, certificates, buffering and filtering, are the three properties that the production log architecture demands. The certificates prove identity and encrypt the transport, the buffering survives the network failures that would otherwise lose entries, and the filtering keeps the transport's scope aligned with the compliance requirements. The result is a log architecture that the team can rely on for the audit trail, the incident investigation and the correlation analysis, with the confidence that the entries are complete, encrypted and delivered.
The forwarding setup is the one that the fleet configures once and forgets, because the buffering and the retry policy mean that the transport survives the failures that would otherwise require attention, and the log architecture runs itself.
And the architecture runs itself, which is the highest compliment a log pipeline can receive.
Running cleanly for years.