Production servers should never hear the public internet knocking on port 22, and the design that makes this true without making engineers miserable has one door and one keyring: a bastion host, exposed and hardened, through which every interactive session to the internal fleet passes. The client-side half of the design lives in the ssh config, where a jump alias turns a two-hop connection into a single command that feels local. This article builds the architecture end to end: the bastion itself and its hardening stack, the client configuration with the -J jump flag and Host aliases, file transfers and tunnels through the same path, key management without agent-forwarding footguns, session auditing on the door, and the fallback route that keeps the team working when the bastion has a bad day.
The one-door principle and what changes on day one
A bastion concentrates risk and control in one place. Before it exists, every production server carries its own exposure: a public address, an SSH listener, a password file and the eternal rain of guessing attempts. After it exists, the fleet sits on addresses no outsider can route to, and the single machine facing the internet becomes the only surface that has to be perfect, which turns a hundred small defenses into one large and inspectable one.
The change for the engineer is smaller than the change for the attacker, and both facts are features. The engineer types one extra hop, or nothing at all once the client config is written, while the attacker's target list collapses from every server to one machine that is, by design, the most paranoid box in the infrastructure. The economics matter too: hardening, patching, rate-limiting and monitoring one exposed host is a budget line, while doing it across every node is a lifestyle nobody sustains.
The principle generalizes beyond SSH. The same door hosts the entry point for administrative tunnels, database ports during incident work, and the reach-in path for support sessions, each an ordered feature of the design rather than an exception to it. The rest of this article assumes the standard shape: one bastion, a private network behind it, and engineers with laptops holding keys that never leave the laptop.
The bastion host and its hardening stack
The bastion runs the leanest system that supports the job, because every installed package is one more thing to patch on the most targeted machine in the fleet. The base is a minimal server installation with automatic security updates, and the SSH service gets the settings that turn the guesser's rain into noise on a sealed window:
PermitRootLogin no
PasswordAuthentication no
PubkeyAuthentication yes
AllowGroups ssh-bastion
LogLevel VERBOSE
Key-only authentication removes the entire password-guessing threat class in one line, and the AllowGroups directive means an account that is not in the ssh-bastion group cannot connect at all, which turns offboarding into a membership change instead of a key hunt. The VERBOSE log level makes every session record the key fingerprint that was accepted, evidence that pays for itself the first time an incident asks which of three keys on a shared account was used.
On top of the base sit the two layers this series covered in its previous articles, and the bastion is the natural first host for both. Fail2ban with its sshd jail, tuned per the earlier article, keeps the public side quiet and the logs readable. Two-factor authentication through TOTP and PAM, from the same series, adds the property that a stolen private key alone no longer opens the door. A bastion without the second factor is half a design, because the key model assumes keys never leave laptops, and laptops are lost exactly as often as phones are.
The network position is the last hardening layer and the one that does the most work. The bastion holds one interface on the public side, or behind the load balancer of the hosting platform, and one on the internal network. Every production host's firewall, or security group in cloud terms, accepts SSH from exactly the bastion's internal address and nothing else, which is the rule that makes the fleet invisible rather than merely unwelcoming. From this point on, the question of whether a server is reachable is answered by the security group, and the answer is a single IP address long.
Client configuration with the jump flag and Host aliases
The jump mechanism in modern OpenSSH is the -J flag, available since release 7.3 and present on every current client, from Linux workstations to recent macOS and Windows builds. Its one-line form already handles the common case:
$ ssh -J
The connection hops through the bastion, and the direct TCP connection to the target is established from the bastion, with the client authenticating end to end. The flag accepts the same user@host:port syntax as the destination, so a bastion on a nonstandard port is written bastion.example.com:2222 inside the -J argument without any ceremony.
The alias layer in ~/.ssh/config removes the typing entirely, and its shape rewards a moment of design. The bastion gets its own Host block, every production destination gets one, and the day-to-day commands become one word each:
Host bastion
HostName bastion.example.com
User admin
Port 22
Host app-01
HostName 10.0.1.21
User deploy
Host db-01
HostName 10.0.2.31
User postgres-admin
With these blocks in place, the working commands become ssh -J bastion app-01 and the longer hop chains stay readable, because every component of the route is a named alias rather than a coordinate. A team convention worth adopting on day one: the aliases live in a shared file in the infrastructure repository, and each engineer symlinks or copies it, so the map of the fleet is the same document for everyone and a new host appears on every laptop with one commit.
File transfers and tunnels through the same door
Everything that travels, travels through the jump path, and the tooling supports this natively once the aliases exist. Secure copies route through the bastion with the same flag:
$ scp -J bastion backup.tar.gz app-01:/tmp/
The -J shorthand stands for the ProxyJump directive introduced in OpenSSH 7.3, and it replaced the older ProxyCommand chains that jump setups required before that version.
while rsync, which wraps ssh, takes the option through its -e switch:
$ rsync -av -e "ssh -J bastion" ./reports/ app-01:/srv/reports/
The pattern generalizes: any tool that invokes ssh accepts the jump either through the flag, where the tool exposes ssh options, or through the config aliases, which is why designing the config file first pays off in every later command. Interactive sessions, file transfers and scripted jobs all use the same door and leave the same trail, and the bastion log becomes the single place where every movement is visible.
Port forwarding composes cleanly with the jump, which matters during incident work when a database console or a metrics page must be reached from a laptop. A forward through the bastion is one command:
$ ssh -J bastion -L 15432:10.0.2.31:5432 app-01
after which the database console connects to localhost:15432 and the traffic flows laptop to bastion to app-01 to database, each hop authenticated and each within the design. The habit worth forming is opening forwards to a named internal target rather than to a database the engineer cannot otherwise reach, because the forward is an extension of the door rather than a hole beside it.
Key management and the agent-forwarding question
The cleanest property of the -J design is where the private keys live: on the laptop, and only on the laptop. The bastion relays the encrypted session but never sees key material, because authentication happens between the client and each server end to end, with the bastion forwarding bytes rather than credentials. An attacker who fully compromises the bastion gains the ability to relay sessions, a serious problem, but not a keyring, which is the difference between a contained incident and a fleet-wide credential rotation.
Agent forwarding, ForwardAgent yes, is the older pattern that solves the same problem with the opposite tradeoff, and it deserves the honest paragraph: with forwarding enabled on the bastion, the laptop's agent socket is reachable from the bastion, and any code running there as root can sign requests with the engineer's identity. The bastion is the one machine an attacker is certain to study, which makes forwarding the convenience that weakens the whole design for a saving of a few keystrokes. The -J configuration achieves the same single-command experience without the bastion ever holding a live credential handle, and it is the correct default for this architecture.
Key distribution follows the one-key-per-person-per-purpose rule. Each engineer holds a personal key, the bastion's authorized_keys holds the public halves, and the same public keys appear on the production fleet, so access to any host proves the same identity and revoking one key on the bastion revokes the route without touching a hundred machines. Host keys deserve the same discipline: the bastion's host key is known to every laptop through the shared config file, and a change to it is a signal event, not a yes-prompt on a tired Friday evening. Setting StrictHostKeyChecking through the shared known_hosts turns the strongest warning the protocol has into a hard stop that no social engineering smooths over.
Auditing the door and keeping the fallback alive
Every session through the bastion is a log line, and a design whose entire point is concentration owes the log some care. The VERBOSE sshd setting records the accepted fingerprint per login, and the session recording stack from the earlier sudoreplay article turns the interactive sessions into replayable transcripts, which on a bastion is the natural place for the strongest recording policy in the infrastructure. The sudo journal on the bastion matters as much as on any production host, because privilege changes on the door are privilege changes for everything behind it.
The fallback path is the part of the design that gets skipped and then gets tested by reality at the worst moment. The hosting console, the out-of-band management port, or the serial console of the platform is the route when the bastion itself is down, and the honest test is a quarterly drill: one engineer, one deliberate bastion outage window, one recovery through the documented path, and one paragraph in the runbook about what surprised them. The alternative console's credentials live in the same secrets manager as everything else, and the drill's true product is the knowledge of whether the console path actually works, which no amount of documentation substitutes for.
The checklist that closes the design is short enough for a card. Bastion: minimal OS, key-only auth, no root login, restricted group, TOTP, fail2ban, VERBOSE logging, session recording. Network: fleet SSH reachable from the bastion's internal address only. Clients: shared config with Host aliases, -J as the universal route, no agent forwarding, strict host key checking from a distributed known_hosts. Transfer and tunnels: the same path for scp, rsync and forwards. Fallback: out-of-band path documented, credentialed, and drilled once a quarter. Built in this order, the architecture holds the line that makes all the other rules easy: one door, watched, and everything else simply out of range.
## Connection multiplexing for the two-hop tax
The one real cost of the jump architecture for daily work is latency: every new SSH invocation pays the handshake to the bastion and then to the target, a second or three that adds up across dozens of quick commands. OpenSSH solves this with connection multiplexing, which keeps the first authenticated session alive as a socket and lets every subsequent command reuse it, skipping both handshakes entirely. The settings live in the client config and cost nothing:
Host bastion app- db-
ControlPath ~/.ssh/cm-%r@%h:%p
ControlMaster auto
ControlPersist 10m
With these lines, the first ssh -J bastion app-01 of the morning opens the road, and everything after, new terminals, scp runs, rsync jobs, rides the existing connections for ten idle minutes at a time. The effect on scripted work is dramatic, a deploy that opens twenty SSH connections completes in a fraction of the wall time, and the effect on morale is not much smaller. The socket directory belongs in the engineer's dotfiles alongside the aliases, and the one caution worth printing in the shared config is the idle timeout: a very long ControlPersist keeps tunnels alive that nobody remembers opening, so ten minutes is the sweet spot where speed meets hygiene.