Mitigating SSH Denial of Service is a basic piece of security hardening for any homelab that exposes SSH. I say basic because the fixes are small and the risk is real: an SSH service can be flooded until it stops responding. Cisco’s IEC6400 advisory shows how a missing flood protection can make SSH go quiet while the rest of the device keeps running. I’ll give concrete, practical steps you can apply tonight to reduce that risk, with exact SSH configuration knobs and simple network controls.
Start with the SSH configuration. Edit sshd_config and set explicit limits that throttle unauthenticated activity. Useful lines I run in my lab are: PermitRootLogin no, PasswordAuthentication no, PubkeyAuthentication yes, LoginGraceTime 30, MaxAuthTries 3, MaxSessions 2, UseDNS no. For flood protection use MaxStartups 10:30:60 which allows bursts but starts dropping new unauthenticated connections once the table fills. Turn off unused options: X11Forwarding no, AllowTcpForwarding no unless you need it, HostbasedAuthentication no. Lock access to specific accounts with AllowUsers or AllowGroups. These changes narrow the attack surface and make an SSH denial of service much harder to trigger from simple connection floods.
Add network-level rate limiting and access controls. If your host runs nftables or iptables, add a connlimit rule for tcp dport 22, for example reject or drop connections above 10 simultaneous from the same /32. A practical nftables example looks like: add rule inet filter input tcp dport 22 ct count over 10 drop. Use fail2ban to parse auth logs and temporarily ban sources that attempt many sessions or failed logins. For a homelab that must allow remote admin, put SSH behind a VPN or a bastion host. I use WireGuard as a jump point; the SSH port is not publicly exposed. Hiding the port is not security by itself, but combining a VPN with connlimits and strict SSH configuration gives defence in depth. If you control a small edge router, push rate limits there rather than relying solely on the server.
Plan the response before an incident happens. Logging and monitoring matter for denial of service as much as for intrusion. Increase OpenSSH LogLevel to VERBOSE during testing so you capture connection patterns, then revert to INFO for normal operation. Forward logs to a small syslog server or central journald instance so you can spot spikes: sudden hundreds of SYNs or many half-open sessions are telltale. Create a short runbook with three actions: apply a temporary firewall block for offending IP ranges, enable stricter connlimits, and capture a pcap of traffic for analysis. If the device supports vendor updates, prioritise installing fixes that mention SSH flood protection. The Cisco IEC6400 advisory shows vendors sometimes release fixes specifically for this class of flaw; disabling SSH is an option only if you truly do not need remote shell access.
Test your hardening regularly. Run scans with ssh-audit and nmap from an isolated test network to see how your service behaves under load. Simulate a denial of service in a controlled environment using hping3 or another traffic generator, watch whether MaxStartups and connlimits hold, and confirm the SSH daemon remains responsive to legitimate connections. Audit your configuration twice a year, check installed OpenSSH versions for CVEs, and keep a short checklist beside your router console: patch date, last config test, fail2ban enabled. Training for a homelab is mostly practice for you; rehearse the runbook so the first response is fast and calm.
Concrete takeaways: patch devices that list SSH flood fixes as a priority, tighten sshd_config (MaxStartups, MaxAuthTries, PermitRootLogin no, PasswordAuthentication no), apply connlimits at the host or router, use fail2ban and a VPN or bastion for remote access, and keep a tested runbook that includes temporary blocks and traffic capture. Do these and an SSH denial of service will become an annoyance rather than a showstopper.
