SSH Hardening on Your VPS: Keys, Sudo User & Fail2Ban

If your VPS only has a root password and port 22 open to the world, it's just a matter of time before something finds it. Bots scan the entire IPv4 range for SSH round the clock, and a weak or reused root password is the single most common way small VPS instances get compromised. Here's how to lock yours down properly — keys, a non-root user, and Fail2Ban — in about 20 minutes.

Symptom: why this matters even on a "small, unimportant" server

You'll usually notice one of these first:

• /var/log/auth.log (or /var/log/secure on CentOS/AlmaLinux) is full of thousands of failed login attempts from IPs you don't recognize.
• Your server suddenly starts sending spam, mining crypto, or scanning other hosts — and your hosting provider emails you about abuse complaints.
• Load average is oddly high with no obvious process, or a cron job you didn't create is running.

None of this needs a sophisticated attacker. It's almost always an automated brute-force bot that got lucky against root + a guessable password.

Cause: the default setup is wide open

Most VPS images ship with three things that make brute-forcing trivial:

1. Root login enabled over SSH — the highest-privilege account is also the most-attacked username on the internet.
2. Password authentication — crackable given enough time and a weak password, and phishable if you ever paste it somewhere you shouldn't.
3. No rate limiting — nothing stops a bot from trying 10,000 password guesses a day against port 22.

Fixing all three is what actually closes the door. Doing just one (say, changing the SSH port) slows bots down slightly but doesn't stop a targeted attack.

Fix, step by step

1. Create a non-root user with sudo

Log in as root one last time and create your day-to-day account:

adduser deploy
usermod -aG sudo deploy      # Debian/Ubuntu
# usermod -aG wheel deploy   # CentOS/AlmaLinux/Rocky

Test that sudo works for this user in a second terminal window before you close the first one — if it fails, you don't want to be locked out.

2. Set up SSH key authentication

On your local machine (not the server), generate a key pair if you don't already have one:

ssh-keygen -t ed25519 -C "deploy@myserver"

Copy the public key to the server:

ssh-copy-id deploy@your-server-ip

No ssh-copy-id on your system? Paste the contents of ~/.ssh/id_ed25519.pub into ~/.ssh/authorized_keys on the server manually, and make sure the permissions are correct:

chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

Now log out and log back in as deploy using the key, to confirm it works, before touching any config files.

3. Lock down sshd_config

Edit /etc/ssh/sshd_config (on Ubuntu 22.04+/RHEL 9 also check for drop-in files under /etc/ssh/sshd_config.d/ that might override these):

PermitRootLogin no
PasswordAuthentication no
PubkeyAuthentication yes
AllowUsers deploy
Port 2222

Changing the port is optional cosmetic noise-reduction, not real security — skip it if it complicates your firewall rules or monitoring. The two settings that matter are PermitRootLogin no and PasswordAuthentication no.

Restart the service and, critically, keep your current session open while you test in a new one:

sudo systemctl restart sshd
ssh deploy@your-server-ip -p 2222

If the new connection works, only then close your original root session.

4. Install and configure Fail2Ban

Fail2Ban watches your auth logs and temporarily bans IPs that fail login too many times — it catches the attempts that get past everything above, including scans against whatever other services you expose.

sudo apt install fail2ban -y
sudo cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local

Edit /etc/fail2ban/jail.local and set, under [sshd]:

[sshd]
enabled = true
port = 2222
maxretry = 4
bantime = 1h
findtime = 10m

Restart and check it's actually watching:

sudo systemctl restart fail2ban
sudo fail2ban-client status sshd

You should see a jail status with a ban count that climbs over the next day or two — that's it working, not a sign of an attack you need to panic about.

5. Firewall as a last layer

Add a firewall rule so only your SSH port is open (plus whatever web ports you actually need):

sudo ufw allow 2222/tcp
sudo ufw allow 80,443/tcp
sudo ufw enable

Double-check the new SSH port is allowed before you enable UFW — enabling it with the wrong port open is the single most common way people lock themselves out of a fresh VPS.

6. Optional: add two-factor for SSH

If you want another layer beyond the key itself — useful for shared team servers — install Google Authenticator PAM and require both the key and a one-time code:

sudo apt install libpam-google-authenticator -y
google-authenticator

Then in /etc/pam.d/sshd add auth required pam_google_authenticator.so, and in sshd_config set:

ChallengeResponseAuthentication yes
AuthenticationMethods publickey,keyboard-interactive

This is optional for a single-admin server, but worth it once more than one person has access.

Prevention: keep it locked down

On Getwebup VPS plans, we can bake key-only access and Fail2Ban into your image at provisioning time if you'd rather not do this by hand on every server — just mention it when you order or open a ticket for an existing instance.