Exposing a server to the open internet comes with two immediate questions: How do I secure it, and how do I spot an intrusion?
Modern DevOps is always a balance between security and convenience. After deploying infrastructure for over a decade, I've learnt the real challenge isn't building maximum security, but it's building a resilient defense that doesn't make operations miserable(I dive deeper into this mindset in the security philosophy behind a hardened Linux server). If your configuration makes day-to-day admin work painful, you will eventually bypass your own rules.
Securing a server behind a private VPN is relatively straightforward. Exposing a public app to the open internet is where the challenges start. This guide is about practical hardening—(intentional em dash) I will accept some risks along the way.
Choosing an SSH Client
Best SSH Client for Windows, Best SSH Client for Mac or Best SSH Client for Linux.
I am sure, you have searched for these terms. So before we even deploy the server, let's configure our local environment. Windows is my primary workstation, but I frequently switch to macOS. And for that reason, maintaining an identical cross-platform setup is essential.
To keep things unified, I will stick to the built-in terminal. While I prefer MobaXterm, it is not cross-platform. So for editing configuration, and deployment tasks, we will use VS Code. It is available for all platforms and with a few key extensions, it will handle everything.
Preparing Your Local PC and Generating Secure SSH Keys
When setting up a cloud instance, providers usually ask you for SSH public key. While some platforms offer a web GUI to generate keys, creating them locally on your own machine is the standard, secure approach.
I plan to generate two separate key pairs. The first is strictly for initial setup and server provisioning. The second is for daily use, assigned to a dedicated non-root user for operations.
I'll name this user 'deployer'. Note that in a multi-admin team, you should never share generic accounts. Provision separate user accounts for each admin which makes audit trails significantly easier.
Go to your user directory and create a folder named .ssh.
ssh-keygen -t ed25519 -C "default"
ssh-keygen -t ed25519 -C "deployer"
In some cases, specially in windows, if you open terminal before creating .ssh folder, it creates these files in the profile directory. Move them inside .ssh folder if this is the case.
Initial VPS Setup: Choosing Ubuntu 24.04 LTS vs Ubuntu 26.04 LTS
Although Ubuntu 26.04 LTS has been recently released, I recommend holding off on the newest version for now. There is a high chance you'll run into package compatibility issues. Ubuntu 24.04 is rock-solid and supported until May 2029, which makes it the safer choice for production.
Initial VPS Setup along with Firewall Configuration
You can choose any cloud provider you like as long as you have SSH access. Ubuntu comes preinstalled with UFW (Uncomplicated Firewall). But I prefer to configure at the provider level with their cloud firewall. This blocks automated scans and brute-force attempts at the edge before traffic ever touches your server.
After you have provisioned your server, restrict inbound traffic on your port 22 to YOUR IP ADDRESS ONLY. Just keep in mind, if your ISP assigns you dynamic IP addresses, you will need to update this rule in your cloud dashboard to gain access.
Server Hardening: Creating a sudo User and Disabling Root Login
Grab the Public IP address from your newly created VM.
On your local machine, navigate to ~/.ssh and create a file called config if it doesn't exist.
Host vps-root
Hostname <IPADDRESS>
User <user>
IdentityFile ~/.ssh/default
Host vps-deployer
Hostname <IPADDRESS>
User deployer
IdentityFile ~/.ssh/deployer
Make sure the key files we create earlier (default and deployer) are in this directory.
Now it's time to connect. Open your terminal and run:
ssh vps-root
you should now have a shell session inside your new server instance.
Create a New User - Deployer
Create the new user and add them to the sudo administrative group
sudo adduser deployer
sudo usermod -aG sudo deployer
Verify group membership:
groups deployer
Now login as the user deployer
Switch to the new deployer user:
su - deployer
Verify your sudo privileges by running an initial package update:
sudo apt update
sudo apt upgrade
Run the following commands to set up SSH key login for the new user:
mkdir -p ~/.ssh
nano ~/.ssh/authorized_keys
Paste the public key of deployer and save authorized_keys
Set the correct permission for the folder and keys
chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys
Disable password Authentication
Check if password authentication is enabled:
sudo sshd -T | grep -i passwordauthentication
If the result says no, you can skip the next step.
Edit the following configuration file.
sudo nano /etc/ssh/sshd_config
Change PasswordAuthentication no
Save and validate the configuration.
sudo sshd -t
sudo systemctl restart ssh
Run our previous command again to verify if password authentication is disabled.
Now that you have verified, your deployer and sudo are working, lets disable root ssh login. Exit the terminal.
Remember the two entries we created in our config file? Let's use the second entry to log into the server before we disable root.
ssh vps-deployer
Note: Make sure this is working before proceeding to the next step. You will be locked out of the server if you proceed to disable root login without the primary user account
Disable root login
Check the current status of root login
sudo sshd -T | grep permitrootlogin
If the result says No, you can skip the next step.
sudo nano /etc/ssh/sshd_config
Add or change PermitRootLogin no inside the file
Check if there are issues with configuration and restart the service
sudo sshd -t
sudo systemctl restart ssh
Block default user
(Skip this step if you don't have a default user ubuntu).
Some cloud providers, AWS in this instance, created a default ubuntu account with the NOPASSWD flag enabled by default. To secure this account and prevent interactive logins, change its shell:
sudo usermod -s /usr/sbin/nologin ubuntu
To restore shell access if needed in the future:
sudo usermod -s /bin/bash ubuntu
Core System Optimization and Automating Security Updates
Enable swap
Allocating a swap file provides an emergency memory buffer, preventing the kernel from terminating critical processes if memory usage spikes.
# Allocate a 4GB swap file allocation footprint
sudo fallocate -l 4G /swapfile
sudo chmod 600 /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile
# Make the configuration survive system reboots
echo '/swapfile none swap sw 0 0' | sudo tee -a /etc/fstab
Keep time zone to UTC
Servers should always run on UTC. It standardizes timestamps, and eliminates timezone confusion if you are parsing logs from multiple regions.
Check what timezone you are on using timedatectl and if it is anything other than UTC, run the following command:
sudo timedatectl set-timezone UTC
Automatic vs. Manual Update
I have mixed feelings about automatic updates. They make server maintenance quite effortless, but I have been burned one too many times by surprise breaks.
Most days of the year, it works perfectly. But for production workloads, I prefer to maintain full control and avoid unexpected downtime. This is purely my opinion based on my past experience. Your workflow and risk tolerance may differ.