How to Set Up SSH Key Authentication on a Linux VPS
Password-based SSH authentication is the most common entry point for brute-force attacks on Linux servers. SSH key authentication replaces the password with a cryptographic key pair: a private key that never leaves your local machine, and a public key that lives on the server. This guide walks through the complete setup, from key generation to…
How SSH Key Authentication Works
When you connect with key authentication, the server checks whether the public key in its authorized_keys file matches the private key on your local machine, using a cryptographic challenge. No password is transmitted over the network. If your private key is protected with a passphrase, you enter the passphrase locally, and it never leaves your machine.
The practical benefit over passwords is twofold: key authentication is not vulnerable to brute-force attacks (there is no password to guess), and it allows automation without storing passwords in scripts or environment variables.
Prerequisites
- A Linux VPS running Ubuntu, AlmaLinux, or Debian with a root or sudo user
- SSH access with password authentication (you’ll switch to key auth by the end of this guide)
- A local machine running Linux, macOS, or Windows with OpenSSH client installed (comes pre-installed on macOS and Windows 10/11)
InMotion’s Cloud VPS plans include Ubuntu 22.04 LTS, AlmaLinux 9, and Debian 12 with root SSH access from provisioning. Managed VPS plans also support SSH access.
Step 1: Generate the SSH Key Pair on Your Local Machine
Run the following on your local machine, not the server.
The Ed25519 algorithm is the current recommended standard. When prompted for a file name, press Enter to use the default (~/.ssh/id_ed25519 on Linux/macOS, %USERPROFILE%\.ssh\id_ed25519 on Windows). Adding a passphrase is strongly recommended for any key used to access production infrastructure.
Two files are created: id_ed25519 (your private key, never share this) and id_ed25519.pub (your public key, this goes on the server).
If your server or organization requires RSA keys, generate a 4096-bit RSA key instead:
Step 2: Copy the Public Key to the Server
Option A: Using ssh-copy-id (Linux/macOS)
This copies the public key to ~/.ssh/authorized_keys on the server and sets the correct permissions automatically.
Option B: Manually (Windows or when ssh-copy-id is unavailable)
Display your public key:
Copy the entire output. Then on the server:
Paste the public key on a new line, save, and exit. Then set the correct permissions:
Permission settings matter. If authorized_keys is world-readable, SSH will refuse to use it.
Step 3: Test Key Authentication Before Disabling Passwords
Open a new terminal window and connect using the key:
If this succeeds without prompting for a password (or after entering your key passphrase if you set one), key authentication is working. Do not close your existing SSH session until you confirm this. Losing both authentication methods simultaneously locks you out of the server.
Step 4: Harden sshd_config
Once key authentication is confirmed working, open the SSH daemon configuration file:
Make the following changes:
PasswordAuthentication no disables password login. Only key-authenticated connections are permitted.
PermitRootLogin no (or prohibit-password) prevents direct root login. Use a standard user with sudo instead.
PubkeyAuthentication yes confirms public key authentication is enabled (usually already set to yes by default).
AuthorizedKeysFile .ssh/authorized_keys specifies the key file location (verify this is not commented out).
Optionally, changing the default SSH port from 22 to a non-standard port (such as 2222 or any port above 1024) reduces the volume of automated scanning traffic. This is security through obscurity, not a substitute for proper authentication, but it significantly reduces log noise.
Save and close the file.
Step 5: Restart the SSH Service
On some distributions, the service name is ssh rather than sshd:
Important: test connectivity from a new terminal session before closing your existing connection. On AlmaLinux and CentOS-based systems, SELinux may block non-standard SSH ports. If you changed the port, allow it through the firewall:
On Ubuntu using UFW:
Step 6: Add Your Key to SSH Agent for Convenience
If you set a passphrase on your key (which you should), entering it on every connection becomes tedious. The SSH agent caches the decrypted key in memory for the duration of your session.
On macOS, add -K to store the passphrase in your keychain:
On Windows, the OpenSSH Authentication Agent service can be configured to start automatically via Services or PowerShell.
Managing Multiple Keys and Servers
A ~/.ssh/config file simplifies connecting to multiple servers without specifying the key file and user on every command.
With this config in place, connecting to production is simply:
Related: How to Setup a VPS Server covers the full VPS provisioning workflow including initial SSH connection.
