In this post you’ll learn why using SSH keys to log into a server is more secure than using passwords, and how to set up key‑based authentication in a few simple steps. It matters because a compromised password can give an attacker full control, while SSH keys make that much harder.

How SSH Works at a Glance

SSH (Secure Shell) is a network protocol that lets you run commands on a remote machine over an encrypted connection. When you type ssh user@host, the client and server perform a handshake that creates a private, encrypted channel.

Password Authentication

  1. The client sends the username.
  2. The server asks for a password.
  3. You type the password; it travels encrypted to the server.
  4. The server checks the password against its stored hash.

If the password matches, you get access.

Key‑Based Authentication

  1. You generate a key pair: a private key (kept on your computer) and a public key (uploaded to the server).
  2. During the handshake, the server sends a random challenge.
  3. Your client signs that challenge with the private key.
  4. The server verifies the signature using the public key stored in ~/.ssh/authorized_keys.
  5. If the signature is valid, you are logged in.

Why Keys Are More Secure

  • No secret transmitted: With passwords, the secret (the password) is sent over the encrypted channel each login. If an attacker ever gets hold of the encrypted traffic and cracks the encryption, they obtain the password. With keys, the private key never leaves your machine; only a signature is sent.
  • Strong cryptography: A typical RSA 4096‑bit or Ed25519 key provides far more entropy than any human‑chosen password. Brute‑forcing such a key is computationally infeasible.
  • Resilience to phishing: A password can be entered on a fake login prompt. A private key requires the actual SSH client to perform the cryptographic operation, making it harder to trick users.
  • Easy revocation: If a private key is lost, you simply remove its public counterpart from authorized_keys. Changing a password requires updating it on every machine that uses it.
  • Automation friendly: Scripts and CI pipelines can use keys without exposing passwords in logs or environment variables.

Common Mistakes to Avoid

  • Leaving the private key unencrypted: Always protect your private key with a passphrase. Even if the file is stolen, the attacker still needs the passphrase to use it.
  • Using the same key everywhere: If one server is compromised, the same key could be reused elsewhere. Generate separate keys for different environments.
  • Storing keys in world‑readable locations: Keep them in ~/.ssh/ with permissions 600 for the private key and 644 for the public key.

Setting Up SSH Key Authentication

  1. Generate a key pair

    ssh-keygen -t ed25519 -C "your_email@example.com"
    • -t ed25519 chooses a modern, fast algorithm.
    • -C adds a comment, usually your email, to identify the key.

  2. Add the public key to the server

    ssh-copy-id user@host
    • This command appends the public key (~/.ssh/id_ed25519.pub) to the server’s ~/.ssh/authorized_keys file.

  3. Test the login

    ssh user@host
    • If you set a passphrase, you’ll be prompted for it now.

  4. Disable password login (optional but recommended)

    sudo sed -i 's/^#PasswordAuthentication yes/PasswordAuthentication no/' /etc/ssh/sshd_config
sudo systemctl restart sshd
    • This edits the SSH daemon configuration to reject password attempts and restarts the service.

Managing Multiple Keys

If you work with several servers that require different keys, use an SSH config file:

# ~/.ssh/config
Host prod-server
    HostName prod.example.com
    User deploy
    IdentityFile ~/.ssh/id_ed25519_prod

Host staging-server
    HostName staging.example.com
    User deploy
    IdentityFile ~/.ssh/id_ed25519_staging
  • Host defines a shortcut name.
  • HostName is the actual server address.
  • User is the login name.
  • IdentityFile points to the private key to use for that host.

Backing Up and Rotating Keys

  • Backup: Store a copy of your private key in a secure password manager or encrypted USB drive.
  • Rotate: Periodically generate a new key pair and replace the old public key on all servers. This limits the impact if a key is later compromised.

FAQ

How strong does my SSH key need to be?

Use Ed25519 or RSA with at least 2048‑bit size. Ed25519 offers strong security with smaller keys and faster operations.

What if I forget my key passphrase?

You’ll need to generate a new key pair and replace the public key on every server. Keeping a backup of the unencrypted key is risky, so it’s better to remember the passphrase.

Can I still use passwords for some accounts?

Yes, you can keep password authentication enabled for specific users or services, but it’s best to disable it globally to reduce the attack surface.

Wrapping up

SSH key authentication gives you stronger cryptography, eliminates the need to type secrets, and makes revoking access easier. Set up a passphrase, keep your private key safe, and consider disabling password logins on your servers. A few minutes of configuration now saves you from headaches and potential breaches later.