How to Create and Manage SSH Keys for Secure Access

Introduction

SSH keys provide a secure, convenient way to authenticate to servers without typing a password every time. This guide walks you through why SSH keys are better, how to generate them, deploy and manage them, and common pitfalls to avoid — with practical commands and examples for beginners.

Why use SSH keys (short overview)

SSH keys use public-key cryptography: you keep a private key on your machine and copy the matching public key to any server you want to access. This is more secure than passwords because:

• Private keys are never transmitted over the network.
• Keys can be protected with passphrases and stored in encrypted formats.
• You can manage per-device access easily and revoke keys by removing the public key from the server.

Example: check whether your SSH client supports modern key types

ssh -V
# Example output: OpenSSH_9.2p1, LibreSSL 3.6.2

Explanation: Newer OpenSSH supports secure key algorithms like ed25519 and RSA with larger key sizes. Prefer ed25519 when available (shorter, faster, secure), otherwise rsa 3072+.

Generate and configure SSH keys

This section shows how to create a key pair, add a passphrase, and store keys in a custom location. We'll also include a quick reference commands table.

Generate a new ed25519 key (recommended):

ssh-keygen -t ed25519 -C "your.email@example.com"
# prompts:
# Enter file in which to save the key (/home/you/.ssh/id_ed25519): [press Enter to accept default]
# Enter passphrase (empty for no passphrase): [type passphrase]
# Enter same passphrase again:

Generate an RSA key (if ed25519 isn't available):

ssh-keygen -t rsa -b 4096 -C "your.email@example.com"

Create a key in a custom file (useful for multiple identities):

ssh-keygen -t ed25519 -f ~/.ssh/id_work -C "work@example.com"

Commands table

Command	Purpose	Example
ssh-keygen -t ed25519 -C "email"	Create ed25519 key pair	ssh-keygen -t ed25519 -C "me@host"
ssh-keygen -t rsa -b 4096	Create RSA 4096-bit key	ssh-keygen -t rsa -b 4096
ssh-add	Add key to ssh-agent	eval "$(ssh-agent -s)"; ssh-add ~/.ssh/id_ed25519
ssh-copy-id user@host	Copy public key to server (Linux)	ssh-copy-id user@server.example.com
scp ~/.ssh/id_rsa.pub user@host:~/.ssh/authorized_keys	Copy public key manually	scp ~/.ssh/id_rsa.pub user@server:~/.ssh/temp.pub

Explanation: The table gives the most common commands for creating and loading keys. Use ssh-add + ssh-agent to avoid entering your passphrase every time.

Add key to ssh-agent (so your passphrase is cached for a session):

# Start agent
eval "$(ssh-agent -s)"

# Add private key
ssh-add ~/.ssh/id_ed25519
# Enter passphrase when prompted

If you need to change or remove a passphrase:

# Change passphrase
ssh-keygen -p -f ~/.ssh/id_ed25519

# Remove passphrase (not generally recommended)
ssh-keygen -p -f ~/.ssh/id_ed25519 -N ""

Deploying keys and connecting to servers

Once you have a public/private key pair, you must copy the public key to the server and ensure correct permissions. There are a few common approaches.

Automatic copy with ssh-copy-id:

ssh-copy-id -i ~/.ssh/id_ed25519.pub user@server.example.com
# You will be prompted for the server password one final time.
# After this you should be able to SSH without a password.
ssh user@server.example.com

Manual installation (if ssh-copy-id not available):

# On local machine: copy public key to server
scp ~/.ssh/id_ed25519.pub user@server.example.com:~/temp_pub_key

# On server: append to authorized_keys and set permissions
ssh user@server.example.com
mkdir -p ~/.ssh
cat ~/temp_pub_key >> ~/.ssh/authorized_keys
chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys
rm ~/temp_pub_key
exit

Example of connecting while using a specific key:

ssh -i ~/.ssh/id_work user@server.example.com

Using the SSH config file to simplify connections:

# ~/.ssh/config
Host myserver
    HostName server.example.com
    User deploy
    IdentityFile ~/.ssh/id_work
    IdentitiesOnly yes

Then connect with:

ssh myserver

Explanation: Permissions on ~/.ssh and authorized_keys are critical; SSH will refuse to use keys if directories are world-writable. The SSH config file lets you manage multiple keys and defaults.

Manage keys, rotate, and secure access

Managing keys means keeping private keys safe, rotating old keys, auditing allowed keys on servers, and removing unused keys.

List authorized keys on a server (for audit):

ssh user@server.example.com 'nl -ba ~/.ssh/authorized_keys'

Remove a key from authorized_keys (edit carefully):

ssh user@server.example.com
nano ~/.ssh/authorized_keys
# Delete the line containing the public key and save

Rotate keys (create a new key, deploy it, then remove the old one):

# Create a new key locally
ssh-keygen -t ed25519 -f ~/.ssh/id_rotated -C "rotated@you"

# Deploy new public key
ssh-copy-id -i ~/.ssh/id_rotated.pub user@server.example.com

# Verify you can connect with the new key
ssh -i ~/.ssh/id_rotated user@server.example.com

# When satisfied, remove old public key from server authorized_keys
ssh user@server.example.com 'grep -v "old-key-comment" ~/.ssh/authorized_keys > ~/.ssh/authorized_keys.new && mv ~/.ssh/authorized_keys.new ~/.ssh/authorized_keys'

Revoke access quickly: remove the offending public key from all servers’ authorized_keys files or manage server-side with authorized_keys2 or an external mechanism like LDAP, configuration management (Ansible, Puppet), or a central CA.

Backing up private keys: store encrypted backups (e.g., encrypted archive, hardware tokens) and never email raw private keys. Prefer hardware-backed keys (YubiKey) for high-security use:

# Example: list keys handled by a YubiKey (requires tooling)
ssh-keygen -D /usr/lib/ssh/yubikeyyubico.so

Explanation: Treat private keys like passwords. Use passphrases and consider hardware tokens for critical accounts. Regularly audit and rotate keys — long-lived keys are a common security gap.

Common Pitfalls

• Wrong file permissions: SSH ignores keys if ~/.ssh is 0777 or authorized_keys is world-writable. Always use chmod 700 ~/.ssh and chmod 600 ~/.ssh/authorized_keys.
• Copying the private key around: Never copy your private key to other machines or share it. Share only the public key (.pub).
• Multiple keys / identity confusion: If multiple keys exist and SSH picks the wrong one, use ssh -i or configure ~/.ssh/config with IdentityFile and IdentitiesOnly yes to force the correct key.

Next Steps

• Use ssh-agent forwarding and/or a hardware token (YubiKey) for improved security and convenience.
• Implement central key management (Ansible, Vault, or an SSH CA) to automate deployment and revocation.
• Audit your servers for old or unused authorized_keys entries and rotate keys regularly.

Final notes

SSH keys are an essential tool for secure server access. Start with ed25519 keys, protect private keys with passphrases and ssh-agent, use the SSH config file to manage multiple identities, and always follow good permission and rotation practices. The commands and examples above will get you from zero to a manageable, secure SSH key setup.