BLOG POSTS

MangoHost Blog / SSH Essentials – Working with SSH Servers, Clients, and Keys

SSH Essentials – Working with SSH Servers, Clients, and Keys

SSH (Secure Shell) is the backbone of secure remote access in modern computing environments, allowing developers and system administrators to securely connect to servers, execute commands, and transfer files over encrypted connections. While many professionals use SSH daily, understanding its underlying mechanics, proper client configuration, and key management practices can dramatically improve both security posture and workflow efficiency. This comprehensive guide will walk you through SSH server configuration, client setup, key-based authentication, troubleshooting common issues, and implementing advanced SSH techniques that will make your server management more secure and streamlined.

How SSH Works – Under the Hood

SSH operates on a client-server model using TCP port 22 by default. The protocol establishes a secure channel through a three-stage process: version negotiation, key exchange, and authentication. During the initial handshake, both client and server agree on encryption algorithms, exchange cryptographic keys, and verify server identity before any sensitive data transmission occurs.

The beauty of SSH lies in its layered security approach. The transport layer handles encryption and server authentication, the user authentication layer manages client verification, and the connection layer multiplexes multiple channels over a single SSH connection. This architecture allows you to run multiple simultaneous operations like command execution, file transfers, and port forwarding over one encrypted connection.

SSH supports several authentication methods:

Password authentication (least secure, but widely supported)
Public key authentication (most common and secure)
Host-based authentication (useful for trusted networks)
Keyboard-interactive authentication (supports multi-factor auth)
GSSAPI authentication (for enterprise environments with Kerberos)

SSH Server Configuration and Setup

Setting up an SSH server properly is crucial for both security and functionality. Most Linux distributions come with OpenSSH server available through their package managers.

Installation on Ubuntu/Debian:

sudo apt update
sudo apt install openssh-server
sudo systemctl enable ssh
sudo systemctl start ssh

Installation on CentOS/RHEL:

sudo yum install openssh-server
sudo systemctl enable sshd
sudo systemctl start sshd

The main SSH server configuration file is located at /etc/ssh/sshd_config. Here’s a security-hardened configuration example:

# Basic Settings
Port 2222                          # Change default port
Protocol 2                         # Use SSH protocol version 2 only
PermitRootLogin no                 # Disable direct root login
MaxAuthTries 3                     # Limit authentication attempts
MaxSessions 2                      # Limit concurrent sessions

# Authentication Settings
PubkeyAuthentication yes           # Enable public key auth
PasswordAuthentication no          # Disable password auth
PermitEmptyPasswords no           # Never allow empty passwords
ChallengeResponseAuthentication no # Disable challenge-response

# Network Settings
ClientAliveInterval 300           # Send keepalive every 5 minutes
ClientAliveCountMax 2            # Disconnect after 2 missed keepalives
TCPKeepAlive no                  # Use SSH keepalives instead of TCP

# Access Controls
AllowUsers deploy webadmin       # Only allow specific users
DenyUsers root guest            # Explicitly deny users
AllowGroups ssh-users          # Allow users in ssh-users group

# Security Enhancements
X11Forwarding no               # Disable X11 forwarding if not needed
AllowTcpForwarding no         # Disable port forwarding if not needed
GatewayPorts no               # Prevent gateway port forwarding
PermitTunnel no              # Disable tun device forwarding

After making configuration changes, always test the configuration and restart the service:

sudo sshd -t                    # Test configuration syntax
sudo systemctl restart sshd     # Restart SSH daemon
sudo systemctl status sshd      # Verify service is running

SSH Client Configuration and Usage

The SSH client configuration file ~/.ssh/config is a powerful tool that can simplify your daily SSH workflow. Here’s a comprehensive example showing various configuration options:

# Global defaults
Host *
    ServerAliveInterval 60
    ServerAliveCountMax 3
    TCPKeepAlive no
    Compression yes
    ControlMaster auto
    ControlPath ~/.ssh/connections/%r@%h:%p
    ControlPersist 10m

# Production web server
Host prodweb
    HostName 192.168.1.100
    User deploy
    Port 2222
    IdentityFile ~/.ssh/prodweb_rsa
    ForwardAgent no
    StrictHostKeyChecking yes

# Development environment with port forwarding
Host devbox
    HostName dev.example.com
    User developer
    LocalForward 3000 localhost:3000
    LocalForward 5432 localhost:5432
    DynamicForward 8080

# Jump host configuration
Host internal-server
    HostName 10.0.1.50
    User admin
    ProxyJump jumphost.example.com
    
# Multiple jump hosts
Host deep-internal
    HostName 10.0.2.100
    User admin
    ProxyJump jumphost.example.com,gateway.internal.com

The ControlMaster settings enable connection sharing, which significantly speeds up multiple SSH connections to the same host. The first connection establishes a master socket, and subsequent connections reuse it, eliminating the handshake overhead.

SSH Key Management and Authentication

Public key authentication is the gold standard for SSH security. Here’s how to generate, deploy, and manage SSH keys effectively:

Generate different types of SSH keys:

# RSA key (traditional, widely supported)
ssh-keygen -t rsa -b 4096 -f ~/.ssh/myserver_rsa -C "user@myserver"

# Ed25519 key (modern, recommended)
ssh-keygen -t ed25519 -f ~/.ssh/myserver_ed25519 -C "user@myserver"

# ECDSA key (good alternative)
ssh-keygen -t ecdsa -b 521 -f ~/.ssh/myserver_ecdsa -C "user@myserver"

Key strength comparison:

Key Type	Key Size	Security Level	Performance	Compatibility
RSA	4096 bits	High	Moderate	Excellent
Ed25519	256 bits	Very High	Excellent	Good (newer systems)
ECDSA	521 bits	High	Good	Good
DSA	1024 bits	Weak (deprecated)	Poor	Legacy only

Deploy your public key to servers:

# Using ssh-copy-id (recommended)
ssh-copy-id -i ~/.ssh/myserver_ed25519.pub user@server.example.com

# Manual deployment
cat ~/.ssh/myserver_ed25519.pub | ssh user@server.example.com "mkdir -p ~/.ssh && cat >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys && chmod 700 ~/.ssh"

# For servers without direct SSH access
scp ~/.ssh/myserver_ed25519.pub user@server.example.com:~/
ssh user@server.example.com "cat ~/myserver_ed25519.pub >> ~/.ssh/authorized_keys && rm ~/myserver_ed25519.pub"

Manage multiple keys with SSH agent:

# Start SSH agent
eval "$(ssh-agent -s)"

# Add keys to agent
ssh-add ~/.ssh/myserver_ed25519
ssh-add ~/.ssh/github_rsa

# List loaded keys
ssh-add -l

# Remove specific key
ssh-add -d ~/.ssh/myserver_ed25519

# Remove all keys
ssh-add -D

For enhanced security, use key passphrases and consider implementing key rotation policies. Many organizations require key rotation every 90-180 days for compliance reasons.

Advanced SSH Features and Techniques

SSH tunneling and port forwarding open up powerful possibilities for secure access to internal services:

# Local port forwarding (access remote service locally)
ssh -L 8080:internal-web-server:80 user@gateway.example.com

# Remote port forwarding (expose local service remotely)
ssh -R 9000:localhost:3000 user@public-server.example.com

# Dynamic port forwarding (SOCKS proxy)
ssh -D 1080 user@proxy-server.example.com

# Tunnel with background execution
ssh -f -N -L 5432:database-server:5432 user@gateway.example.com

SSH can also handle file transfers efficiently:

# SCP examples
scp file.txt user@server.example.com:/path/to/destination/
scp -r local-directory/ user@server.example.com:/remote/path/
scp user@server.example.com:/remote/file.txt ./local-file.txt

# SFTP for interactive file transfer
sftp user@server.example.com
# SFTP commands: get, put, ls, cd, mkdir, rm, etc.

# Rsync over SSH (most efficient for large transfers)
rsync -avz -e ssh /local/path/ user@server.example.com:/remote/path/
rsync -avz --delete -e ssh /local/path/ user@server.example.com:/remote/path/

Real-World Use Cases and Examples

SSH shines in numerous practical scenarios that developers and system administrators encounter daily:

Development Environment Setup: Many developers use SSH to connect to remote development servers, enabling consistent environments across teams while leveraging more powerful hardware than local machines.

# Connect to dev environment with port forwarding for local testing
ssh -L 3000:localhost:3000 -L 5432:postgres-dev:5432 developer@dev-server.company.com

Database Administration: DBAs frequently use SSH tunnels to securely access database servers that aren’t directly exposed to the internet:

# Connect to production database through bastion host
ssh -L 5432:prod-db-internal:5432 -J bastion.company.com dba@db-gateway.internal

Container and Kubernetes Management: SSH provides secure access to container hosts and can be used alongside kubectl for comprehensive cluster management:

# Access Kubernetes master node
ssh -i ~/.ssh/k8s-master.pem ubuntu@k8s-master.company.com

# Port forward to access Kubernetes dashboard
ssh -L 8001:localhost:8001 -N -f ubuntu@k8s-master.company.com
kubectl proxy --address='0.0.0.0' --accept-hosts='^*$'

Automated Deployment Pipelines: CI/CD systems leverage SSH for deployment automation, using key-based authentication to securely deploy applications:

# Example deployment script
#!/bin/bash
ssh-add ~/.ssh/deploy_key
ssh deploy@prod-server.com "cd /var/www/app && git pull origin main && docker-compose restart"

Troubleshooting Common SSH Issues

SSH problems can be frustrating, but systematic troubleshooting usually reveals the root cause quickly. Here are the most common issues and their solutions:

Connection Refused or Timeout:

# Check if SSH service is running
sudo systemctl status sshd

# Verify SSH is listening on expected port
sudo netstat -tlnp | grep :22
sudo ss -tlnp | grep :22

# Test connectivity
telnet server.example.com 22
nc -zv server.example.com 22

Permission Denied (Public Key):

# Check SSH key permissions (common issue)
chmod 700 ~/.ssh
chmod 600 ~/.ssh/id_rsa
chmod 644 ~/.ssh/id_rsa.pub
chmod 600 ~/.ssh/authorized_keys

# Verify public key is correctly deployed
ssh-copy-id -i ~/.ssh/id_rsa.pub user@server.example.com

# Debug authentication process
ssh -vvv user@server.example.com

SSH Verbose Debugging: The verbose flags (-v, -vv, -vvv) provide increasingly detailed output about the SSH connection process:

# Basic debugging
ssh -v user@server.example.com

# Detailed debugging
ssh -vvv user@server.example.com 2>&1 | tee ssh-debug.log

Server-Side Troubleshooting:

# Check SSH daemon logs
sudo journalctl -u sshd -f
sudo tail -f /var/log/auth.log

# Test SSH configuration
sudo sshd -T | grep -i "setting you want to check"

# Run SSH daemon in debug mode (use separate terminal)
sudo /usr/sbin/sshd -d -p 2223

Common SSH error messages and their meanings:

Error Message	Common Cause	Solution
Connection refused	SSH daemon not running	Start SSH service
Permission denied (publickey)	Key not authorized or wrong permissions	Check key deployment and file permissions
Host key verification failed	Server key changed	Remove old key from known_hosts
Bad permissions	SSH files too permissive	Fix file/directory permissions
Connection timed out	Firewall or network issue	Check firewall rules and network connectivity

SSH Security Best Practices

Implementing proper SSH security requires attention to both server and client configurations. Here are essential security practices:

Server Hardening:

Change the default SSH port to reduce automated attacks
Implement fail2ban or similar intrusion prevention systems
Use strong ciphers and disable weak algorithms
Enable two-factor authentication where possible
Regularly audit SSH access logs
Implement network-level restrictions using firewalls

# Install and configure fail2ban
sudo apt install fail2ban
sudo cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local

# Example fail2ban SSH configuration
[sshd]
enabled = true
port = ssh
filter = sshd
logpath = /var/log/auth.log
maxretry = 3
bantime = 3600
findtime = 600

Client Security:

Use strong passphrases for SSH keys
Implement key rotation policies
Use SSH agent forwarding cautiously
Verify host keys before first connection
Use jump hosts for accessing internal networks
Implement certificate-based authentication for large environments

Network Security:

# Configure firewall to limit SSH access
sudo ufw allow from 192.168.1.0/24 to any port 22
sudo ufw deny 22

# Or using iptables
sudo iptables -A INPUT -p tcp -s 192.168.1.0/24 --dport 22 -j ACCEPT
sudo iptables -A INPUT -p tcp --dport 22 -j DROP

SSH Alternatives and When to Use Them

While SSH is the standard for secure remote access, several alternatives exist for specific use cases:

Tool	Use Case	Pros	Cons
Mosh	High-latency connections	Survives connection drops, low latency	UDP-based, limited protocol support
Telnet	Local network debugging	Simple, widely supported	No encryption, insecure
RDP/VNC	Graphical remote access	Full desktop experience	Higher bandwidth, less secure than SSH
WinRM/PSRemoting	Windows environments	Native Windows integration	Windows-specific, complex setup
Ansible/Fabric	Automation and orchestration	Built for automation	Not interactive, learning curve

Mosh (Mobile Shell) deserves special mention as an SSH alternative designed for mobile and high-latency connections:

# Install Mosh
sudo apt install mosh

# Connect using Mosh
mosh user@server.example.com

# Mosh with specific port range
mosh --server="mosh-server -p 60001" -p 60001 user@server.example.com

For organizations managing hundreds or thousands of servers, consider SSH certificate authorities. This approach scales better than individual key management:

# Generate CA key pair
ssh-keygen -t rsa -b 4096 -f ssh-ca-key

# Sign user certificate
ssh-keygen -s ssh-ca-key -I user-cert -n username -V +1w ~/.ssh/user-key.pub

# Sign host certificate
ssh-keygen -s ssh-ca-key -I host-cert -h -n server.example.com -V +365d /etc/ssh/ssh_host_rsa_key.pub

SSH remains the cornerstone of secure remote administration due to its robust security model, extensive feature set, and universal support. Understanding its internals, proper configuration, and advanced features enables you to build more secure and efficient infrastructure management workflows. Whether you’re managing a single VPS or orchestrating thousands of containers across multiple cloud providers, mastering SSH fundamentals and advanced techniques will significantly improve your operational capabilities.

For additional information, consult the official OpenSSH documentation at https://www.openssh.com/manual.html and the comprehensive SSH protocol specifications in RFC 4251 through RFC 4254.

This article incorporates information and material from various online sources. We acknowledge and appreciate the work of all original authors, publishers, and websites. While every effort has been made to appropriately credit the source material, any unintentional oversight or omission does not constitute a copyright infringement. All trademarks, logos, and images mentioned are the property of their respective owners. If you believe that any content used in this article infringes upon your copyright, please contact us immediately for review and prompt action.

This article is intended for informational and educational purposes only and does not infringe on the rights of the copyright owners. If any copyrighted material has been used without proper credit or in violation of copyright laws, it is unintentional and we will rectify it promptly upon notification. Please note that the republishing, redistribution, or reproduction of part or all of the contents in any form is prohibited without express written permission from the author and website owner. For permissions or further inquiries, please contact us.