SSH access: keys, firewall, tunnels and file transfer

#Create an ed25519 key

Create the key on your own computer, never on the server. Ed25519 keys are short, fast and supported by every current OpenSSH. Give the key a passphrase; an SSH agent saves you from typing it on every connection.

ssh-keygen -t ed25519 -C "alice@laptop"
# private key: ~/.ssh/id_ed25519    public key: ~/.ssh/id_ed25519.pub
ssh-add ~/.ssh/id_ed25519           # optional: load it into your SSH agent

Copy the public key to the account you log in with. From Linux or macOS:

ssh-copy-id -i ~/.ssh/id_ed25519.pub [email protected]

Windows has no ssh-copy-id. In PowerShell:

type $env:USERPROFILE\.ssh\id_ed25519.pub | ssh [email protected] "umask 077; mkdir -p ~/.ssh; cat >> ~/.ssh/authorized_keys"

Only the .pub file ever leaves your computer. If a private key may have been exposed, delete the matching line from ~/.ssh/authorized_keys on the server and create a new key.

#Shortcuts in ~/.ssh/config

A host entry saves typing and applies the same options to ssh, scp and rsync:

Host gpu1
    HostName 203.0.113.10
    User alice
    IdentityFile ~/.ssh/id_ed25519
    IdentitiesOnly yes
    ServerAliveInterval 30
    ServerAliveCountMax 4

Now ssh gpu1 is enough. ServerAliveInterval sends a keep-alive every 30 seconds, so idle sessions are not dropped by routers along the way. On Linux and macOS the file must not be writable by others (chmod 600 ~/.ssh/config); on Windows it lives in %USERPROFILE%\.ssh\config.

#Turn off password logins

Once key login works for your user, turn off password and keyboard-interactive logins and stop root from logging in with a password. Ubuntu reads drop-in files from /etc/ssh/sshd_config.d/ before the rest of the configuration, and for most options the first value found wins. A file whose name starts with 00- is read first and overrides later ones, such as a cloud-init file that turns passwords back on.

sudo tee /etc/ssh/sshd_config.d/00-hardening.conf >/dev/null <<'EOF'
PasswordAuthentication no
KbdInteractiveAuthentication no
PermitRootLogin prohibit-password
EOF
sudo sshd -t && sudo systemctl restart ssh
sudo sshd -T | grep -Ei '^(passwordauthentication|kbdinteractiveauthentication|permitrootlogin) '

Restarting the SSH service does not close open sessions. From a second terminal, check that a password is now refused and that your key still works:

ssh -o PubkeyAuthentication=no -o PreferredAuthentications=password [email protected]   # must fail
ssh gpu1                                                                                  # must work

When your sudo user works, you can go one step further with PermitRootLogin no.

#A basic firewall with ufw

Deny all incoming traffic except SSH, then open other ports one by one when you need them:

sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw allow OpenSSH
sudo ufw enable
sudo ufw status verbose

For a public HTTPS endpoint, for example a reverse proxy, open sudo ufw allow 80,443/tcp. Keep everything else private and reach it through SSH tunnels (below). Note that Docker publishes container ports through its own firewall rules, which ufw does not filter: publish them on 127.0.0.1 only, as explained in Containers.

#fail2ban for SSH

With passwords off, guessing attacks cannot succeed, but they still fill the logs. fail2ban bans addresses that keep failing. Ubuntu’s package enables its SSH jail out of the box; a small override sets the ban policy:

sudo apt install -y fail2ban
sudo tee /etc/fail2ban/jail.d/sshd.local >/dev/null <<'EOF'
[sshd]
enabled  = true
maxretry = 5
findtime = 10m
bantime  = 1h
# never ban your own fixed address:
# ignoreip = 127.0.0.1/8 ::1 198.51.100.7
EOF
sudo systemctl restart fail2ban
sudo fail2ban-client status sshd

To lift a ban: sudo fail2ban-client set sshd unbanip 198.51.100.7.

#Keep long jobs alive with tmux

A job started in a plain SSH session stops when the connection drops. Inside tmux it keeps running, and you can reattach from any new login:

tmux new -s train                     # start a named session
python train.py 2>&1 | tee train.log  # run the job, keep a log file
# detach: press Ctrl-b, then d
tmux ls                               # list sessions
tmux attach -t train                  # reattach later

Scroll back with Ctrl-b [ (then the arrow keys, q to leave). The log file lets you check progress without attaching: tail -f train.log.

#Port forwarding for Jupyter and TensorBoard

Keep notebooks and dashboards bound to 127.0.0.1 on the server and reach them through SSH. Nothing extra is exposed to the internet.

# on the server, inside tmux
jupyter lab --no-browser --ip=127.0.0.1 --port=8888
tensorboard --logdir runs --host 127.0.0.1 --port 6006

# on your computer
ssh -N -L 8888:127.0.0.1:8888 -L 6006:127.0.0.1:6006 gpu1

Then open http://localhost:8888 (with the token Jupyter printed) and http://localhost:6006 in your browser. To forward every time you connect, add LocalForward 8888 127.0.0.1:8888 to the gpu1 entry in ~/.ssh/config.

#Copy data with rsync and scp

# upload: the trailing slash copies the contents of dataset/ into /data/dataset/
rsync -avhP ./dataset/ gpu1:/data/dataset/

# download results
rsync -avhP gpu1:/data/checkpoints/ ./checkpoints/

# single files
scp ./config.yaml gpu1:~/
scp gpu1:~/results.csv .

rsync skips files that are already identical, so running it again only sends what changed; -P keeps partial files, which lets an interrupted transfer resume, and shows progress. Add -z for compressible data such as text or CSV, not for images, video or archives. On Windows, scp is built in and rsync is available through WSL.

For large datasets and backups to object storage, see Storage and data.