Your first hour on a new GPU server

#What you receive

A CryptGPU server is a dedicated machine rented by the month. It comes with two things: the server’s address and root access over SSH. The GPUs are yours alone for the whole term, with no time-slicing and no other tenant on the cards.

Everything above that is your choice: GPU driver, CUDA or ROCm, containers and frameworks. The commands in these guides assume Ubuntu 22.04 or 24.04 LTS. On another distribution the steps are the same, but package names and the package manager differ (for example dnf instead of apt).

In the examples, 203.0.113.10 stands for your server’s address and alice for your user name. Replace both.

#First login

Connect from a terminal on your computer. An OpenSSH client is built into Linux, macOS and Windows 10 and 11 (PowerShell or Windows Terminal).

ssh [email protected]

On the first connection SSH shows the server’s host key fingerprint and asks you to accept it. SSH then remembers the key and warns you if it ever changes. Check which system you are on before running anything else:

cat /etc/os-release
uname -r

If you logged in with a password, set a new one and move to key authentication. Create a key on your computer as shown in SSH access, then copy it to the server:

# on the server: choose a new root password
passwd

# on your computer (Linux or macOS; for Windows see the SSH guide)
ssh-copy-id -i ~/.ssh/id_ed25519.pub [email protected]

#Check the hardware

Compare what the system reports with the configuration you ordered: GPU model and count, vCPUs, memory and NVMe capacity.

GPUs

If a GPU driver is already installed:

# NVIDIA: model, memory, driver version, processes
nvidia-smi
nvidia-smi -L          # one line per GPU
nvidia-smi topo -m     # how the GPUs are connected: NVLink (NV#) or PCIe

# AMD Instinct MI355X
amd-smi list
amd-smi topology

The topology matrix, and what it means for training and serving, is explained in Multi-GPU.

If nvidia-smi or amd-smi is not found, no driver is installed yet. The GPUs are still visible on the PCI bus (10de is NVIDIA’s vendor ID, 1002 is AMD’s):

lspci -d 10de: | grep -Ei '3d controller|vga compatible'   # NVIDIA GPUs
lspci -d 1002:                                             # AMD GPUs

Then install a driver: NVIDIA drivers and CUDA or ROCm on MI355X.

CPU, memory and disks

lscpu | grep -E 'Model name|^CPU\(s\)|NUMA node\(s\)'
free -h
lsblk -o NAME,SIZE,TYPE,FSTYPE,MOUNTPOINTS,MODEL
df -h

free -h shows a little less than the installed memory, because firmware and the kernel reserve part of it. lsblk lists every NVMe drive; if one has no filesystem yet, Storage and data shows how to format and mount it.

#Update the operating system

sudo apt update
sudo apt full-upgrade -y
sudo apt install -y build-essential git curl tmux htop nvtop
[ -f /var/run/reboot-required ] && sudo reboot

A reboot after a kernel update is normal; reconnect once the server is back. Do this before you install a GPU driver, so the driver is built for the kernel you will actually run.

Ubuntu installs security updates automatically with unattended-upgrades. If your GPU driver comes from Ubuntu’s own archive, such an update can replace the driver while jobs are running, and new processes then fail with a driver/library version mismatch until you reboot. NVIDIA drivers and CUDA explains how to lock the driver version.

#Create a non-root sudo user

Work as a normal user and use sudo only when needed. As root, create the user, give it sudo rights and copy your SSH key to it:

adduser alice                  # asks for a password; sudo will use it
usermod -aG sudo alice
install -d -m 700 -o alice -g alice /home/alice/.ssh
install -m 600 -o alice -g alice /root/.ssh/authorized_keys /home/alice/.ssh/authorized_keys

On an AMD server, also add the user to the groups that may open the GPUs: usermod -aG render,video alice.

Test from a second terminal on your computer before you change anything else:

ssh [email protected]
sudo -v && echo "sudo works"

Next, turn off password logins, restrict root and enable a firewall, as described in SSH access.

#Where to go next

• SSH access: key-only login, ufw, fail2ban, tmux, tunnels and rsync.
• NVIDIA drivers and CUDA, or ROCm on AMD Instinct MI355X.
• Containers: Docker with the NVIDIA Container Toolkit or ROCm images.
• Storage and data: format the NVMe drives and plan your backups.
• Multi-GPU training and inference serving.
• Billing and payments: renewals and what happens at the end of a term.

Questions go through the contact form. To plan another server, use the configurator or the price list.