Run ComfyUI on a GPU server

ComfyUI on a GPU server: what you need

ComfyUI is an open-source (GPL-3.0) engine for image, video, audio and 3D generation models, such as Flux.2, Qwen Image, Wan 2.2 and LTX-Video 2.3: you build a workflow from nodes in the browser and queue it on the GPU. To run ComfyUI on a GPU server, install PyTorch built for CUDA 13 (ROCm on AMD) in a Python environment, clone ComfyUI, start it on 127.0.0.1:8188 and open it through an SSH tunnel. It needs an NVIDIA driver from the 580 series or newer and enough GPU memory for the models you load.

The steps work on Ubuntu 22.04 and 24.04, on a dedicated server or a cloud GPU instance. If you picked ComfyUI in the Software step of your order, it is already installed; this guide covers installing and running it yourself.

Memory decides which model versions fit. The Qwen-Image-2512 diffusion model, for example, is a 20.4 GB file in FP8 and 40.9 GB in BF16: the FP8 file fits on all three cards, the BF16 file only on the L40S and the RTX PRO 6000. Weights that do not fit are offloaded to system RAM, at a cost in speed. See the image and video generation page for suggested configurations and Best GPU for Stable Diffusion, Flux and AI video for a comparison.

Install ComfyUI on Ubuntu

These steps follow the manual install in the ComfyUI README; run them as your sudo user, not as root. The README requires a PyTorch build for CUDA 13.0 or newer (cu130) on NVIDIA GPUs from the 20 series on, which includes every NVIDIA GPU we rent. The wheels bring their own CUDA libraries, so the host only needs a driver from the 580 series or newer (CUDA release notes):

nvidia-smi        # the header must show CUDA Version: 13.0 or higher

If not, upgrade the driver first (NVIDIA drivers and CUDA). PyTorch added support for Blackwell GPUs such as the RTX 5090, RTX PRO 6000 and B200 in version 2.7, with its CUDA 12.8 builds; use the cu130 build below, not the CUDA 12.6 option the PyTorch selector also lists:

curl -LsSf https://astral.sh/uv/install.sh | sh     # then open a new shell so uv is on PATH
uv venv --python 3.13 --seed --managed-python ~/comfy-env
source ~/comfy-env/bin/activate

git clone https://github.com/Comfy-Org/ComfyUI.git ~/ComfyUI
cd ~/ComfyUI
pip install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu130
pip install -r requirements.txt
pip install -r manager_requirements.txt             # ComfyUI-Manager, see below

python -c "import torch; print(torch.__version__, torch.cuda.is_available(), torch.cuda.get_device_name(0))"

Expect a version ending in +cu130, True and the GPU’s name. ComfyUI’s documentation recommends Python 3.13; --seed adds pip, which ComfyUI-Manager uses for the requirements of custom nodes. comfy-cli (pip install comfy-cli, then comfy install) is an alternative that picks the PyTorch build for you.

On an AMD MI355X, set up the GPU driver first (ROCm on MI355X) and use the README’s ROCm line instead: pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/rocm7.2. torch.cuda.get_arch_list() must then include gfx950.

Download models to the NVMe disk

ComfyUI looks for each kind of model in its own folder under models/:

Keep the files on the NVMe data disk at /data (Storage and data), safe from a reinstall of ComfyUI. hf download recreates the repository’s subfolders under --local-dir, so use a staging folder and move the files. Example: Qwen-Image-2512 from Comfy-Org’s repackaged repository (Apache 2.0, about 30 GB in FP8):

curl -LsSf https://hf.co/cli/install.sh | bash      # installs the hf command
mkdir -p /data/models/{checkpoints,diffusion_models,text_encoders,vae,loras,upscale_models}

hf download Comfy-Org/Qwen-Image_ComfyUI \
  split_files/diffusion_models/qwen_image_2512_fp8_e4m3fn.safetensors \
  split_files/text_encoders/qwen_2.5_vl_7b_fp8_scaled.safetensors \
  split_files/vae/qwen_image_vae.safetensors \
  --local-dir /data/models/incoming
cd /data/models
mv incoming/split_files/diffusion_models/* diffusion_models/
mv incoming/split_files/text_encoders/* text_encoders/
mv incoming/split_files/vae/* vae/
rm -rf incoming

ComfyUI reads extra_model_paths.yaml from its folder at start-up (extra_model_paths.yaml.example lists every key). Point it at the disk; is_default puts these folders first, also for downloads:

cat > ~/ComfyUI/extra_model_paths.yaml <<'EOF'
nvme:
    base_path: /data/models/
    is_default: true
    checkpoints: checkpoints/
    diffusion_models: diffusion_models/
    text_encoders: text_encoders/
    vae: vae/
    loras: loras/
    upscale_models: upscale_models/
EOF

Gated repositories, such as black-forest-labs/FLUX.1-dev, need you to accept the conditions on the model page, then a token: hf auth login, or HF_TOKEN in the environment. Licenses differ: FLUX.1 [dev] comes under a non-commercial license.

Start ComfyUI and keep it running

Start it by hand first, inside tmux (see SSH access):

source ~/comfy-env/bin/activate
cd ~/ComfyUI
python main.py --listen 127.0.0.1 --port 8188 --enable-manager

The log then shows To see the GUI go to: http://127.0.0.1:8188. 127.0.0.1 and 8188 are the defaults, written out for clarity. python main.py --help lists every option, also documented on the startup flags page.

For a server that comes back after a crash or a reboot, use a systemd template unit: the instance number is the GPU, and the port is 8188 plus that number. Stop the tmux copy first, and replace alice with your user:

mkdir -p /data/comfyui/output
sudo tee /etc/systemd/system/comfyui@.service >/dev/null <<'EOF'
[Unit]
Description=ComfyUI on GPU %i
Wants=network-online.target
After=network-online.target

[Service]
User=alice
WorkingDirectory=/home/alice/ComfyUI
# $$ passes a literal $ to the shell: GPU 0 listens on 8188, GPU 1 on 8189
ExecStart=/bin/sh -c 'exec /home/alice/comfy-env/bin/python main.py --listen 127.0.0.1 --port $$((8188 + %i)) --cuda-device %i --output-directory /data/comfyui/output/gpu%i --enable-manager'
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target
EOF
sudo systemctl daemon-reload
sudo systemctl enable --now comfyui@0
sudo journalctl -u comfyui@0 -f       # the log; Ctrl-C stops following

Each instance saves its images in its own folder under /data/comfyui/output, so two instances never write the same file.

Remote access: SSH tunnel or authenticated proxy

The ComfyUI server has no authentication (ComfyUI’s self-hosting notes): its API queues workflows, accepts uploads and serves your outputs to anyone who can reach it. Keep it on 127.0.0.1 and use one of two ways in.

SSH tunnel: for you and a few trusted people

# on your computer, then open http://localhost:8188
ssh -N -L 8188:127.0.0.1:8188 gpu1

# two instances: one -L per port
ssh -N -L 8188:127.0.0.1:8188 -L 8189:127.0.0.1:8189 gpu1

gpu1 is the host entry from SSH access; LocalForward 8188 127.0.0.1:8188 in that entry opens the tunnel on every connection.

Reverse proxy with TLS and a password

For browser access without SSH, point a DNS name at the server and install Caddy from its official repository. Caddy gets the TLS certificate, asks for a password with basic_auth and also proxies the WebSocket connection the interface uses:

caddy hash-password       # type a long random password; prints its bcrypt hash
sudo tee /etc/caddy/Caddyfile >/dev/null <<'EOF'
comfy.example.com {
    basic_auth {
        alice <paste-the-hash-here>
    }
    reverse_proxy 127.0.0.1:8188
}
EOF
sudo systemctl reload caddy
sudo ufw allow 80,443/tcp

Run ComfyUI on several GPUs

By default a workflow runs on one GPU. On a server with 2, 4 or 8 GPUs, run one instance per GPU: --cuda-device hides the other GPUs from each instance, and each has its own port and queue. With the template unit above:

sudo systemctl enable --now comfyui@1     # GPU 1 on port 8189; add comfyui@2, @3 and so on
nvidia-smi --query-gpu=index,memory.used,utilization.gpu --format=csv -l 5

Four GPUs then run four jobs at once. Since May 2026, a built-in MultiGPU CFG Split node (MultiGPU_WorkUnits) can also speed up one workflow by sharing its CFG sampling between identical GPUs: place it just before the sampler and give that instance several GPUs (--cuda-device 0,1, or no flag). Its documentation describes two-GPU setups and speedups of up to 1.95×, depending on the workflow. The CFG must be above 1, so distilled models that sample at CFG 1 gain nothing, and each GPU loads its own copy of the model (pull request #7063): two cards do not hold a model too large for one.

Custom nodes with ComfyUI-Manager

ComfyUI-Manager installs, updates and removes custom nodes, finds the nodes a workflow is missing and saves snapshots of your setup. It comes with ComfyUI: install manager_requirements.txt (done above) and start with --enable-manager. Its new interface installs only node packs published in the Comfy Registry, which scans them for malicious behavior, and does not install from Git URLs.

Packs that are not in the registry go in by hand:

source ~/comfy-env/bin/activate
cd ~/ComfyUI/custom_nodes
git clone https://github.com/<author>/<pack>.git
pip install -r <pack>/requirements.txt
sudo systemctl restart 'comfyui@*'       # every running instance

To update ComfyUI, move to the latest stable release tag; the README warns that commits between releases can break custom nodes:

source ~/comfy-env/bin/activate
cd ~/ComfyUI
git fetch --tags
git checkout $(git describe --tags `git rev-list --tags --max-count=1`)
pip install -r requirements.txt -r manager_requirements.txt
sudo systemctl restart 'comfyui@*'

Memory and FP8 options

Start without memory flags. ComfyUI turns on dynamic VRAM where it is supported, automatically on NVIDIA GPUs, and moves weights between GPU memory and system RAM as needed. Change the defaults only for a reason:

--highvram and --gpu-only turn dynamic VRAM off, and --lowvram does nothing while it is on. FP8 weights take half the memory of BF16 ones; when a model is published in FP8, as Qwen-Image-2512 is, ComfyUI’s documentation recommends that file for most users.

Back up workflows and outputs before the term ends

If the term is not renewed, the server stops at the end of the paid month and its disks are wiped. Models can be downloaded again; your work cannot. Copy these off the server a few days before the end date:

• /data/comfyui/output: generated images and videos. Unless you start ComfyUI with --disable-metadata, it saves the workflow inside the files it generates, so an image restores its full workflow when you load it.
• ~/ComfyUI/user: workflows saved in the interface, settings and ComfyUI-Manager snapshots.
• ~/ComfyUI/input, extra_model_paths.yaml, the systemd unit and any LoRA you trained.

# from your computer
rsync -avhP gpu1:/data/comfyui/output/ ./comfyui/output/
rsync -avhP gpu1:ComfyUI/user/ ./comfyui/user/

Copy the other items the same way, or to S3-compatible storage with rclone as shown in Storage and data. To move to a larger GPU, order it in the configurator, install ComfyUI as above and copy these folders across while both servers run.