B300, B200, H200 or MI355X: which flagship GPU to rent

#Four flagships, four profiles

The B300 (Blackwell Ultra) is the most powerful GPU that can be rented on demand today: eight providers list it, while NVIDIA’s next generation, Rubin, is only available through reservations. The B200 is its predecessor in the Blackwell family. The H200 is Hopper with 141 GB of HBM3e. The MI355X is AMD’s current flagship, built on the CDNA 4 architecture.

All four come in servers of 1, 2, 4 or 8 GPUs, at the same price per GPU whatever the size. The choice depends on four things: how much memory your model and context need, which number formats you train or serve in, whether you need FP64, and whether your software runs on CUDA only.

#The specifications side by side

Sources: NVIDIA’s HGX specifications for the B300 and B200 (figures for an 8-GPU board divided by eight; NVIDIA quotes FP8 with sparsity, so dense is half), the NVIDIA H200 product page (dense FP8 is half the sparse figure), NVIDIA’s Blackwell Ultra technical brief for maximum power, and AMD’s ROCm documentation for the MI355X. Power figures are vendor maximums; the actual limit depends on the server.

Two details stand out. Hopper has no FP4 support at all, while Blackwell and CDNA 4 both run 4-bit and 6-bit formats in hardware. And the B300 keeps very little FP64, which matters for scientific code.

#What a GB of GPU memory costs

For inference, and for training jobs limited by memory rather than compute, the useful metric is the monthly price of each GB of GPU memory. With our prices per GPU per month:

The MI355X costs $4.89 per GB, 35% of the B300’s $13.95 for the same 288 GB and 8 TB/s. Among the NVIDIA parts the B300 is the cheapest per GB and the B200 the most expensive: it costs 86% of a B300 for 62.5% of its memory.

#Training at scale: B300 or B200

Both Blackwell parts connect each GPU to the others at 1.8 TB/s over NVLink 5, twice the 900 GB/s of Hopper. That matters in distributed training, where gradients and activations cross GPUs at every step.

NVIDIA’s HGX figures give the B300 and B200 the same dense FP8 and BF16 throughput per GPU. The differences are memory (288 GB against 180 GB) and FP4: the B300 is rated 13.5 PFLOPS dense FP4 per GPU, the B200 9 PFLOPS. For FP8 or BF16 training that fits in 180 GB per GPU, the B200 offers the same rated throughput for 14% less ($3,439 against $4,019). The B300 earns its price when larger batches, longer sequences or bigger models need the extra memory, or when you train or serve in FP4.

For an 8-GPU node, see multi-GPU setup.

#Long-context inference: memory decides

During inference, every token in the context keeps its keys and values in GPU memory. For Llama 3.1 70B that KV cache is 327,680 bytes per token in BF16, so a single 128K-token conversation needs about 43 GB on top of the weights. The arithmetic is in our GPU memory guide.

With the weights in FP8 (about 70 GB) and 92% of memory counted as usable, one H200 holds the model and one full-length conversation. A B300 or MI355X holds the model and about four. Bandwidth adds to the gap: generating each token reads the weights and the cache from memory, and the three 8 TB/s parts have 1.67 times the bandwidth of the H200. The B200 sits between them: 180 GB at 8 TB/s, room for about two such conversations.

#FP64 science: not the B300

Many simulation and scientific codes rely on double precision, and here the four GPUs diverge sharply. The MI355X is rated 78.6 TFLOPS in FP64. The H200 reaches 34 TFLOPS, or 67 TFLOPS on its Tensor Cores, and the B200 37 TFLOPS per GPU. The B300 gave most of its FP64 up: NVIDIA lists 10 TFLOPS for a whole 8-GPU HGX B300 board, about 1.25 TFLOPS per GPU.

For mixed AI and HPC work, the MI355X delivers 78.6 TFLOPS for $1,409 a month, against 67 TFLOPS for $2,279 on the H200. The H200 and B200 are the choices on CUDA. Avoid the B300 for FP64-heavy code.

#CUDA or ROCm

The three NVIDIA GPUs run the CUDA ecosystem as it is, including NVIDIA-only tools such as TensorRT-LLM. The MI355X runs ROCm, AMD’s open software stack, which lists it as supported under the architecture name gfx950. The main frameworks are there: PyTorch publishes ROCm builds, vLLM supports the MI350 series with ROCm 7.0 or later, and SGLang publishes Docker images built for MI350-class GPUs.

The risk lies in the rest of your stack. Custom CUDA kernels have to be ported to HIP, which AMD’s HIPIFY tool only partly automates, and libraries that ship CUDA builds only will not run. Check each dependency before you commit, or test on a single GPU for a month first. Setup notes are in AMD ROCm and NVIDIA drivers and CUDA.

#Which one to rent

Start from memory and software, then compare the price per GB and per GPU. Each GPU has its own page: B300, B200, H200 and MI355X. The comparison page puts them side by side, and when you have made your choice, configure a server.