#The specifications
The MI355X is AMD’s flagship on the CDNA 4 architecture (target name gfx950). It is built from eight compute dies with 256 compute units in total. The figures below are AMD’s peak specifications.
| Specification | MI355X |
|---|---|
| Memory | 288 GB HBM3E, in 8 stacks of 36 GB |
| Memory bandwidth | 8 TB/s |
| GPU-to-GPU | 7 Infinity Fabric links, 1.07 TB/s in total |
| FP8 (OCP), dense | 5.0 PFLOPS |
| MXFP6 and MXFP4 | 10 PFLOPS |
| BF16 and FP16, dense | 2.5 PFLOPS |
| FP64, vector and matrix | 78.6 TFLOPS |
| Board power | 1,400 W |
Next to NVIDIA’s parts, that is the memory of a B300, 1.67 times the bandwidth of an H200 and more FP64 than any NVIDIA GPU we rent: the H100 and H200 reach 67 TFLOPS on their Tensor Cores.
#Why the price is low
Our MI355X costs $1,409 per GPU per month, against $1,779 for an H100 and $2,279 for an H200. It follows the same rule as every GPU we rent, 30% below the market reference, and that reference is simply low. No provider in our survey publishes an on-demand price for the MI355X. The two public starting prices we found are $2.59 and $2.95 per GPU-hour, below the $3.49 median of the H100.
Our reading of why:
- Limited on-demand availability. Few providers sell the MI355X by the hour. In our survey the offers were quotes or entry prices, so the public reference is set by the lowest advertised prices rather than by a broad on-demand market.
- A younger software ecosystem. CUDA has a long head start, and some tools and kernels exist only for NVIDIA GPUs. Teams that cannot move their stack do not compete for AMD capacity, which keeps prices down.
Neither point says anything about the hardware itself. Both are reasons to check your software before you switch.
#Memory per dollar against the H100 and H200
| GPU | Memory | Bandwidth | Price per GPU / month | Price per GB / month | GB per $1,000 a month |
|---|---|---|---|---|---|
| MI355X | 288 GB | 8 TB/s | $1,409 | $4.89 | 204 GB |
| H200 | 141 GB | 4.8 TB/s | $2,279 | $16.16 | 62 GB |
| H100 | 80 GB | 3.35 TB/s | $1,779 | $22.24 | 45 GB |
Per GB, the MI355X costs 30% of an H200 and 22% of an H100: each dollar buys 3.3 times the memory of an H200 and 4.5 times that of an H100. Per GPU it is 21% cheaper than an H100 while holding 3.6 times as much memory.
#Where it wins: memory-bound inference
Large-model inference is usually limited by memory: the model has to fit, and each generated token reads the weights and the KV cache. That is where 288 GB at 8 TB/s pays. Take Llama 3.1 405B in FP8. Our sizing rule (1.2 GB per billion parameters, 92% of memory usable) puts it at 486 GB:
| Server | Usable memory | Price per month |
|---|---|---|
| 2× MI355X | 530 GB | $2,818 |
| 2× B300 | 530 GB | $8,038 |
| 4× H200 | 519 GB | $9,116 |
| 8× H100 | 589 GB | $14,232 |
Two MI355X hold the model for 35% of the price of two B300s, 31% of four H200s and 20% of eight H100s. The pattern repeats for DeepSeek-V3 or R1 in FP8 (805 GB by the same rule): 4× MI355X at $5,636 a month, against 4× B300 at $16,076 or 8× H200 at $18,232. And Llama 3.1 70B in BF16 (168 GB) fits on a single MI355X, where CUDA users need two RTX PRO 6000 ($1,918), one B300 ($4,019) or two H200 ($4,558).
Fitting is not the same as speed. Tokens per second depend on the engine, the kernels and how the model is split, so benchmark your own model before you move a production workload. The sizing method is explained in how much GPU memory your model needs.
#What runs on ROCm today
- ROCm lists the MI355X (gfx950) as supported on Ubuntu 22.04 and 24.04, RHEL 9 and 10, SLES 15 SP7, Debian 13 and Oracle Linux 9 and 10.
- PyTorch publishes ROCm wheels, and AMD maintains ready-made
rocm/pytorchDocker images. - vLLM supports the MI350 series (gfx950) with ROCm 7.0 or later, ships ROCm wheels and Docker images, and its FP8 KV cache (
fp8_e4m3) works on AMD GPUs. - SGLang publishes Docker images built for MI350-class GPUs.
For PyTorch code without custom CUDA kernels, and for serving open-weight models with vLLM or SGLang, these are supported paths. You get root SSH access and install your own stack; setup steps, from the driver to a first container, are in AMD ROCm and containers.
#Caveats before you switch
- CUDA-only software. NVIDIA tools such as TensorRT-LLM do not run on AMD GPUs, and some libraries ship CUDA builds only. List every dependency and check it.
- Custom kernels. Hand-written CUDA kernels must be ported to HIP. AMD’s HIPIFY tool translates much of the source but not all of it, and ported kernels still need testing and tuning.
- Recent versions only. The MI355X needs a recent stack: vLLM requires ROCm 7.0 or later for it, and libraries must include kernels for gfx950. Containers built only for older AMD GPUs may not.
- Quantized formats. The MI355X implements FP8 and the open MX formats (MXFP8, MXFP6, MXFP4). Check that the format of your quantized checkpoint is supported on ROCm before you plan around it.
- Scale-up bandwidth. Seven Infinity Fabric links give 1.07 TB/s, more than the 900 GB/s of NVLink 4 on the H100 and H200, and less than the 1.8 TB/s of NVLink 5 on the B200 and B300.
#When to choose the MI355X
Choose the MI355X when memory per dollar decides: large models in FP8 or BF16, long contexts, several models on one server, or FP64 work next to AI. Choose an H100 or H200 when your stack is tied to CUDA, when you depend on TensorRT-LLM or custom CUDA kernels, or when the model fits comfortably in 80 or 141 GB.
If you are unsure, rent one MI355X for a month and run your own benchmark, then order 2, 4 or 8 GPUs at the same price per GPU. See the MI355X page, compare it with the H100 and H200, read our flagship comparison, or configure a server.