Hardware
The GPU.
NVIDIA H200 NVL: 141 GB of fast memory on every card, in a datacentre in the Kathmandu Valley.
Coming soonEnterprise early access is open
The GPU
NVIDIA H200 NVL. The headline numbers, from NVIDIA.
Tensor Core figures are NVIDIA's peak numbers with sparsity2.
- GPU memory
- 141 GB
- Memory bandwidth
- 4.8 TB/s
- FP8 Tensor Core
- 3,341 TFLOPS
- MIG instances, up to
- 7
Memory you can feel
141 GB in about 29 milliseconds.
At 4.8 TB/s, an H200 NVL can read its entire memory in about 29 milliseconds. A blink takes 100 to 400.
Arithmetic from the datasheet3
What comes next
Starting with H200. Built for what follows.
- NowNVIDIA H200 NVLComing soon · enterprise early access
- NextSixteen GPUsMore H200 NVL as early-access customers come on board.
- LaterMore GPUsOther NVIDIA GPUs and new generations, as demand grows.
Questions
About the hardware.
Why the H200 NVL?
141 GB of HBM3e at 4.8 TB/s fits large models and long contexts on a single card, and it splits into up to seven isolated MIG slices, so smaller jobs pay for a slice rather than a whole GPU.
How many slices can one GPU make?
Up to seven, each with its own memory and compute, using NVIDIA Multi-Instance GPU. See MIG slicing for how they are offered.
Can we reserve several GPUs?
Yes. Tell us what you need and we will plan it with you.
Why are the Tensor Core figures so high?
They are NVIDIA's peak figures with sparsity. Real throughput depends on the model, the precision and the code; we are happy to help you measure yours.