GPU Generations: Hopper, Ada, Blackwell, Rubin, Grace

A GPU model name such as H100 or B200 looks like a random code. It is not. The letter names the architecture (the design generation of the chip), and each generation changes how fast the chip computes, how fast its memory is, and which number formats it supports. Picking the wrong generation costs real money: a chip that is too old runs the workload slowly, and a chip that is too big wastes budget.

This lesson explains how GPU names work, then walks through the generations that matter for inference: Hopper, Ada Lovelace, Blackwell, and the coming Rubin and Feynman. It ends with the Grace and Vera CPUs, which NVIDIA attaches to its GPUs. Afterwards you can read a spec table for these GPUs, compute the ratio of arithmetic speed to memory speed for each one, and name the use case for which each is a good fit. The terms FLOPS, VRAM, and bandwidth come from Inside a GPU.

Example 1: why GPU generations arrive slowly

Hardware iteration is slow. Years pass between finishing an architecture design and shipping the GPUs, because the design must go through tapeout (sending the final chip design to the factory) and testing. The AI industry moves fast, so even a next-generation GPU was designed at a time when model capabilities looked very different from today. A GPU architecture that stays useful for years has to predict how use cases will change over its lifetime.

For years, training AI models was the main use of GPUs. Inference is now the dominant use, so recent architectures add features meant for inference.

Every one to two years NVIDIA releases a new architecture that powers all of its products, from datacenter servers to personal computers. A new generation brings faster base compute and memory plus features for more efficient inference. Since 1998 NVIDIA has named its architectures after prominent scientists.

timeline
  title NVIDIA GPU architectures
  2017 : Volta
  2018 : Turing
  2020 : Ampere
  2022 : Hopper : Lovelace
  2024 : Blackwell
  2026 : Rubin announced
  2028 : Feynman announced

Hopper and Lovelace share the year 2022: Hopper launched in March 2022, and Lovelace six months later. Rubin and Feynman are announced and not yet shipping at the time of writing.

Example 2: reading a GPU name

A GPU name has two parts.

• Letter: the architecture generation used in the chip.
• Number: identifies a model within the generation.

The H100 directly replaces the A100 of the previous generation. The H200 is a larger GPU of the same generation as the H100, and the B200 replaces it in turn. The numbers look arbitrary because each generation uses its own set of numbers. The general rule is that a bigger number means a larger, faster, more expensive GPU. The letter is the part that carries meaning. Splitting each name into its letter and number and looking up the architecture gives this table.

name   letter -> architecture   model number
T4     T -> Turing              4
A10    A -> Ampere              10
A100   A -> Ampere              100
L4     L -> Lovelace            4
L40    L -> Lovelace            40
H100   H -> Hopper              100
H200   H -> Hopper              200
B200   B -> Blackwell           200
B300   B -> Blackwell           300

The table shows the point: the same letter always means the same architecture, and comparing numbers is only meaningful inside one letter (an H200 against an H100, not an L40 against an H100).

Inference engineers generally work within the three to five most recent generations. Even for cost-sensitive workloads, newer chips are often cheaper at large scale because they are more efficient, and they perform better. Older Turing (T4) and Ampere (A10, A100) GPUs still appear in low-traffic or legacy systems. Most deployments today use Lovelace (L4, L40), Hopper (H100, H200), or Blackwell (B200, B300).

Example 3: Hopper, the widely used baseline

The Hopper architecture is named for Rear Admiral Grace Hopper. It launched in March 2022 with the H100.

Hopper introduced FP8, a floating point number format that uses 8 bits for each number. FP8 Tensor cores are twice as fast as FP16 Tensor cores, and moving FP8 values takes half the memory bandwidth, because each value is half the size. The gain does not translate to exactly double the speed in practice, as Quantization shows, but for workloads that can run in FP8 it is a large improvement.

Hopper also adds fourth-generation Tensor cores inside more and faster Streaming Multiprocessors. Dynamic programming instructions, thread block clusters, and distributed shared memory give kernel engineers more tools for writing fast kernels (a kernel is a function that runs on the GPU, see CUDA, kernels, and kernel fusion). One kernel that uses them is FlashAttention 3, which improves the speed and memory efficiency of attention on Hopper by using the new asynchronous data transfer and execution features. Attention is covered in Attention and Mixture of Experts and Optimizing attention.

The H100 and H200 are among the most widely used inference accelerators. The architecture is new enough to be fast and established enough to have industry-wide support and heavily optimized kernels. Both GPUs are right-sized for common workloads across all modalities.

The two Hopper rows have identical compute and differ in memory: the H200 has 141 GB against 80 GB, and 4.8 TB/s against 3.35 TB/s. The next calculation computes, for every GPU in this lesson, the ops:byte ratio, which is FLOPS divided by bandwidth. It says how many arithmetic operations the chip can do for each byte it reads from memory. Finding the bottleneck: compute or memory explains why this ratio decides whether a workload is compute-bound or memory-bound. The Blackwell rows use the approximate figures from the tables in this lesson. The calculation also derives the FP16 rate of the Hopper GPUs from the fact that FP8 is twice as fast.

ops per byte = FLOPS / bandwidth

L4:   242 x 10^12 / (300 x 10^9)   = 807
L40:  362 x 10^12 / (864 x 10^9)   = 419
H100: 1,979 x 10^12 / (3,350 x 10^9) = 591
H200: 1,979 x 10^12 / (4,800 x 10^9) = 412
B200: 5,000 x 10^12 / (8,000 x 10^9) = 625
B300: 5,000 x 10^12 / (8,000 x 10^9) = 625

H100 FP16 dense = 1,979 / 2 = 989.5 teraFLOPS (FP8 is twice as fast as FP16)
H200 vs H100: memory 141 / 80 = 1.76x, bandwidth 4.8 / 3.35 = 1.43x

Read the ops:byte column. A value near 600 means the H100 can do about 600 operations in the time it takes to read one byte. A workload that does fewer operations per byte than that leaves the arithmetic units waiting, and one that does more leaves memory waiting. The L4 has the highest ratio of the group, because its memory is slow compared with its compute. The H200 has a lower ratio than the H100 for the same compute, which means it is better balanced for memory-bound work such as decode.

Example 4: Ada Lovelace, the graphics-oriented counterpart

The Ada Lovelace architecture is named for the first computer programmer. It launched about six months after Hopper. The two are similar, and Lovelace is more a counterpart than a successor. Lovelace also supports FP8 inference. Hopper focuses on AI applications, while Lovelace GPUs are more graphics oriented.

One limit is major: Lovelace GPUs do not support the NVLink interconnect, the very fast direct link between GPUs described in Instances, multi-GPU, MIG, and other accelerators. Nodes of eight Hopper or Blackwell GPUs use NVLink to share one model efficiently. Lovelace GPUs must be used one at a time or through less efficient methods such as pipeline parallelism, which splits the layers of a model across GPUs (Model parallelism covers the methods).

The L4 is a cheap and convenient way to run small models for modalities such as text embeddings and computer vision. The L40 is generally not a good choice for inference. For a similar memory size, a multi-instance slice of an H100 (described in Instances, multi-GPU, MIG, and other accelerators) offers much higher compute and memory bandwidth. A 3-slice H100 instance has about 3/7 of 1,979 teraFLOPS, around 848 teraFLOPS, against 362 teraFLOPS for a whole L40.

Example 5: Blackwell, the current gold standard

Blackwell is named for the mathematician David Blackwell. It launched in November 2024 with the B200, followed by the B300. A B100 exists but is not common for inference.

Where Hopper introduced FP8, Blackwell goes further with FP4, a 4-bit floating point format. It also adds a set of microscaling formats, MXFP8, MXFP4, and NVFP4, which keep more output quality during inference at low precision. Quantization explains these formats.

Blackwell extends Hopper's asynchronous programming model with more features for loading and storing data between the Tensor cores and global memory. The newer FlashAttention 4 kernel relies heavily on tiling its loads, computations, and writes into asynchronous pipelines.

The B200 and B300 are the new standard for inference. They give the highest performance for large language models and for demanding work such as video generation. Software support, optimized kernels, and general availability for Blackwell all arrived only in recent months, which marks an important transition in the industry. The B300 has the same compute as the B200 and 288 GB of memory against 192 GB. The next calculation shows the memory side of the FP8 gain by timing one read of a 70B model's weights at 16 bits and at 8 bits on an H100.

16-bit weights: 70 x 2 = 140 GB, one read = 140 / 3,350 GB/s = 0.0418 s = 41.8 ms
 8-bit weights: 70 x 1 =  70 GB, one read =  70 / 3,350 GB/s = 0.0209 s = 20.9 ms
FP8 values are half the size, so each read of the weights moves half the bytes.

One pass over 70 GB of weights at 3.35 TB/s takes about 21 ms, and the 140 GB at 16 bits takes about 42 ms. Moving FP8 values takes half the bandwidth because the weights are half as big. This is the memory side of Hopper's FP8 gain. The compute side is the Tensor cores, which run twice as fast on FP8 as on FP16.

Example 6: Rubin, CPX, and Feynman

The Rubin architecture is named for the astronomer Vera Rubin. NVIDIA plans to launch it in 2026 as the next generation of GPUs. New architectures should be judged only after real-world benchmarks, because a rollout, followed by industry-wide software support, takes about a year to saturate.

At the time of writing, some details about Rubin were known:

• Rubin uses HBM4, an upgrade over the HBM3 and HBM3e of recent GPUs. Inference tasks that are limited by memory bandwidth, such as LLM decode, benefit from the higher-throughput VRAM.
• Rubin introduces the CPX, a separate chip built for compute-bound tasks such as LLM prefill. The CPX is part of NVIDIA's rack-scale systems for high-volume inference.

The pairing mirrors the split between the two LLM phases. Prefill is compute-bound and decode is memory-bound, so one chip is tuned for each. Prefill-decode disaggregation shows how engines place the two phases on separate hardware.

After Rubin, NVIDIA plans to release Feynman in 2028. Few details are known. It is likely to support larger, higher-performance chips with a faster memory architecture.

Example 7: Grace and Vera CPUs

NVIDIA also makes its own ARM-based CPUs. They are integrated with its GPUs on superchips such as the GH200 and GB200. The "G" stands for "Grace", as in Grace Hopper, to match the Hopper GPUs. Grace CPUs have strong compute performance overall, and for inference the important feature is a much faster connection between the CPU and the GPU. Grace uses NVLink Chip to Chip (NVLink C2C), which gives up to 900 GB/s of bi-directional bandwidth between CPU memory and GPU memory. This is several times faster than PCIe, the standard connection between a CPU and a GPU.

This matters for inference because some setups offload (move out of GPU memory) data that is not needed right now, such as LoRA fine-tune weights (small add-on weights that adapt a model, see Choosing a model) and KV caches from earlier requests (KV caching). CPU memory is far larger than GPU memory, so it is a good place to keep them. The cost is the time to bring the data back to the GPU when it is needed again. With Grace, retrieval is much faster.

flowchart LR
  subgraph Standard["Standard server"]
    CPU1["CPU memory"] -- "PCIe (several times slower)" --> GPU1["GPU memory"]
  end
  subgraph Superchip["Grace superchip (GH200, GB200)"]
    CPU2["Grace CPU memory"] -- "NVLink C2C, up to 900 GB/s bi-directional" --> GPU2["GPU memory"]
  end

The diagram shows the two paths that offloaded data takes back to the GPU. The calculation compares them by moving 20 GB of cached data. The PCIe figure is an assumption for this illustration: PCIe 5.0 with 16 lanes moves about 64 GB/s in each direction, so about 128 GB/s in both directions together, which is the comparable number to 900 GB/s bi-directional.

data moved: 20 GB
NVLink C2C: 900 GB/s bi-directional = 450 GB/s one way, 20 / 450 = 0.044 s = 44 ms
PCIe 5.0 x16 (assumed): 64 GB/s one way, 20 / 64 = 0.3125 s = 313 ms
speedup: 450 / 64 = 7.0x

Retrieving the cache takes about 44 ms over C2C and about 313 ms over PCIe. A user waiting for the first token of a reply feels that difference. For the Rubin architecture, the Vera CPU (named for Vera Rubin) replaces the Grace CPU used with Hopper and Blackwell.

Example 8: choosing by generation

The next table gathers the choices of this lesson.

Older Turing and Ampere GPUs appear in legacy systems. For Rubin and Feynman, wait for real benchmarks.

Interview points

• A GPU name has a letter for the architecture and a number for the model. The same letter means the same generation (H100 and H200 are both Hopper), and a bigger number means a larger, more expensive GPU.
• Architectures: Volta (2017), Turing (2018), Ampere (2020), Hopper (2022), Lovelace (2022), Blackwell (2024), Rubin (2026), Feynman (2028). Most deployments use Lovelace, Hopper, or Blackwell.
• Hopper (H100: 1,979 FP8 teraFLOPS, 80 GB, 3.35 TB/s; H200: 141 GB, 4.8 TB/s) introduced FP8, fourth-generation Tensor cores, thread block clusters, and distributed shared memory, and runs FlashAttention 3.
• Lovelace (L4: 242 teraFLOPS, 24 GB, 300 GB/s; L40: 362 teraFLOPS, 48 GB, 864 GB/s) supports FP8, has no NVLink, and fits small models.
• Blackwell (B200 and B300: about 5 petaFLOPS FP8, 192 GB and 288 GB, up to 8 TB/s) adds FP4 and the MXFP8, MXFP4, and NVFP4 microscaling formats, and runs FlashAttention 4.
• Rubin uses HBM4 and adds the CPX chip for compute-bound prefill. Feynman follows in 2028.
• Grace (and Vera with Rubin) are NVIDIA ARM CPUs. NVLink C2C gives up to 900 GB/s bi-directional between CPU and GPU, several times faster than PCIe, which speeds up retrieval of offloaded LoRA weights and KV caches.
• Judge a new architecture by real benchmarks, because full software support takes about a year.

Summary

• GPU architectures are named after scientists. NVIDIA has done so since 1998 and ships a new architecture every one to two years.
• Inference engineers work with the three to five most recent generations.
• Hopper: FP8, fourth-generation Tensor cores, FlashAttention 3. The H100 and H200 are the widely used baseline.
• Ada Lovelace: graphics oriented, FP8, no NVLink. The L4 suits small models and the L40 is generally not recommended.
• Blackwell: FP4, microscaling formats, FlashAttention 4. The B200 and B300 are the current top choice. The B100 exists and is uncommon for inference.
• Rubin (2026): HBM4, the CPX for prefill, and the Vera CPU. Feynman (2028): larger chips and faster memory are likely.
• Grace CPUs pair with Hopper and Blackwell on the GH200 and GB200 superchips and connect over NVLink C2C.
• Ops:byte (FLOPS divided by bandwidth) summarizes how a GPU balances compute and memory.