In short: Intel has sparked a rivalry between its new Gaudi2 accelerator and the two-year-old Nvidia A100 market leader. In two tests suited to its niche, the flashy-named new accelerator comes out ahead.

- Advertisement -

Gaudi2 was created for Intel by the Israeli company Habana Labs, which acquired at the end of 2019 for $2 billion. In fact, Habana makes two types of specialized accelerators: some for training neural networks, such as Gaudi2; and others to launch them (i.e., “bring them out”), such as Goya and Greco.

Performance

- Advertisement -

Habana and Intel launched Gaudi2 in May but waited until last week to release their test results. MLPerf database. On their charts they compare their Gaudi2 system scores compared to Nvidia and Dell’s publicly available A100 system scores.

- Advertisement -

ResNet-50 tests the hardware’s ability to teach AI about image classification. It took just 18 minutes for the Habana Gaudi2 system to train the AI ​​well enough for it to pass the test, easily outperforming the Nvidia A100 system, which took almost half an hour.

The Habana Gaudi2 system took only 17 minutes to train the BERT model, which is about a minute faster than the Nvidia A100 system. BERT is a natural language processing model, and in this test, it is trained on articles from Wikipedia.

In both tests, all systems used eight accelerators/GPUs. The Habana system paired them with a pair of 40-core Intel Xeon 8380 processors, while Nvidia used two 64-core AMD Epyc 7742 processors.

Characteristics

Gaudi2 has 24 TPCs (Tensor Processing Cores) and two MMEs (Matrix Multiplication Engines) that run partially in parallel. It supports a wide range of data types including FP32, TF32, BF16, FP16 and FP8. It also has a dedicated media engine for processing audio and video as input.

In terms of memory, Gaudi2 has six 16GB HBM2e stacks for a total of 96GB and 2.45TB/s of total memory bandwidth. Inside, it has a 48MB cache. It uses a x16 PCIe 4.0 connection for connectivity and has 24 100Mbps RoCE2 (RDMA over Converged Ethernet 2) ports.

Competition

Habana has clearly created a real competitor to the A100 for Intel. The timing could have been better given that Nvidia announced H100 three months ago, but the two products are so different that even if they can compete in benchmarks, they may not actually compete for motherboard slots.

While the A100 and H100 are versatile behemoths, the Gaudi2 is an optimized accelerator trying to do something different and it will be interesting to see if it succeeds or not.