Nvidia Gooses Grace-Hopper GPU Memory, Gangs Them Up For LLM

Nvidia has announced the second generation GH200 superchip, which combines the Grace CPU and the Hopper GPU, offering increased memory capacity and bandwidth for AI training and inference workloads. The upgraded superchip uses HBM3e memory, enabling a 76.3% increase in memory capacity and a 49.3% increase in memory bandwidth compared to the original Hopper SXM5 device.

nextplatform.com

Relevant topic timeline:

8/29/2023

Wide Horizons: NVIDIA Keynote Points Way to Further AI Advances

Bill Dally, NVIDIA's chief scientist, discussed the dramatic gains in hardware performance that have fueled generative AI and outlined future speedup techniques that will drive machine learning to new heights. These advancements include efficient arithmetic approaches, tailored hardware for AI tasks, and designing hardware and software together to optimize energy consumption. Additionally, NVIDIA's BlueField DPUs and Spectrum networking switches provide flexible resource allocation for dynamic workloads and cybersecurity defense. The talk also covered the performance of the NVIDIA Grace CPU Superchip, which offers significant throughput gains and power savings compared to x86 servers.

8/31/2023

Intel Habana Gaudi Beats Nvidia's H100 in Visual-Language AI Models: Hugging Face

Intel's Gaudi 2 silicon has outperformed Nvidia's A100 80GB by 2.5x and H100 by 1.4x in a benchmark for the Vision-Language AI model BridgeTower, with the results attributed to a hardware-accelerated data-loading system.

9/9/2023

Nvidia Soars on Booming AI Demand; New Chips Target Data Centers

Nvidia's revenue has doubled and earnings have increased by 429% in the second quarter of fiscal 2024, driven by the high demand for its data center GPUs and the introduction of its GH200 Grace Hopper Superchip, which is more powerful than competing chips and could expand the company's market in the AI chip industry, positioning Nvidia for significant long-term growth.

9/11/2023

Nvidia Unveils New AI Chips With Big Leaps in GPT Performance

Nvidia has submitted its first benchmark results for its Grace Hopper CPU+GPU Superchip and L4 GPU accelerators to MLPerf, demonstrating superior performance compared to competitors.

9/15/2023

Micron Poised for AI-Driven Growth Spurt as Memory Demand Rebounds

Nvidia's strong demand for chips in the AI industry is driving its outstanding financial performance, and Micron Technology could benefit as a key player in the memory market catering to the growing demand for powerful memory chips in AI-driven applications.

9/17/2023

Nvidia's New Grace Chip Dominates MLPerf Benchmarks, But Rivals Are Gaining Ground

Large language models like Llama2 and ChatGPT perform well on datacenter-class computers, with the best being able to summarize more than 100 articles in a second, according to the latest MLPerf benchmark results. Nvidia continues to dominate in performance, though Intel's Habana Gaudi2 and Qualcomm's Cloud AI 100 chips also showed strong results in power consumption benchmarks. Nvidia's Grace Hopper superchip, combined with an H100 GPU, outperformed other systems in various categories, with its memory access and additional memory capacity contributing to its advantage. Nvidia also announced a software library, TensorRT-LLM, which doubles the H100's performance on GPT-J. Intel's Habana Gaudi2 accelerator is closing in on Nvidia's H100, while Intel's CPUs showed lower performance but could still deliver summaries at a decent speed. Only Qualcomm and Nvidia chips were measured for datacenter efficiency, with both performing well in this category.

Topics

Posted 8/22/2023, 5:54:32 PM