In an attempt to compete with Nvidia’s dominant position in the generative AI market, AMD is developing a new GPU. This GPU, known as the AMD Instinct MI300X, is equipped with the capability to handle up to 192GB of HBM3 memory. It has been specifically designed to efficiently train large language models, which are crucial for various programs like ChatGPT.
In the realm of chips that can train generative AI programs, Nvidia has long been the dominant player. However, AMD is now making its move to secure a portion of the market by introducing a new enterprise-grade GPU.
Today, the company made an announcement about the AMD Instinct MI300X, which is a chip specifically developed to enhance the training of extensive language models. These models play a crucial role in driving programs like OpenAI’s ChatGPT.
According to Lisa Su, the CEO of AMD, AI is the crucial technology that is influencing the future of computing. In fact, it is considered as AMD’s biggest and most important opportunity for long-term growth in the market of generative AI.
In an attempt to outperform its competitors, the MI300X stands out in the generative AI market by offering an impressive 192GB of HMB3 memory. This advanced technology is built on AMD’s CDNA 3 architecture, specifically designed for AI workloads in data centers. With the capability to accommodate eight MI300X accelerators in one system, users can now train more extensive AI models compared to their competitors.
According to Su, the utilization of generative AI in the market can lead to a reduction in the number of required GPUs for operating large models. This reduction not only enhances the performance, particularly for inference, but also contributes to lowering the overall costs associated with ownership.
The MI300X is a product that belongs to the generative AI market and is developed by AMD. It shares similarities with another AI-focused chip, the MI300A, which is designed for supercomputers. However, the main distinction lies in the fact that AMD replaced the Zen 4 CPU chiplets in the MI300A with a dedicated GPU processor, transforming the MI300X into a solely GPU-based chip.
The newly designed product appears quite similar to MI300A, as the company removed three chiplets and replaced them with two GPU chiplets, while also increasing the amount of HBM3 memory. This product has been specifically developed for the generative AI market.
During a demonstration, Su showcased the capabilities of a single MI300X device that was enhanced with 192GB of memory. This device was able to run Falcon-40B, an open-source large language model. The demonstration involved requesting the program to compose a poem about San Francisco, and remarkably, it generated the text within a matter of seconds.
Our editors highly recommend this demo because it is the first time that a large language model of this scale can be executed solely in the memory of a single GPU.
Nvidia anticipates a significant increase in sales in the upcoming quarters due to the rising demand for generative AI chatbots. To meet this demand, companies in the industry have been purchasing Nvidia’s A100 GPU, which has a price tag of approximately $10,000. Furthermore, Nvidia is offering the H100 GPU, which now has the capability to be customized with a maximum of 188GB of HMB3 memory.
In the third quarter, AMD plans to distribute samples of its upcoming product, the MI300X, to important customers. AMD foresees significant growth in the generative AI market for data center chips, estimating it to reach $150 billion by 2027. This is a substantial increase from the current market value of $30 billion.
