The memory semiconductor market is undergoing a seismic shift. The craze for generative artificial intelligence (AI), including ChatGPT, is sparking an explosive increase in demand for memory optimized for AI processors, extending beyond traditional markets such as mobile, PC, and servers. The South Korean memory industry, historically focused on productivity competition with a strategy of “mass-producing a small variety of products,” is now facing a changed “rule of the game.” Collaboration with major tech companies in the United States, along with advancements in microfabrication technology, have emerged as key success factors in exploring new markets. ChosunBiz has compared and analyzed the competitiveness of Samsung Electronics and SK Hynix, the world’s leading memory semiconductor companies, forecasting the future market landscape. [Editor’s note]
“High bandwidth memory (HBM) stands as the sole technology capable of processing information closest to the CPU (central processing unit) and GPU (graphics processing unit), positioning itself as a ‘game changer’ in the semiconductor industry.”
In 2016, Lee Seok-hee, who was then vice president of SK Hynix and is now president of SK On, spent nearly 30 minutes explaining the value of HBM after a lecture at Hanyang University’s Seoul campus. Semiconductor giants like NVIDIA and AMD now incorporate HBM as their core memory, but at that time, the commercialization of HBM was practically nonexistent, contributing to less than 1% of SK Hynix’s total sales.
HBM is a high-performance memory semiconductor that notably enhances data processing speed by vertically stacking multiple DRAM (dynamic random-access memory) chips using Through-Silicon Vias (TSVs). It maximizes bandwidth, the pathway for input and output data, facilitating high-speed parallel operations. Unlike conventional DRAM, which follows a mass production model for a limited variety of products, HBM is a sort of “customized memory” created through collaboration with clients. Therefore, the competitive edge of HBM lies not just in efficiently manufacturing and stacking DRAM but in the custom optimization tailored to each processor. HBM has been developed and mass-produced from the first generation (HBM) through to the current fifth generation (HBM3E), including the second (HBM2), third (HBM2E), and fourth (HBM3) generations.
HBM, with its maximized data processing speed, has become essential in the memory chip industry, amid the boom in generative AI led by ChatGPT. While the overall HBM market is divided between SK Hynix and Samsung Electronics, SK Hynix is the leader, commanding over 90% market share (according to market researcher TrendForce) in the mainstream fourth-generation HBM (HBM3) segment. The average selling price of HBM is about five times higher than that of DDR4 DRAM. SK Hynix’s swift return to profitability in the global semiconductor memory sector in the fourth quarter of last year is largely attributed to HBM.
Until the early 2010s, when HBM first entered the market, the technological competitiveness of SK Hynix and Samsung Electronics was at almost the same level. Although SK Hynix developed the industry’s first HBM prototype in collaboration with AMD, it did not lead to mass production and remained an experimental product for a while. Samsung Electronics, not SK Hynix, was the first to mass-produce HBM2, the actual commercialized HBM product, succeeding in mass production at the end of 2015.
Prompted by Samsung’s success, SK Hynix ramped up its investment in HBM research and development (R&D) more aggressively. In 2017, the company decided to build a new R&D center in Icheon, Gyeonggi Province. Two years later, upon completion, significant funding was allocated to stabilize the TSV process, a crucial technology in HBM manufacturing. The early HBM development efforts initiated under the leadership of Lee Seok-hee have been successfully continued by former Vice Chairman Park Jung-ho and the current President Kwak Noh-jung at SK Hynix.
The third-generation HBM (HBM2E) was the deciding factor between the two companies in the HBM market. SK Hynix preemptively introduced Mass Reflow Molded Underfill (MR-MUF) technology which is the most technically challenging among TSV process techniques, in HBM2E. SK Hynix’s proprietary MR-MUF technology is a process in which protective materials are inserted between stacked chips and cured all at once. It is more efficient and stronger in heat dissipation than the process used by Samsung Electronics and Micron Technology (which layers a film-like material on each chip as it is stacked one by one).
Based on the stabilized process, SK Hynix gained an early advantage by being the first to mass-produce HBM3 in June 2022, when the HBM market began to grow. The company was the first in the industry to offer HBM3, which is optimized for the H100 GPU, Nvidia’s most advanced AI chip. Samsung Electronics, on the other hand, mass-produced HBM3 in July last year, but has yet to supply full-scale products to Nvidia, its largest customer.
It has been pointed out that Samsung Electronics, the ‘undisputed number one’ in the memory semiconductor market, lost the early lead to the compatriot company because it underestimated the competitiveness of the HBM market. Samsung, indeed, made the mistake of disbanding its dedicated HBM R&D team in 2019, before the AI semiconductor market took off under the executives’ skepticism towards investing in HBM. Coincidentally, around the same time, rival SK Hynix was ramping up its HBM-related technology capabilities at a new R&D center.
“In terms of the HBM strategy, Samsung Electronics’ serious blunder was the failure to establish a research and development center and integrate related research personnel promptly,” said a senior Samsung official. “When the AI memory market was showing signs of dawning in the 2020s, it was definitely the failure to establish a next-generation memory research and development organization and systematize development capabilities in a focused manner.”
Samsung’s internal analysis suggests that Samsung’s bureaucratized semiconductor division has hindered the company’s ability to respond quickly to changing market demands. In the case of the memory division, the executives and senior management, who are traditionally D-RAM specialists, are deeply involved in the company’s decision-making, making it difficult for them to accept unconventional ideas or preemptive investments.
“The reason why it is difficult for Samsung to agilely pivot according to rapidly changing market trends is that the experts in the D-RAM and NAND flash fields, which have been in place since the 1980s, have become bureaucratic leaders,” said an executive in Samsung’s semiconductor (DS) division. “The traditional memory semiconductor companies’ strategies focusing on manufacturing technologies, which devote themselves to mass production of small-scale products like D-RAM and NAND flash, are completely different from the AI memory market which is created through active collaboration with external companies.”
In fact, Samsung’s semiconductor division has been led by people who have worked in the D-RAM development lab. Many of the key leaders of Samsung’s semiconductor business, such as former Vice Chairman Kim Ki-nam (head of DS) and former Chairman Kwon Oh-hyun (head of DS), have come from the D-RAM development lab or design team.
SK Hynix, on the other hand, has experts from various backgrounds who joined the company during its integration into SK Group from LG Semicon, Hyundai Microelectronics, and Hynix, creating an open organizational culture.
This article was originally published on March 20, 2024.
