Why AI Chips Made In The U.S. Are Being Sent To Taiwan — Creating A Major Bottleneck

Taiwan Semiconductor Manufacturing Company (TSMC) is addressing a critical bottleneck in AI chip production: advanced packaging. While TSMC manufactures silicon chips, even those made in the US, almost all are currently sent back to Taiwan for the packaging stage. This process involves protecting, testing, and bonding smaller chips together to form a modern high-performance chip. This step, historically an afterthought, is now recognized as crucial, akin to the chip die itself, and is predominantly performed in Asia. However, this is set to change as TSMC plans to build two new advanced packaging plants near its chip fabs in Arizona. Intel, another major player, also conducts advanced packaging in the US, in New Mexico and Arizona, in addition to its operations in Vietnam and Malaysia. These developments highlight the growing importance of advanced packaging, driven by the insatiable demand for AI chips and the evolving complexity of microchips. Manufacturing advanced microchips is an expensive and intricate process, with only three global companies—TSMC, Intel, and Samsung—operating leading-edge fabs and also leading in advanced packaging. Whether it's GPUs or CPUs from NVIDIA, AMD, or Intel, or custom ASICs from Amazon or Google, every chip requires assembly onto a system to interact with the external world. Paul Rousseau, head of packaging solutions in North America for TSMC, likens chip packaging to soda bottling, emphasizing its primary function of protecting and containing the product while enabling efficient distribution and connection to a larger system. As chips have become exponentially more complex, particularly for AI applications, standard packaging has evolved into advanced packaging. Previously, chips were manufactured on a single wafer. Now, multiple chips, potentially made with different technologies and in various locations, are integrated into one larger chip. This integration necessitates advanced packaging to create the necessary interconnects, a process that was not widely adopted until five or six years ago. This development is seen as a natural extension of Moore's Law into the third dimension. The advanced packaging process begins with "bumping" individual chips or dies, where micron-scale metallic bumps are added to the silicon surface to serve as electrical connection points. These bumped dies are then picked, placed, and attached to the substrate, the base layer of the package, which connects them to a larger system like a circuit board. The substrate, composed of multiple layers of resin-type material, routes signals and distributes power using copper. Companies like Intel are exploring glass substrates for improved control and finer features, critical for the growing demands of AI systems that require increased chip density and performance. The rapid advancements by AI leaders like NVIDIA are accelerating the evolution of advanced packaging needs, pushing for more GPUs and high-bandwidth memory (HBM) within a single package, leading to capacity shortages due to unexpected AI demand. While all packaging is essential, the most advanced types are currently in short supply. Simple chips, like CPUs, use 2D or flip-chip packaging, where they are mounted directly onto the base substrate. However, complex AI chips, such as GPUs, require 2.5D packaging. TSMC pioneered 2.5D packaging in 2012 with its Chip on Wafer on Substrate (CoWoS) technology, now widely adopted by customers including NVIDIA, Google, Amazon, and MediaTek. In 2.5D packaging, chips are placed side-by-side, but an "interposer" underneath provides an extra half-dimension for communication. This interposer, with its high-density wiring, acts as a bridge between multiple chips on a single substrate, enabling tight interconnections crucial for stacking HBM directly around the main chip. This innovation addresses the "memory wall" challenge by efficiently placing HBM memory right beside the compute chip. TSMC has developed three generations of CoWoS: CoWoS-S, CoWoS-R, and CoWoS-L. CoWoS-L, featuring two large silicon chunks surrounded by 12 HBM units, was first used in NVIDIA Blackwell GPUs. The capacity for CoWoS-L is particularly strained, with NVIDIA reportedly reserving the majority of it, raising concerns among NVIDIA's competitors. While TSMC aims to serve all customers, capacity remains tight. Intel offers an alternative 2.5D packaging solution called EMIB (Embedded Multi-Dye Interconnect Bridge), which embeds a silicon bridge within the substrate to facilitate communication between HBM and the logic chip, offering a cost advantage by connecting only the edges of the chips rather than requiring a full interposer. Samsung also has its 2.5D packaging, iCube. Looking beyond 2.5D, all major players are developing 3D packaging technologies. Samsung's is xCube, Intel's is Foveros Direct, and TSMC's is SOIC (System on Integrated Chips). True 3D technology involves stacking chips one on top of another, allowing them to behave as a single unit. This reduces the space between chips, leading to lower power consumption for communication, which is vital for data centers limited by power availability. TSMC anticipates products utilizing SOIC will emerge in a couple of years. Memory companies like Samsung, SK Hynix, and Micron already use 3D packaging to stack dies into HBM within their own advanced packaging fabs. They are also exploring hybrid bonding, replacing traditional bumps with flat copper pads to boost chip density and enhance power and electrical performance by shortening connection paths. The vast majority of advanced packaging currently occurs in Asia. This concentration poses national security concerns, particularly for the US, given geopolitical tensions in regions like Taiwan. Historically, packaging migrated to Asia in the 1970s due to lower labor costs, as it was a manually intensive process. Today, much of the process is robotic, and there's a push to reshore manufacturing steps. TSMC is moving to establish its first two US packaging facilities in Arizona to eventually provide high-volume capacity closer to its US chip fabs, which would significantly reduce turnaround times compared to shipping chips to Asia and back. Intel, while performing most of its final packaging assembly in China, Vietnam, and Malaysia, completes certain EMIB and Foveros operations entirely in the US, in New Mexico, Oregon, and Arizona. Intel is also expanding its advanced packaging services to external customers, including Amazon, Cisco, and potentially SpaceX, XAI, and Tesla through Elon Musk's initiatives. NVIDIA is also reportedly considering Intel for packaging as part of a $5 billion investment. This suggests that advanced packaging could serve as an "inroad" for Intel to attract major chip manufacturing customers. Third-party packagers, known as OSATs (Outsourced Semiconductor Assembly and Test), like Amkor (a TSMC and Intel partner building a site in Arizona), handle simpler and increasingly advanced steps in the packaging process. This is particularly important as TSMC grapples with unprecedented demand, increasing its CoWoS capacity at an 80% compound annual growth rate (CAGR) and SOIC capacity at 100% CAGR. ASE, the world's largest OSAT, projects advanced packaging sales to double by 2026 and is building a new CoWoS site in Taiwan. As TSMC brings advanced packaging plants to the US, this crucial, lesser-known step in chip manufacturing could help the US regain some of its long-lost leadership in silicon production, although a specific timeline for high-volume production in the US has not been provided.