Behind the AI Boom: The Overlooked Hidden Costs of Supply Chains

AI competition is usually described as a race in algorithms, compute, and data, but Harvard Business School professor Willy Shih reminds us that the foundation of this boom is deeply embedded in global supply chains. In his forthcoming book Where Does My Stuff Come From? The Powerful Economics Behind Global Supply Chains, Shih systematically breaks down the complex networks behind modern goods—and AI hardware is precisely the most strained link in this network.

How Compute Demand Translates into Supply Chain Pressure

The voracious demand for GPUs, high-bandwidth memory, and advanced packaging capacity from large model training and inference is transmitting pressure up the value chain. Behind a single AI accelerator lies the coordination of multiple stages: wafer foundry, advanced packaging, substrates, power management, and thermal modules. Any bottleneck in any of these links is amplified into delays in finished system delivery.

In conversation, Shih emphasizes that supply chain costs are never just procurement prices. When a particular link becomes locked as a scarce resource, the premiums companies pay to secure capacity, the logistics costs incurred to shorten lead times, and the redundant inventory maintained to diversify risk all settle as hidden costs. These costs do not appear on chip price lists, yet they genuinely reshape the profit structure of the AI industry.

High-bandwidth memory (HBM) is a key component of AI accelerators. It vertically stacks multiple layers of DRAM chips using through-silicon via (TSV) technology to achieve extremely high bandwidth and low power consumption within a limited area. HBM is far more difficult to manufacture than conventional memory; currently only a few manufacturers worldwide can achieve stable mass production, and capacity expansion cycles are long with enormous capital investment. Consequently, when demand for large model training surges, HBM often becomes one of the first links to experience shortages. Its tight supply directly constrains GPU shipments and drives up overall system costs.

Advanced packaging refers to the process of integrating multiple chips (such as GPU cores and HBM stacks) into a single package via interposers or bridge technologies. Typical examples include TSMC's CoWoS (Chip on Wafer on Substrate). Advanced packaging not only affects chip compute density and energy efficiency but also determines the interconnect bandwidth between HBM and logic chips. Because advanced packaging lines have long construction cycles and expensive equipment, their capacity ramp-up often cannot keep pace with the growth in AI chip demand, making them another major bottleneck in the supply chain.

Underestimated Hidden Costs: Redundancy, Logistics, and Geopolitical Risk

There are three easily overlooked pieces of the AI hardware supply chain cost puzzle.

First, redundancy. When key components are available from only a few suppliers, downstream manufacturers often need to place orders with multiple vendors simultaneously to ensure supply—this duplicated investment is itself a cost.

Second, logistics. High-value, time-sensitive AI hardware is sensitive to transport conditions. A higher proportion of air freight and faster inventory turnover both drive up unit costs.

Third, geopolitical risk. Changes in export controls and trade policies mean supply chain layout is no longer purely a matter of efficiency but has become a compliance and backup engineering effort requiring continuous investment. Shih's analytical framework suggests that these costs are not short-term fluctuations but structural features of the AI infrastructure build-out period.

Export controls refer to a government's legal or administrative restrictions on the export of specific goods, technologies, or services to particular countries or entities. In AI hardware, the United States has in recent years implemented a series of export control measures on advanced-node chips, EDA tools, semiconductor equipment, and more, requiring upstream and downstream supply chain companies to comply. These controls not only directly limit the sales scope of relevant products but also force companies to adjust global capacity layouts and build backup supply chains, generating additional compliance costs and efficiency losses. Geopolitical risk thus transforms from an abstract political issue into a concrete supply chain cost item.

AI Economics Through a Supply Chain Lens

Viewing AI through the lens of supply chains yields a conclusion different from the "software eats the world" narrative: AI's marginal cost is not as low as imagined. Model inference continuously consumes compute, and behind compute are real materials, energy, and logistics.

Shih has long studied manufacturing and the global division of labor. His view is that understanding where a product comes from means understanding who prices it and who constrains it within the global network. For the AI industry, whoever controls capacity and alternatives in key links holds the initiative in the cost game. This is why leading manufacturers are extending upstream, investing in packaging capacity, and developing in-house chips—essentially paying for supply chain uncertainty.

Implications for AI Practitioners

For startups and application-layer teams, the practical significance of this perspective is that compute budgets cannot be estimated based on unit price alone; they must also factor in the time cost and risk premium required to obtain compute. For investors, valuation logic based on the AI narrative that ignores supply chain constraints may overestimate short-term supply elasticity.

Shih's conversation does not offer simple answers, but it provides a more solid way of asking questions: as we pursue leaps in AI capabilities, do we clearly see where the materials and capacity supporting this leap come from, and what price will be paid?