The Supply and Demand of AI Tokens | Dylan Patel Interview

The conversation highlights a dramatic shift in the AI landscape, moving from a time when execution was difficult and ideas were cheap, to a present where ideas are abundant and execution is significantly easier. This ease of implementation means only truly exceptional ideas can justify the cost of development. A key personal anecdote illustrates this shift: the speaker's firm, initially a heavy AI user with a spend of tens of thousands of dollars on subscriptions like ChatGPT, has seen its AI expenditure "skyrocket" to a $7 million annual run rate. This surge began in late December with the adoption of tools like Claude, particularly by non-technical personnel for coding tasks. This $7 million spend on AI now represents over 25% of the firm's $25 million salary expense, a trajectory that, if continued, could see AI expenditure surpass salaries by year-end. While this rapid growth allows the company to hire less aggressively and grow faster, the speaker anticipates other businesses will soon face the difficult decision of cutting staff as AI enables individuals to perform the work of many. Several compelling use cases demonstrate the transformative power of AI. In a reverse engineering lab, a single individual, with a few thousand dollars in AI token spend, developed a GPU-accelerated application capable of overlaying material information onto chip images, a task that previously required an entire team and a year to build. Similarly, an economist at a major bank, Malcolm, single-handedly processed vast amounts of economic data, ran regressions, and analyzed the impact of economic revolutions. He also created a metric to assess which of the Bureau of Labor Statistics' 2,000 tasks are AI-doable, identifying about 3% as currently feasible, and calculated the deflationary impact of AI adoption, termed "phantom GDP." This monumental undertaking would have taken a team of 200 economists a year. The speaker, as a business owner, views AI not as a cost to be controlled, but as a necessity for survival and growth in an increasingly commoditized information business. The core challenge is to constantly innovate and move up the value chain to avoid being outpaced by competitors who will inevitably adopt AI. The analogy is drawn to the energy sector, where a year-long effort to build an energy model was accelerated into a few weeks by AI, enabling the creation of a comprehensive US grid map and demand source analysis that rivals decade-long projects by 100-person teams. This rapid development allows the firm to commoditize existing energy data services and forces the question of who will commoditize them if they don't continue to move fast. The conversation then delves into the economics of tokens and compute. Anthropic's revenue has surged from $9 billion to an estimated $40-45 billion ARR, with gross margins reported to have grown from around 30% to over 72%, attributed to extremely high demand allowing for price increases and usage limits. The scarcity of tokens and the need for enterprise contracts to secure access and rate limit increases are highlighted. The speaker emphasizes that the value generated by tokens far exceeds their cost, and the key is to effectively leverage them. Companies that fail to create significant value with AI will be priced out. The rapid advancement of AI models, exemplified by Anthropic's Mythos, is both exciting and slightly daunting. Mythos, rumored to be significantly more capable than previous models, is being held back from broad release due to concerns about its impact, with a deliberately "worse" version being released to the public. The speaker advises securing enterprise contracts with per-token pricing to avoid rate limiting and focus on leveraging tokens for the highest value tasks, suggesting that in a few years, the business may be about arbitraging tokens themselves. The cost reduction in achieving specific capability tiers is staggering, with models like Deepseek being 1/600th the cost of GPT-4 class models, though demand is driven by frontier models creating economically valuable applications. The supply side of the AI infrastructure is also a critical bottleneck. Despite soaring demand, the supply chain for components like GPUs, memory, and specialized chips faces significant lead times. While useful life of GPUs is extending, prices are increasing, leading to expanding margins across the cloud and hardware layers. Memory prices are expected to double or triple due to insufficient capacity growth relative to demand. TSMC's massive capital expenditure plans highlight the long-term strain on semiconductor manufacturing, with downstream supply chains also facing constraints. CPUs are also in high demand, crucial for reinforcement learning environments and for deploying AI models that generate code and useful outputs. The hardest area to model is "tokconomics" – the actual usage and adoption of tokens and the diffusion of value into the broader economy. While infrastructure costs and model margins are relatively understood, quantifying the "phantom GDP" generated by AI remains a challenge. Looking ahead, the speaker predicts large-scale protests against AI within three months, fueled by public fear and a lack of understanding. They suggest the AI industry needs to improve its public image by showcasing uplifting applications, avoiding overly charismatic spokespeople, and focusing on present benefits rather than future existential threats. The conversation concludes by emphasizing that to escape the "permanent underclass," individuals must not only use more tokens but also generate and capture economic value from them.