In this podcast episode, Nathan joins the host in person to discuss the macro-level dynamics of AI progress, the frontier labs, open-source models, the Chinese AI ecosystem, and Nathan's new lab, Trillium.
Nathan breaks down model progress into three forms of capital: compute, data, and talent. While closed frontier labs like OpenAI and Anthropic have secured massive amounts of capital and taken ambitious bets, Chinese labs face compute shortages but possess comparable talent. Nathan argues that distillation is not a major factor in current post-training based on scientific literature, viewing Chinese open-weight releases largely as a practical commercialization and distribution strategy. He notes that distillation simply serves competition and helps labs bridge a few months of lag.
When comparing open and closed models, Nathan explains that they operate on different timescales of exponentials. Closed labs are integrated, efficient at inference, and reach high-value markets quickly. The open-source and open-weight model economy takes much longer to build because it involves multiple fragmented players—builders, inference companies, and post-training startups—taking time to match private enterprise data and find ROIs. However, open-source models are essential for the startup ecosystem, providing a reliable Pareto frontier for fine-tuning.
Nathan shares insights on data and environments, highlighting that frontier labs invest heavily in buying high-quality data and RL environments, though this often introduces issues like reward hacking. To counter this, Nathan's new nonprofit, Trillium—named after a spring ephemeral flower that blooms before the canopy grows—aims to build open-source environments and data stacks for high-leverage tasks like terminal operations. By making these resources open, Trillium seeks to democratize research, demystify recursive self-improvement (RSI), and help society better understand automation and agentic systems.
Addressing RSI and the fear of closed labs pulling away, Nathan believes that while inference scaling and efficiency will continue to improve drastically, exponential increases in compute yield diminishing returns on performance, making a sudden "fast takeoff" less likely. However, he warns that while capability plots do not worry him, society is largely unprepared for cross-public cybersecurity and other diffuse risks that follow as open models catch up to closed ones.
Finally, Nathan touches on pre-training and the human aspect of the industry. He believes pre-training science remains accessible and that the hard work is shifting to post-training and orchestrating agents. Despite the secrecy of closed labs, knowledge naturally travels through the tech community, keeping open-source research competitive. Ultimately, Nathan advocates for open research, education, and infrastructure to ensure the broader tech ecosystem can participate and benefit from AI progress.