Why AI’s Next Breakthroughs Could Come from Outside the Big Labs

该播客以对人工智能早期监管的批判性讨论开场，特别辩论了“放缓”人工智能发展的概念。一位发言者称“放缓”是“不真诚的”和“一派胡言”，认为在没有已知发展速度的情况下，根本不可能放慢其发展。这种“放缓”被视为对“暂停派”和监管者双方的妥协，最终却两边不讨好。他们打了个比方，指出缓慢建造核武器并不会使其更安全。 一个核心争议点是“X风险”或物种灭绝叙事，特别是一名员工声称的“物种灭绝有10%的可能性”，Dario似乎对此表示同意。主持人质疑为何人工智能实验室尚未就X风险采取明确立场。他们认为，如果风险确实是生存性的，那么这项技术就应该国有化。如果这是关于招聘或留住人才的担忧，那它就是“人事问题”，而不是对一项有前景的技术进行国家层面封锁的理由。他们强调了绝对风险和边际风险之间的区别，认为关注后者会更具成效。 对话强调了“实验室与安全社区之间的一道鸿沟”。安全社区常常发现实验室对事故的事后分析“草率”且“不完整”，缺乏CVE（计算机病毒与漏洞报告）等流程中结构化的报告方式。这与1986年《计算机犯罪和欺诈法》等历史案例形成对比，该法案源于具体且有充分记录的违规事件。 提出的一个关键担忧是人工智能带来的新型网络安全风险，特别是“智能体群”。它们被描述为“数量达10,000倍的漫游无人机”，能够轻易将好的任务误认为坏任务，并不知疲倦地尝试所有可能的漏洞利用。这要求对内部安全层、身份验证和API追踪进行根本性的重新评估。发言者指出，当前的操作系统和安全模型对于这种新现实来说不够精细或性能不足，他们建议在操作系统、网络和计算机语言方面进行一场“复兴”，以构建“设计即安全”的系统。他们引用了历史案例，例如早期个人电脑中病毒的普遍存在以及安全措施的演变（例如iPhone的强制更新），以说明安全如何随技术发展而演进。 从历史上看，监管通常是在重大创新和事件之后才出现的，而非在此之前。例子包括汽车、航空和制药行业，这些行业的健全监管（如FAA或FDA）都是在最初发展几十年后才出现的。例如，如果在1910年就成立FAA，那将会扼杀航空业的发展。互联网在其早期阶段也基本处于不受监管的状态。发言者担心“过早”监管人工智能可能会阻碍创新，同时又无法有效解决潜在风险。 播客还讨论了政治格局，预测2028年的选举可能成为“人工智能选举”。有人担心，美国在过去“停止了在科技反垄断方面的领导地位”后，可能会将监管领导权让给欧洲。人们担心会出现“AI版GDPR”的情景，导致过多的提示、合规疲劳和创新受阻，这由欧洲“没什么可失去的”态度以及对责任归属的热衷所驱动。当前的公众辩论，使用“暂停”、“群集”和“失控”等词语，被认为是受反人工智能者影响，使得支持人工智能的叙事难以获得关注。 最后，播客介绍了“Jeff”的创新，这一点被强调为一个积极的发展。与传统的“文本输入，文本输出”的大型语言模型不同，Jeff的方法实现了“文本输入，选项输出”，使得将语言模型集成到传统软件中更快、更便宜、更准确。这涉及到回归到“概率编程”——一个早期计算机科学的概念，其中“if语句”是基于概率而非绝对条件。这一创新表明“创新中心已经转移”到核心人工智能实验室之外，预示着未来将由应用层面的进步推动该领域。

The podcast opens with a critical discussion on the early regulation of AI, specifically debating the concept of "pacing" AI development. One speaker labels "pacing" as "disingenuous" and "utter nonsense," arguing that without a known rate of development, it's impossible to slow it down. This "pacing" is seen as a capitulation to both "pause folks" and regulators, ultimately satisfying neither. They draw a parallel, stating that slowly building a nuclear weapon doesn't make it safer. A central point of contention is the "X-risk" or species extinction narrative, particularly an employee's claim of a "10% chance of species extinction," which Dario seems to agree with. The hosts question why AI labs haven't taken a clear position on X-risk. If the risk is truly existential, they argue, the technology should be nationalized. If it's a concern about recruiting or retaining talent, it's an "HR problem," not a justification for national-level lockdowns on a promising technology. They emphasize the difference between absolute risk and marginal risk, suggesting a focus on the latter would be more productive. The conversation highlights a significant "rift between the labs and the security community." The security community often finds lab postmortems on incidents "sloppy" and "incomplete," lacking the structured reporting seen in processes like CVE (Computer Virus and Vulnerability reporting). This contrasts with historical examples like the 1986 Computer Crime and Fraud Act, which arose from specific, well-documented breaches. A key concern raised is the novel cybersecurity risks posed by AI, particularly "agent swarms." These are described as "roaming drones but times 10,000," capable of easily mistaking good tasks for bad ones and trying all possible exploits without fatigue. This necessitates a fundamental re-evaluation of internal security layers, authentication, and API tracking. The speakers note that current operating systems and security models aren't granular or performant enough for this new reality, suggesting a "renaissance" in operating systems, networks, and computer languages to build "secure by design" systems. Historical examples like the prevalence of viruses in early PCs and the evolution of security measures (e.g., iPhone's mandatory updates) are cited to illustrate how security evolves with technology. Historically, regulation has typically followed significant innovation and incidents, not preceded it. Examples include the automotive, airline, and pharmaceutical industries, where robust regulation (like the FAA or FDA) came decades after initial development. Starting the FAA in 1910, for instance, would have stifled aviation. The internet also developed largely unregulated in its early stages. The speakers worry that regulating AI "too early" might hinder innovation without effectively solving underlying risks. The political landscape is also discussed, with the prediction that the 2028 election could become the "AI election." There's concern that the US, having previously "stopped leading in tech antitrust," might cede regulatory leadership to Europe. The fear is a "GDPR for AI" scenario, leading to excessive prompts, compliance fatigue, and stifled innovation, driven by Europe's "nothing to lose" approach and love for liability assignment. The current public debate, using terms like "pause," "swarms," and "rogue," is seen as being shaped by those who are anti-AI, making it hard for pro-AI narratives to gain traction. Finally, the podcast introduces "Jeff's" innovation, which is highlighted as a positive development. Unlike traditional LLMs that do "text in, text out," Jeff's approach allows for "text in, option out," making it faster, cheaper, and more accurate for integrating language models into traditional software. This involves a return to "probabilistic programming," a concept from early computer science, where "if statements" are based on probabilities rather than absolute conditions. This innovation signifies that "the center of innovation has just moved" outside the core AI labs, suggesting a future where application-level advancements drive the field.