在一场关于人工智能未来的广泛讨论中，Databricks 的 Ali Goetze 和主持人 Martine 辩论了人工智能的快速发展步伐、其潜在风险和实际应用。对话以“公地悲剧”困境开场：企业迫于竞争压力，感到不得不快速创新，即使有些人认为为了安全有必要放慢速度。 Ali 强调领导者有责任不要“不必要地吓唬人们”关于生存风险，并指出目前的风险“接近于零”。他批评公众人物在社交媒体上发布可怕的警告，认为这会导致心理健康问题和公众恐慌，并举例说明 Elizabeth Warren 和 Bernie Sanders 呼吁暂停人工智能发展。Martine 虽然同意存在公众恐慌，但强调了政治的复杂性，指出不同的派别，包括那些有经济利益（如 IPOs）的派别，也试图影响叙事。 “放慢前沿发展速度”（pacing the frontier）这一说法成为了一个争议点。Martine 认为这是一个公关失误，与安全无关，并建议“保障前沿安全”（secure the frontier）会更真诚。然而，Ali 为这个概念辩护，认为实验室中实际的安全措施本身就会减缓发展，而且公司可能会寻求外部监管作为一种“护栏”，以避免被那些不优先考虑安全的竞争对手单方面置于不利地位。他引用了 Hugging Face OpenAI 事件作为证据，表明实验室“本应该放慢”他们的实验。 Ali 区分了他认为的低生存风险和真实的、迫在眉睫的威胁，主要是网络风险。他承认当今互联基础设施的规模前所未有，这使得它容易受到 AI 代理发现漏洞的攻击。然而，他也指出，与早期互联网蠕虫爆发相比，目前缺乏由 AI 驱动的重大网络攻击，这让 Martine 质疑其中的脱节。Ali 认为这是一个“工程问题”，可以通过快速自动化安全来解决，因为人类的安全运营中心（SOCs）无法跟上新漏洞出现的速度。 关于“超级智能”（superintelligence）和递归式自我改进（Recursive Self-Improvement, RSI），Ali 列出了四个标志着这种风险真正转变的标准：模型需要更少的资源、更短的训练时间、智能水平不断提高，以及能够无限期地重复这个循环。他目前没有看到任何证据表明这些标准正在被满足，并指出前沿模型仍然需要巨大且不断增加的资源和人力投入，这使得它们脆弱且昂贵。他强调了一个“非常好的观点”，即前沿模型所需的算力“只会不断增加”。 Ali 提议为人工智能实验室设立第三方“检查员”，并提及 Jan Bakun 等人物，他们的独立评估将具有分量。他驳斥了实验室自我监管或相互审查的 F 想法，因为存在固有的竞争偏见。他还谈到了 Elon Musk 的“四维国际象棋”（4D chess）言论，承认一些人工智能警告可能是为了吸引注意力或影响对公司有利的监管而采取的玩世不恭的营销策略。 在益处方面，Ali 强调了展示人工智能积极用例的重要性。他引用了危机求助热线拯救生命、用于糖尿病管理的 Omnipod、Zipline 向难民运送血液以及 Merck 的 TEDDY 模型加速药物发现等例子。他认为这些主要基于当前人工智能能力的应用不应该被“放慢”。 为了让企业获得价值，Ali 强调了“本体”（ontology）的关键作用——创建一个组织隐性知识、关系和流程的数字图谱。他解释说，这种上下文理解是当前人工智能所缺乏的，阻碍了它们自动化复杂任务（超出简单的聊天机器人）的能力。Databricks 内部构建了这样一个本体，从而显著提高了生产力，并改变了员工获取信息的方式。 最后，Ali 谈到了人工智能的成本管理。Databricks 开发了 Unity Gateway 和 Omnigent，以帮助管理令牌使用并优化模型选择，使他们能够在人工智能采用增加的情况下降低成本曲线。他指出，创业公司中存在一种趋势，即从昂贵的前沿模型转向更具成本效益的开源或针对特定任务的专业模型，他认为企业一旦掌握了“评估方法”（evals），最终也会效仿这种做法。 Ali 总结说，他个人对“毁灭概率”（Pdoom）的估计“接近于零”，而 Martine 则提供了一个独特的视角：他“没有人工智能的毁灭概率远高于拥有人工智能的毁灭概率”。

In a wide-ranging discussion on AI's future, Ali Goetze of Databricks and host Martine debated the rapid pace of AI development, its potential risks, and practical applications. The conversation kicked off with the "tragedy of the commons" dilemma: businesses feel compelled to innovate rapidly due to competition, even if some believe slowing down is necessary for safety. Ali emphasized the responsibility of leaders not to "freak people out unnecessarily" about existential risks, stating that the current risk is "close to zero." He criticized public figures for broadcasting dire warnings on social media, arguing it causes mental health issues and public hysteria, pointing to examples like Elizabeth Warren and Bernie Sanders calling for pauses in AI development. Martine, while agreeing on public hysteria, highlighted the political complexities, noting that different factions, including those with financial interests (like IPOs), are also trying to influence the narrative. The term "pacing" the frontier became a point of contention. Martine views it as a PR misstep, orthogonal to safety, suggesting "secure the frontier" would have been more genuine. Ali, however, defended the concept, arguing that practical security measures in labs inherently slow down development, and companies might seek external regulation as a "guardrail" to avoid being unilaterally disadvantaged by competitors who don't prioritize safety. He cited incidents like the Hugging Face OpenAI case as evidence that labs "should have paced" their experiments. Ali distinguished between what he considers low existential risk and real, immediate threats, primarily cyber risks. He acknowledged the unprecedented scale of interconnected infrastructure today, making it vulnerable to AI agents finding exploits. However, he also noted the lack of significant AI-driven cyber attacks compared to the early internet worm outbreaks, leading Martine to question the disconnect. Ali believes this is an "engineering problem" that can be solved with rapid automation of security, as human Security Operations Centers (SOCs) cannot keep up with the speed of new vulnerabilities. Regarding "superintelligence" and Recursive Self-Improvement (RSI), Ali laid out four criteria that would signify a genuine shift towards such a risk: models requiring less resources, less time to train, increasing intelligence, and the ability to repeat this cycle indefinitely. He currently sees no evidence of these criteria being met, noting that frontier models still require enormous, increasing resources and human effort, making them brittle and expensive. He highlighted the "really good point" that the compute needed for a frontier model "just keeps going up." Ali proposed third-party "inspectors" for AI labs, referencing figures like Jan Bakun, whose independent assessment would carry weight. He dismissed the idea of labs self-regulating or cross-checking each other due to inherent competitive biases. He also addressed Elon Musk's "4D chess" comment, acknowledging that some AI warnings might be cynical marketing ploys to garner attention or influence regulation to a company's advantage. On the benefit side, Ali stressed the importance of showcasing AI's positive use cases. He cited examples like crisis text lines saving lives, Omnipod for diabetes management, Zipline delivering blood to refugees, and Merck's TEDDY model accelerating drug discovery. He argued these applications, largely built on current AI capabilities, should not be "paced." For enterprises to gain value, Ali emphasized the critical role of "ontology" – creating a digital graph of an organization's tacit knowledge, relationships, and processes. He explained that this contextual understanding is what current AIs lack, hindering their ability to automate complex tasks beyond simple chatbots. Databricks internally built such an ontology, leading to significant productivity gains and a shift in how employees access information. Finally, Ali touched on cost management in AI. Databricks developed Unity Gateway and Omnigent to help manage token usage and optimize model selection, allowing them to bend the cost curve despite increased AI adoption. He noted a trend among startups to move from expensive frontier models to more cost-effective open-source or specialized models for specific tasks, a practice he believes enterprises will eventually follow once they master "evals" (evaluation methods). Ali concluded by stating his personal "Pdoom" (probability of doom) is "close to zero," while Martine offered a unique perspective: his "Pdoom without AI is much higher than my Pdoom with AI."