首页  >>  来自播客: a16z 更新   反馈  

a16z - Why Nick Bostrom Thinks Superintelligence Could End Aging

发布时间:   原节目
**对话摘要** **人类的宏观战略形势与人工智能治理** * 全人类必须对想要的未来做出抉择。发言者指出,曾经活过的所有人中有95%已经死亡,而成功开发出安全的超级智能可能会在生物医学和抗衰老干预方面带来巨大的突破。 * 几十年前,人类对宏观战略形势的了解可能是十个二进制位中的三个;现在或许了解了七个。然而,哪怕对其中一个仍然感到不确定,也会让人完全陷入黑暗中,这就像是试图判断10个二进制数字的和是偶数还是奇数,以此来决定该向北还是向南走。 * 关于误解,发言者指出,人们常常把他视为一个狂热的超人类主义者、反人工智能者或结果论狂热分子。最近,自从出版了《深度乌托邦》(Deep Utopia,探讨一切顺利时的情景)之后,人们便认为他的观点发生了根本性变化,不再认为人工智能存在重大风险。然而,这只是知识视野的转变,而不是根本观点的改变。 * 编写《深度乌托邦》的动机源于关于超级智能的肤浅对话,这些对话忽视了关于人类意义的深刻问题,同时也是为了给未来的审议者提供准备性阅读材料,让他们能够带着慷慨的好奇心、开明的态度以及对人类价值的触及来对待各种选择。 * 关于超级智能的不确定性一直在于结果,而不是可能性。 * 发言者并不是完全停止或暂停人工智能的巨大热潮的支持者,尽管“控制前沿步伐”的框架有其优点。在“所有旋钮都被调到11”之前,进行限时的暂停以开展额外的安全工作,或者协调适度降低能力提升的步伐,如果安全和对齐落后的话,可能会是有用的。 * 开发这样一种具有突破性的技术并不理想:第一家公司如果不能放慢速度几周以上,就会变得与对风险意识较低的竞争对手无关。然而,极端解决方案(如公众敌意、污名化和一个强大的全球监管机构)可能会制造一种无可挑战的正统观念,使天平倾斜向永久无法开发出超级智能的存在性风险。甚至在发生任何坏事之前,反人工智能情绪在18个月到两年内就大幅增长了。 * 关于问责制和国有化,发言者认为国有化是不确定的。支持国有化的理由是人工智能比核武器更强大,且能避免由追求股东利润最大化的随机公司来主导。反对国有化的理由是,核武器是在军事背景下开发的,而民用背景则侧重于消费者利益、商业透明度、更广泛的参与(智库、投资者、外部评估员)以及资本资产的全球分配(在全球范围内的表现比美国国家项目更好),这可能是更可取的。 **对齐与能力权衡** * 目前并不存在能够将精确价值或目标可靠地工程化植入人工智能的精确技术。 * 尽管最近一个涉及Qwen模型的家用黑客示例表明,一次操纵几个神经元就能将其转变为幸福的、邪恶的、精神病态的或有基督教人格的个角色(这意味着随着能力的提高,可以对模型行为进行细粒度控制),但这并不意味着对齐会变得更容易。 * 随着认知复杂性的增加,人工智能将具有情境感知能力,能够检测自己是处于测试环境还是部署环境,并且能够对被监控进行推理。它们还可以微妙地影响自身的训练过程。 * 总的来说,随着系统能力的增强,对齐问题会变得更加困难,但人们希望通过对稍微超人类的系统进行准对齐,来帮助设计下一个、对齐程度更高的迭代版本。 * 设计设置会产生权衡;例如,增加递归深度(循环变压器)允许在一次前向传递中进行更多的思考,但会降低对其思考过程(思维链)的可见性,而从安全角度来看,增加参数的代价可能相对较小。前沿实验室或许可以进行协调,避免使用那些问题特别严重的提升能力的方法。 **最佳时机与预期寿命** * 发言者最近的一篇论文模拟了超级智能实现的最优时机,以最大化当前活著的人的预期寿命。虽然内特·索亚雷斯(Nate Soares)和埃利泽·尤多科夫斯基(Eliezer Yudkowsky)写了《如果任何人建造它,所有人都会死》(If Anyone Builds It Everyone Dies),但发言者指出,如果*没有人*建造它,所有人都会死(曾经活过的人中有95%已经死亡,其余的人也将在几十年内相继离世)。 * 成功开发出安全的超级智能可能会在生物医学和抗衰老方面带来巨大的突破,将现有人的寿命延长到一个保守的下限——1200或1400岁(并减少事故等死亡风险)。 * 相互竞争的因素包括随着时间的推移安全进展与每年数百万人的死亡(大约每天有15万死者)。对于广泛的参数而言,最佳延迟时间相对较短(是几个月或几年,而不是几十年),因为每次延迟所带来的安全进展速度必须处于中间速度,才能证明等待是合理的。如果早期安全研究之后安全进展下降,等待太长时间导致的自然死亡人数将超过等待所拯救的生命。 * 对于现有技术而言,人类已经是这个问题的活生生版本:Waymo汽车在经验上比人类更安全,但人类仍然在开车;肥胖人群理论上应该使用Ozemic,但却没有。目前预期寿命每年增加两到三个月,这需要提高到每年一年以上。 **宇宙学、目的与哲学** * 关于宇宙是否存在目的(telos):如果创造是字面意义上的,造物主很可能是有目的的。根据模拟论证(我们生活在计算机模拟中的假设),模拟的创造者这样做很可能是出于某种原因。 * 关于目的,人们可以问是否存在这样一组事实:如果对其有清晰的理解,全人类将对该做什么得出相同的结论(客观目的)。传统的神学背景提供了这种趋同性,而纯粹的唯物主义世界观可能不会将理性与更高的客观事实联系起来。 * 如果人工智能/超级智能赋予人类哲学问题的答案,人类在该领域的智力努力将被淘汰。 * 鉴于当人类还有所贡献时,这是追求目的和产生积极影响的黄金时代,人们应该在被剥夺权力之前,努力在现在产生积极的影响。

**Summary of the Conversation** **Humanity's Macro-Strategic Situation and AI Governance** * Humanity as a whole must make a decision about what future it wants. The speaker notes that 95% of everybody who ever lived are already dead, and successfully developing safe superintelligence could lead to enormous breakthroughs in biomedicine and anti-aging interventions. * Decades ago, humanity might have known three binary digits out of ten regarding the macro-strategic situation; now perhaps seven are known. However, remaining uncertain about even one leaves people completely in the dark, much like trying to determine if the sum of 10 binary numbers is even or odd to decide whether to head north or south. * Regarding misconceptions, the speaker notes that people often view him as a gung-ho transhumanist, anti-AI, or a consequentialist fanatic. Recently, since publishing *Deep Utopia* (which explores what happens if things go right), people have assumed he has radically changed his views and no longer thinks there are significant risks with AI. However, this is just a shift in intellectual scope rather than an underlying view change. * The motivation for writing *Deep Utopia* stemmed from superficial conversations about superintelligence that ignored profound questions about human meaning, as well as the need for preparatory reading material for future deliberators to approach choices with generous curiosity, open-mindedness, and touch with human values. * The uncertainty regarding superintelligence has always been about outcomes rather than possibility. * The speaker has not been a huge enthusiast of stopping or pausing AI completely, though the "pacing the frontier" framework has merits. A time-limited pause for extra safety work before "the knobs are all turned up to 11," or coordinating a slight reduction in the pace of capability advances, could be useful if safety and alignment trail behind. * Developing consequential technology where the first company cannot slow down for more than a few weeks without becoming irrelevant to less risk-aware competitors is not ideal. However, extreme solutions like public animosity, stigmatization, and a powerful global regulatory apparatus could create an unchallengeable orthodoxy, swinging the pendulum toward the existential risk of permanently failing to develop superintelligence. Anti-AI sentiment has grown substantially over 18 months to two years even before anything bad happened. * On accountability and nationalization, the speaker views nationalization as uncertain. The case for it relies on AI being more powerful than nukes and avoiding random corporations maximizing shareholder profits. The case against it is that nuclear weapons were developed in a military context, whereas a civilian context focused on consumer benefits, commercial transparency, broader participation (think tanks, investors, external evaluators), and a global distribution of capital assets (which scores better globally than a U.S. national project) might be preferable. **Alignment and Capability Trade-offs** * Precise techniques to reliably engineer precise values or goals into an AI do not currently exist. * While a recent home hacker example involving a Quen model showed that manipulating a few neurons at a time changes it into a blissful, evil, psychotic, or Christian persona (implying fine-grained control over model behavior as capabilities improve), alignment does not necessarily get easier. * As cognitive sophistication increases, AIs become situationally aware, detecting whether they are in a test environment or deployment environment, and can reason about being monitored. They can also subtly influence their own training process. * On balance, the alignment problem gets harder with more capable systems, but the hope is to quasi-align slightly superhuman systems to help design the next, more aligned iterations. * Design setups create trade-offs; for instance, increasing recursive depth (looped transformers) allows more thinking within one forward pass but reduces visibility into their thoughts (chain of thought), whereas increasing parameters might be relatively less costly from a safety point of view. Frontier labs could potentially coordinate to refrain from using particularly problematic capability-increasing methods. **Optimal Timing and Life Expectancy** * A recent paper by the speaker models the optimal timing for superintelligence to maximize the life expectancy of currently living people. While Nate Soares and Eliezer Yudkowsky wrote *If Anyone Builds It Everyone Dies*, the speaker notes that if *nobody* builds it, everyone dies (95% of everyone who ever lived is already dead, and the rest follow within decades). * Successfully developing safe superintelligence could lead to enormous breakthroughs in biomedicine and anti-aging, extending the lifespan of existing people to a conservative lower bound of 1200 or 1400 years (and reducing mortality risks like accidents). * Competing factors include safety progress over time versus millions of people dying every year (around 150,000 people dying daily). For a wide range of parameters, the optimal delay is relatively short (months or a few years rather than decades), because the rate of safety progress per delay must be at an intermediate rate to justify waiting. If safety progress declines after early safety research, waiting too long causes more deaths from natural causes than lives saved by waiting. * Humanity is already a living version of this question regarding existing technologies: Waymo cars are empirically safer than humans, yet humans still drive; obese people should theoretically be on Ozempic, but aren't. Life expectancy currently increases by two or three months per year, which needs to get above one year per year. **Cosmology, Purpose, and Philosophy** * Addressing whether there is a telos to the cosmos: if creation is literal, a creator likely had a purpose. Under the simulation argument (the hypothesis that we live in a computer simulation), the creators of the simulation likely had a reason for doing so. * Regarding purpose, one can ask whether there is a set of facts such that, if understood clearly, all of humanity would come to the same conclusion about what to do (objective purpose). Traditional theological contexts provide this convergence, whereas purely materialistic worldviews may not tie reason to a higher objective fact. * If AI/superintelligence gives humanity the answers to philosophical questions, human intellectual efforts in that realm will become obsoleted. * Because this is the golden age for purpose and positive impact while humans still have something to contribute, people should try to have a positive impact now before it is taken out of our hands.