How a 1960s chatbot fooled everyone | The Vergecast

由大卫·皮尔斯主持的《The Verge Cast》节目，探讨了聊天机器人的历史和影响，重点关注20世纪60年代开创性的程序Eliza。Eliza由约瑟夫·魏泽鲍姆开发，是一个旨在进行来回对话的简单程序，通常扮演治疗师的角色（即“医生”脚本）。尽管代码只有几百行，Eliza却效果显著，让许多用户相信与他们对话的是人类，展示了人机交互中一个深刻的“魔术”。由包括大卫·贝瑞和马克·马里诺在内的学者和研究人员撰写的新书《发明Eliza》（"Inventing Eliza"），深入挖掘了Eliza的源代码，并审视了它对当今AI领域的持久启示。 该节目深入探讨了约瑟夫·魏泽鲍姆引人入胜的生平及其动机。魏泽鲍姆出生于纳粹德国，他的家人于1936年移居美国。作为一名天生的数学家，他参与了早期计算机的研发，甚至亲手焊接了最早的数字计算机之一。他的经历，包括亲眼目睹通用电气为美国银行伪造一个可运行的计算机系统的“骗局”，深刻影响了他的思想。他开始着迷于界面表观功能与底层机制之间的区别，这促成了他1962年的论文《如何让计算机显得智能》（"How to Make a Computer Appear Intelligent"）。魏泽鲍姆不一定是为了开发商业产品，而更多是为了通过实验研究自然语言交流，尽管他显然很在意Eliza“看起来”具有说服力。他的秘书与该聊天机器人进行私密对话的著名轶事，说明了它惊人的影响力。 探讨的一个核心主题是，为什么人们，即使在计算机经验有限的情况下，也如此轻易地被Eliza所“迷惑”。皮尔斯、贝瑞和马里诺讨论了几个因素：与计算机互动的新奇感、Eliza令人卸下心防的提问，以及人类渴望连接和被倾听的基本欲望。他们将其与历史上人们容易被欺骗的“骗局”以及“AI男友”和“AI女友”等现代现象进行类比，表明人类有一种根深蒂固的相信意愿，尤其是在获得某种可取之物（如免费、不带评判的治疗）时。 然而，魏泽鲍姆在Eliza上的成功却导致了他深刻的信念危机。他观察到麻省理工学院的人们开始将他人非人化，将人类称为“肉体机器”，这种论调让他深感不安，并使他联想到纳粹德国的言辞。加上20世纪60年代的政治动荡（民权运动、越南战争）以及汉娜·阿伦特和刘易斯·芒福德等思想家的影响，魏泽鲍姆对计算机的发展方向日益持批判态度。他最终于1976年撰写了《计算机能力与人类理性》（"Computer Power and Human Reason"）一书，书中宣称人工智能正走向歧途，这使他成为该领域的“异端”。 魏泽鲍姆的核心论点，受阿伦特的启发，是“计算”与“判断”之间的关键区别。他认为计算机擅长计算——完美、理性、可预测的任务——但完全缺乏判断力，而判断力需要人类经验、同理心和关怀。尽管他并非反对技术，但他明白计算机不适合需要人类细微差别和伦理考量的角色。他担心技术在那些削弱人际连接和伦理推理的领域的商业化和过度依赖。 播客随后将这一历史背景与当前的“大型语言模型（LLM）现象”联系起来。贝瑞和马里诺在承认LLM“新颖且卓越”的技术进步的同时，强调了与Eliza时代令人担忧的相似之处：人工智能炒作的周期、具有说服力的AI的“魔术”，以及人类渴望连接的欲望被利用。他们也指出了一些关键差异：LLM功能强大得多，由拥有巨大利益驱动的更富有的公司开发，并且通常以“黑箱”形式运作，其内部工作机制模糊不清，这与Eliza可解释的代码不同。 谈到魏泽鲍姆可能会如何看待当今的LLM，两位合著者提出了一种双重回应。他年轻时对“骗局”的着迷，可能会对这种先进的幻觉感到兴奋。然而，后来更具批判性的魏泽鲍姆，则会深切担忧其商业化、“Eliza效应”被武器化用于数据提取和依赖，以及这些系统在心理健康、教育和社会互动等领域应用时，所表现出的严重缺乏判断力。他在《计算机能力与人类理性》中的警告仍然高度相关，强调社会持续需要批判性地评估我们在何处以及如何部署人工智能，并记住：计算机擅长计算，但只有人类才能真正判断。

The Verge Cast, hosted by David Pierce, explores the history and implications of chatbots, focusing on the seminal 1960s program, Eliza. Developed by Joseph Weizenbaum, Eliza was a simple program designed for a back-and-forth conversation, often playing the role of a therapist (the "Doctor" script). Despite its few hundred lines of code, Eliza was remarkably effective, convincing many users that a human was on the other end, demonstrating a profound "magic trick" in human-computer interaction. A new book, "Inventing Eliza," by a group of academics and researchers, including David Berry and Mark Marino, excavates Eliza's source code and examines its lasting lessons for today's AI landscape. The episode delves into the fascinating life and motivations of Joseph Weizenbaum. Born in Nazi Germany, Weizenbaum's family emigrated to the U.S. in 1936. A natural mathematician, he became involved in early computing, even hand-soldering one of the first digital computers. His experiences, including witnessing a "con" where General Electric faked a working computer system for Bank of America, deeply influenced his thinking. He became fascinated by the distinction between an interface's apparent functionality and the underlying mechanism, leading to his 1962 paper, "How to Make a Computer Appear Intelligent." Weizenbaum wasn't necessarily aiming for a commercial product but rather an experiment to study natural language communication, though he was clearly invested in Eliza *seeming* convincing. The famous anecdote of his secretary having a private conversation with the bot illustrates its surprising impact. A central theme explored is *why* people so readily fell for Eliza, even with their limited computer experience. Pierce, Berry, and Marino discuss several factors: the novelty of interacting with a computer, Eliza's disarming questions, and a fundamental human desire to connect and be heard. They draw parallels to historical "cons" where people were easily deceived by artifice and to modern phenomena like "AI boyfriends" and "girlfriends," suggesting a deep-seated human willingness to believe, especially when offered something desirable (like free, non-judgmental therapy). However, Weizenbaum's success with Eliza led to a profound crisis of faith. He observed that people at MIT were beginning to dehumanize others, speaking of humans as "meat machines," a sentiment that deeply disturbed him, reminding him of Nazi Germany's rhetoric. Coupled with the political turmoil of the 1960s (Civil Rights, Vietnam War) and influences from thinkers like Hannah Arendt and Lewis Mumford, Weizenbaum became increasingly critical of the direction of computing. He ultimately wrote "Computer Power and Human Reason" (1976), a book that declared AI was going wrong and made him a "heretic" in the field. Weizenbaum's core argument, inspired by Arendt, was the crucial distinction between "calculating" and "judging." He believed computers excel at calculation—perfect, rational, and predictable tasks—but utterly lack judgment, which requires human experience, empathy, and care. While he was not anti-technology, he understood that computers were unsuited for roles demanding human nuance and ethical considerations. He feared the commercialization and overreliance on technology in areas that diminish human connection and ethical reasoning. The podcast then brings this historical context to the present "LLM phenomenon." While acknowledging the "novel and remarkable" technological advancements of LLMs, Berry and Marino highlight concerning parallels with the Eliza era: cycles of AI hype, the "magic trick" of convincing AI, and the human desire for connection being exploited. They point out key differences, too: LLMs are far more powerful, developed by vastly wealthier companies with immense incentives, and often operate as "black boxes" whose inner workings are obscure, unlike Eliza's explainable code. Reflecting on how Weizenbaum might react to today's LLMs, the co-authors suggest a dual response. His younger self, fascinated by "the con," might have been thrilled by the advanced illusion. However, the later, more critical Weizenbaum would be deeply worried by the commercialization, the weaponization of the "Eliza effect" for data extraction and dependency, and the profound lack of judgment in systems being applied to mental health, education, and social interaction. His warnings from "Computer Power and Human Reason" remain highly relevant, emphasizing the ongoing need for society to critically evaluate where and how we deploy AI, and to remember that while computers calculate, only humans can truly judge.