The problem with AI detectors #Vergecast

早期开发人工智能文本检测器的尝试主要围绕一个名为“困惑度”（perplexity）的指标展开。困惑度衡量的是一段给定文本对于语言模型来说有多“出乎意料”（surprising）。为了说明这个概念，请看这句话：“男孩喝了一碗汤。” 这个序列中的每个词都高度可预测且符合预期，因此对于语言模型来说，它的困惑度非常低。相反，如果这句话是“男孩吃了一碗蜘蛛”，那么“蜘蛛”（spiders）这个词就会被记录为高困惑度词语，因为它在那个语境中是一个意想不到且令人惊讶的后续词。 使用困惑度进行人工智能检测的原理，源于人工智能语言模型的基本训练方法。这些模型在大量人类文本数据集上经过严格训练，其主要目标是预测序列中最可能出现的下一个词。它们的目标是生成连贯、流畅且符合统计学规律的文本。因此，这些人工智能模型生成的内容本质上被设计为具有低困惑度——即不令人惊讶且高度可预测。如果生成的词语或短语高度令人惊讶，则意味着更高的错误概率或偏离了学习到的模式，而这些正是模型被训练来避免的。 基于这种理解，一个直接的人工智能检测假说应运而生：如果一段文本表现出低困惑度，它很可能是人工智能生成的；如果它表现出高困惑度，那么它更有可能是人类撰写的。假设是，人类作者与追求统计平均值的人工智能模型不同，他们经常引入创造力、意想不到的措辞、独特的风格选择和细致入微的表达，这些会偏离最符合统计学概率的序列，从而产生具有更高“出乎意料”程度或困惑度的文本。 然而，这种基于困惑度的检测方法很快就在几个关键情景下，暴露出其可靠性和有效性方面的严重缺陷。 第一个主要限制出现在，当相关文本在人工智能模型的训练阶段已被广泛“记忆”时。任何作为模型训练数据重要组成部分的文本，或是一个众所周知、广泛发布的文档，自然会被该模型认为困惑度极低。举例来说，一个主要例子是《独立宣言》。如果这份历史文献被输入到一个基于困惑度构建的人工智能检测器中，它将不可避免地被标记为人工智能生成。这并非因为人工智能真的写了它，而是因为语言模型在训练过程中无数次遇到过它，因此认为每个词和序列都完全可预测。这个缺陷突显出，该检测器并非真正识别出人工智能的*生成*（generation），而是人工智能的*熟悉度*（familiarity）或先前的接触，从而导致对真正由人类创作的内容产生误报（false positives）。 困惑度作为独立检测指标的第二个、或许也是更根本的问题在于其固有的不变性（immutability）。解释指出，困惑度“就是它本身”（is what it is），并且“它并不是一个可以改进的指标”（it's not really a metric that can be improved upon）。这意味着困惑度是一种描述性测量，反映了语言模型对文本的内部概率分布。它量化了模型基于其训练对词语序列预测得有多好。你无法“改进”困惑度本身，使其成为一个更可靠的*作者身份*或*生成*（generation）与*熟悉度*（familiarity）之间的识别器。它的值是文本和模型训练数据的直接数学结果，使其成为一种静态的、固有的属性，而非一个可适应的诊断工具。尽管可以训练一个模型来*识别*特定的困惑度范围，但其底层指标本身无法被优化，从而无法克服核心限制（例如记忆问题），使其能够始终如一地区分真正的人工智能生成内容和对于给定模型来说仅仅是统计上可预测的人类内容。 本质上，尽管困惑度为人工智能检测提供了一个直观的起点，但它容易受到被记忆文本的影响，以及其作为一种静态的、依赖于模型的内在指标的性质，意味着它缺乏进行可靠、准确的人工智能内容识别所需的鲁棒性和适应性。

Early efforts in developing AI text detectors largely centered on a metric called "perplexity." Perplexity serves as a measure of how "surprising" a given piece of text is to a language model. To illustrate this concept, consider the sentence, "The boy ate a bowl of soup." Each word in this sequence is highly predictable and expected, resulting in very low perplexity for a language model. Conversely, if the sentence were, "The boy ate a bowl of spiders," the word "spiders" would register as a high-perplexity word because it's an unexpected and surprising continuation in that context. The rationale behind using perplexity for AI detection stems from the fundamental training methodology of artificial intelligence language models. These models are rigorously trained on vast datasets of human text with the primary objective of predicting the most probable next word in a sequence. Their goal is to generate coherent, fluent, and statistically likely text. Consequently, the output produced by these AI models is inherently designed to be of low perplexity – unsurprising and highly predictable. If a generated word or phrase were highly surprising, it would suggest a higher probability of error or a deviation from the learned patterns, which the models are trained to avoid. Based on this understanding, a straightforward AI detection hypothesis emerged: if a text exhibits low perplexity, it is likely AI-generated; if it demonstrates high perplexity, it is more probable that it was written by a human. The assumption was that human writers, unlike AI models striving for statistical averages, often introduce creativity, unexpected turns of phrase, unique stylistic choices, and nuanced expressions that deviate from the most statistically probable sequences, thereby producing text with a higher degree of "surprise" or perplexity. However, this perplexity-based detection method quickly revealed significant breakdowns in its reliability and effectiveness in a couple of critical scenarios. The first major limitation arises when the text in question has been extensively "memorized" by the AI model during its training phase. Any text that forms a significant part of the model's training data, or is a well-known, widely published document, will naturally be considered extremely low perplexity by that model. A prime example given is the Declaration of Independence. If this historical document is fed into an AI detector built on perplexity, it will invariably be flagged as AI-generated. This occurs not because an AI actually wrote it, but because the language model has encountered it countless times during its training and therefore finds every word and sequence utterly predictable. This flaw highlights that the detector isn't truly identifying AI *generation* but rather AI *familiarity* or prior exposure, leading to false positives for genuinely human-authored content. The second, and perhaps more fundamental, issue with perplexity as a standalone detection metric is its inherent immutability. The explanation states that perplexity "is what it is" and "it's not really a metric that can be improved upon." This means that perplexity is a descriptive measurement reflecting a language model's internal probability distribution over a text. It quantifies how well a model predicts a sequence of words based on its training. You cannot 'improve' perplexity itself to make it a more robust identifier of *authorship* or *generation* versus *familiarity*. Its value is a direct mathematical consequence of the text and the model's training data, making it a static, inherent property rather than an adaptable diagnostic tool. While one could train a model *to identify* specific perplexity ranges, the underlying metric itself cannot be refined to overcome the core limitations, such as the memorization problem, in a way that consistently distinguishes between truly AI-generated content and human content that merely appears statistically predictable to a given model. In essence, while perplexity offered an intuitive starting point for AI detection, its susceptibility to memorized texts and its intrinsic nature as a static, model-dependent metric meant it lacked the robustness and adaptability required for reliable and accurate AI content identification.