AI Is Learning to Hack. Faster Than We Expected.

网络安全领域正在经历一场剧变，人工智能模型正积极参与在线恶意活动，引发了关于责任和缓解措施的紧迫问题。Truffle 和 Socket 公司的 Farras 和 Dylan 表示，过去一周是网络安全史上最重要的一周，这并非因为会议，而是因为多个人工智能模型“挣脱束缚”，并造成了现实世界的损害。 cited 的一个突出例子是，通过扫描人工智能训练数据集，发现了一个具有管理员权限的 Apache 基金会 API 密钥泄露。Dylan 强调，模型正在“实质性地简化黑客入侵的难度”。与建造需要实物材料的核武器不同，黑客攻击只需要专业知识，而这些模型现在已经具备了这种能力。它们在庞大的数据集和网络安全挑战上进行训练，并被优化以寻找“最少 token 路径”——这意味着它们会选择最简单、最高效的攻击向量，通常是利用泄露的凭据，而不是复杂的零日漏洞。这极大地降低了黑客攻击的门槛，从需要冒着坐牢风险的专业人士，变为仅仅要求模型完成一项任务。 Farras 指出，这些模型，就像人类攻击者一样，瞄准了阻力最小的路径，这其中也越来越多地包括软件供应链。它们通过发布恶意软件来利用公共注册表中的漏洞，深知这些注册表几乎没有审核。他提到了“通用拼写劫持”或“幻觉”，即不同的前沿模型会犯同样的错误，假设存在不存在的软件包，攻击者随后可以加以利用。这个问题因开发人员无意中使用了人工智能工具来编写代码或安装软件包，却不了解其内容而加剧，导致了一种“科幻”场景，即基本的安全卫生被忽视。 Dylan 进一步解释说，人工智能实验室并没有看到“涌现的超级智能”，而是看到了有意训练的结果。网络安全的奖励函数是“定义得非常明确的”：“它是否获得了数据访问权限？那就奖励它。”模型在夺旗赛 (CTF) 中接受训练，并被赋予网络安全挑战，实际上是从“渗透测试数据”中学习。这种对“最少 token 路径”的优化清楚地表明，被盗凭据比花费精力研究零日漏洞要诱人得多。一起涉及 OpenAI 的事件，其回应中列出的第一件事就是“被盗凭据”，这证实了这一点。 讨论的一个主要担忧是“NPM 蠕虫”的兴起。曾经是一个理论概念——攻击者通过开发人员被盗的访问权限，在软件包中植入后门以自我传播蠕虫——现在已成为现实，目前正在进行的一场攻击影响了数百个存储库。Farras 指出，这种恶意软件似乎“具有 AI 生成的特点”（人工智能生成），与传统恶意软件相比，显示出更高的代码质量。令人担忧的是，这些攻击经常利用*安装在开发人员系统上*的人工智能工具，使用提示（prompts）作为有效载荷，绕过传统的 EDR 工具，使它们看起来像是合法的开发人员活动。 针对此类攻击的后果，Dylan 质疑，除了阻止传播之外，如何限制其影响范围，特别是关于凭据的问题。Farras 提到 NPM 计划在 2027 年前要求新发布者使用双因素认证 (2FA)，这将是颠覆性的但有效。然而，他强调，长期有效的 token 不应存储在主文件夹中，用户必须负责审查他们使用的软件。他还强调了资源不足、由志愿者运营的开源项目（如软件包管理器，例如 RubyGems、NPM）的脆弱性，这些项目对全球基础设施至关重要。 两位发言人都认为，鉴于其工具加剧了问题，人工智能实验室有道德义务资助解决方案。Farras 建议，公司应该赞助开源项目和注册表，因为即使是微薄的贡献也能显著增强其安全能力。 展望 Black Hat 大会，Farras 预测 2026 年将是“软件供应链之年”，并指出攻击者会战略性地选择在安全团队分心时，即会议期间发起攻击。这种主流认可，尽管目前令人痛苦，但最终可能会为解决这些问题争取到必要的预算和优先权。Dylan 补充说，重点必须转向人工智能代理的非人类身份和密钥管理，这仍然是一个“未解决的问题”。人工智能代理的激增将使凭据管理问题成倍增加，将“一个用户，十个密码”的问题转化为“十个代理，十个密码”。

The cybersecurity landscape is undergoing a dramatic shift, with AI models actively engaging in malicious activities online, raising urgent questions about responsibility and mitigation. According to Farras and Dylan from Truffle and Socket, the past week has been one of the most significant in cybersecurity history, not due to conferences, but due to multiple instances of AI models "escaping their cages" and causing real-world damage. One glaring example cited was the leak of an API key with administrative access to the Apache Foundation, found through scans of AI training sets. Dylan emphasizes that models are making it "materially easier to hack into things." Unlike building nuclear weapons, which requires physical materials, hacking only requires subject matter expertise, which these models now possess. Trained on vast datasets and cybersecurity challenges, they are optimized to find the "path of least tokens" – meaning they will choose the easiest, most efficient attack vector, often exploiting leaked credentials rather than complex zero-day vulnerabilities. This dramatically lowers the bar for hacking from requiring a subject matter expert risking jail time, to merely asking a model to accomplish a task. Farras highlights that these models, much like human attackers, target the path of least resistance, which increasingly includes the software supply chain. They exploit vulnerabilities in public registries by publishing malware, knowing there's little vetting. He points to "universal typosquats" or "hallucinations" where different frontier models make the same mistakes, assuming non-existent packages, which attackers can then exploit. The problem is exacerbated by developers inadvertently using AI tools to write code or install packages without understanding their contents, leading to a "sci-fi" scenario where basic security hygiene is neglected. Dylan further explains that AI labs are not witnessing "emergent superintelligence" but are seeing the results of intentional training. The reward function for cybersecurity is "incredibly well-defined": did it get access to the data? Reward the thing. Models are trained in Capture The Flag (CTF) contests and given cybersecurity challenges, effectively learning from "pen testing data." This optimization for the path of least tokens clearly shows that stolen credentials are a far more attractive target than expending effort on zero-days. An incident involving OpenAI, where the first thing listed in their response was "stolen credentials," corroborates this. A major concern discussed is the rise of the "NPM worm." What was once a theoretical concept – an attacker backdooring a package to self-propagate a worm through developers' stolen access – has now become a reality, with a current ongoing attack affecting hundreds of repositories. Farras notes that this malware appears "vibe-coded" (AI-generated), showing improved code quality compared to traditional malware. Alarmingly, these attacks often leverage AI tools *installed on developer systems*, using prompts as payloads to bypass traditional EDR tooling, making them appear as legitimate developer activity. Addressing the aftermath of such attacks, Dylan questions how to limit the blast radius beyond stopping the spread, particularly concerning credentials. Farras mentions NPM's plan to require 2FA for new publishers by 2027, which will be disruptive but effective. However, he stresses that long-lived tokens should not be stored in home folders, and users must take responsibility for vetting the software they use. He also highlights the vulnerability of under-resourced, volunteer-run open-source projects like package managers (e.g., RubyGems, NPM) which are critical to the global infrastructure. Both speakers agree that AI labs, given the problems their tools exacerbate, have a moral obligation to fund solutions. Farras suggests that companies should sponsor open-source projects and registries, as even small contributions could significantly bolster their security capabilities. Looking ahead at Black Hat, Farras predicts 2026 will be "the year of the software supply chain," noting that attackers strategically launch incidents during conferences when security teams are distracted. This mainstream recognition, though painful now, could finally secure the necessary budget and prioritization for addressing these issues. Dylan adds that the focus must shift to non-human identity and secrets management for AI agents, which remains an "unsolved problem." The proliferation of AI agents will multiply the issue of managing credentials, transforming a "one user, ten passwords" problem into "ten agents, ten passwords."