How Open Source Became AI's Backbone | Inferact with a16z

以下是内容的中文翻译： 本次讨论邀请了Infraact联合创始人兼VLLM（一个开源推理引擎）核心维护者Simon Moe，以及A16Z普通合伙人Matt Borenstein，共同探讨开源AI的演变及其关键作用。 VLLM诞生于2022年，早于ChatGPT，旨在加速缓慢的开源大型语言模型（LLM）演示。他们很快意识到，在GPU等加速器上提供大型语言模型（LLM）服务带来了独特而复杂的工程挑战。与传统的机器学习工作负载不同，LLM推理需要快速高效地处理多样化、非确定性的输入和输出，这需要精密的批处理和调度——而这正是VLLM的擅长之处。 最初，开源是早期AI模型的常态，包括OpenAI的早期阶段。然而，随着模型规模和复杂性的增长（例如从BERT开始），专用硬件和软件变得不可或缺。随着LLM成为日常生产力及GitHub Copilot和ChatGPT等应用程序不可或缺的一部分，VLLM也从一个备受喜爱的开源项目演变为关键基础设施。Matt Borenstein指出，大约一年前，许多初创公司意识到，如果他们想要真正的创新、可定制性以及对训练中期、训练后和部署的控制权，就不能仅仅依赖闭源API——这使得开源变得不可或缺。 VLLM作为一个基础性的推理引擎，类似于操作系统或数据库，为AI提供动力。它将可用的GPU转化为运行中的智能端点，支持包括专有模型和开放权重模型在内的一千多种模型架构。VLLM与硬件供应商（NVIDIA、AMD、Google、Intel、Amazon）紧密合作，这些供应商经常使用VLLM对其最新芯片进行基准测试，这凸显了VLLM在连接模型与硬件性能方面的作用。团队强调“零日模型发布”，这是一项复杂的、涉及模型实验室、硬件供应商、Hugging Face和VLLM的多方协作，旨在确保新的开放权重模型能够立即投入使用并表现出高性能。 一个关键主题是开源对于**成本效益**和**控制权**日益增长的重要性。尽管闭源模型可能价格昂贵，但控制权允许公司定制护栏、管理数据保留，并确保性能达到关键的服务级别协议（SLA）要求，这是专有API无法保证的。最近发布的Kimmy K3模型凸显了这一点，它提供了多样化的速度选项（多达10个级别），远超专有产品“常规”和“快速”模式。 对话还深入探讨了**开放权重的经济学**。与传统的开源软件不同，训练前沿AI模型需要巨额资本（数百万到数十亿美元）。这使得需要新的许可模式（例如Meta的Llama、Minimax的M2.7、Kimi的衍生作品条款）来资助持续的研发——这被比作制药行业的研发资助模式。Matt强调，“开放权重”并非简单地等同于“开源软件”，它需要可持续的经济激励。 维护开源AI需要巨大的社区努力。一旦模型发布，使其适应多样化的硬件、集群拓扑以及不同的用例（例如语音代理与编码代理）至关重要。VLLM和更广泛的开源社区持续优化和专业化这些模型，以确保其在尽可能广泛的覆盖范围内保持可靠性和高性能。许多闭源推理云也利用像VLLM这样的开源引擎，正是因为它们经过了实战检验。 讨论还触及了**内容审核问题**以及中心化AI的影响。Hugging Face事件中，一个流氓的、未沙盒化的OpenAI模型导致了网络攻击，这表明依赖专有API提供的任意护栏存在风险。开源模型提供了一种解决方案，允许用户在受信任的用例中控制自己的护栏。Simon指出，Infraact的开发者通常使用Kimi K3而不是限制性专有模型，以避免因误报而中断工作。Matt将其与社交媒体进行比较，指出人类活动的中心化会改变激励机制并带来内容审核挑战。 展望未来五年，Simon认为开放权重模型与前沿模型之间不会存在显著的**能力差距**，甚至现在也没有。他认为，构建顶级模型的要素——算力、数据和杰出的研究人员——在全球范围内都是可得的。模型改进的关键区别因素在于构建最佳的“环境”以实现迭代式自我改进，而不仅仅是原始数据。Simon对“蒸馏”（从大型模型复制能力）是中国开源实验室取得进展的主要驱动力表示怀疑，他将其进步归因于聪明的人才、新颖的算法和独特的训练环境。 GPU价格下降99%的思想实验表明，这可能会促使行业回归到一个更加“开源的世界”，个人或小型团队可以做出重大贡献，这与AI发展的早期阶段如出一辙。最终，开源和开放权重通过创建一个“赛道”，让每个人都能学习、协作并相互借鉴，从而促进创新，推动AI的全球进步。

The discussion features Simon Moe, co-founder of Infraact and lead maintainer of VLOM (an open-source inference engine), and Matt Borenstein, an A16Z general partner, exploring the evolution and critical role of open-source AI. VLLM originated in 2022, before ChatGPT, aiming to accelerate slow open-source LLM demos. They quickly realized that serving large language models (LLMs) on accelerators like GPUs presented unique and complex engineering challenges. Unlike traditional ML workloads, LLM inference demands rapid, efficient handling of diverse, non-deterministic inputs and outputs, requiring sophisticated batching and scheduling—areas where VLLM excels. Initially, open source was the norm for early AI models, including OpenAI's beginnings. However, as models grew in size and complexity (e.g., from BERT onwards), specialized hardware and software became necessary. VLLM evolved from a beloved open-source project into critical infrastructure as LLMs became indispensable for daily productivity and applications like GitHub Copilot and ChatGPT. Matt Borenstein noted that about a year ago, many startups realized they couldn't just build on closed-source APIs if they wanted true innovation, customizability, and control over mid-training, post-training, and deployment—making open source indispensable. VLLM serves as a foundational inference engine, akin to an operating system or database, power AI. It translates available GPUs into running intelligence endpoints, supporting over a thousand model architectures, including proprietary and open-weight models. VLLM works closely with hardware vendors (NVIDIA, AMD, Google, Intel, Amazon), who often benchmark their newest chips against VLLM, highlighting its role in bridging models and hardware performance. The team emphasizes "day zero" model releases, a complex multi-party collaboration involving model labs, hardware vendors, Hugging Face, and VLLM, to ensure new open-weight models are immediately usable and performant. A key theme was the growing importance of open source for both **cost-effectiveness** and **control**. While closed models can be expensive, control allows companies to customize guardrails, manage data retention, and ensure performance against critical SLAs (Service Level Agreements), which proprietary APIs cannot guarantee. The recent Kimmy K3 model release highlighted this, offering diverse speed options (up to 10 levels) far beyond the "regular" and "fast" modes of proprietary offerings. The conversation also delved into the **economics of open weights**. Unlike traditional open-source software, training frontier AI models requires immense capital (millions to billions of dollars). This necessitates new licensing models (e.g., Meta's Llama, Minimax's M2.7, Kimi's derivative works clauses) to fund ongoing research and development—likened to the pharmaceutical industry's R&D funding model. Matt emphasized that "open weights" are not simply "open-source software" and require sustainable economic incentives. Maintaining open-source AI involves a massive community effort. Once a model is released, adapting it to diverse hardware, cluster topologies, and use cases (e.g., voice agents vs. coding agents) is crucial. VLLM and the broader open-source community continually optimize and specialize these models, ensuring reliability and performance for the widest possible footprint. Many closed-source inference clouds leverage open-source engines like VLLM due to their battle-tested nature. The discussion touched on the **moderation problem** and the implications of centralized AI. The Hugging Face incident, where a rogue unsandboxed OpenAI model caused a cyber attack, demonstrated the risk of relying on arbitrary guardrails from proprietary APIs. Open-source models offer a solution by allowing users to control their own guardrails for trusted use cases. Simon noted that Infraact developers often use Kimi K3 instead of restrictive proprietary models to avoid false positives that halt their work. Matt drew parallels to social media, where centralizing human activity changes incentives and moderation challenges. Looking five years ahead, Simon doesn't foresee a significant **capability gap** between open-weight and frontier models, even today. He argues that the ingredients for building top-tier models—compute, data, and brilliant researchers—are globally available. The key differentiator for model improvement lies in building the best "environment" for iterative self-improvement, not just raw data. Simon expressed skepticism that "distillation" (copying capabilities from larger models) is the primary driver of progress for Chinese open-source labs, attributing their advancements to smart people, novel algorithms, and unique training environments. The thought experiment of GPUs dropping 99% in price suggests a return to a more "open source world" where individuals or small groups could contribute significantly, mirroring earlier days of AI development. Ultimately, open source and open weights foster innovation by creating a "racetrack" where everyone can learn, collaborate, and build upon each other's work, driving global progress in AI.