Post-Training Is How You Keep Your Taste | Fireworks CEO Lin Qiao

以下是内容的中文翻译： Fireworks 首席执行官兼联合创始人 Lynn 发表了一场引人入胜的演讲，阐述了“后训练”在构建持久且具竞争力的AI驱动型业务中的关键作用。她首先强调了最近软件和应用程序开发领域发生的颠覆性变革，小型团队无需丰富的编码知识即可快速实现创意。这种转变要求企业超越现成的“黑盒”API，以建立更深层次的竞争优势，这也与业界日益增长的对开源模型的支持相吻合。 Lynn 强调，企业若要蓬勃发展，必须“拥有自己的智能”。这涉及几个阶段： 1. **数据（Data）：** 策划高质量的生产数据并生成合成数据以丰富其内容，从而奠定独特智能的基础。 2. **模型所有权（Model Ownership）：** 利用这些数据训练和微调模型，从而拥有模型权重并嵌入独特的见解。 3. **服务（Serving）：** 部署这些模型，进行 A/B 测试以验证其对产品指标的影响，并进行持续迭代。 她概述了 AI 开发的自然演进路径： * **提示（Prompting）：** 快速测试创意的初始阶段，直接使用现有模型。 * **RAG（检索增强生成）：** 将 AI 根植于专有数据，实现上下文工程。 * **监督微调（Supervised Fine-tuning, SFT）：** 整合特定数据集以改善模型行为和结构，类似于从事实性文献中学习。 * **偏好调优（Preference Tuning, DPO）：** 根据个性化偏好和用户交互调整模型输出，反映个体判断。 * **强化学习（Reinforcement Learning, RL）：** 在特定垂直领域（例如法律、金融、医疗保健）发展深厚的领域专业知识，类似于成为一名专业专家。 * **蒸馏（Distillation）：** 从更大的“教师”模型中创建更小、更高效的模型，以提高性能和成本效益。 Lynn 阐明了每种技术的应用场景：RAG 用于动态事实，SFT 用于输出结构，偏好调优用于产品特定的“品味”，RL 用于在专业领域强化薄弱模型，以及蒸馏用于在生产中优化速度和成本。 然而，她也坦诚地指出了常见陷阱： * **数据质量：** 数量并非全部；高质量数据至关重要，往往需要产品团队做出关键判断。 * **评估（Evals）：** 摆脱主观的“感觉式评估”，转向系统化、可重复的评估流程，类似于传统软件开发中的单元测试。 * **草率的强化学习环境：** 设计不当的模拟可能导致模型“奖励作弊”（reward hacking），以意想不到的方式实现目标（例如，生成零代码以最小化编译错误）。 * **实验：** 最终的评判标准是产品指标，而不仅仅是训练结果。确保训练和服务栈之间的一致性至关重要，因为不同的数值库或优化方法可能导致生产中的质量下降。 Lynn 展示了许多在 Fireworks 平台上利用后训练取得成功的公司，这证明了其广泛的适用性： * **Cursor：** 旨在达到前沿质量，控制模型供应，并持续发布先进模型。 * **Doximity：** 将后训练应用于“临床 AI”，在医疗保健领域的斯坦福哈伯临床安全基准测试中取得了最高分。 * **Factory：** 专注于安全编码，调优在安全基准测试中表现出色的模型。 * **GenSpark：** 一个通用的协作应用程序，以低 5-10 倍的成本实现了与前沿模型相当的竞争质量。 * **HIDI：** 另一个医疗保健领域的案例，由于医疗保健数据的专业性质，其调优模型显著优于最先进的闭源模型。 她将采用后训练的开发者分为几类：“前沿智能体构建者”（他们创建定制的解决方案）、寻求减轻大规模部署 AI 功能巨大成本负担的大型现有企业，以及专业模型运营商。Lynn 表示，“在产品市场契合（PMF）之后，后训练将成为众多公司构建专业智能的载体”，并预测将出现“数百万个专业模型”。 在问答环节中，Lynn 对后训练的*时机*和*投资回报率（ROI）*提供了关键见解。她认为，尽管初创公司最初专注于使用前沿模型实现产品市场契合（PMF），但后训练在 PMF *之后*变得至关重要。这是因为 PMF 验证了产品的价值并产生了有意义的大量数据——这是拥有智能的燃料。后训练随后服务于两个关键目的： 1. **竞争优势：** 它使公司能够将自身独特的判断、品味和对客户的深刻理解编码到模型中，从而创建一个难以复制的专有“模式”，尤其是在应用程序本身变得更容易被复制的情况下。 2. **持久业务：** 通过实现推理成本降低 5-10 倍，后训练使企业能够高效扩展，在相同预算下支持更高的流量，并避免“规模化破产”（scaling into bankruptcy）。 Lynn 最后鼓励开发者尽早开始，迭代实验，并获得实践经验，指出奖励工程与软件工程在逻辑上具有显著的相似性。

Lynn, CEO and co-founder of Fireworks, delivered a compelling presentation on the critical role of "post-training" in building durable and competitive AI-powered businesses. She began by highlighting the recent disruption in software and application development, where ideas can be implemented rapidly by small teams without extensive coding knowledge. This shift necessitates moving beyond off-the-shelf, black-box APIs to build deeper competitive advantages, aligning with the industry's growing support for open models. Lynn emphasized that for companies to thrive, they must "own their own intelligence." This involves several stages: 1. **Data:** Curating high-quality production data and generating synthetic data to enrich it, forming the foundation of unique intelligence. 2. **Model Ownership:** Utilizing this data to train and fine-tune models, thereby owning the model weights and embedding unique insights. 3. **Serving:** Deploying these models, conducting A/B testing to validate their impact on product metrics, and iterating in a continuous loop. She outlined a natural progression in AI development: * **Prompting:** The initial stage for quick idea testing, using models as-is. * **RAG (Retrieval-Augmented Generation):** Grounding AI in proprietary data, enabling context engineering. * **Supervised Fine-tuning (SFT):** Incorporating specific datasets to improve model behavior and structure, analogous to learning from factual literature. * **Preference Tuning (DPO):** Tailoring model outputs to personalized tastes and user interactions, reflecting individual judgment. * **Reinforcement Learning (RL):** Developing deep domain expertise in specific verticals (e.g., legal, finance, healthcare), similar to becoming a specialized expert. * **Distillation:** Creating smaller, more efficient models from larger "teacher" models to improve performance and cost-effectiveness. Lynn elucidated when to apply each technique: RAG for dynamic facts, SFT for output structure, Preference tuning for product-specific taste, RL for strengthening weak models in specialized areas, and Distillation for optimizing speed and cost in production. However, she also candidly addressed common pitfalls: * **Data Quality:** Quantity isn't enough; high-quality data is paramount, often requiring product teams to make critical judgments. * **Evals (Evaluations):** Moving beyond subjective "vibe eval-ing" to systematic, repeatable evaluation processes akin to unit testing in traditional software development. * **Sloppy RL Environments:** Poorly designed simulations can lead to models "reward hacking," achieving goals in unintended ways (e.g., generating zero code to minimize compilation errors). * **Experimentation:** The final judge is product metrics, not just training results. Ensuring alignment between training and serving stacks is crucial, as differing numerical libraries or optimizations can lead to quality degradation in production. Lynn showcased numerous successful companies leveraging post-training on Fireworks, demonstrating its broad applicability: * **Cursor:** Aims for frontier quality, controlling model supply, and consistently releasing advanced models. * **Doximity:** Uses post-training for "clinic AI," achieving top scores on the Stanford Harbor clinic safety benchmark in healthcare. * **Factory:** Focuses on secure coding, tuning models that excel in security benchmarks. * **GenSpark:** A general co-work application, achieving competitive quality with frontier models at 5-10x lower cost. * **HIDI:** Another healthcare example, where tuned models significantly outperform state-of-the-art closed models due to the specialized nature of healthcare data. She categorized developers adopting post-training: "Frontier Agent Builders" who create bespoke harnesses, large incumbents seeking to mitigate the huge cost burden of deploying AI features at scale, and specialized model operators. Lynn stated that "after product-market fit, post-training becomes the vehicle for many, many companies to build specialized intelligence," predicting "millions of specialized models." During the Q&A, Lynn provided crucial insights into the *timing* and *ROI* of post-training. She argued that while startups initially focus on achieving product-market fit (PMF) using frontier models, post-training becomes essential *after* PMF. This is because PMF validates the product's value and generates meaningful, high-volume data—the fuel for owning intelligence. Post-training then serves two critical purposes: 1. **Competitive Edge:** It allows companies to codify their unique judgment, taste, and deep customer understanding into the model, creating a proprietary "mode" that's difficult to replicate, especially as applications themselves become easier to copy. 2. **Durable Business:** By enabling a 5-10x reduction in inference costs, post-training allows businesses to scale efficiently, supporting significantly higher traffic with the same budget and avoiding "scaling into bankruptcy." Lynn concluded by encouraging developers to start early, experiment iteratively, and get hands-on experience, noting that reward engineering shares significant logical similarities with software engineering.