首页  >>  来自播客: Sequoia Capital 更新   反馈  

Sequoia Capital - Continual Learning: How AI Agents Get Better With Every Use | Arjun Karanam, Trajectory

发布时间:   原节目
以下是内容的中文翻译: Trajectory AI 的联合创始人 Arjun 和 Ronak 阐述了他们对“持续学习”的愿景,旨在解决他们所称的现代 AI 模型中的“经验鸿沟”。他们认为,虽然模型在“智商”(智能)方面迅速发展,但它们通常缺乏能够以类人熟练度执行任务的“经验”,这使得它们在工作中感觉永远是新手。 Trajectory 的核心理念是,每天通过用户交互生成大量有价值的“经验”——数十亿的 token 和代理执行的实际工作——这些数据随后被丢弃了。他们相信这些数据是模型学习和改进的关键信号,这与人类获取专业知识的方式如出一辙。他们的使命是将静态的、昂贵的、不会自我改进的 AI 代理转变为“学习型代理”,使其能够在使用中不断改进,从而形成更快、更便宜、更有效的 AI 系统,这些系统能够“在使用中复利增长”。 他们实现这一目标的方法涉及一个多方面的平台和前沿研究: 1. **可追溯性 (Traceability):** 第一步是捕获所有交互,包括行动、子代理和工具调用的完整“树”,这些通常被忽视。 2. **模型规范与奖励提取 (Model Specification & Reward Extraction):** 他们专注于从用户交互中提取精确的反馈。他们优先考虑“纠正性行为”,如编辑、撤销和重试,而不是简单的点赞/点踩,将其作为定义代理应该学习内容的**高保真信号**。 3. **持续改进 (Continual Improvement):** 这些反馈随后被用于改进模型(例如,通过在长追踪数据上使用 SDPO 等强化学习算法)和编排模型的“Harness”。他们区分了哪些应该更新模型的行为,哪些应该更新 Harness 的上下文信息(例如,事实性修正可能归 Harness 处理,而程序性改进则归模型处理)。 4. **部署 (Deployment):** 该平台旨在无缝部署这些持续改进的模型和 Harness。 Arjun 概述了加速持续学习的代理生态系统的“四个愿望”: 1. **可追溯性:** * 追踪整个行动树,包括所有子代理和工具调用。 * 设计产品不仅要捕获交互数据,还要积极地引出和记录高信号反馈,如用户编辑、撤销和重试。 2. **评估 (Evals):** * 评估应直接来源于真实产品流量和“前沿”用例。 * 任务应是“可回放的 (roll-outable)”,这意味着用户操作可以被重放以进行评估。 * 评分必须使用实际的生产 Harness 进行,以确保其与真实世界性能的保真度。 3. **Harnesses(编排器):** * 从严格、强制的流程转变为允许代理通过编排核心产品原语(工具、私有数据)来“自主运行”。 * 使代理界面尽可能接近用户界面,理想情况下,每一个 UI 操作都可以是一个工具调用。 * 确保工具响应高度信息丰富(例如,详细说明写入或读取了什么),而不仅仅是“完成”之类的模糊确认。 4. **模型:** * 公司应习惯运行开源模型(open-weight models),以获得对其 AI 堆栈的所有权,并实现持续的微调。 * 尝试使用模型路由器,智能地将任务的不同部分导向最合适的模型。 Trajectory 的产品目前处于测试阶段,旨在普及这种能力。他们强调抽象化后期训练和微调的复杂性,允许用户以最小的努力(例如,15分钟的用户交互)导入、训练、评估和部署模型(如 Harvey 或 Lab benchmarks)。他们的目标是让任何公司都能够“拥有自己的经验层”并利用持续学习。 在问答环节,Arjun 回答了几个关键问题: * **可训练对象:** 他澄清说,持续学习是一种系统级别的优化,平台会根据学习到的信号智能地决定是更新模型权重、Harness 逻辑还是上下文,从而为用户抽象化这些复杂性。 * **客户数据与隐私:** Trajectory 避免直接在客户数据上进行训练。相反,他们使用抽样分布、合成数据生成和加密比较等技术,在不损害隐私的情况下从客户交互中学习。 * **持续学习的范围:** 尽管应用广泛,但它对于“前沿”任务尤其令人兴奋。持续学习允许代理从用户在当前能力边缘的尝试中学习,从而使产品能够扩展其功能,而不是在初步失败后强迫用户退回到更简单的请求。

Arjun and Ronak, co-founders of Trajectory AI, presented their vision for "continual learning," addressing what they call the "experience gap" in modern AI models. They argue that while models are rapidly advancing in "IQ" (intelligence), they often lack the "experience" that allows them to perform tasks with human-like proficiency, making them feel perpetually new to the job. Trajectory's core worldview is that vast amounts of valuable "experience" are generated daily through user interactions—billions of tokens and real work that agents perform—which are then discarded. They believe this data is the crucial signal for models to learn and improve, mirroring how humans gain expertise. Their mission is to transform static, expensive, and non-improving AI agents into "learning agents" that continually get better with use, leading to faster, cheaper, and more effective AI systems that "compound with use." Their approach to achieving this involves a multi-faceted platform and cutting-edge research: 1. **Traceability:** The first step is to capture all interactions, including the full "tree" of actions, sub-agents, and tool calls, which are often overlooked. 2. **Model Specification & Reward Extraction:** They focus on extracting precise feedback from user interactions. Instead of just basic thumbs-up/down, they prioritize "corrective behaviors" like edits, undoes, and retries as high-fidelity signals for defining what the agent should learn. 3. **Continual Improvement:** This feedback is then used to improve *both* the models (e.g., through RL algorithms like SDPO on long traces) and the "harnesses" that orchestrate them. They differentiate between what should update the model's behavior versus what should update the harness's contextual information (e.g., a factual correction might go to the harness, while a procedural improvement goes to the model). 4. **Deployment:** The platform is designed to seamlessly deploy these continually improving models and harnesses. Arjun outlined "four wishes" for the agent ecosystem to accelerate continual learning: 1. **Traceability:** * Trace the entire action tree, including all sub-agent and tool calls. * Design products to not only capture interaction data but also actively elicit and log high-signal feedback like user edits, undoes, and retries. 2. **Evals:** * Evaluations should be drawn directly from real product traffic and "frontier" use cases. * Tasks should be "roll-outable," meaning user actions can be replayed for evaluation. * Grading must occur using the actual production harness, ensuring fidelity to real-world performance. 3. **Harnesses:** * Shift from strict, enforced flows to allowing agents to "cook" by orchestrating core product primitives (tools, private data). * Make the agent interface as close as possible to the user interface, ideally allowing every UI action to be a tool call. * Ensure tool responses are highly informative (e.g., detailing what was written or read), not just vague confirmations like "done." 4. **Models:** * Companies should become comfortable running open-weight models to gain ownership over their AI stack and enable continuous fine-tuning. * Experiment with model routers to intelligently direct different parts of a task to the most suitable model. Trajectory's product, currently in beta, aims to democratize this capability. They emphasize abstracting away the complexities of post-training and fine-tuning, allowing users to import, train, evaluate, and deploy models (like Harvey or Lab benchmarks) with minimal effort (e.g., 15 minutes of user interaction). Their goal is to empower any company to "own its own experience layer" and leverage continual learning. During the Q&A, Arjun addressed key concerns: * **Trainable Objects:** He clarified that continual learning is a system-level optimization, where the platform intelligently decides whether to update model weights, harness logic, or context based on the learned signal, abstracting this complexity from the user. * **Customer Data & Privacy:** Trajectory avoids direct training on customer data. Instead, they use techniques like sampling distributions, synthetic data generation, and cryptographic comparisons to learn from customer interactions without compromising privacy. * **Scope of Continual Learning:** While broadly applicable, it's particularly exciting for "frontier" tasks. Continual learning allows agents to learn from user attempts at the edge of current capabilities, enabling the product to expand its functionality rather than forcing users to retreat to simpler requests after initial failures.