Ben Wellington – Complex Feature Engineering at Two Sigma (S7E32)

以下是这段内容的中文翻译： 本期“与模型调情”节目由Corey Hofstein主持，特邀Two Sigma复杂特征引擎主管Ben Wellington。本次对话深入探讨了量化策略背后的人为因素，尤其侧重于特征生成，以及人工智能（AI）在该领域带来的变革性影响。 以下是讨论的详细摘要： **1. Ben Wellington的背景与职责：** * **教育背景：** Ben拥有纽约大学的自然语言处理（NLP）博士学位，在Siri/Alexa出现之前的时代，他专注于机器翻译，当时这些应用还是新鲜事物。他选择这个领域是因为它“听起来很酷”。 * **Two Sigma的职业生涯：** 他于2007年加入Two Sigma，当时该公司规模尚小（125人）。他的初始职责是数据工程团队，专注于数据收集、摄入和清洗。 * **转向建模：** Ben发现了一些有趣的数据集，并进行了一项内部的“新闻地域政治分析”，这引起了Two Sigma一位建模师的注意。这促使他将重心转向研究文本数据如何预测市场，这一职责他已经担任了10多年，旨在预测资产在不同时间范围内的走势（上涨/下跌）。 * **现任职位：** 复杂特征引擎主管，这突显了他的核心工作重点。 **2. 量化投资中“特征”的定义：** * **核心概念：** “特征”是“关于世界中值得注意的事实”，对Two Sigma而言，它必须“具有经济意义”。 * **例子：** * **常见例子：** 过去一周股价的变化，自去年以来的股价走势，波动性。 * **更具趣味性的例子：** 某分析师最近上调了一只股票的评级。 * **文本/另类特征（Ben的专长）：** 一家公司的新闻报道量与其历史相比，新闻稿发布日期/时间（例如，周五收盘后）。 * **目的：** 特征封装了假设。它们将人类的智慧和直觉叠加到原始数据之上，帮助计算机聚焦于潜在有意义的信息，而不是简单地将所有原始数据一股脑地扔给算法。这使得这项工作变得可行，并利用了人类的知识。 **3. Two Sigma的运营模式与投资小组模式（Pod Shops）的对比：** * **共享平台模式：** 与投资小组（pod shops）中团队（pod）相互竞争不同，Two Sigma采用的是一种将建模师的预测集中起来并整合到公司更大的投资组合中的模式。他们的目标不是控制某个特定的投资组合，而是做出尽可能最好的预测。 * **全局优化：** 这种结构鼓励“全局优化”，而非（像投资小组那样）的“局部优化”。团队被激励将他们最好的工作成果（特征、算法）提升到共享平台供他人使用，从而使整个公司变得更好。这被描述为一个“循环”过程，团队不断地向平台贡献改进。 **4. 阿尔法（Alpha）的来源（数据、特征、预测）：** * Ben指出，潜在的阿尔法来源主要有三个方面： 1. **数据获取：** 由于商品化程度提高，在过去的15-20年里，其相对重要性已经“下降”。尽管Two Sigma仍拥有专有数据（长期历史数据、众包交易想法），但仅仅通过数据获取获得的优势已不如从前。 2. **特征生成（Ben的重点）：** 这一点“极其重要”。将数据创造性地、独特地转化为有洞察力的观察结果的能力是巨大的优势，这由“人类智慧”驱动。Ben举例说，研究分析师时，不仅仅是简单的评级上调，还要考虑他们的地理位置、毕业院校、与CEO的联系等。每一个微小而巧妙的问题（例如，200个问题=200个特征）共同创造出显著的阿尔法信号。 3. **预测/机器学习算法：** 这同样是“非常重要的一部分”，有一个专门的“技术”团队专注于先进的算法，以从特征中提取最大信息。 * **协同作用：** 以特征为重点的团队（识别巧妙数据）与技术团队（应用先进算法）的结合，才是“奇迹发生的地方”，这种结合是由方法的多元性驱动的。 **5. 特征构思与挑战管理：** * **构思过程：** 它需要“坚持不懈”和人类层面来辨别什么是有意义的（例如，分析师的眼睛颜色可能无关紧要，但他们的训练背景或地理位置可能有关）。这需要来自不同科学背景的多元化思维。 * **实例泛化：** 将现实世界的事件（例如，影响一家公司的火车脱轨事件）进行泛化，并问“这是否可以泛化？”，以创建更广泛的特征。 * **科学方法：** 遵循科学方法，即提出假设，进行测试，而不是事后合理化与先验知识相矛盾的结果。 * **共线性：** 这是“信号的杀手”。研究人员不断寻找*正交性*，以避免那些仅仅重新解释现有信号的特征（例如，仅仅跟踪财报发布后表现的新闻情绪）。他们使用工具和阈值来确保新的阿尔法信号具有足够的正交性。 * **预测周期：** 研究人员会注意与特征相关的时限（例如，商店访问数据可能预测几个月后的销售额）。 * **特征生命周期：** 随着越来越多的市场参与者根据某个信号采取行动，其预测周期可能会加快，从而可能侵蚀其有效性。Two Sigma旨在开发那些能抵御这种压力的创新特征，并监测“阿尔法衰减”。 **6. 招聘创意和创新人才：** * **超越技术技能：** 虽然数学、统计和编程很重要，但特征工程需要创造力、创新性思维和横向思维。 * **面试方法：** 采用开放式问题来评估创造力，例如“如果你能访问社交媒体的实时数据流，你能预测什么？”目的是观察候选人如何在没有固定“正确”答案的情况下，产生新颖有趣的 F想法。 **7. 人工智能/大型语言模型（LLMs）与特征生成的未来：** * **Ben的核心洞察：** “自然语言处理是将语言转化为数字，而有了大型语言模型，我就可以将数字转化为语言。现在，任何事物都可以是语言。” * **影响：** 大型语言模型可以“生成文本数据”，这为Two Sigma（一家拥有15年以上文本数据交易经验的公司）带来了“机会的革命”。专有数据现在包括“生成”的数据。 * **成本骤降：** 大型语言模型极大地降低了探索想法的成本和时间投入。一个假设的例子：以前，研究CEO眨眼来预测价值（需要计算机视觉和6个月的工作）投资回报率很低。现在，大型语言模型可以使此类探索速度提高100倍，使得以前“过于昂贵”的想法变得可行。这“将思维从对实现细节的过多担忧中解放出来”。 * **优势的民主化（批判）：** Ben承认，如果大型语言模型使得特征创建对所有人来说都更便宜，那么这种优势可能会更快地被侵蚀。 * **Two Sigma的应对：** 由于其深厚的经验和系统化的方法，公司已经“做好了充分准备”（shovel ready），能够应对这种技术。虽然数据“创建”的能力可能民主化，但“然后怎么办”（如何有效利用它）仍然是基于经验的关键区别。 **8. 降低输出熵（同质化）的风险：** * **危险：** 生产线自动化（例如，纸箱生产）要求输出完全相同。然而，对于阿尔法生成，*正交性*和*原创性*至关重要。如果人工智能工具导致每个人都生成相同的信号，就会使投资组合同质化并侵蚀阿尔法。 * **Two Sigma的策略：** 将人工智能视为原创性的“放大工具”。工具应该赋予研究人员倍增他们独特见解的能力。如果两位背景不同的研究人员使用相同的工具，它应该能产生不同的、具有上下文感知能力的答案，从而强化了思维的多样性而非使其同质化。 **9. 权衡：迎难而上与等待能力成熟：** * **突破界限：** 重要的是“比工具出现的速度更快地推动界限”，以了解局限性（“看到墙”）。这样，当新工具出现时，研究人员就能确切知道该将它们应用于何处。 * **竞争优势：** 等待12-18个月让人工智能能力成熟可能会节省工程投入，但率先将新的阿尔法信号推向市场，能带来显著的（尽管可能是短暂的）优势。 **10. 基础设施选择（本地部署与供应商模型）：** * **多样化：** 关键在于避免被任何单一方法所束缚。 * **灵活性：** 技术堆栈必须允许在供应商、外部或内部模型之间进行切换。 * **生命周期控制：** 一个主要担忧是供应商模型被停止或升级，这迫使公司不断适应。内部模型提供更多控制。 * **未来信息偏差：** 不了解模型的训练数据（尤其是外部模型）可能导致虚假关联或偏差（例如，一个基于安然事件后数据训练的LLM，可能会将任何关于安然的信息都贴上“坏”的标签，即使是查询历史数据时也是如此）。 * **平衡：** 在内部模型的控制力与前沿实验室令人瞩目的创新速度之间取得平衡至关重要。 **11. P值操纵（P-Hacking）和人工智能的过拟合：** * **自然保障：** 无法创建能够预测未来（因为过拟合了过去）模型的 H研究人员，自然会面临职业挫折。 * **机构措施：** Two Sigma拥有长期存在的工具、统计测试和最佳实践，以对抗 P值操纵和过拟合，这在规模化实验中非常重要。 * **过拟合的细微之处：** 所有模型“从定义上讲都存在过拟合”，因为未来永远不会与过去完全相同。目标是在“可接受且可衡量”的程度上进行过拟合，以确保交易环境的稳健性。 * **人工智能的影响：** 人工智能由于其可实现的实验数量庞大，*加剧了* P值操纵的风险。因此，随着规模的扩大，教育、检查和稳健性方面的警惕性也需提高。 **12. 数据升级与模型重建：** * **学术与商业思维：** 在学术界，目标通常是在一个项目上达到100%的完美。而在Two Sigma这样多元化投资组合的背景下，“在该过程中最后10%的收益，却需要90%的工作量。” * **收益递减：** 通常，完成10个90%的想法比完成1个95%的想法更有益。一个优秀的量化分析师知道“何时放手”。 * **投资回报率驱动的决策：** 当数据源（例如，供应商V3升级到V4）或算法改进时，重建或重新训练现有模型的决策是以商业投资回报率的视角进行的。通常，更好的做法是探索新的、正交的信号，而不是对一个已经表现良好的模型进行边际改进。 **13. 规模化特异性（人工智能前沿）：** * **激发特异性：** Ben用“规模化特异性”来鼓励进行详细的、特定公司的分析，这在传统上由于样本量小而受到规避。 * **人工智能的作用：** 人工智能提供了独特的能力，可以深入挖掘特定公司的数据，构建高度定制化的特征。这些定制化的洞察力可以同时推广到成千上万的其他公司。 * **模仿主观投资：** 人工智能使Two Sigma这样的规模化公司能够模拟主观投资者通常发现的“深层知识”和“公司特定机会”，这些信息通常不会出现在广泛聚合的数据中。这使得对公司数据中那些定制化且难以建模的部分进行更深入的探索成为可能。 **14. 初级工程师的未来技能：** * **价值转移：** 技术编程技能的相对价值将下降。阐述想法并“告诉人工智能该做什么”（提示工程）的能力将增加。 * **主要建议：** 1. 熟悉人工智能及其功能；成为“人工智能原生”。 2. 将人工智能视为个人独特技能和想法的“放大工具”，而不是替代品。 3. 专注于培养原创性、正交性和创造力。 4. 想法以及有效利用人工智能来实现这些想法将是至关重要的。 **15. Ben的个人爱好：** * **公共数据和政府发布的信息：** Ben对大量的公共数据（停车罚单、超速记录、餐厅健康检查）及其改善城市和社会的潜力感到着迷。 * **人工智能对公共数据的影响：** 他认为，人工智能能够使这些信息的获取民主化，消除了对专业数据科学技能的需求，这令人鼓舞。城市规划师、律师和普通大众都可以直接提问，从而使政府数据更易于获取和利用。 * **Corey的幽默反驳：** 随着这些数据变得更容易获取，它可能会凸显政府的低效，从而可能导致数据被撤回。

This episode of "Flirting with Models," hosted by Corey Hofstein, features Ben Wellington, Head of Complex Feature Engines at Two Sigma. The conversation delves into the human factor behind quantitative strategy, specifically focusing on feature generation and the transformative impact of AI in this space. Here's a detailed summary of the discussion: **1. Ben Wellington's Background and Role:** * **Education:** Ben has a PhD in Natural Language Processing (NLP) from New York University, specializing in machine translation during an era (pre-Siri/Alexa) when such applications were novel. He chose the field because it "sounded cool." * **Career at Two Sigma:** He joined Two Sigma in 2007 when it was a smaller firm (125 people). His initial role was on the data engineering team, focused on collecting, ingesting, and cleaning data. * **Shift to Modeling:** Ben discovered interesting datasets and performed an internal "political analysis of news places," which caught the attention of a modeler at Two Sigma. This led him to shift his focus to studying how textual data predicts markets, a role he's held for over 10 years, aiming to predict asset movements (up/down) over various time horizons. * **Current Title:** Head of Complex Feature Engines, highlighting his core focus. **2. Defining "Feature" in Quant Investing:** * **Core Concept:** A "feature" is a "fact about the world that is worth itself noting," and for Two Sigma, it must be "economically meaningful." * **Examples:** * **Familiar:** Change in stock price over the last week, stock price movement since last year, volatility. * **More Interesting:** An analyst recently upgrading a stock. * **Textual/Alternative (Ben's expertise):** Amount of news coverage about a company compared to its history, the day/time a press release is issued (e.g., Friday after hours). * **Purpose:** Features encapsulate hypotheses. They layer human intelligence and intuition onto raw data, helping computers focus on what's potentially meaningful, rather than just throwing all raw data at an algorithm. This makes the job tractable and leverages human knowledge. **3. Two Sigma's Operating Model vs. Pod Shops:** * **Shared Platform Model:** Unlike pod shops where teams (pods) compete, Two Sigma operates on a model where modelers' predictions are centralized and aggregated for the firm's larger portfolios. Their goal is not to control a specific portfolio, but to make the best possible predictions. * **Global Optimization:** This structure encourages "global optimization" rather than "local optimization" (as in pod shops). Teams are incentivized to elevate their best work (features, algorithms) to a shared platform for others to use, making the overall firm better. This is described as a "cyclical" process where teams constantly push improvements to the platform. **4. Where Alpha Lives (Data, Features, Forecasting):** * Ben identifies three potential silos for alpha: 1. **Data Access:** Its proportional weight has "dropped" due to increased commoditization over the last 15-20 years. While Two Sigma still has proprietary data (long history, crowdsourced trading ideas), the edge from *just* data access is less than before. 2. **Feature Generation (Ben's Focus):** This is "just huge." The ability to creatively and uniquely transform data into insightful observations is a massive edge, driven by "human ingenuity." Ben gives the example of studying analysts: beyond simple upgrades, considering their location, school, connection to CEOs, etc. Each tiny, clever question (e.g., 200 questions = 200 features) collectively creates a significant alpha signal. 3. **Forecasting/ML Algorithms:** This is also a "very big part," with a dedicated "techniques" team focusing on advanced algorithms to extract maximum information from features. * **Synergy:** The intersection of feature-focused teams (identifying clever data) and techniques teams (applying advanced algorithms) is where "amazing things happen," driven by diversity of approach. **5. Ideating Features and Managing Challenges:** * **Ideation Process:** It requires "persistence" and a human layer to discern what's meaningful (e.g., analyst's eye color is likely irrelevant, but their training or location might be). It involves diversity of thought from various scientific backgrounds. * **Generalizing Instances:** Taking real-world events (e.g., a train derailment affecting a company) and asking "Is this generalizable?" to create broader features. * **Scientific Method:** Adhering to the scientific method by stating hypotheses, testing them, and not post-hoc rationalizing results that contradict priors. * **Collinearity:** This is "the death of a signal." Researchers constantly search for *orthogonality* to avoid features that merely re-explain existing signals (e.g., news sentiment that just tracks post-earnings reports). Tools and thresholds are used to ensure new alpha signals have sufficient orthogonality. * **Forecast Horizon:** Researchers are mindful of the time horizon relevant to a feature (e.g., store visit data might predict sales months out). * **Feature Lifecycle:** As more market participants act on a signal, its forecast horizon can speed up, potentially eroding its viability. Two Sigma aims for creative features that resist this pressure and monitors for "alpha decay." **6. Hiring for Creativity and Ingenuity:** * **Beyond Technical Skills:** While math, stats, and coding are important, feature engineering requires ingenuity, creative, and lateral thinking. * **Interview Approach:** Open-ended questions are used to assess creativity, such as "If you had access to a social media firehose, what could you predict?" The goal is to see how candidates generate novel, interesting ideas without a fixed "right" answer. **7. AI/LLMs and the Future of Feature Generation:** * **Ben's Core Insight:** "NLP was about turning language into numbers, with LLMs I can turn numbers into language. Anything can be language now." * **Impact:** LLMs *generate textual data*, which is a "revolution of opportunity" for Two Sigma, a firm with 15+ years of experience trading textual data. Proprietary data now includes *generated* data. * **Cost Collapse:** LLMs dramatically reduce the cost and time investment for exploring ideas. A hypothetical example: previously, studying CEO blinking for predictive value (requiring computer vision, 6 months of work) had low ROI. Now, LLMs can make such explorations 100x faster, making previously "too expensive" ideas viable. This "frees the mind from having to worry so much about implementation." * **Democratization of Edge (Critique):** Ben acknowledges that if LLMs make feature creation cheaper for everyone, the edge could erode faster. * **Two Sigma's Counter:** The firm is "shovel ready" due to its deep experience and systematic approach to using such technologies. While the ability to *create* data might be democratized, the "then what" (how to use it effectively) remains a key differentiator rooted in experience. **8. Risk of Lowering Entropy of Output (Homogenization):** * **The Danger:** Automation for a production line (e.g., cardboard boxes) requires identical output. However, for alpha generation, *orthogonality* and *originality* are crucial. If AI tools lead everyone to generate the same signals, it homogenizes portfolios and erodes alpha. * **Two Sigma's Strategy:** View AI as an *amplification tool* for originality. Tools should empower researchers to multiply their unique insights. If two researchers with different backgrounds use the same tool, it should produce different, context-aware answers, reinforcing diversity of thought rather than homogenizing it. **9. Trade-off: Leaning into Difficulty vs. Waiting for Capability:** * **Pushing Boundaries:** It's important to "push the boundaries faster than the tools come" to understand limitations ("see the wall"). This way, when new tools emerge, researchers know exactly where to apply them. * **Competitive Advantage:** Waiting 12-18 months for AI capabilities to mature might save engineering effort, but being first in the market with new alpha signals provides a significant, potentially short-lived, advantage. **10. Infrastructure Choices (On-Premise vs. Vendor Models):** * **Diversification:** The key is to avoid being held hostage by any single approach. * **Flexibility:** The technical stack must allow for swapping between vendor, external, or internal models as needed. * **Control over Lifecycle:** A major concern is vendor models being discontinued or upgraded, forcing continuous adaptation. Internal models offer more control. * **Look-Ahead Bias:** Not understanding a model's training data (especially external ones) can lead to spurious correlations or biases (e.g., an LLM trained on post-Enron data might inherently label anything about Enron as "bad," even when querying historical data). * **Balance:** Balancing the control of internal models with the impressive pace of innovation from frontier labs is crucial. **11. P-Hacking and Overfitting with AI:** * **Natural Safeguards:** Researchers failing to create models that predict the future (due to overfitting the past) will naturally face career setbacks. * **Institutional Measures:** Two Sigma has long-standing tools, statistical tests, and best practices to combat p-hacking and overfitting in scaled experiments. * **Nuance of Overfitting:** All models "overfit by definition" because the future is never exactly like the past. The goal is to overfit to a "comfortable and measurable" degree, ensuring robustness to trading environments. * **AI's Impact:** AI *exacerbates* the risk of p-hacking due to the sheer volume of experiments possible. Therefore, increased vigilance in education, checks, and robustness is necessary as scale increases. **12. Data Upgrades and Model Rebuilding:** * **Academic vs. Business Mindset:** In academia, the goal is often 100% perfection on one project. In a diverse portfolio context like Two Sigma, "the last 10% of gains in that journey is 90% of the work." * **Diminishing Returns:** It's often more beneficial to pursue 10 different ideas at 90% completion than one idea at 95% completion. A good quant knows "when to move on." * **ROI-Driven Decisions:** When data sources (e.g., vendor V3 to V4) or algorithms improve, the decision to rebuild or retrain an existing model is made with a business ROI lens. Often, the time is better spent exploring new, orthogonal signals rather than marginally improving an already well-performing model. **13. Idiosyncrasy at Scale (AI Frontier):** * **Inspiring Specificity:** Ben uses "idiosyncrasy at scale" to encourage detailed, company-specific analysis, traditionally avoided due to small sample sizes. * **AI's Role:** AI offers the unique ability to dive deeply into a specific company's data, building highly tailored features. These bespoke insights can then be generalized across thousands of other companies simultaneously. * **Mimicking Discretionary Investing:** AI enables a scale-oriented firm like Two Sigma to model the "deep knowledge" and "company-specific opportunities" often identified by discretionary investors, which typically don't show up in broad, aggregated data. This allows for deeper exploration of "bespoke and hard-to-model parts of company data." **14. Skills for the Future for Junior Engineers:** * **Shift in Value:** Technical coding skills will decrease in relative value. The ability to articulate ideas and "tell the AI what to do" (prompting) will increase. * **Key Advice:** 1. Get comfortable with AI and its capabilities; become "AI native." 2. View AI as an *amplification tool* for unique personal skills and ideas, rather than a replacement. 3. Focus on developing originality, orthogonality, and creativity. 4. Ideas and the effective use of AI to implement them will be paramount. **15. Ben's Personal Obsession:** * **Public Data and Government Releases:** Ben is fascinated by the wealth of public data (parking tickets, speeding stops, restaurant health inspections) and its potential for improving cities and society. * **AI's Impact on Public Data:** He finds it inspiring that AI can democratize access to this information, removing the need for specialized data science skills. Urban planners, lawyers, and the general public can ask questions directly, making government data more accessible and useful. * **Corey's Humorous Counter:** As this data becomes more accessible, it might highlight government inefficiencies, potentially leading to the data being pulled.