在这个播客节目中，Lenny Rachitsky 采访了 OpenAI 的首席产品官 Kevin Weil，讨论了公司的运营、人工智能的影响以及如何在人工智能生态系统中构建产品。Weil 强调了人工智能发展的快速步伐，指出人工智能模型不断改进，以至于现在的模型是“你未来一生将使用的最糟糕的人工智能模型”。这种持续变化要求产品构建者快速适应并以不同的方式思考他们的工作。 Weil 讨论了 OpenAI 的内部文化，强调其专注于快速行动和拥抱实验。他强调了授权的、自下而上的团队的重要性，以及接受错误是学习过程的一部分。他提到了 OpenAI 的迭代部署理念，即他们尽早发布产品，与公众共同发展，并根据用户反馈快速迭代。 一个中心主题是“evals”（评估）在人工智能产品开发中的日益重要性。Evals 是用于衡量模型在特定任务或主题材料上的表现的测试。Weil 认为，编写 evals 正在成为产品经理和人工智能开发人员的一项核心技能，因为它们有助于确定模型的优势和劣势。了解模型的性能（例如，60% 的准确率与 95% 或 99.5%）对于设计合适的产品至关重要。定制的 evals 对于衡量公司特定或用例特定场景中的性能至关重要。 Weil 消除了人们对 OpenAI“扼杀”初创公司的担忧，他强调公司外部聪明的人比内部多。他还指出，OpenAI 的重点是构建一个强大的 API，以赋能各个行业的开发人员。这为初创公司创造了巨大的机会，可以构建改进现有技术水平的基于人工智能的产品。 Weil 强调了人工智能产品开发的一些反直觉的方面，例如将人工智能视为人类进行推理。这种方法对于设计用户界面和交互模式来说可能出奇地有效。他还认为，“聊天”将成为人工智能的多功能界面，因为它具有多功能性，并且可以适应各种交流方式。 展望未来，Weil 预测人工智能将深入整合到产品的各个方面。这将导致微调过程，因此研究人员将被纳入产品团队以进行持续改进。他还建议，人们将越来越关注针对特定应用进行定制调整的模型。这将是对定制训练和其他工具日益普及的回应。 关于未来，Weil 对人工智能的影响持乐观态度，同时他也承认对工作岗位流失和其他挑战的担忧。他对个性化辅导非常乐观。人工智能将使辅导更易于获得，并为所有人降低成本。然而，他认为总体而言，技术推动着经济和地缘政治的进步。他还认为优质教育可以改变世界。 Weil 提到了 Facebook 的 Libra 加密货币的失败，表示由于监管障碍和 Facebook 的声誉，它从未发布而感到失望。他希望 Meta 现在可以重新考虑构建它。 对话以一个闪电问答结束，涵盖了 Weil 最喜欢的书籍、电视节目和产品。他建议持续不断地完成伟大的工作，因为他相信要专注于每天的工作，而不是最终的结果。关于提示的最后提示是，给模型提供你想要的东西的例子，它们会听从你的指示并学习，你不需要成为提示方面的专家就能得到你想要的东西。

In this podcast episode, Lenny Rachitsky interviews Kevin Weil, Chief Product Officer at OpenAI, about the company's operations, the implications of AI, and how to build products in the AI ecosystem. Weil emphasizes the rapid pace of AI development, stating that AI models are constantly improving, rendering the current models "the worst AI model you will ever use for the rest of your life." This constant change requires product builders to adapt quickly and think differently about their work. Weil discusses OpenAI's internal culture, highlighting its focus on moving fast and embracing experimentation. He stresses the importance of empowered, bottoms-up teams and the acceptance of mistakes as part of the learning process. He mentioned OpenAI's iterative deployment philosophy, where they ship products early, co-evolve with the public, and rapidly iterate based on user feedback. A central theme is the growing importance of "evals" (evaluations) in AI product development. Evals are tests used to gauge how well a model performs on specific tasks or subject material. Weil argues that writing evals is becoming a core skill for product managers and AI developers, as they help determine a model's strengths and weaknesses. Understanding a model's performance (e.g., 60% accuracy vs. 95% or 99.5%) is critical for designing appropriate products. Custom evals are vital for measuring performance in company-specific or use-case-specific scenarios. Weil dispels concerns about OpenAI "squashing" startups, emphasizing that there are more smart people outside the company than inside. He also notes that OpenAI's focus is on building a robust API to empower developers across various industries. This creates immense opportunities for startups to build AI-based products that improve upon the state-of-the-art. Weil highlights some counterintuitive aspects of AI product development, such as reasoning about AI as if it were a human. This approach can be surprisingly effective for designing user interfaces and interaction patterns. He also suggests that "chat" will be a versatile interface for AI due to its versatility and ability to accommodate diverse communication styles. Looking ahead, Weil predicts that AI will become deeply integrated into every facet of the products. This will result in the fine tuning process, therefore researchers will be included into the product teams for continuous improvement. He also suggested that there'll be an increasing focus on custom-tuned models for specific applications. This will be a response to the growing accessibility to custom training and other tools. Regarding the future, Weil is optimistic about the impact of AI, acknowledging concerns about job displacement and other challenges. He is very optimistic towards personalized tutoring. AI will make tutoring more accessible and cheaper for all. However, he believes that technology, in general, drives economic and geopolitical advancements. He also believes that quality education can change the world. Weil touches on the failure of Facebook's Libra cryptocurrency, expressing disappointment that it never launched due to regulatory hurdles and Facebook's reputation. He hopes that Meta might reconsider building it now. The conversation concludes with a lightning round covering Weil's favorite books, TV shows, and products. He advises to do great work over a sustained period of time, as he believes in doing the daily work and not focusing on the final result. His final tip about prompting is give models examples of the kinds of things you want and they will listen to you and learn and you do not need to be an expert in prompting to get what you want.