AI Safety Fact vs. Fiction | YC Paper Club

This week’s Paper Club is all about AI safety - not the headlines, the science. AI safety has been a research field for decades, with its own conferences, methods, and a track record in systems like aircraft collision avoidance, where the standard is one catastrophic failure per billion flight hours. In this episode we'll hear from Clark Barrett and Mykel Kochenderfer, Stanford professors and co-directors of the Stanford Center for AI Safety.  Together with Francois they'll discuss what actually happened in the OpenAI–Hugging Face incident versus what got reported, how automated reasoning can mathematically prove that code cannot be hacked, how the stress-testing methods built to validate aircraft are now being turned on language models, and why we might need an NTSB for AI. Join us for the next Paper Club on October 21st, where we'll talk Inference: https://events.ycombinator.com/yc-paperclub-oct21 Chapters: 00:00 - Science fiction vs. science fact 04:32 - Engineering problem, or P(doom) 06:04 - What actually happened at Hugging Face 07:53 - Where "10% in 10 years" comes from 11:21 - The real risk: open weights in the wrong hands 17:25 - Clark Barrett: when AI met automated reasoning 20:35 - CSLib: proving code correct in Lean 24:32 - From English spec to verified code in one click 27:55 - Mykel Kochenderfer: how do we know a system is safe 32:11 - Adaptive stress testing, from aircraft to LLMs 36:57 - Verifying neural networks 40:30 - Runtime monitoring and BetterBench 43:53 - Panel: reacting to the past three weeks 45:55 - An NTSB for AI 52:18 - Is cybersecurity solvable in five years 55:00 - If you were CISO of Bank of America 1:03:06 - The Swiss cheese model 1:05:43 - Audience Q&A Apply to Y Combinator: https://www.ycombinator.com/apply Work at a startup: https://www.ycombinator.com/jobs

GPT-4正在为你翻译摘要中......

翻译全文