Who Feeds the GPUs? Inside AI's Hidden $30B Layer | Renen Hallak, VAST Data

Everyone talks about GPUs. Almost nobody talks about the layer that feeds them. Renen Hallak is the founder & CEO of VAST Data — the $30 billion company powering xAI and some of the world's biggest AI clouds — and he sits in the hidden layer of the AI stack. In this episode, we cover what an AI factory actually is, why every company will eventually own its own AI, the architecture bet behind VAST (DASE, explained simply), KV caches and agent memory, and DataEnclave — VAST's brand-new confidential AI announcement with NVIDIA that lets leading models run on the world's most sensitive data. Plus: the demand signal that scares even him (a customer went from 500 petabytes to 2 exabytes), circular financing, which neoclouds survive, sovereign AI, working with NVIDIA and Elon Musk's xAI — and why the next 10 years will bring more change than the last 1,000. Renen Hallak LinkedIn — https://www.linkedin.com/in/renenh/ VAST Data Website — https://www.vastdata.com/ X / Twitter — https://x.com/VAST_Data LinkedIn — https://www.linkedin.com/company/vast-data/ Matt Turck (General Partner) Blog - https://mattturck.com LinkedIn - https://www.linkedin.com/in/turck/ X - https://x.com/mattturck FirstMark Capital Website - https://firstmark.com X - https://x.com/FirstMarkCap Listen on: Spotify - https://open.spotify.com/show/7yLATDSaFvgJG80ACcRJtq Apple - https://podcasts.apple.com/us/podcast/the-mad-podcast-with-matt-turck/id168623872 00:00 Intro 00:51 The hidden software layer in NVIDIA's AI stack 02:28 What actually makes an "AI factory"? 05:13 Should Walmart and Goldman Sachs build their own AI? 06:20 "We infer during the day, fine-tune at night" 13:16 The announcement: models become a resource to manage 15:32 From P vs. NP to founding VAST Data 17:32 OpenAI, Navier–Stokes and 10,000 collaborating agents 20:25 The pre-transformer insight behind VAST 21:55 DASE: VAST's "shared everything" architecture explained 25:18 "Storage was where startups go to die" 27:39 Trillions of vectors: why old databases break 29:01 Are S3, Snowflake and Databricks ready for AI? 31:29 Data gravity, vendor lock-in and zero churn 33:13 Training vs. inference: why the infrastructure changes 34:46 Model routing, KV caches, RAG and agent memory 36:59 Identity, permissions and security for AI agents 40:27 Can multi-agent systems unlock scientific discovery? 41:55 DataEnclave: how confidential AI protects data and weights 45:16 Who should be AI's trust layer? 46:37 "Sometimes it scares me": 500 petabytes to 2 exabytes 50:08 Is circular AI financing creating systemic risk? 51:32 Why VAST is profitable when AI infra isn't 53:24 What separates the winning neoclouds? 55:06 "Their lunch is being eaten": why hyperscalers lag 59:10 Where will the trillions accrue across the AI stack? 1:01:18 NVIDIA: "There's no legal document between us" 1:03:59 What VAST learned from xAI and Elon Musk 1:05:36 "Bad things loudly and often": building at AI speed 1:06:53 More change in 10 years than the previous 1,000? 1:08:39 VAST's endgame: all the data in the world

GPT-4正在为你翻译摘要中......

翻译全文