首页 >> 来自播客: Investment videos - YouTube 更新反馈

Mark Zuckerberg - Llama 3, $10B Models, Caesar Augustus, & 1 GW Datacenters

发布时间 2024-04-19 00:13:14 来源

摘要

Zuck on: - Llama 3 - open sourcing towards AGI - custom silicon, synthetic data, & energy constraints on scaling - Caesar Augustus, intelligence explosion, bioweapons, $10b models, & much more Enjoy! Timestamps 00:00:00 Llama 3 00:09:15 Coding on path to AGI 00:26:07 Energy bottlenecks 00:34:03 Is AI the most important technology ever? 00:38:04 Dangers of open source 00:54:40 Caesar Augustus and metaverse 01:05:36 Open sourcing the $10b model & custom silicon 01:16:02 Zuck as CEO of Google+ Links Apple Podcasts: https://podcasts.apple.com/us/podcast/mark-zuckerberg-llama-3-open-sourcing-%2410b-models-caeser/id1516093381?i=1000652877239 Spotify: https://open.spotify.com/episode/6Lbsk4HtQZfkJ4dZjh7E7k?si=GOqj7hUdSaWSgi7ULWXjMA Transcript: https://www.dwarkeshpatel.com/p/mark-zuckerberg Me on Twitter: https://twitter.com/dwarkesh_sp Sponsors If you’re interested in advertising on the podcast, fill out this form: https://airtable.com/appxGOvFLDLP5dlzv/pagFVrbHRohW6F2bZ/form - This episode is brought to you by Stripe, financial infrastructure for the internet. Millions of companies from Anthropic to Amazon use Stripe to accept payments, automate financial processes and grow their revenue. Learn more at https://stripe.com/ - V7 Go is a tool to automate multimodal tasks using GenAI, reliably and at scale. Use code DWARKESH20 for 20% off on the pro plan. Learn more at https://www.v7labs.com/go?utm_campaign=Dwarkesh%20Podcast%20Newsletter&utm_source=Dwarkesh-Podcast&utm_medium=Newsletter&utm_term=Paid-Email - CommandBar is an AI user assistant that any software product can embed to non-annoyingly assist, support, and unleash their users. Used by forward-thinking CX, product, growth, and marketing teams. Learn more at https://www.commandbar.com/

GPT-4正在为你翻译摘要中......

中英文字稿

That's not even a question for me, whether we're going to go take a swing at building the next thing. I'm just incapable of not doing that. There's a bunch of times when we wanted to launch features, and then Apple's just like, nope, you're not launching that. It's just like, that sucks. Are we set up for that with AI, where you're going to get a handful of companies that run these closed models that are going to be in control of the APIs, and therefore going to be able to tell you what you can build? Then when you start getting into building a data center that's like 300 megawatts, or 500 megawatts, or a gigawatt, just no one has built a single gigawatt data center yet. From wherever you sit, there's going to be some actor who you don't trust if they're the ones who have like the super strong AI. I think that that's potentially a much bigger risk. Mark, welcome to the podcast. Hey, thanks for having me. Big fan of your podcast. Oh, thank you. That's very nice of you to say. Okay, so let's start by talking about the releases that will go out when this interview goes out. Tell me about the models. Tell me about MedAI. What's new? What's exciting about them? Yeah, sure. I think the main thing that most people in the world are going to see is the new version of MedAI. Right? The most important thing about what we're doing is the upgrade to the model. We're rolling out Lama3. We're doing it both as open source for the dev community, and it is now going to be powering MedAI. So there's a lot that I'm sure we'll go into around Lama3, but I think the bottom line on this is that with Lama3, we now think that MedAI is the most intelligent AI assistant that people can use that's freely available. We're also integrating Google and Bing for real-time knowledge. We're going to make it a lot more prominent across our apps. So basically, at the top of WhatsApp and Instagram and Facebook and Messenger, you'll just be able to use the search box right there to ask any questions. There's a bunch of new creation features that we added that I think are pretty cool that I think people enjoy.

对我来说，这甚至不是一个问题，我们是否要着手构建下一个项目。我根本无法不这样做。有很多次我们想要推出功能，然后苹果就会说，不，你不能推出那个。这真是糟透了。我们是否已做好准备应对人工智能，那些运行封闭模型的少数公司将控制API，从而告诉你可以构建什么？当你开始建造一个300兆瓦、500兆瓦或一吉瓦的数据中心时，迄今为止还没有人建造过一个吉瓦的数据中心。无论你身在何处，总会有一些你不信任的行动者，如果他们是拥有超强人工智能的那些人。我认为这可能是一个更大的风险。马克，欢迎来到播客。嘿，谢谢邀请我。我很喜欢你的播客。哦，谢谢你这么说，太好了。好的，让我们从谈论这次采访发布时会放出的新版本开始。告诉我关于这些模型。告诉我关于MedAI。它们有什么新的？有什么令人兴奋的？是的，当然。我认为全世界大部分人会看到的主要事情是MedAI的新版本。对我们正在做的最重要的事情是对模型的升级。我们正在推出Lama3。我们将其作为开源供开发社区使用，现在它将为MedAI提供动力。因此，我相信我们将围绕Lama3展开讨论，但对此的底线是，有了Lama3，我们现在认为MedAI是人们可以使用的最智能的免费AI助手。我们还在集成Google和必应的实时知识。我们将在我们的应用程序中更加突出地展示它。因此，基本上，在WhatsApp、Instagram、Facebook和Messenger的顶部，你只需使用搜索框就可以立即提出任何问题。我们添加了许多新的创作功能，我认为这些功能相当酷，我认为人们会喜欢。

I think animations is a good one. You can basically just take any image and animate it. But I think one that people are going to find pretty wild is it now generates high-quality images so quickly. I don't know if you've gotten a chance to play with this, that it actually generates it as you're typing and updates it in real-time. So you're typing your query and it's honing in on. It's like, okay, here, show me a picture of a cow in a field with mountains in the background. It's just like everything's populating. You didn't get any nice drinking beer. And it's updating the image in real-time. It's pretty wild. I think people are going to enjoy that. So that, I think, is that's what most people are going to see in the world. We're rolling that out. Not everywhere, but we're starting in a handful of countries and we'll do more over the coming weeks and months. So that, I think, is going to be a pretty big deal. And I'm really excited to get that in people's hands. It's a big step forward for MedAI. But I think, if you want to get under the hood a bit, the llama 3 stuff is obviously the most technically interesting. So we're basically, for the first version, we're training three versions. You know, an 8 billion and a 70 billion, which we're releasing today. And a 405 billion dense model, which is still training. So we're not releasing that today. But the 8 in 70, I mean, I'm pretty excited about how they turned out. I mean, they're leading for their scale. You know, it's, I mean, we'll release a blog post with all the benchmarks that people can check it out themselves. And obviously, it's an open source, so people get a chance to play with it.

我认为动画是一个很好的选择。你基本上可以拿任何图像来制作动画。但我认为人们会觉得很惊奇的一点是它现在可以快速生成高质量的图像。我不知道你是否有机会尝试过这个功能，它实际上是在你输入时生成图像，并实时更新。所以当你输入你的查询时，它会逐步完善。比如，当你要求展示一张有山丘背景的田地里有一头牛的图片时，所有东西都会被填充进去。你没有得到一个漂亮的喝啤酒的图片。并且图像会实时更新。这相当惊人。我认为人们会喜欢这个功能。这将是大多数人在世界上看到的。我们正在推出这个功能。虽然不是所有地区，但我们正从一些国家开始，接下来的几周和几个月我们会开展更多。我认为这将是一个相当重要的事件。我很期待人们使用它。这是MedAI迈出的一大步。但我认为，如果你想深入了解一下，llama 3可能是最有技术含量的部分。所以我们基本上在第一个版本中训练了三个模型。你知道，一个是80亿，一个是700亿，这两个我们今天发布。还有一个4050亿的密集模型，目前正在训练，所以今天不会发布。但80亿和700亿，我对它们的成果感到非常兴奋。就他们各自的规模来看，它们是领先的。我们将发布一篇博客文章，里面有所有的基准测试供人们自己检验。而且它是开源的，所以人们有机会尝试使用它。

We have a roadmap of new releases coming that are going to bring multi modality, more multi-linguality, bigger context windows to those as well. And then, you know, hopefully, sometime later in the year, we'll get to roll out the 405, which I think is, is, you know, in training, it's still training. But for where it is right now in training, it is already at around 85 MMOU. And just, we expect that it's going to have leading benchmarks on a bunch of the benchmarks. So I'm pretty excited about all that. I mean, the 70 billion is great too. I mean, we're releasing that today. It's around 82 MMOU and has leading scores on math and reasoning. So I mean, it's, I think just getting this in people's hands is going to be pretty wild. Oh, interesting. Yeah, that's the first time here. That's super impressive. Yeah. And it'll be billion is, the 8 billion is, um, is nearly as, as powerful as the biggest version of llama two that we released. So it's like the smallest llama three is basically as powerful as the biggest llama two. Okay.

我们有新版本的路线图即将推出，将为用户带来多模态、更多语言和更大的上下文窗口。然后，希望在今年晚些时候，我们将开始推出 405，我认为它目前仍处于训练阶段。但就目前的训练进展来看，它的MMOU已经达到了大约85。我们预计它将在许多基准测试上达到领先水平。我对所有这些都感到非常兴奋。70亿也很棒。我们今天将发布它。它的MMOU约为82，并在数学和推理方面取得了领先成绩。我认为将这些交到用户手中会非常激动人心。哦，有趣。这是我第一次听说。真是令人印象深刻。而且，这80亿几乎与我们发布的llama two最大版本一样强大。所以说，最小的llama three基本上就像是最大的llama two一样强大。

So before we dig into these models, I actually want to go back in time. 2022 is, I'm assuming when you started acquiring these H 100s, um, or you can tell me when, uh, we were like, stock prices getting hammered. People are like, what's happening with all this cat, because people aren't buying the metaverse. And presumably you're spending that cat, but you get these H 100s. How back then, how did you know to get the H 100s? How did you know we'll need the GPUs? Um, I think it was, it was because we were working on reels. So, you know, we got into this situation where, um, you know, we always want to have enough capacity to build something that we can't quite see that were on the horizon yet. Um, and we got into this position with reels where we needed more GPUs to train the models, right? It was, it was this big evolution for our services where instead of just ranking content from people who you follow or your friends and whatever pages you follow, um, we made this big push to basically start recommending what we call unconnected content, basically connected content from people or pages that you're not following.

因此，在我们深入研究这些模型之前，我实际上想回到过去。2022年，我假设您开始收购这些H100，或者您可以告诉我什么时候，我们当时的股价受到打击。人们都在问，这些猫到底发生了什么，因为人们并没有购买元宇宙。而你显然正在花费这些猫，但你得到了这些H100。那时候，你是如何知道要买H100的？你怎么知道我们需要GPU？我想是因为我们当时正致力于短视频。所以，你知道，我们总是想要足够的容量来构建一些我们尚不能看到的东西，但这在地平线上却已经出现了。我们遇到了这种情况，我们需要更多的GPU来训练模型，对吧？这对我们的服务来说是一个巨大的进步，我们不再只是对你关注的人或朋友以及你关注的页面的内容进行排名，我们做出了巨大努力，基本上开始推荐我们称为不相关内容的内容，即你不关注的人或页面的相关内容。

So now kind of the, the corpus of, of kind of content candidates that we could potentially show you expanded from, you know, on the order of thousands to on the order of hundreds of millions. So completely different infrastructure. And we, um, started working on, on doing that. And we were constrained on, um, on basically the infrastructure that we had to catch up to what TikTok was doing as quickly as we would have wanted to. Um, so I basically looked at that and I was like, Hey, we have to make sure that we're never in this situation again. So let's order enough GPUs to do what we need to do on reels and ranking content and feed. But let's also, let's double that. Right. cause again, like our normal principle is there's going to be something on the horizon that we can't see. As you know, it would be a, um, well, we thought it would be, we thought it was going to be something that I had to do with training large models. Right. I mean, but at the time I thought it was probably going to be more something that I had to do with content, but I don't know. I mean, it's, it's almost just the pattern matching and running the company is there's always another thing.

所以现在，我们可以潜在展示给你的内容候选者的数量从几千个扩大到数亿个。所以基础设施完全不同了。我们开始着手解决这个问题。我们受到约束，基本上是我们必须赶上TikTok所做的一切，但我们希望能更快地赶上。所以我看了看，我想，嘿，我们必须确保再也不会遇到这种情况了。所以让我们订购足够的GPU来在Reels上做我们需要做的事情，排名内容和提供内容。但同时，我们也加倍数量。因为我们的常规原则是，总有一天会有我们看不见的东西出现在地平线上。正如你知道的，这可能是，我们认为会是，我们认为可能会涉及到训练大型模型。但当时我认为可能更多的是与内容有关，但我不知道。我是说，公司的运营几乎都是在不断匹配模式，永远都有下一个问题需要解决。

Right. So I'm not even sure I had, at that time I was so deep and just, you know, trying to get, you know, the recommendations working for reels and other content. Cause I mean, that's just such a big unlock for Instagram and Facebook and now being able to show people content that's interesting to them that they're from people that they're not even following. But um, yeah, I, that, that ended up being a very good decision in retrospect. Yeah. Yeah. Okay. And it came from being behind. So then it wasn't like I was, you know, I wasn't like, oh, I was so far ahead. Actually, most of the times I think where we kind of make some decision that ends up seeming good is because we messed something up before and just didn't want to repeat the mistake. Uh, this is a total detour, but I actually want to ask about this while we're on this. We'll get back to you and I in a, yeah, in a second. So you didn't suffer one billion, but presumably there's some amount you would have sold for, right? cause you write down in your head, like, I think the actual valuation of Facebook at the time is this and they're not actually getting the valuation. Right. I mean, out of $5 trillion, of course you would have sold. So what, like, how did you think about that choice?

对。所以当时我甚至不确定我有没有做过那个决定，那时我很投入，尽力让推荐功能为社群和其他内容发挥作用。因为这对Instagram和Facebook来说是一个很重要的突破，现在能够向用户展示有趣的内容，而这些内容来自他们甚至没有关注的人。但是，事实证明那是一个非常正确的决定。是的。好。这决定是因为之前有过困难。所以当时我并不是说，我并不像是领先太多。其实大多数时候，我认为我们做出看起来正确的决定是因为之前出过错，不想再重复那个错误。这话题有点跑题，但我确实想趁现在问一下。我们稍后会回到你和我在一起的话题。你并没有损失10亿美元，但我相信你当时一定有卖出的价码吧？因为你会自己在心里估算，根据Facebook当时的实际估值，他们并没有达到这个估值。我是说，如果价值5万亿美元，那当然你会卖出。所以，你是如何考虑这个选择的？

Yeah. I don't know. I think some of these things are just personal. Um, I, I don't know at the time that I was sophisticated enough to do that analysis, but I had all these people around me who were making all these arguments for how, like, a billion dollars was, you know, it's like, here's the revenue that we need to make and here's how big we need to be. And like, it's clearly so many years in the future. Like, and it was, it was very far ahead of where we were at the time. And I don't know. I didn't, I didn't really have the financial sophistication to really even engage with that kind of debate. I just, I think I sort of deep down believed in what we were doing. And I did some analysis. Um, I was like, okay, well, what would I go do if I wasn't doing this? It's like, well, I really like building things and I like helping people communicate and I like understanding what's going on with people and the dynamics between people. So I think if I sold this company, I'd just go build another company like this. And I kind of like the one I have. So, um, so I mean, you know, what's, why, why, right? But I don't know. I think a lot of the biggest bets that people make are often just based on conviction and values. Um, not, it's, it's actually usually very hard to do the analyses trying to connect the dots forward. So you've had, um, Facebook AI research for a long time. Uh, now it's become seemingly central to your company. At what point did making AGI or whatever, however you consider that mission, at what point is that like, this is a creek priority of what meta is doing.

是的，我不知道。我觉得有些事情是很个人的。在那个时候，我不觉得我足够复杂去做那种分析，但是我周围有很多人为我们需要赚取多少收入以及我们需要变得多么庞大这样的论点。明显这是很多年后的事情。那时候我们还很远。我不知道。我并没有太多的财务经验去进行这种辩论。我想我内心深处相信我们正在做的事情。我做了一些分析。我想，如果我不做这个，我会去做什么？我真的喜欢建设事物、帮助人们沟通，我喜欢了解人们之间发生的事情，以及人与人之间的动态。所以我觉得如果我把这家公司卖掉，我会再去建立一家类似的公司。而且我挺喜欢我现在的这家公司。那么，为什么呢？但是我不知道。我觉得人们做出最大的赌注通常都是基于信念和价值观。其实，试图将点连接到未来是很困难的。很久以前您就创建了Facebook人工智能研究部门。现在它似乎已经成为您公司的核心。在什么时候使得实现人工智能智能或者无论您如何看待那个任务，成为像meta正在做的事情的一个重要优先级？

Yeah. I mean, it's been a big deal for a while. So we started fair, um, about 10 years ago. And the idea was that along the way to general intelligence or AI, like full AI, whatever you want to call it, there are going to be all these different innovations. And that's going to just improve everything that we do. So we didn't kind of conceive it as a product. It was more kind of a research group. And over the last 10 years, it has created a lot of different things that have basically improved all of our products, um, and advanced the field. And allowed other people in the field to create things that have improved our products too. So I think that that's been great. But there's obviously a big change.

是的。我的意思是，这已经是一个大问题了很久了。所以大约10年前我们开始了公平，嗯，起初的想法是在通往通用智能或人工智能（AI）的过程中，无论你想如何称呼它，都会涌现出各种创新。这将大大改善我们所做的一切。因此，我们并没有将其设想为一个产品，它更像是一个研究小组。在过去的10年中，它创造了许多不同的东西，基本上改进了我们所有的产品，并推动了领域的发展。还允许领域内的其他人创建改进我们产品的东西。所以我认为这是很棒的。但显然有一个很大的变化。

Um, yeah. In the last few years when, you know, chat GPT comes out, um, the diffusion models or an image creation come out and like, I mean, this is some pretty wild stuff. Right. That, that I think is like pretty clearly going to affect how, how people interact with like every app that's out there. So I, at that point, we started a second group, um, the, the gen AI group, um, with the goal of basically bringing that stuff into our product. So building leading foundation models that would, that would sort of power all these different products.

嗯，是的。在过去几年中，你知道，聊天GPT问世了，扩散模型或图像创作问世了，这些都是相当疯狂的东西。我认为，这明显会影响人们与每个应用程序的互动方式。所以，在那时，我们成立了第二个团队，也就是GEN AI团队，旨在将这些内容引入我们的产品中。我们构建领先的基础模型，这些模型将为各种不同的产品提供动力。

And initially when we started doing that, um, the theory at first was, Hey, a lot of the stuff that we're doing is, is pretty social. Right. So, you know, it's helping people interact with creators, helping, um, people interact with businesses to, you know, so the businesses can sell things or do customer support or, um, you know, basic assistant functionality for, um, you know, whether it's for apps or the smart glasses or VR, like all these different things. So initially it wasn't completely clear that you were going to need kind of full AGI, um, to be able to support those use cases. But then through working on them, I think it's actually become clear that you do.

起初当我们开始做这个的时候，最初的理论是，嘿，我们正在做的很多事情是相当社会化的。对吧。所以，你知道，它帮助人们与创作者互动，帮助人们与企业互动，这样企业就可以销售东西或提供客户支持，或者完成一些基本的助手功能，无论是针对应用程序、智能眼镜还是虚拟现实等等。所以最初并不完全清楚你是否需要一种全面的人工智能来支持这些用例。但通过对它们的工作，我认为实际上变得很清楚你确实需要。

Right. And all these subtle ways. So for example, you know, for llama two, when we were working on it, we didn't prioritize coding. And the reason why we didn't prioritize coding is because people aren't going to ask meta AI a lot of coding questions and WhatsApp. Now they will. Right. Well, I don't know. I'm not sure that WhatsApp is like the UI that people are going to be doing a lot of coding questions. So we're like, all right, look, in terms of the things that, you know, or, or Facebook or Instagram or, you know, those, those different services, maybe, maybe the website or meta data. I that we're launching, I think. But, but the, the thing that was sort of a, I think has been a, you know, somewhat surprising result over the last, um, you know, 18 months is that it, it turns out that coding is important for a lot of domains, not just coding. Right. So even if people aren't asking coding questions, the models, um, training the models on coding helps them, um, just be more rigorous and answer the question and kind of, um, help reason across a lot of different types of domains.

对。还有一些微妙的方法。例如，当我们在开发 llama two 的时候，我们没有将编码作为优先任务。我们之所以没有优先考虑编码，是因为人们不会在 WhatsApp 上向 meta AI 提出大量关于编码的问题。但现在他们会。对。嗯，我不知道。我不确定 WhatsApp 是否是人们会对编码问题进行大量提问的用户界面。所以我们说，好吧，在我们要推出的事物中，可能是网站或 meta 数据。但是，在过去的18个月里，有一件事情，我认为有点出乎意料的是，原来编码对很多领域都很重要，不仅仅是编码领域。对。所以即使人们不问编码问题，对模型进行编码训练可以帮助它们更加严谨地回答问题，并在许多不同领域之间进行推理。

Okay. So that's one example where it's like, all right. So for llama three, we like really focused on training it with a lot of coding because it's like, all right, that's going to make it better on all these things. Even if people aren't answering or are an asking primarily coding questions. Reasoning, I think is another example. It's like, okay. Yeah. Maybe you want to chat with a creator or, you know, you're a business and you're trying to interact with a customer. You know, that interaction is not just like, okay, the person sends you a message and you just reply, right? It's a, it's like a multi step interaction where you're trying to think through, how do I accomplish the person's goals? And, um, you know, a lot of times when a customer comes, they don't necessarily know exactly what they're looking for or how to ask their questions. So it's not really the job of the AI to just respond to the question. It's like, you need to kind of think about it more holistically. It's really becomes a reasoning problem.

好的。这是一个例子，就像，好吧。所以对于llama three，我们真的很注重用很多编码来训练它，因为这样做会使它在所有这些方面都变得更好。即使人们不是在回答或主要询问编码问题。推理我认为是另一个例子。就像，好的。也许你想与一个创作者聊天，或者你是一个企业，试图与客户进行互动。这种互动不仅仅是，好的，这个人给你发了消息，你就回复了，对吧？这是一个多步互动，在这个过程中你要思考，如何实现这个人的目标。而且，你知道，很多时候，当客户来时，他们并不一定知道他们在寻找什么，或者如何提问他们的问题。因此，AI的工作并不只是回答问题，而是需要更全面地思考。这实际上变成了一个推理问题。

Right. So if someone else solves reasoning or makes good advances on reasoning and we're sitting here with a basic chat bot, then like our product is lame compared to what other people are building. So it's like, so, okay. So at the end of the day, we've got, we, you know, I, we basically realized we've got to solve general intelligence. Um, and we just kind of up the ante and the investment to make sure that we could do that. So the version of llama that, um, that, uh, that's going to solve all these use cases for users. Is that the version that will be powerful enough to like replace a programmer you might have in this building? I mean, I just think that all this stuff is going to be progressive over time, but in case llama 10, um, I mean, I think that there's a lot baked into that question.

对。所以如果有人解决了推理或取得了很好的推理进展，而我们只是拥有一个基本的聊天机器人，那么与其他人正在构建的产品相比，我们的产品就显得很糟糕。所以，就像，嗯。那么最终，我们意识到，我们必须解决通用智能的问题。我们只是不断提高赌注和投资，确保我们能够做到这一点。因此，那个将解决所有用户用例的“羊驼”版本，是否足够强大，足以替代您在部门中可能拥有的程序员？我认为所有这些东西会随着时间的推移而逐渐发展，但在“羊驼10”号的情况下，我认为这个问题有很多内涵。

I'm not sure that we're replacing people as much as making people tools to do more stuff. Is a programmer in this building 10x more productive after that? I would have more, but, um, but no, I mean, look, I, I'm not, I don't believe that there's like a single threshold of intelligence for, for humanity because, I mean, people have different skills. Oh, and at some point I think that AI is going to be, um, is, is probably going to surpass people at most of, of those things. I'm depending on how powerful the models are, but, um, but I think it's progressive. And I don't think AGI is one thing. I think it's, you're basically adding different capabilities.

我不确定我们是在取代人类，还是让人类成为做更多事情的工具。在这座建筑物里的程序员经历这样之后会提高生产力了吗？我可能会有更多，但是，嗯，但不，我的意思是，我不相信人类有一个单一智力门槛，因为，我是说，人们有不同的技能。噢，我认为在某个时候人工智能可能会在大多数事情上超过人类。取决于模型有多强大，但我认为这是渐进的。我认为通用人工智能不是一件事。我认为这是基本上添加不同的功能。

So multimodality is, is kind of a key one that we're focused on now initially with photos and images and text, but eventually with videos. And then because we're so focused on the metaverse kind of 3D type stuff is important. Um, one modality that I'm pretty focused on that I haven't seen as many other people in the industry, um, focus on this is sort of like emotional understanding. Like, I mean, so much of, of the human brain is just dedicated to understanding people and kind of like understanding your expressions and emotions and that that's like its own whole modality, right? That, um, I mean, you could say, okay, maybe it's just video or image, but it's like clearly a very specialized version of those two.

因此，多模态是我们现在关注的一个关键点，最初主要是通过照片、图像和文字，但最终会发展到视频。由于我们非常关注元宇宙这种3D类型的东西，所以这也很重要。另外，有一个模态我比较关注，其他行业的人好像没有太多注意到，那就是情感理解。我是说，人脑的很大一部分专门用来理解人类、理解表情和情绪，这本身就是一个完整的模态，对吧？你可能会说，这或许只是视频或图像，但显然它是这两者的一个非常专业化的版本。

So there's all these different capabilities that I think you want to basically train the models to focus on as well as, um, getting a lot better at reasoning, getting a lot better at memory, which I think is, is kind of its own whole thing. It's, I mean, I don't think we're going to be, you know, primarily shoving context or, or, or kind of things into a query context window, um, in the future to ask more complicated questions. I think that there will be kind of different stores of memory or different custom models that, um, that are maybe more personalized to people. But I know that I think that these are all just different capabilities.

因此，我认为你基本上想要训练模型专注于各种不同的能力，以及在推理方面取得更大的进步，在记忆方面取得更大的进步，我认为这是一个独立的整体。我不认为我们将会主要将上下文或其他内容输入到查询上下文窗口中，以便提出更加复杂的问题。我认为会有不同的记忆存储或不同的定制模型，也许更个性化地对人们进行服务。但我知道，我认为这些都只是不同的能力。

And then obviously making them big and small, we care about both because, you know, we want to, you know, if you're running something like meta AI, then we have the ability to, that's pretty server based. Um, but we also want it running on smart glasses and, you know, there's not a lot of space in smart glasses. So, um, you want to have someone that's very efficient for that. What is the use case that if you're doing tens of billions of dollars with inference or even eventually hundreds of billions of dollars worth of inference, using intelligence in an industrial scale? What is the use case? Is it simulations? Is it the EIs that will be in the metaverse? Where, where, where, what will we be using the data centers for?

然后显然地，我们让它们既大又小，因为我们关心两者，你懂的，我们想要，你知道的，如果你在运行类似于元AI这样的东西，那么我们就有能力做到，那主要是基于服务器的。但我们也想让它能在智能眼镜上运行，你知道的，智能眼镜空间有限。所以，你想要有一个非常高效的程序来处理这个。如果你在进行数百亿美元的推理，甚至最终是数千亿美元的推理，使用工业规模的智能技术，那么使用情况是什么？是模拟吗？是在元宇宙中的EIs吗？我们将用数据中心做什么呢？

Um, I mean, our bet is that it's going to, this is basically going to change all of the products, right? So I think that there's going to be a kind of meta AI general assistant product. And I think that that will shift from something that feels more like a chat bot where it's like you just ask a question that kind of formulates an answer to things where you're increasingly giving it more complicated tasks and that goes away and does them. Mm. So that's going to take a lot of inference. It's going to take a lot of compute in other ways too.

嗯，我的意思是，我们的赌注是这会改变所有的产品，对吧？所以我认为会有一种类似META AI智能助手产品。我认为这将从像聊天机器人那样只需提问就能得到答案的形式转变为越来越多地赋予它更复杂的任务，它会去完成这些任务。嗯。所以这将需要很多推理，同时也会需要大量的计算。

Then I think that there's a big part of what we're going to do that is, um, like interacting with other agents for other people, so whether it's businesses or creators. Um, I guess a big part of my theory on this is that there's not just going to be like one singular AI that you interact with because I think, um, you know, every business is going to like want an AI that represents their interests. They're not going to like want to primarily interact with you through an AI that is going to sell their competitors customers. So, uh, sorry, their competitors products.

然后我认为我们要做的一大部分是与其他人的代理互动，无论是企业还是创造者。我想我对此的理论的一个重要部分是，你不仅仅只会与一个AI互动，因为我认为，每个企业都会希望有一个代表他们利益的AI。他们不希望主要通过一个销售竞争对手产品的AI与您互动。

Um, so, um, uh, so yeah, so I think creators is going to be a big one. I mean, there are about 200 million creators on our platforms. They all basically have the pattern where, um, they want to engage their community, but they're limited by hours in the day and their community generally wants to engage them, but they don't know they're limited by hours in the day. Um, so if you could create something where, um, an AI could basically, where that creator can basically own the AI and train it in the way that they want, um, and can engage their community. I think that that's going to be super powerful too.

嗯，所以，嗯，嗯，对，我认为创作者们会是一个重要的群体。我是说，我们的平台上大约有2亿创作者。他们基本上都有这样的模式，想要与社区互动，但受到了一天中时间的限制，而他们的社区通常想要与他们互动，但并不知道时间的限制。所以，如果你能够创造出一种方式，让人工智能可以基本上，让那个创作者可以拥有这个人工智能并训练它以他们想要的方式，并能够与他们的社区互动。我认为这也将是非常强大的。

So, um, so I think that there's going to be a ton of engagement across all these things. Um, but these are just the consumer use cases. I mean, I think when you think about stuff like, I mean, you know, I run like our foundation, right? A Chan Zuckerberg initiative with my wife and, you know, we're doing a bunch of stuff on science and, um, and there's obviously a lot of AI work that, where I think is going to advance science and healthcare and all these things too. So I think that it's like, there's a, this is, I think, an end up affecting basically every area of the products and, and, and the, and the, uh, the economy.

所以，我认为在所有这些方面会有大量的参与。但这些只是消费者使用情况。我指的是，你知道，我和妻子共同运营我们的基金会，即陈-扎克伯格倡议，我们正在从事一些关于科学的工作，显然也有很多人工智能的工作，我认为将推动科学和医疗保健等方面。所以我认为，这样做将影响到产品和经济的每个领域。

The thing you mentioned about an AI that can just go out and do something for you, that's multi-step. Is that a bigger model? Is that you'll make like, LAMA four will still, there'll be a version that's still 70 B, but we'll just be, you'll just train it on the right data and that will be super powerful. How do like, what does the progression look like? Is it scaling? Is it just same size, but different banks like you were talking about? Um, I don't know that we know the answer to that.

关于你提到的可以为你做一些事情的人工智能，那是一个多步骤的过程。那是一个更大的模型吗？就像，LAMA四会继续存在，还会有一个版本仍然是70亿，但你只需在正确的数据上进行训练，那将会非常强大。进展会是怎样的？会扩张吗？还是只是相同规模，但不同的领域，就像你说的那样？嗯，我不知道我们是否知道答案。

So I think one thing that is seems to be a pattern is that you have the LAMA, that's sorry, the, the LAMA model. And then you build some kind of other application specific code around it. Right. So some of it is, is the fine tuning for the use case, but some of it is just like logic for, okay, how, um, like how MedAI should integrate that should work with tools like Google or Bing to bring in real time knowledge. I mean, that's not part of the base LAMA model. That's like part of a, okay. So for LAMA two, we had some of that. And it was a little more kind of hand engineered. And then part of our goal for LAMA three was to bring more of that into the model itself.

因此，我认为一个看起来是一个模式的东西是你有LAMA，抱歉，就是LAMA模型。然后你会围绕它构建某种其他的应用程序特定代码。对。所以其中一些是针对使用案例的微调，但其中一些只是逻辑，比如，MedAI应该如何集成，应该如何与像谷歌或必应这样的工具协作，带来实时知识。我的意思是，这不是基本LAMA模型的一部分。这是一部分，好吧。因此，对于LAMA 2，我们有一些。它有点更多地是手工设计的。因此，我们LAMA3的目标之一是将更多的内容带入模型本身。

And, but for LAMA three, as we start getting into more of these agent like behaviors, I think some of that is going to be more hand engineered. And then I think our goal for LAMA four will be to bring more of that into the model. So I think at each point, like at each step along the way, you kind of have a sense of what's going to be possible on the horizon. You start messing with it and hacking around it. Um, and then I think that that helps you hone your intuition for what you want to try to train, train into the next version of the model itself. Interesting. Which makes it more general, because obviously anything that your hand coding is, um, you know, you can unlock some use cases, but it's just inherently brittle and non-general.

而对于拉马三来说，当我们开始涉及更多类似代理的行为时，我认为其中一些会更多地由人工设计。然后我认为我们对拉马四的目标是将更多这方面的内容融入模型中。因此，我认为在每个阶段，你会对将来可能实现的东西有一定的了解。你开始尝试并探讨它。然后我认为这有助于你锻炼直觉，以便尝试训练到模型的下一个版本中。有趣的是，这使得模型更加通用，因为显然手工编码的任何东西都会解锁一些用例，但它只是本质上脆弱且不通用。

Hey, everybody. Real quick, I want to tell you about a tool that I wish more applications used. So obviously you've noticed every single company is trying to add an AI chat bot to their website. But as a user, I usually find them really annoying because they have these long generic, often useless answers. Command bar is a user assistant that you can just embed into your website or application. And it feels like you're talking to a friendly human support agent who is browsing with you and for you. And it's much more personalized than a regular chat bot. It can actually look up users history and respond differently based on that. It can use APIs to perform actions. It can even proactively nudge users to explore new features. One thing that I think is really cool is that instead of just outputting text, command mark and kind of just say here, let me show you and start browsing alongside the user.

大家好。我想快速告诉大家一个工具，我希望更多应用程序会使用。显然，你已经注意到每家公司都试图在他们的网站上添加一个AI聊天机器人。但作为用户，我通常觉得它们很烦人，因为它们有这些冗长的、通常是无用的答案。Command bar是一个用户助手，你可以把它嵌入到你的网站或应用程序中。它感觉就像你正在和一个友好的人类支持代理人交谈，他正在和你一起浏览和为你服务。而且它比普通的聊天机器人更加个性化。它可以查看用户的历史记录并根据情况作出不同的回应。它可以使用API执行操作。它甚至可以主动提示用户去探索新功能。我认为真的很酷的一点是，它不仅仅是输出文本，Command bar可以说：“让我来展示给你看”，并开始和用户一起浏览。

Anyways, there are a bunch of great products already. You can learn more about them at command bar.com. Thanks to them for sponsoring this episode. And now back to Mark. When you say into the model itself, you train it on the thing that you want in the model itself. But what do you mean by into the model itself? Well, I mean, I think like the example that I gave for Alama to where, you know, it's we we really, I mean, for Alama to the tool use was very, very specific. Whereas Loma three has the ability to has much better tool use, right? So so we don't have to like hand code all the stuff to have it use Google to to go do a search. It just kind of can do that. So in similarly for coding and kind of running code and just a bunch of stuff like that and.

无论如何，已经有很多优秀的产品。您可以在command bar.com上了解更多信息。感谢他们赞助本集。现在回到马克。当你说进入模型本身时，你是指在模型本身训练它所需的内容。但你所指的进入模型本身是什么意思？嗯，我是说，就像我给Alama的例子一样，我们对于Alama这个工具的使用非常特定。而Loma three则具有更好的工具使用能力，对吧？因此，我们不必手动编码所有的内容，让它使用谷歌去进行搜索。它可以自己做到。同样地，对于编码和运行代码等等，都是如此。

But I think once you kind of get that capability, then you get a peak of, OK, well, what can we start doing next? OK, well, I don't necessarily want to wait until Loma fours around to start building those capabilities. So let's start hacking around it. And so you do you do a bunch of hand coding and that makes the the products better for the interim, but then that also helps show the way of what we want to try to build into the next version of the level. What is the community fine to an Alama through? You're most excited by maybe not the one that will be most useful to you, but Jess, you will just enjoy playing it with the most. They like fine to run it on antiquity and you'll just be like talking to Virgil or something. What are you excited about? I don't know. I mean, I think the nature of the stuff is it's like. You get surprised, right? So I think like any any specific thing that I sort of. Thought would be valuable. We'd probably be building, right? So, but. I think you'll get distilled versions. I think you'll get kind of smaller versions. I mean, I mean, one thing that I think is. Eight billion, I don't think is quite small enough for for a bunch of use cases, right?

但是我认为一旦你掌握了那种能力，你就会达到一个高峰，好吧，我们可以开始做些什么？好吧，我不一定想等到下一个版本才开始构建这些能力。所以让我们开始尝试。所以你会完成一堆手工编码，这使产品在过渡阶段变得更好，同时也有助于展示我们想要尝试构建到下一个版本的方向。社区在Alama中找到的东西，也许你最感兴趣的不是对你最有用的，但你会很享受其中的大部分。他们喜欢在历史悠久的地方运行它，然后你会像和维吉尔交谈一样。你对什么感到兴奋？我不知道。我的意思是，这样的东西的本质就是。你会被惊喜到，对吧？所以我认为我认为任何我认为会有价值的具体东西，我们可能会去构建的，对吧？所以，但是。我认为你会得到精简版本。我认为你会得到更小的版本。我的意思是，我认为某样东西。80亿，我觉得对于很多用例来说，还不够小。

I think like over time, I'd love to get, you know, a billion parameter model or a two billion parameter model or even like a, I don't know, maybe like a 500 million parameter model and see what you can do with that. Because I mean, as they start getting. If it if with eight billion parameters were basically nearly as powerful as the largest Lama to model, then the billion parameters, you should be able to do something that's interesting, right? And faster, good for classification or a lot of kind of like basic things that people do before kind of understanding the intent of a user query and feeding it to the most powerful model to kind of hone what what the what the prompt should be. So I don't know, I think that's one thing that maybe the community can help fill in, but I mean, we'll also we'll also thinking about getting around to distilling some of these ourselves, but right now the GPUs are trading the four or five. So what okay, so you have all these GPUs. These are 350,000 by the end of the year. That's the whole fleet. I mean, I was we we built two, I think it's like 22, 24,000 clusters that are kind of the single clusters that we have for training the big models. I mean, obviously across a lot of the stuff that we do, a lot of our stuff goes towards training, like reels models and sure, and like Facebook news feed and Instagram feed and then inference is a huge thing for us because we serve a ton of people, right? So our ratio of inference compute required to to training is probably much higher than most other companies that are doing this stuff just because the sheer volume of the community that we're serving. Yeah. Yeah. Yeah.

我觉得随着时间的推移，我会喜欢得到一个十亿参数模型，或者一个二十亿参数模型，甚至一个五亿参数模型，看看你能做什么。因为我是说，随着他们开始变得强大。如果拥有八十亿个参数基本上几乎和最大的Lama模型一样强大，那么十亿个参数，你应该能做一些有趣的事情，对吧？更快，对分类有益，或者许多人在了解用户查询意图之前做的基本事情，然后将其提供给最强大的模型，来磨练该提示应该是什么。所以我不知道，我觉得这可能是社区可以帮助填补的一件事情，但我是说，我们也在考虑自己提炼其中的一部分，但眼下显卡正在交易四五。所以好吧，那么你拥有所有这些显卡。到年底会有35万张显卡。这是整个机群。我是说，我们建造了两个，我想是22、24,000个单一集群，用于训练大型模型。我是说，显然我们所做的很多事情，很多我们的东西都用于训练，比如真实模型和Facebook新闻动态和Instagram动态，然后推理对我们来说是一大事，因为我们为大量人服务，对吧？我们的推理计算量所需与训练的比率可能比其他正在做这些事情的公司要高得多，只是因为我们为服务的社区规模太庞大了。是的。是的。是的。

That was really interesting in the material they shared with me before that you trained it on more data than is compute optimal just for training because the inference is such a big deal for you guys and also for the community that it makes sense to just have this thing and have trillions of tokens in there. Yeah. Yeah. Although in one of the interesting things about it that we saw even with the 70 billion as we we thought it would get more saturated at, you know, it's like we trained on around 15 trillion tokens. Yeah. We, I guess our prediction going in was that it was going to ask some tote more, but even by the end, it was still learning, right? It's like we probably could have fed it more tokens and it would have gotten somewhat better. But I mean, at some point, you know, you're running a company.

在他们之前与我分享的资料中，他们对我进行的培训数据比计算优化更多，因为对你们来说推理是如此重要，也对社区而言，所以有意义的是只要拥有这个东西，并且有数万亿的令牌。是的。尽管有趣的是，即使我们使用了700亿个令牌进行训练，我们认为它会更加饱和，你知道，就好像我们在大约15万亿个令牌上进行了训练。是的。我想我们当初的预测是它会要求更多令牌，但即使到最后，它仍在学习，对吧？我们可能本可以再给它更多令牌，它会变得更好一些。但是，我是说，在某个时候，你在经营一家公司。

You need to do these meta reasoning questions of like, all right, how do I want to spend our GPUs on like training this 70 billion model further? Do we want to kind of get on with it so we can start testing hypotheses for llama for? So we kind of needed to make, to make that call. And I think we got it. We, I think we got to a reasonable balance for, for this version of the 70 billion. Um, there will be others in the future where, you know, 70 billion multimodal one that'll come over the next period. But, um, but yeah, I mean, it's, that was, that was fascinating. That you could just, that it's the architectures at this point can just take so much data. Yeah, that's really interesting. So what does this imply about future models? I, you mentioned that the llama three eight B is better than the llama 270 B. No, it's nearly as well. Okay. I don't know what to do. But does that mean like the llama four to magnitude? There's, there's, I mean, like the llama four 70 B will be as good as the llama three, four or five B like, what is this? One of the great questions, right? That I think no one knows, um, is, is basically, you know, it's, it's one of the trickiest things in the world to plan around is when you have an exponential curve, how long does it keep going for? Yeah. And, um, I think it's likely enough that it will keep going, that it is worth investing the, um, tens or, you know, 100 billion plus in building the infrastructure to, um, assume that if that kind of keeps going, you're going to get some really amazing things that are just going to make amazing products. Mm. But I don't think anyone in the industry can really tell you that it will continue scaling at that rate for sure. Right. And in general, you know, in history, you hit bollumlex at certain points. And now there's so much energy on this that maybe those bollumlex get knocked over pretty quickly, but, um, but I don't know. I think that's, that's an interesting question. What does the world look like where there aren't these bottlenecks?

你需要做这些类似的元推理问题，比如，我们如何更好地利用我们的GPU来进一步训练这个700亿模型？我们是否想要继续下去，以便可以开始为llama进行假设测试？所以我们需要做出这个决定。我认为我们做到了。我认为我们在这个700亿版本中达到了一个合理的平衡。未来会有其他版本，可能会有700亿多模型会在未来到来。但是，这是一件很有趣的事情，就是在这一点上的架构可以处理这么多数据。是的，这真的很有趣。那么这对未来模型意味着什么？你提到llama 38B比llama 270B要好。不，它差不多。我不知道该怎么做。但是，这是否意味着像llama 40倍之类的情况呢？这是一个很大的问题，对吧？我认为没有人知道，基本上，当你有指数曲线时，这是世界上最棘手的事情之一是，这种情况会持续多久？是的。我认为很可能会继续下去，值得投资数百亿甚至数千亿在建立基础设施上，假设如果这种情况持续下去，你将会得到一些非常惊人的产品，这将是令人惊叹的产品。但是我认为没有人可以确切地告诉你，它将继续以这种速度扩展的。总的来说，在历史上，你会在某些时候遇到瓶颈。现在对此有很多关注，也许这些瓶颈很快就会被打破，但是，我不知道。我觉得这是一个有趣的问题。如果不存在这些瓶颈，世界会是什么样子？

As you know, suppose like progress just continues, uh, at this pace, which seems like plausible, um, like zooming out. Well, they're going to be different bottlenecks. Right. So if not training, then, like, oh, yeah, go ahead. Well, I think at some point, you know, over the last few years, I think there was this issue of, um, GPU production. Yeah. Right. So even companies that had the models, uh, sorry, that had the money to pay for the GPUs, um, couldn't necessarily get as many as they wanted because there was there were all these supply constraints. Yeah. Now I think that's sort of getting less. So now I think you're seeing a bunch of companies think about, wow, we should just like really invest a lot of money in building out these things. And I think that that will go for, um, for some period of time. Um, I think there's a, there is a capital question of like, okay, at what point does it stop being worth it to put the capital in? But I actually think before we hit that, you're going to run into energy constraints. Right. Because, um, I just, I mean, I don't think anyone's built a gigawatt single training cluster yet. Right. And, um, and then you run into these things that just end up being slower in the world, like getting energy permitted is like a very heavily regulated government function. Right. So you're going from on the one hand software, which is somewhat regulated. I, I'd argue that it is more regulated than I think a lot of people in the, in the tech community feel, although it's obviously different. If you're starting a small company, maybe you feel that less. If you're a big company, you know, we just interact with people, but different governments and regulators are, you know, we have kind of lots of rules or that we need to kind of follow and make sure we do a good job with around the world. Um, but I think that there's no doubt that like energy, and if you're talking about building large new power plants or large build outs and then building transmission lines that cross. Other private or public land, that is just a heavily regulated thing. So you're talking about many years of lead time.

正如你所知，假设进展继续像这样，在这个速度上，这似乎是有可能的，就好像在放大。嗯，它们会遇到不同的瓶颈。所以如果不是培训，那么，嗯，就像，是的，继续吧。我觉得在过去几年里，有一个问题，就是GPU的生产。是的。对吧。所以即使是那些有钱支付GPU的公司，也不一定能得到他们想要的那么多，因为存在供应限制。是的。现在我觉得这种情况好像有所改善。所以现在我觉得你会看到很多公司在考虑，哇，我们应该投入大量资金来建设这些东西。我认为这将持续一段时间。我认为这里有一个资本问题，就是在什么时候投资不值得，但我认为在我们遇到这个问题之前，你会遇到能源限制。因为，我只是，我的意思是，我不认为有人已经建立了一吨的单一训练集群。然后你会遇到这些在世界中变慢的事物，比如能源许可证是一个非常被监管的政府职能。从某种程度上说，软件也受到监管。我认为它比很多科技界的人认为的更受到监管，尽管显然情况不同。如果你开了一家小公司，也许你会觉得少一些。如果是一家大公司，你知道，我们必须与不同的政府和监管机构互动，那么我们就会遇到很多规则，需要我们遵循并确保我们在世界各地做好工作。但我认为毫无疑问，如能源这类问题，如果你谈论建设大型新的发电厂或大规模建设，然后修建穿越其他私人或公共土地的输电线路，那就是一个高度受监管的事物。所以你要经历多年的前期准备。

So if we wanted to stand up just some like massive facility, um, to power that. I think that that is, that's, that's a very long term project. Right. And, um, so I don't know. I think that that's, I think people do it. I don't, but I don't think that this is like something that can be quite as magical as just like, okay, you get a level of AI and you get a bunch of capital and you put it in and then like all of a sudden the models are just going to kind of like, it just, like, I think you, you do hit different bottlenecks along the way. Yeah. Is there something a project? Maybe I really did. Maybe not that even a company like meta doesn't have the resources for it. Like if you're R&D budget or cap X budget was 10 X what it is now, then you could pursue it. Like it's in the back of your mind, but meta today, and maybe you could, like, because even you can't even issue stock or bond for it. It's like just 10 X bigger than your budget.

如果我们想要建立一个大型设施来支持这个，我觉得那是一个非常长期的项目。我不知道。我觉得人们会做到这一点。但我不认为这样的事情会像魔法一样简单，你只需要得到一定水平的人工智能，投入一大笔资金，然后模型就会突然变得像这样。我觉得在这条路上会遇到不同的瓶颈。也许这是一个项目吧？也许我真的做到了。也许像meta这样的公司甚至没有这样的资源。如果你们的研发预算或资本支出预算是现在的十倍，那么你们可能会去追求这个。像现在这样的meta，也许你们可以，因为你们甚至不能发行股票或债券来支持它。它就是你们预算的十倍。

Well, I think energy is one piece. Yeah. Right. Um, I think we would probably build out bigger clusters than we currently can. If we could get the energy to do it. So I think that's, um, that's fundamentally money bottlenecked in the limit. Like if you had a trillion dollars, it's time. Yeah. Right. Um, well, if you look at it in terms of, but it depends on how far the, the exponential curves go. Right. Like I think a number of companies are working on, you know, right now, I think I did like a lot of data centers around the order of 50 megawatts or a hundred megawatts or like a big one might be a hundred, 50 megawatts. Okay. So you take a whole data center and you fill it up with just all the stuff that you need to do for training and you build the biggest cluster you can. I think you're, that's kind of, I think a bunch of companies are running at stuff like that. Um, but then when you start getting into building a data center, that's like 300 megawatts or 500 megawatts or a gigawatt. I just, I mean, just known as built a single gigawatt data center yet.

我认为能源是一个重要因素。是的，没错。我觉得如果我们能够获得足够的能源，我们可能会构建比目前更大的集群。所以我认为，这基本上是受到资金限制。如果有一万亿美元，就能做到。是的，没错。但这取决于指数曲线的走势。目前有许多公司在研究，我想现在大约有50兆瓦或100兆瓦的数据中心，或者最大的可能是150兆瓦。所以你把整个数据中心填满所需的训练材料，构建最大的集群，我认为许多公司正在进行类似的工作。但当你开始建造300兆瓦、500兆瓦或1吉瓦的数据中心时，我觉得，至今还没有谁建造过单个吉瓦级数据中心。

So I think it will happen. Right. I mean, this is only a matter of time, but it's, it's not going to be like next year. Right. It's, um, I think that some of these things will take, I don't know, some, some number of years to build out. And then the question is, okay, well, if you, I mean, just to, I guess, put this in perspective, I think a gigawatt, it's like around the size of like a meaningful nuclear power plant only going towards training a model. Didn't, didn't Amazon do this? There's like, they have a 950 gig a megawatt. Uh, yeah, I'm not exactly sure what you did. You have to, what they did, you don't have to ask them. Um, um, but it doesn't have to be in the same place, right? If distributed training works, it can be distributed. That I think is a big question. Yeah. Right. Just is basically how that's going to work. And I do think in the future, it seems quite possible that more of what we call training for these big models is actually more along the lines of inference, generating synthetic data to then go feed into the model.

所以我认为这会发生。是的。我的意思是，这只是一个时间问题，但不会像明年那样就会发生。我觉得这些事情会花费一些年份来建设。然后问题就是，好吧，如果你，我想一吉瓦特，大概相当于一个有意义的核电站的规模，只用来训练一个模型。亚马逊不是也做了吗？他们有一个950吉瓦特的。是的，我不太清楚他们做了什么。你得问问他们。但是不一定要在同一个地方，对吧？如果分布式训练起效，就可以分布式进行。这我认为是一个很大的问题。对吧。基本上就是这样的工作方式。我认为未来可能更多的所谓训练大型模型的内容实际上更多地倾向于推断，生成合成数据然后输入模型。

So I don't know what that ratio is going to be, but I consider, um, the generation of synthetic data to be more inference than training today. But obviously if you're doing it in order to train a model, it's, it's part of the broader training process. So, um, I don't know. That's an, that's an open question is to, to kind of wear what the balance of that and how that plays out. If that's the case, would that potentially also be the case with llama three and maybe like llama four onwards where you put this out and if somebody has a ton of compute, then using the models that you've put out, you can just keep making these things arbitrarily smarter, like some, a Kuwait or UA or some random country has a ton of compute, um, and they can just, uh, actually just use llama for it to just make something much smarter. Um, I, I do think that there are going to be dynamics like that, but I also think that there is a fundamental limitation on, um, on kind of the network architecture, right? Or the, the kind of model architecture, right?

所以我不知道这个比例将会是多少，但我认为目前合成数据的生成更多是推断而不是训练。但显然，如果你是为了训练模型而这样做，它是更广泛的训练过程的一部分。所以，我不知道。这是一个悬而未决的问题，关于如何平衡以及这将如何发挥作用。如果是这样的话，那是否也会出现在llama三，以及之后的llama四等版本中，在这种情况下，如果有人有大量计算资源，那么他们可以利用你所发布的模型，不断使这些东西变得更加智能，像某个科威特或乌克兰或其他一些随机国家拥有大量计算资源，他们可以直接利用llama来使某些东西变得更加智能。我认为会有这样的动态，但我也认为在网络架构的基础上存在着根本性的限制，或者说模型架构的限制。

So I think like a 70 billion model that kind of we trained with a llama three architecture can get better, right? It can, it can keep going. Like I was saying, it's, you know, we felt like if we kept on feeding it more data or, or rotated the high-value tokens through again, then, then, you know, it would, it would continue getting better. But. And we've seen a bunch of other people around the world, um, you know, different companies basically take the llama to 70 billion base, like take that model architecture and then build a new model. Um, it's still the case that when you make a generational improvement to the kind of llama three 70 billion or the llama three four hundred and five, there's nothing open source, anything like that today. Right. Like a, it's, it's not, I think that that's like, it's a big step function and what people are going to be able to build on top of that. I don't think can go infinitely from there. I think it can, there can be some optimization in that until you get to the next step function. yeah.

所以我认为，像我们用美洲驼三种架构训练的七百亿模型可以变得更好，对吧？它可以，它可以持续改进。就像我说的那样，我们觉得如果继续给它输入更多数据，或者再次旋转高价值的标记，它就会继续变得更好。但是。我们看到世界各地的许多其他人，就是不同的公司基本上采用了美洲驼七百亿基础模型，然后建立了一个新的模型。当你对美洲驼七百亿或美洲驼三四百零五进行一代改进时，今天没有任何开源的东西。对。我认为那是一个巨大的跃迁，人们将能够在其基础上构建的东西。我认为不可能无限制地从那里继续发展。我认为在达到下一个跃迁之前，可能会有一些优化。是的。

Okay. So let's zoom out a little bit from, uh, specific models and even the many years, Lee times you would need to get energy approvals and so on, like big picture. These next couple of decades. Sure. What's happening with AI, um, does it feel like another technology, like metaverse or social or does it feel like a fundamentally different thing in the course of human history? Um, I think it's going to be pretty fundamental. I think it's going to be more like the creation of computing in the first place. Right. So, um, you'll get all these new apps. In the same way that when you got the web or you got mobile phones, you got like people basically rethought all these experiences and a lot of things that weren't possible before now became possible. Um, something that will happen, but I think it's a much lower level innovation. It's, um, it's going to be more like going from people didn't have computers to people have computers is my, my sense.

好的。那么让我们从具体的模型和需要获得能源批准等多年的时间中稍微放大镜头，看看大局。在接下来的几十年里，人工智能会发生什么呢？它感觉像是另一种技术，比如元宇宙或社交，还是感觉像是人类历史进程中的一个根本性不同的东西呢？我认为它将是相当根本的。我认为它会更像是计算机的创造。因此，你将会得到所有这些新应用程序。就像当你得到了互联网或手机时，人们基本上重新思考了所有这些体验，之前不可能实现的许多事情现在变得可能了。会发生某些事情，但我认为这是一个更低层次的创新。它更像是从人们没有计算机到人们有计算机的变化，这是我对此的看法。

Um, but it's also it's, it's a. I don't know. It's, it's very hard to reason about exactly how this goes. I tend to think that. You know, in like the cosmic scale, obviously it'll happen quickly over a, you know, a couple of decades or something, but I, I do think that there, there is some set of people who are afraid of like, you know, it really just kind of spins and goes from being like somewhat intelligent to extremely intelligent overnight. And I just think that there's all these physical constraints that make that so that that's unlikely to happen. um, I, I just don't, I don't really see that, that playing out. So I think you'll have, I think we'll have time to kind of acclimate a bit, but it will really change the way that we work and give people all these creative tools to do different things that they, uh, yeah, I think, I think it's going to be. It's, it's going to really enable people to do the things that they want a lot more as it is my view.

嗯，但也是如此。我不知道。确切地推理这个过程是非常困难的。我倾向于认为，你知道，在宇宙尺度上，显然会在几十年的时间内迅速发生，但我认为有一些人害怕，就好像是从稍微聪明到极其聪明的转变发生得很快。我觉得有很多物理限制，使得这种情况不太可能发生。我只是觉得这种情况不太可能发生。所以我认为我们会有时间来适应一下，但这将会真正改变我们工作的方式，并给人们提供许多创造性的工具来做他们想做的事情，我认为这会使人们更多地实现他们想要的目标。

Um, okay. So maybe not overnight, but is it your view that like on a cosmic scale, if you think like humans evolved and then like AI happened and then they like went out through the galaxy or maybe it takes too many decades, maybe it takes a century, but like, is that like the grand scheme of what's happening right now in history? um, sorry. In what sense? I mean, in the sense that there were other technologies, like computers and even like fire, but like the AI happening is as significant as like humans evolving in the first place. I think that's tricky. Um, I think people like to, you know, the history of humanity, I think has been people basically, you know, thinking that certain aspects of humanity are like really unique in different ways. And then coming to grips with the fact that that's not true, but humanity is actually still super special. Right. So it's, um, it's like we thought that the earth was the center of the universe. And it's like it's not, but like it's like humans are still pretty awesome, right? And pretty unique.

嗯，好的。也许不是一夜之间，但你是否认为在宇宙尺度上，如果你想像人类进化，然后人工智能出现，然后它们像穿越星系那样，也许需要太多的十年，也许需要一个世纪，但是，像这样是不是现在历史上正在发生的宏伟计划？对不起，在什么意义上？我的意思是，是否有其他技术，如计算机甚至火，但人工智能的出现和人类进化一样重要。我觉得这很棘手。我觉得人们喜欢，你知道，人类的历史，我认为人们基本上一直认为人类的某些方面在某种程度上非常独特。后来他们意识到这不是真的，但人类实际上仍然非常特殊。对，所以，我们曾认为地球是宇宙的中心。但是，人类还是相当了不起的，相当独特的。

Um, I think that another bias that people tend to have is thinking that intelligence is somehow kind of fundamentally connected to life. And it's not actually clear that it is. Right. I think like, like people think that, um, I mean, I don't know that we have a clear enough definition of consciousness or, um, or, or life to kind of fully, um, interrogate this, but I know there was all the science fiction about, okay, you create intelligence and now it like starts taking on all these human like behaviors and, and things like that. But I actually think that the current incarnation of all this stuff at least kind of feels like it's going in a direction where intelligence can be pretty separated from consciousness and agency and things like that. That, um, I think just makes it a super valuable tool.

嗯，我认为人们往往会有另一种偏见，即认为智力在某种程度上与生命基本上有关联。但实际上并不清楚是否确实如此。我认为人们认为，我是指，我不知道我们是否对意识或生命有一个足够清晰的定义，或者或者为了完全推翻这种观念，但我知道有很多科幻小说，比如，你创造了一个智能体，现在它开始表现出所有这些人类行为之类的行为。但我实际上认为，目前所有这些东西的化身至少似乎是朝着智力可以与意识和行动等相分离的方向发展的。我认为这仅仅使它成为一个非常有价值的工具。

So I don't know. I mean, obviously it's, it's, um, it's very difficult to predict what direction the stuff goes in over time, which is why I, I don't think anyone should be dogmatic about, you know, how they plan to develop it or what they plan to do. I think you want to kind of look at like each release, you know, it's like, we're obviously very pro open source. Yeah. But I haven't committed that we're going to like release every single thing that we do, but it's basically we, like, I'm, I'm just generally very inclined to thinking that open sourcing it is going to be good for the community and also good for us. Right. Cause we'll, we'll benefit from, from the innovations. Um, but if it's at some point, like there's some qualitative change in what the, the thing is capable of and we feel like it's just not responsible to open source it then we won't, but, um, so I don't know. It's, it's all, it's all very difficult to predict. Yeah. Um, what is a kind of qualitative change, like a specific thing, you're training Lamify, Lamaphore, and you've seen this and like, you know what? I'm not sure about open sourcing it. Um, I think that that it's a little hard to answer that in the abstract because there are negative behaviors that any product can exhibit that as long as you can mitigate it, it's like, it's okay. Right. So, um, I mean, there's bad things about social media that we work to mitigate. Right. There's bad things about Lamatou that we spend a lot of time trying to make sure that it's not like, you know, helping people commit violent acts or things like that. Right. I mean, that doesn't mean that it's like a kind of autonomous or intelligent agent. It just means that it's learned a lot about the world and it can answer a set of questions that we think it would be unhelpful for it to answer.

所以我不知道。我的意思是，显然很难预测事物随着时间发展会走向何方，这就是为什么我觉得任何人都不应该对他们计划的发展或他们计划做的事情持教条主义态度。我认为你应该像看待每一个发布版本一样，我们显然非常支持开源。是的。但我还没有承诺我们会释放我们所做的每一件事，但基本上我们，我，我总体上非常倾向于认为开源对社区和我们自己都是有益的。对。因为我们会受益于创新。但如果在某个时候，像是在能力上出现了某种质变，我们觉得开源不负责任，那么我们就不会这样做，但我不知道。这一切都很难预测。是的。什么是质变，就像你在训练Lamify，Lamaphore，你看到了这一点，你知道吗？我不确定是否要开源它。我认为在抽象层面上很难回答这个问题，因为任何产品都可以表现出负面行为，只要你可以减轻它，就没问题。对。所以，我是说，社交媒体有不好的地方，我们努力减轻这些问题。对。有关Lamatou也有不好的地方，我们花了很多时间来确保它不会像帮助人们犯下暴力行为之类的事情。对。这并不意味着它是一种类似智能的代理程序，而是意味着它对世界了解很多，可以回答一些我们认为它回答没有帮助的问题。

Um, so I, um, I don't know. I think the question isn't really what behaviors would it show? It's what things would we not be able to mitigate after it shows that and, um, and I don't know. I, I think that there's so many ways in which something can be good or bad that it's hard to actually enumerate them all upfront. If you even look at like what we've had to deal with in, um, you know, social media and like the different types of harms we've basically gotten to it's like, there's like 18 or 19 categories of, of harmful things that, that people do. And we've basically built AI systems to try to go identify what those things are that people are doing and try to make sure that that, you know, it doesn't happen on our network as much as possible. So, um, yeah, I think you can, over time, I think you'll be able to break down, um, this into more of a taxonomy too. And I think this is a thing that we spend time researching too. Cause we want to make sure that we understand that.

哦，所以，嗯，我不知道。我认为问题并不是它会展示什么行为，而是在展示后，我们将无法缓解的是什么问题，嗯，我不知道。我认为有很多方面可以是好的或坏的，很难事先完全罗列出来。如果你看看我们在社交媒体上所面对的问题，以及我们基本上已经面对的不同类型的伤害，就像，有18或19类伤害。我们基本上已经建立了人工智能系统来尝试识别人们正在做的那些事情，并尽可能地确保这些事情不会在我们的网络上发生。所以，是的，我认为随着时间的推移，我认为你将能够更细分，将这一切归类为更多种类。我认为这也是我们花时间研究的事情，因为我们希望确保了解这一点。

So one of the things I asked Mark is what industrial scale use of LLM's would look like. You see this in previous technological revolutions where at first they're thinking in a very small scale way about what's enabled. And I think that's what chat bots might be for other lums. And I think the large scale use case might look something like what V seven go is. And by the way, it's made by V seven labs who are sponsoring this episode. So it's like a spreadsheet. You put in raw information like documents, images, whatever, and they become rows and the columns are populated by an LLM of your choice. And in fact, I used it to prepare for Mark. So I fed in a bunch of blog posts and papers from Meta's AI research. And as you can see, if you're on YouTube, it summarizes and extracts exactly the information I want as columns. And obviously mine is a small use case. But you can imagine, for example, a company like FedEx has to process half a million documents a day. Obviously a chat bot can't do that a spreadsheet can because this is just like a fire hose of intelligence in there, right? Anyways, you can learn more about them at v7 labs.com slash go or the link in the description back to Mark.

所以我问马克的一件事是LLM的工业规模使用会是什么样子。在以往的技术革命中，最初的想法往往局限在很小的范围内。我认为像聊天机器人对其他LLM可能会是什么。而大规模使用情况可能类似于V七。顺便说一句，这是由赞助本集的V七实验室制造的。它就像一个电子表格。你输入原始信息，如文件、图像等，它们变成行，列由你选择的LLM填充。实际上，我用它来为马克做准备。我输入了一堆Meta的人工智能研究的博文和论文。如果你在YouTube上，你可以看到，它总结并提取我想要的信息作为列。很显然，我的使用情况很小。但你可以想象，比如联邦快递这样的公司每天必须处理五十万份文件。显然，聊天机器人无法做到，电子表格可以，因为这就像里面有一股智慧的火管，对吧？无论如何，你可以在v7labs.com/go或描述中的链接中了解更多关于它们的信息。回到马克。

Yeah. Like it seems to me it would be a good idea. I would be disappointed in a future where AI systems aren't broadly deployed and everybody doesn't have access to them. Yeah. At the same time, I want to better understand the mitigations. Yeah. Because if the mitigation is the fine tuning, well, the whole thing about open weights is that you can then remove the fine tuning, which is often superficial on top of these capabilities.

是的。对我来说，这似乎是一个不错的主意。我会对未来没有广泛部署AI系统，每个人都无法接触到它们的情况感到失望。是的。与此同时，我希望更好地了解缓解措施。是的。因为如果缓解措施是微调，那么关于开放权重的整个概念就是，您可以随后移除微调，这通常只是这些能力之上的肤浅处理。

Like if it's like talking on Slack with a biology researcher, and I think like models are very far from this that right now they're like Google search. But it's like I can show them my petri dish and they can next land like, here's why you're a smallpox sample thing grow. Here's what to change. How do you mitigate that? Because somebody can just like fine tune that in there, right? Yeah. I mean, that's true.

就像是在Slack上与一位生物研究者交谈一样，我认为模型目前与这种情况还有很大距离，它们就像谷歌搜索一样。但我可以向他们展示我的培养皿，他们可以接着说，这是为什么你培养的是天花样本。这里需要改变的是什么。你如何减轻这种情况？因为有人可能会对此进行微调，对吧？是的。我想这是真的。

I think a lot of people will basically use the off the shelf model and some people who have basically bad faith are going to try to strip out all the bad stuff. So I do think that's an issue. The, um, the flip side of this is that, and this is one of the reasons why I'm kind of philosophically so pro open source is I do think that a concentration of AI in the future has the potential to be as dangerous as kind of it being widespread. So I think a lot of people are, they think about the questions of, okay, well, if we can do this stuff, is it bad for it to be out wild? Like just in kind of widely available.

我认为很多人基本上会使用现成的模型，而一些心怀不良意图的人会试图剥去所有不好的东西。所以我认为这是一个问题。另一方面是，这也是我哲学上非常支持开源的原因之一，我认为将来人工智能的集中可能和普及一样危险。所以很多人在思考这个问题，就是，好吧，如果我们可以做这些事情，它被广泛传播出去是不是不好的？

Um, I think another version of this is like, okay, well, it's probably also pretty bad for one institution to have an AI that is way more powerful than everyone else's AI. Right. So if you look at like, like, I guess one security analogy that I think of is, um, you know, it doesn't take AI to basically, okay, there's security holes and so many different things. And if you could travel back in time a year or two years, right? It's like, that's not AI. It's like you just, let's say you just have like one year or two years more knowledge of the security holes. It's pretty much hacking to like any system.

嗯，我认为这句话的另一个版本可能是，好吧，一个机构拥有比其他人更强大的人工智能可能也不太好。对吧。所以，如果你想想，比如，我觉得一个安全的类比是，你知道，并不需要人工智能来发现安全漏洞，那么可以穿越时间回到一年或两年前，是的，那并不是人工智能，只是你对安全漏洞有了一年或两年的更多了解。就像是可以轻松入侵到任何系统一样。

Right. So it's not that far fetched to believe that a very intelligent AI would probably be able to identify some holes and basically be like a human who could potentially go back in time a year or two and compromise all these systems. Okay. So how have we dealt with that as a society? Well, one big part is open source software that makes it so that when improvements are made to the software, it doesn't just kind of get stuck in one company's products, but it can kind of be broadly deployed to a lot of different systems, whether it's banks or hospitals or government stuff and like, just everyone can kind of.

没错。因此，认为非常聪明的人工智能可能能够识别一些漏洞，基本上像一个可能能够回到一两年前并危害所有这些系统的人类一样，并不是那么不切实际。那么，我们作为一个社会是如何应对这种情况的呢？其中一个重要的部分是开源软件，使得当软件进行改进时，它不仅仅停留在某一个公司的产品中，而是可以广泛应用到许多不同的系统中，无论是银行、医院还是政府机构等，每个人都可以受益于此。

Like as the software gets hardened, which happens because more people can see it and more people can bang on it. Um, and there are, and there are standards on how this stuff works. Um, the world can kind of get upgraded together pretty quickly. And I kind of think that a world where AI is very widely deployed in a, in a way where it's gotten hardened, um, progressively over time and is one where all the different systems will be in check.

随着软件变得更加稳定，是因为更多的人可以看到它，更多的人可以进行测试。嗯，还有关于这些工作方式的标准。世界可以相对快速地一起升级。我认为，在一个人工智能被广泛部署的世界里，它逐渐变得稳定，并且所有不同的系统将会相互制衡。

Yeah. In a way that seems like it is fundamentally more healthy to me than one where this is more concentrated. So there are risks on all sides, but I know that's one risk that I think. People, I don't hear them talking about quite as much. I think like there's sort of the risk of like, okay, well, what if the AI system does something bad? I, I am more like, you know, I stay up at night more worrying.

是的。在我看来，这种方式似乎从根本上比集中更健康。因此，各方面都存在风险，但我知道这是我认为的一个风险。人们并不经常谈论这个问题。我认为有一种风险，就是，好吧，如果人工智能系统做了坏事怎么办？我更多的是，你知道的，晚上睡不着觉地担心。

Well, what if like some actor that, you, whatever, it's like from wherever you sit, there's going to be some actor who you don't trust. If they're the ones who have like the super strong AI, whether it's some, like other government that we, that, that is sort of like an opponent of, of our country or some company that you don't trust or whatever, whatever it is. Um, like I think that that's potentially a much bigger risk as in they could like overthrow our government because they have a weapon that like nobody else has cause a lot of mayhem.

如果有某个演员，从你坐的位置来看，总会有些演员是你不信任的。如果他们拥有超强的人工智能，不管是来自其他国家的政府还是你不信任的公司，这可能是一个更大的风险。我认为这个风险潜在地比较严重，因为他们可能推翻我们的政府，因为拥有其他人没有的武器，造成许多混乱。

Right. It's, I think it's like, I, I, I think the intuition is that this stuff ends up being pretty kind of important and, and, um, and valuable for both kind of economic and, and kind of security and other things. And, um, I don't know. I just think, yeah, if, if like if someone who you don't trust or is an adversary of you, get something that is more powerful than, um, then I think that that could be an issue. I think the probably the best way to mitigate that is to have good open source, um, AI that, that basically becomes the standard.

是的。我认为，直觉告诉我这些东西最终会变得非常重要和有价值，不仅对经济和安全等方面，还有其他方面也是如此。我不知道。我只是认为，是的，如果你不信任或是你的对手得到比你更强大的东西，那可能会成为问题。我认为最好的缓解方式可能是拥有良好的开源AI，这基本上会成为标准。

Um, and in a lot of ways kind of can become the leader. And, um, in that way, it just, it just ensures that it's a much more kind of, even and balanced playing field. Yeah. That seems plausible to me. And if that works out, that would be the future I prefer. Um, I guess I want to understand like mechanistically how if somebody was going to cause mayhem with AI systems, how the fact that there are other open source systems in the world prevents that. Like the specific example of like somebody coming with a bio weapon, um, is it just that we'll do a bunch of like R and D in the rest of the world to like figure out the vaccines really fast? Like what's happening? Like the computer, the security one that I was talking about, I think someone with a weaker AI trying to hack into a system that is like protected by a stronger AI will succeed less. Mm. Right.

嗯，在很多方面，他们可能成为领导者。以这种方式，可以确保更加公平和均衡的竞技场。是的，这对我来说似乎是有道理的。如果能实现，那将是我偏爱的未来。嗯，我想了解的是，如果有人想通过人工智能系统制造混乱，那么全球存在其他开源系统会如何阻止这种情况。比如说，有人拿着生物武器，是不是我们会研发出疫苗来迅速解决问题？发生了什么事情？就像我之前提到的那个安全的计算机，我认为一个使用较弱人工智能尝试入侵由更强人工智能保护的系统会成功的可能性较低。对。

So, so I think that that's, um, I mean, that's like in terms of how do you know everything in the world is like that? Like what if bio weapons aren't like that? No, I mean, I don't know that everything in the world is like that. Um, um, I think that that's, I guess one of the bio weapons are one of the areas where I think the people who are most worried about the stuff are focused. And I think that that's a, I think that makes a lot of sense to think about that. Um, the, I mean, I think that there are certain mitigations you can try to not train certain knowledge into the model, right? There's different things, but, um, yeah, I mean, it's some level. I mean, if you get a sufficiently bad actor and you don't have other AI that can sort of balance them, um, and understand what's going on and what the threats are, then, um, then that could be a risk.

所以，我认为，那个，我是说，那就像是世界上的一切都是那样吗？比如生物武器不是那样的吗？不，我是说，我不知道世界上的一切都是那样的。我认为那个，我觉得生物武器是最让人担心的领域之一。我认为重点应该放在那里是有道理的。我认为有一些缓解方法可以尝试不把某些知识灌输到模型中，对吧？有不同的方法，但是，在某种程度上，如果你遇到一个非常坏的行为者，而且你没有其他能够平衡他们并理解正在发生的事情和威胁的人工智能，那么那可能是一个风险。

So I think that that's one of the things that we need to watch out for. Mm. Um, is there something you could see in the deployment of these systems where, uh, you, you observe like you're training llama for and it's like light you because you thought you weren't noticing or something and you're like, whoa, I, what, what's going on here? Um, not that you, this is probably not likely with a lama for test system, but is there something you can imagine like that where you'd like we really concerned about deceptiveness and, and if like billions of copies of things are on the wild? Um, yeah. I mean, I think that that's not necessarily, I mean, right now it's where you see a lot of hallucinations. Yeah. Right. It's more, more that, um, um, I think it's an interesting question how you would tell the difference between a hallucination and deception.

所以我认为这是我们需要注意的事情之一。嗯。呃，在这些系统部署中，你能看到一些什么，比如你在训练羊驼时，突然感觉好像被轻视了因为你以为你没注意到什么，然后你会觉得“哇，我，发生了什么？”。不是说，这在一个羊驼测试系统中可能不太可能发生，但有没有一种你能想象的情况，你会真的很担心欺骗和，如果像这样的事情在野外有成千上万的拷贝存在？嗯。我想这并不一定是，我是说现在你会看到很多幻觉。是的。对。更多的是，我认为有趣的问题是如何区分幻觉和欺骗。

But yeah, I mean, I, look, I mean, I think there's a lot of risks and things to, to think about the, um, the flip side of all of this is that there are also a lot of, I try to, in, in, in running our company at least balance what I think of as these longer term theoretical risks, um, with what I actually think are quite real risks that exist today. So like when you talk about deception, the form of that that I worry about most people using this to generate misinformation and then like pump that through whether it's our networks or others. So the way that we've basically. Combated a lot of this type of harmful content is by building AI systems that are smarter than the adversarial ones. And like this is part of this kind of informs part of my theory on this, right? Is if you look at like the different types of harm that people do or try to do through, through social networks, um, there are ones that are not very adversarial. So for example, like, uh, hate speech, I would say is not super adversarial in the sense that like people aren't getting better at being racist, right? They're just like, it's, you just like, okay, if you, you kind of, that's one where I think the AIs are generally just getting way more sophisticated, faster than people are at those issues.

但是，我是说，我想说的是，我认为有很多风险和需要考虑的事情。另一方面，也存在许多……至少在经营我们公司时，我试图平衡自己认为是长期理论风险与实际存在的相当真实的风险。比如，当谈到欺骗时，我最担心的是大多数人利用这一点来生成虚假信息，然后通过我们的网络或其他渠道传播。我们基本上通过建立比对手更聪明的AI系统来对抗这种有害内容。我的理论的一部分。如果你看看人们通过社交网络做的或试图做的不同类型的伤害，有些并不是非常对抗性的。例如，仇恨言论并不是在种族主义方面变得更好，人们只是，你仅仅认为，我认为AI通常在这些问题上比人们更快更复杂。

So we have, and we have issues both ways. It's like people do bad things that, whether they're trying to incite violence or something. Um, but we also have a lot of false positives, right? So where we, where we basically sense our stuff that we shouldn't. And I think understandably make a lot of people annoyed. So I think having an AI that just gets increasingly precise on that, that's going to be good over time. But let me give you another example, which is like nation states trying to interfere in elections. That's an example where they are absolutely, they have cutting edge technology and absolutely get better each year. So we block some technique. They learn what we did. They come at us with a different technique, right? It's not like a person trying to, you know, say, say mean things. Right. It's like, it's, it's the, they're, they're basically they have a goal. They're sophisticated. They have a lot of technology. Um, in those cases, I still think the ability to kind of have RAI systems grow and in sophistication to faster rate than theirs have. It's an arms race, but I think we're at least currently winning that arms race. Um, so I don't know, I think that that's, but this is like a lot of the stuff that I, that I spend time thinking about is like, okay. Yes, it is possible that whether it's llama four or llama five or llama six. Yeah. We need to think about like what behaviors were, were observing. And it's not just us. I think part of the reason why you make this open source is that there are a lot of other people who study this too. So yeah, we want to see what other people are observing, what we're observing, what we can mitigate, and then we'll make our assessment on whether we can make it, um, open source. But I, I think for the foreseeable future, I'm optimistic we will be able to. And in the near term, I don't want to take our eye off the ball of what our actual bad things that people are trying to use the models for today, even if they're not existential, but they're like, they're like pretty bad kind of day to day harms that we are familiar with and running our services. Um, that's actually a lot of what we have to, I think, spend our time on as well. Yeah. Yeah. Um, actually, I found this synthetic data thing really curious. Uh, I'm actually interested in why you don't think, uh, like current models, it makes sense why there might be an asymptote with just doing the synthetic data again and again, if it gets smarter and uses a kind of techniques you talk about in the paper or the blog post that's coming out, um, on the day this will be released, where it goes to the thought chain that is the most, um, correct. Why you, why this wouldn't like lead to a loop that, of course, it wouldn't be overnight, but over many months or years of training potentially with a smarter model, it gets smarter, makes better output, gets smarter and so forth.

所以我们有这样的问题，两种方式都有。就好像有些人做坏事，不管是企图煽动暴力或其他什么。但我们也有很多错误的判断，对吧？所以我们有时对我们不应该审查的内容感到恼火，我想这会让很多人感到不耐烦。所以我认为，拥有一个越来越精确的人工智能系统，随着时间推移会带来好处。但让我举个例子，比如国家试图干涉选举。这就是一个例子，他们绝对拥有尖端技术，而且每年都会变得更好。所以我们阻止了一些技术。他们学习我们的方法，然后用不同的技术攻击我们，对吧？这不像一个人试图说或者做一些恶意的事情。他们有一个目标，他们很复杂，拥有很多技术。在这些情况下，我依然认为，我们能够让人工智能系统的增长和成熟速度超过他们是至关重要的。这是一场军备竞赛，但我认为我们目前至少在这场军备竞赛中处于领先地位。所以我认为这是一些我花时间思考的很多问题之一。, 无论是 llama 4、llama 5 还是 llama 6，我们需要考虑我们正在观察的行为。而且不只是我们，我认为我们选择开源的原因之一是有很多其他人也在研究这个问题。所以是的，我们想看看其他人正在观察什么，我们正在观察什么，我们可以如何减少研究，然后我们会评估我们是否可以将其开源。但在可预见的未来，我乐观地认为我们将能够实现这一点。在短期内，我不希望我们忽视那些人们试图用模型做出的实际坏事，即使它们并非存在主义问题，但它们是那些我们熟悉并且正在运行服务的日常伤害。这实际上也是我们需要花时间处理的很大一部分内容。是的，我真的觉得这个合成数据很有趣。我很感兴趣，为什么你会认为，现在的模型，如果一遍又一遍地使用合成数据，可能会出现渐近线的情况，如果它变得更聪明并使用你在论文或者即将发布的博客文章中提到的那种技术，会走向最正确的思维链。为什么你觉得这不会导致一个循环，当然这不会一夜之间发生，但在训练很多个月或者几年，潜在地用一个更智能的模型，变得更聪明，做出更好的输出，变得更聪明，如此往复。

Um, well, I think it could be within the parameter of whatever the model architecture is. It's just that like it's some level. I don't know. I think that today is eight billion parameter models. I just don't think you're going to be able to get to be as good as the state of the art, multi-hundred billion parameter models that are incorporating new research into the architecture itself. Um, but those will be open source as well, right? Well, yeah, but I think that that's if, I mean, subject to all the questions that we just talked about. Sure. Yes. I mean, we would, we would hope that that'll be the case, but I, but I think that at each point, I don't know. It's like when you're building software, there's like a ton of stuff that you can do with software, but then at some level, you're constrained by the chips that it's running on, right? So there are always going to be different physical constraints. And it's like how big are the models is going to be constrained by how much energy you can get and, um, and use for inference. Um, so I guess I'm simultaneously very optimistic that this stuff will continue to improve quickly. And also a little more measured than I think some people are about kind of it's, I just don't think the runaway case is like a particularly likely one. I think it makes sense to keep your options open. Like there's so much we don't know. There's a case in which like it's really important to keep the balance of power. So when nobody becomes like a totalitarian dictator, there's a case in which like, you don't want to open source, uh, the architecture because like China's catch, can use it to catch up to America's AIs and like there is an Intel explosion. And they like win that. Yeah. A lot of things can be possible. Just like keeping your options open considering all of them seems reasonable. Yeah. Let's talk about some other things. Okay.

嗯，我觉得模型的架构可能会在某个范围内。这只是在某种程度上。我不知道。我认为今天是八十亿参数模型。我不认为你能够达到像包含新研究成果的架构一样优秀的最先进的数百亿参数模型。但这些也会是开源的，对吧？嗯，是的，但我认为那是如果，我是说，受到我们刚刚讨论的所有问题的影响。当然。是的。我指的是，我们希望会是这样，但我想在每个阶段，我不知道。就像你正在构建软件一样，有很多可以用软件做的事情，但在某种程度上，你会受到它运行的芯片的限制，对吧？所以总会有不同的物理限制。就像模型有多大会受到你能获得并用于推断的能量的限制一样。所以我同时非常乐观地认为这些东西会继续快速改进。也比一些人更加审慎，我认为它不太可能像一个特别可能的情况一样失控。我认为保留选择的空间是有意义的。因为有很多我们不知道的事情。有一种情况下，保持权力平衡非常重要。因此，当没有人成为极权独裁者时，是很重要的。还有一种情况是，你不希望开源架构，因为中国可以用它来赶上美国的人工智能，并且获得主导地位。很多事情都是可能的。像保持选择的开放性，考虑所有这些可能性似乎是合理的。好的，让我们谈谈其他事情。好的。

Metaverse, what time period in human history would you be most interested in going into? A 100,000 BCE to now. You just want to see what it was last for the past. Yeah. It has to be the past. Oh, yeah. It has to be the past. Um, I don't know. I mean, I have the periods of time that I'm interested. I'm really interested in American history and classical history. And, um, I'm really interested in the history of science too. So I actually think seeing in trying to understand more about how some of the big advances came about. I mean, all we have are like somewhat limited writings about some of that stuff. I'm not sure the metaverse is going to let you do that because I mean, it's, um, you know, we can't, we're, it's going to be hard to kind of go back in time for things that we don't have records of, but, uh, I'm actually not sure that going back in time is going to be that, that important thing for them. I mean, I think it's going to be cool for like history classes and stuff, but, um, that's probably not the use case that I'm most excited about for the, for the metaverse overall.

元宇宙，你最感兴趣去的是人类历史中的哪个时期？从公元前10万年到现在。你只想看看过去是什么样的。是的，一定是过去。哦，是的，一定是过去。嗯，我不知道。我对美国历史和古典历史很感兴趣。我也对科学史很感兴趣。所以我想看看并努力理解一些重大进步是如何产生的。我是说，我们对这些事情只有一些有限的文字记录。我不确定元宇宙是否会让你这样做，因为我们很难回到那些没有记录的时代，但我其实并不确定倒退时间对他们来说是否那么重要。我认为这对历史课程等方面很酷，但对我来说，这可能不是我对元宇宙最感兴趣的用例。

I mean, it's, um, I mean, the main thing is just the ability to feel present with people, no matter where you are. I think that's going to be killer. I mean, there's, um, I mean, in the AI conversation that we, that we're having, I mean, it's, uh, you know, so much of it is about physical constraints that kind of underlie all of this, right? And you want to move, I mean, one lesson of technology is you want to move things from the physical constraint realm into software as much as possible because software is so much easier to build and, and evolve. And like you can democratize it more because like not everyone is going to have a data center, but like a lot of people can, can kind of write code and take open source code and modify it. Um, the metaverse version of this is, I think, enabling realistic digital presence is going to be just an absolutely huge difference for, um, for making it so that, um, people don't feel like they have to physically be together for as many things. Um, now, I mean, I think that there are going to be things that are better about being physically together. Um, so it's not, I mean, these things aren't binary. It's not going to be like, okay, now it's, you don't need to do that anymore. But, um, but overall, I mean, I think that this, it's just going to be really powerful for, for socializing, for feeling connected with people, for working, um, for, I don't know, parts of industry, for medicine, for like, a lot of, like so many things.

我的意思是，嗯，我的意思是，最重要的是要能够在任何地方与人们保持当下的感觉。我觉得这将是致命的。我是说，在我们正在进行的人工智能对话中，其实很多都涉及到潜在的物理约束，对吧？你想要移动的一项技术教训就是尽可能将事情从物理约束的领域转移到软件中，因为软件的建设和演化都要容易得多。并且你可以让更多的人参与，因为并非每个人都会拥有数据中心，但是很多人可以编写代码，可以使用开源代码并进行修改。虚拟现实版本的这一点是，我认为实现逼真数字化存在感将会产生巨大的影响，使人们无需在很多事情上感觉一定要亲自在一起。当然，我认为有些事情在亲自在一起时会更好。所以，这些事情并不是非此即彼的。不会像是，好了，现在你不需要再这样做了。但总的来说，我觉得这对社交、与人感情联系、工作，甚至对于某些产业、医疗等方面都将是非常强大的。

I want to go back to something you said at the beginning of the conversation where, you didn't sell the company for a billion dollars and like the metaverse, you knew we were going to do this even though the, the, the market was hammering you for it.

我想回到谈话开始时你说的一些内容，你没有以十亿美元的价格出售公司，就像元宇宙一样，你知道我们会这样做，尽管市场一直在指责你。

And then I'm actually curious, like what is the source of that edge? And you said like, Oh, values, I have this intuition, but like everybody says that, right? Like what, if you had to say something that's specific to you, what is, how would you express what that is? Like, why are you so convinced about the metaverse?

然后我实际上很好奇，那种优势的来源是什么？你说像是价值观，我有这种直觉，但是每个人都会说吧？如果你不得不说一些特定于你的东西，你会怎么表达呢？为什么你对元宇宙如此坚信？

Um, well, I think that those are different questions. So what, what, what are the things that, that kind of power me? Um, I think we've talked about it once the theme. So it's, I mean, I, I just really like building things. Um, I specifically like building things around how people communicate and sort of understanding how people express themselves and how people work.

嗯，我觉得这些是不同的问题。那么，那么，那么是什么样的事情让我感到有力量呢？嗯，我想我们曾经谈论过这个主题。所以，我的意思是，我真的很喜欢建造东西。具体来说，我喜欢建造围绕人们如何交流以及理解人们如何表达自己和工作的事物。

Right. I was, everyone was in college. I was, I was studying computer science and psychology. I think a lot of other people in the industry started studying computer science. Right.

对，我当时在大学，大家都在大学。我当时主修计算机科学和心理学。我觉得很多行业里的人开始学习计算机科学。没错。

So, um, it's, uh, it's always been sort of the intersection of those two things for me. But I think it's also sort of this, like really deep drive. I don't know how to explain it, but I just feel like in the constitutionally, like I'm doing something wrong if I'm not building something new. Right.

所以，嗯，对我来说，这两者的交集一直是重要的。但我觉得也是一种深刻的动力。我不知道该如何解释，但我只是感觉到，如果我不建立新事物，就好像违反了本性一样。

And, um, so I think that there's like, you know, even when we're putting together the business case for, you know, investing like a hundred billion dollars in AI or some huge amount in the metaverse.

嗯，所以我认为，你懂的，即使是在我们为投资人工智能或元宇宙等大量资金的商业案例中。

It's like, yeah, I mean, we have plans that I think make it pretty clear that if our stuff works, it'll be a good investment, but like you can't know for certain from the outset. And, um, so there's all these arguments that people have, you know, whether it's like, you know, with advisors or, or different folks.

就好像，是的，我的意思是，我们有计划，我觉得这些计划明确表明，如果我们的东西有效果，这将是一个不错的投资，但是你不能从一开始就确定。所以，人们对此有各种不同的观点和争论，无论是与顾问还是其他人。

It's like, well, how, how could you like it's a, how, how are you confident enough to do this? And it's like, well, the day I stop trying to build new things, I'm just done. I'm going to go build new things somewhere else. Right.

这就像，嗯，你怎么能喜欢这个，你怎么能有足够的信心做这件事？就像，嗯，如果有一天我停止努力去创造新的东西，那我就结束了。我会去其他地方创造新的东西。对吧。

It's like, um, it's like it is, I'm fundamentally incapable of running something or in my own life and like not trying to build new things that I think are interesting. It's like, that's not even a question for me. Right.

这就像，嗯，就像它现在的样子，我本质上无法运行某件东西，或者在我的生活中不试图建造我认为有趣的新事物。这对我来说根本不是问题。是的。

It's like whether, like, whether we're going to go take a swing at like building the next thing. It's like, it's like, it's like, I'm, I'm just incapable of not doing that. Um, and I don't know. I, I'm kind of like this in like all the different aspects of my life. Right.

就像是，无论我们是否要尝试着去构建下一个项目。就好比我无法不去做那件事一样。我不知道。我在生活的各个方面都是这样的。

It's like, we built this like, you know, family built this ranch and coie and like, I just like worked like design all these buildings and like, kind of trying to like, we started raising cattle and I'm like, all right, well, I want to make like the best cattle in the world.

这感觉就像是，我们像是建立了一个家族牧场，我就像是设计了所有这些建筑，然后我们开始养牛，我想要培育世界上最好的牛。

Right. So it's like, how do we, like, how do we architect this so that where we can figure this out and like and build and call the stuff up that we need to try to do that.

对。就像，我们要如何设计这个架构，以便能够弄清楚这些并构建并调用我们需要尝试做的东西。

Um, so I don't know. That's me. Um, what was the other part of the question? Look, meta is just a really amazing tech company. Right. They have all these great software engineers and even they work with Stripe to handle payments.

嗯，我不知道。那就是我。嗯，问题的另一部分是什么？看，Meta只是一家非常厉害的科技公司。他们拥有许多优秀的软件工程师，甚至与Stripe合作处理付款。

And I think that's just a really notable fact that Stripe's ability to engineer these checkout experiences is so good that big companies like Ford, Zoom, a meta, even open AI, they work with Stripe to handle payments.

我认为这是一个非常显著的事实，即Stripe能够打造出如此出色的结账体验，以至于像福特、Zoom、a meta甚至是Open AI这样的大公司都选择与Stripe合作处理付款。

Because just think about how many different possibilities you have to handle. If you're in a different country, you'll pay a different way. And if you're buying a certain kind of item, that might affect how you decide to pay.

因为想象一下你需要处理的多种可能性。如果你在不同的国家，你可能会以不同的方式支付。如果你购买特定类型的物品，这可能会影响你决定如何支付。

And Stripe is able to test these fine grained optimizations across tens of buildings or transactions a day to figure out what will convert people and obviously conversion means more revenue for you.

而Stripe能够每天测试这些精细的优化措施，涵盖数十个建筑物或交易，以找出能够让人们转化的方法，显然，转化意味着对你来说更多的收入。

And look, I'm not a big company like meta or anything, but I've been using Stripe since long before they were advertisers. Stripe Atlas was just the easiest way for me to set up an LLC and they have these payments and invoicing features that make it super convenient for me to get money from advertisers.

看，我不像meta那样的大公司，但我在他们成为广告商之前就开始使用Stripe了。Stripe Atlas只是对我来说成立有限责任公司最简单的方式，他们提供的付款和发票功能使我能够很方便地从广告商那里获得钱款。

And obviously without that, it would be much harder for me to earn money from the podcast. And so it's been great for me. Go to stripe.com to learn more. Thanks to them for sponsoring the episode. Now back to Mark.

很明显，如果没有这个，我要靠播客赚钱就会更困难。所以对我来说，这是个很好的机会。请访问stripe.com了解更多。感谢他们赞助这一集。现在回到Mark。

I'm not sure, but I'm actually curious about something else, which is so in 19 year old Mark reads a bunch of like antiquity and classics, high school college. What important lessons did you learn from it? Not just interesting things you found, but like, there aren't that many tokens who consume by the time you're 19, a bunch of them were about the classics.

我不确定，但我实际上对另一件事很感兴趣，那就是19岁的马克读了一大堆古代和经典文学，这对他有什么重要启发？不仅仅是你发现的有趣事情，而是像，到19岁时，没有那么多人会花时间去阅读的古典文学。

Clearly that was important in some way. tokens. I don't know. That's a good question. I mean, one of the things that I thought was really fascinating is so when Augustus was first, so he became emperor and he was trying to establish peace. And the was no real conception of peace at the time. Like the people's people's understanding of peace was it is the temporary time between when your enemies will inevitably attack you again. So you get like a short rest. And and he had this view, which is like, look, like we want to change the economy from instead of being so mercenary and like, and kind of militaristic to like actually this positive something. It's like a very novel idea at the time. I don't know. I think that there's like something that's just really fundamental about that. It's like in terms of the the bounds on like what people can conceive at the time of like, what are rational ways to work.

显然，那在某种程度上是重要的。代币。我不知道。那是一个很好的问题。我的意思是，我觉得很有趣的一点是，当奥古斯都第一次成为皇帝时，他试图建立和平。当时人们对和平没有真正的概念。人们对和平的理解是敌人再次攻击你之间的临时时间。所以你可以得到一个短暂的休息。他有这样的观点，就是，看，我们想要改变经济，不再像以前那样专注于搜寻金钱和军事，而是真正追求一种积极的东西。这在当时是一种非常新颖的想法。我不知道。我认为这是一种非常基础的想法。在当时，人们能够构想出什么是合理的工作方式方面，这就是人们的限制了。

And I'm going back to like, I'm this applies to both the metaverse and the AI stuff, but like a lot of investors and just different people just can't wrap their head around why we would open source this. And it's like, like, I don't understand, it's like open source that much just be like the temporary time between which you're making things proprietary. And it's, but I actually think it's like this very profound thing in tech that has actually it creates a lot of winners, right? And it's and and I'm so I don't want to strain the analogy too much. But but I do think that there's there's a lot of times, I think, ways where you can that are just like models for building things that people can't even like, they just like often can't wrap their head around how that would be a valuable thing for people to go do, or like a reasonable state of the world that it's I mean, it's I think there's more reasonable things than people think. That's super fascinating.

我觉得这个既适用于元宇宙也适用于人工智能领域，但很多投资者和其他人仍然不能理解为什么我们要开源这些东西。我认为，开源只是在你将产品专有化之间的临时阶段。实际上，我觉得这是科技领域一个非常深刻的事情，实际上会创造很多赢家。我不想过度引申这个比喻，但我认为有很多时候，有一些建立事物的模式，人们甚至无法理解这如何能为人们提供有价值的东西，或者是一个合理的世界状态。我觉得这真的很有趣。

Can I give you my answer when I was thinking, what you might have gotten from it? I took this is probably totally off. But just how young some of these people are who have very important roles in the empire, like Cesar Augustus, like by the time he's 19, he's actually incredibly one of the most prominent people in Roman politics. And he's like leading battles and forming the second time remember it. I wonder if you were like the 19 year old is like, I can actually do this because like Cesar. I think that's I think that's an interesting example, both from a lot of history and American history.

当我在思考时，我能否给你我的答案，你可能从中得到了什么？我觉得这可能完全不对。但是有些这样的人很年轻，却在帝国中担任着非常重要的角色，比如凯撒·奥古斯都，比如他19岁时就是罗马政治中最显要的人物之一。他领导战斗，组织第二次记忆。我想知道，如果你像那个19岁的少年一样，是否也认为自己可以办到这件事，就像凯撒一样。我认为这是一个很有趣的例子，无论是从历史还是美国历史的角度来看。

Yeah, I mean, it's I mean, one of my favorite quotes is it's this Picasso quote that all children are artists and the challenges, how do you remain an artist when you grow up? And it's like basically, I think because when you're younger, I think it's just easier to have kind of wild ideas and you're not, you know, you have no, there are all these analogies to the innovators dilemma that exist in your life as well as your company or whatever you've built, right? So you're kind of earlier on your trajectory, it's easier to pivot and take in new ideas without it disrupting other commitments that you've made to different things. And so I don't know, I think that's an interesting part of running a company is like, how do you, how do you kind of stay dynamic?

是的，我的意思是，我最喜欢的一句话是毕加索说的，所有的孩子都是艺术家，难题是，当你长大后如何保持自己是一个艺术家？基本上，我认为是因为当你年轻的时候，很容易产生狂野的想法，而且你没有，你的生活中也存在着许多关于创新困境的类比，就像你的公司或你建立的任何其他东西一样。因此，在你的发展轨迹上早期阶段，更容易转变并接受新思想，而不会干扰你对其他事物所做的承诺。所以我不知道，我认为这是经营一家公司的有趣之处，就是如何保持活力。

Going back to the investors in open source, the $10 billion model, suppose it's totally safe, you've done these evaluations. And unlike in this case, the evaluators can also fine tune the model, which hopefully will be the case in future models. Would you open source that the $10 billion model? Well, I mean, as long as it's helping us, then yeah. But would it like the $10 billion of R&D? And then now it's like open source, right? Well, I think here's, I think a question, which we'll have to evaluate this as time goes on too. But we have a long history of open sourcing software, right? We don't tend to open source our product.

对于开源投资者而言，这个100亿美元的模型，假设是完全安全的，你已经做了这些评估。与本例不同的是，评估者还可以微调模型，希望将来的模型也会这样。你会开源这个100亿美元的模型吗？嗯，我是说，只要它对我们有帮助，那就是的。但它会像100亿美元的研发一样吗？然后现在又像是开源的，对吧？嗯，我觉得这里有一个问题，我们将不得不评估这一点，随着时间的推移。但我们有着长期开源软件的历史，对吧？我们不倾向于开源我们的产品。

Right. So it's not like we take, we don't take like the code for Instagram and make it open source, but we take like a lot of the low level infrastructure and we make that open source, right? The, the probably the biggest one in our history was open compute project where we took the designs for kind of all of our, our servers, network switches and data centers and made it open source and ended up being super helpful because, you know, I mean, a lot of people can design servers, but now like the industry standardized on our design, which meant that the supply chains basically all got built out around our design, the volumes went up. So it got cheaper for everyone and saved us billions of dollars. So awesome, right?

对的。所以我们并不像拿 Instagram 的代码然后开源，但是我们会拿很多低层次的基础架构来开源，对吧？我们历史上可能最大的一个就是开放计算项目，我们拿出了所有我们服务器、网络交换机和数据中心的设计并进行开源，结果非常有帮助，因为，你知道的，很多人可以设计服务器，但现在整个行业都采纳了我们的设计，这意味着供应链基本上都围绕我们的设计展开，生产量也增加了。所以对每个人来说都变得更便宜，而且我们也省了数十亿美元。很棒，对吧？

Okay, so there's multiple ways we're open source, I think could be helpful for us. One is if people figure out how to run the models more cheaply, well, we're going to be spending tens or like a hundred billion dollars or more over time on all this stuff. So if we can do that 10% more effectively, we're saving billions or tens of billions of dollars. Okay, that's probably worth a lot by itself. Especially if there's other competitive models out there, it's not like our thing is like, be giving away some kind of crazy advantage.

好的，我们在开源方面有多种方式，我认为这对我们可能会很有帮助。其中之一是，如果有人找出了如何更便宜地运行模型的方法，我们将会在未来花费数十亿甚至上百亿美元来进行所有这些工作。因此，如果我们可以更有效地做到这一点，我们将节省数十亿甚至数百亿美元。好的，这本身可能就值得一提。尤其是如果还有其他竞争模型存在，我们并不是在放弃某种疯狂的优势。

So is your view that the trading will be commodified? I think there's a bunch of ways that this could play out. That's one. The, the other is, is that so commodity kind of implies that it's going to get very cheap because there's lots of options. The other direction that this could go in is qualitative improvements. So, so you mentioned fine tuning, right? It's like right now it's, it's, you know, it's pretty limited. What you can do with fine tuning, you major other models out there, and there are some options, but generally not for the biggest models.

你认为交易会被商品化吗？我认为这有很多可能的发展方向。其中一个可能性是，商品化可能意味着价格会变得很便宜，因为有很多选择。另一个可能性是定性改进。你提到了微调，对吧？目前微调的功能还相对有限。虽然有其他模型可以选择，但通常对于最大的模型来说并不适用。

So I think being able to do that and, and be able to kind of do different app specific things or use case specific things or build them into specific toolchains. I think we'll not only enable kind of more efficient development, it could enable qualitatively different things. Here's one analogy on this is, so one thing that I think generally sucks about the mobile ecosystem is that like you have these two gatekeeper companies, Apple and Google that can tell you what you're allowed to build.

因此，我认为能够做到这一点，能够在特定应用或使用情况下做一些不同的事情，或者将它们构建到特定的工具链中。我认为这不仅可以实现更高效的开发，还可以使质量上产生不同的影响。一个类比是，我认为移动生态系统普遍糟糕的一点是，你有这两个守门者公司，苹果和谷歌，他们可以告诉你允许构建什么。

And there are lots of times in our history. So there's the economic version of that, which is like, all right, we build something there just like I'm going to take a bunch of your money. But then there's the, there's the qualitative version, which is actually what kind of upsets me more, which is there's a bunch of times when we've launched or wanted to launch features, and then Apple's just like, nope, you're not launching that. It's like that sucks. Right?

在我们的历史上，有很多这样的情况。所以这是经济层面的问题，就像是，好吧，我们在那里建立了一些东西，就好像我要拿走你的一大笔钱。但还有一种定性版本，这才是让我更加不满意的。有很多时候我们推出或想要推出某些功能，然后苹果就说，不，你不能推出那个。这真糟糕，对吧？

And so the question is what is like, are we kind of set up for a world like that with AI where like, you're going to get a handful of companies that run these closed models that are going to be in control of the APIs and therefore going to be able to tell you what you can build. Well, for one, I can say for us, it is worth it to go build a model ourselves to make sure that we're not in that position. Right? Like, I don't want any of those other companies telling us what we can build. But from an open source perspective, I think a lot of developers don't want those companies telling them what they can build either.

因此，问题就是，我们是否被设置成了一个由人工智能主导的世界，其中只有少数几家公司运行着这些封闭的模型，从而控制着API，并因此能够告诉你可以构建什么。嗯，对于我们来说，值得去构建自己的模型，以确保我们不处于那种被控制的位置。对吧？就像，我不希望其他公司告诉我们可以构建什么。但从开放源代码的角度来看，我认为很多开发者也不希望这些公司告诉他们可以构建什么。

So the question is, what is the ecosystem that gets built out around that? What are interesting new things? How much does that improve our products? I think that there's a lot of cases where if this ends up being like, you know, like our databases or caching systems or architecture, we'll get valuable contributions from the community that will make our stuff better.

因此，问题是，围绕这一生态系统构建了什么？有什么有趣的新事物？这会如何提升我们的产品？我认为在很多情况下，如果这变成了像我们的数据库、缓存系统或架构等，我们将从社区中得到宝贵的贡献，从而使我们的产品变得更好。

And then our app specific work that we do will still be so differentiated that it won't really matter. Right? It's like, we'll be able to do what we do. We'll benefit in all the systems ours and the communities. We better because it's open source. There is one world where maybe it's not that.

然后，我们做的应用程序特定工作仍将具有如此明显的差异性，以至于这并不重要。对吧？就像，我们将能够做我们所做的事情。我们将从我们自己和社区的所有系统中受益。因为它是开源的，所以我们会更好。也许有一个世界，那里可能不是这样。

I mean, maybe the model just ends up being more of the product itself. In that case, then I think it's a trickier economic calculation about whether you open source that because then you are kind of commoditizing yourself a lot. But I don't, from what I can see so far, it doesn't seem like we're in that zone. Do you expect to earn significant revenue from licensing your model to the cloud providers? So they have to pay you a fee to actually serve the model? We want to have an arrangement like that, but I don't know how significant it will be.

我的意思是，也许模型最终会成为产品本身的一部分。在这种情况下，我认为关于是否开源的经济计算会变得更加棘手，因为这样你就会把自己商品化了很多。但从我目前所见，似乎我们并不处于那个领域。您是否期望通过向云服务提供商授权您的模型来赚取可观的收入？因此，他们必须向您支付费用才能实际提供模型？我们希望有这样的安排，但我不知道它会有多大影响。

And we have this, this is basically our license for for llama. Yeah. You know, in a lot of ways, it's it's like a very permissive open source license, except that we have a limit for the largest companies using it. And this is why we put that limit in is we're not trying to prevent them from using it. We just want them to come talk to us because if they're going to just basically take what we built and resell it and make money off of it, then it's like, okay, well, if you're like, you know, Microsoft Azure or Amazon, then yeah, if you're going to reselling the model, then we should have some revenue share on that.

这是我们的羊驼许可证，基本上类似于一个非常宽松的开源许可证，但对于最大的公司使用它有一定限制。我们之所以设置这个限制是因为我们并不想阻止他们使用，我们只是希望他们与我们沟通，因为如果他们只是拿我们建立的东西重新销售并从中赚钱，那么，嗯，如果你是微软Azure或亚马逊，那么是的，如果你要转售该模型，那么我们应该分享一些收入。

So just come talk to us before you go do that. And that's how that's played out. So for llama too, it's I mean, we basically just have deals with all these major cloud companies and llama too is available as a hosted service on all those clouds. And I assume that as we as we release bigger and bigger models, that'll become a bigger thing. It's not the main thing that we're doing, but I just think of others. So those companies are going to be selling our models. It makes sense that we should, you know, share the upside of that somehow. Yeah.

所以在你去做那件事之前先来找我们谈谈。这就是情况。对于llama来说，我们基本上与所有这些主要的云公司都有合作，并且llama也作为托管服务在所有这些云上可用。我认为随着我们发布越来越大型的模型，这将成为一个更大的事情。这并不是我们正在做的主要事情，但我认为要考虑到其他人。所以这些公司将会销售我们的模型。我们应该有某种方式分享其中的好处。是的。

With the rest of the other open source dangers, I think I'm like, genuinely legitimate points about the balance of power stuff. And potentially, like the harms you can get rid of because we have better alignment techniques or something. I wish there was some sort of framework that meta had like other labs have this where they say, like, if we see this is a concrete thing, then that's a no go on the open source or like, even potentially in deployment, just like writing it down. So like, the company is ready for it. People have expectations around it and so forth. Yeah.

与其他开源风险一样，我觉得自己提出了关于权力平衡的真正合理的观点。还有可能，我们可以通过更好的对齐技术之类的方法消除一些伤害。我希望有一种像其他实验室那样的元框架，他们会说，如果我们看到这是一种具体的事情，那么在开源上是不可行的，甚至在部署时也是如此，只要把它写下来。这样，公司就做好了准备，人们对此有了期望，等等。是的。

No, I think that that's a fair point on the existential risks side. Right now, we focus more on the types of risks that we see today, which are more of these content risks. So, you know, we have lines on we don't want the model to be basically doing things that are helping people commit violence or fraud or, you know, just harming people in different ways. So in practice for today's models, and I would guess the next generation, and maybe even the generation after that, I think while it is somewhat more maybe intellectually interesting to talk about the existential risks, I actually think the real harms that need more energy being mitigated are things that are going to like have someone take a model and do something to hurt a person with today is parameters of and kind of the types of kind of more mundane harms that we see today, like people kind of committing fraud against each other, things like that.

不，我认为在存在风险方面这是一个公平的观点。现在，我们更多关注的是我们今天看到的风险类型，这些风险更多是内容方面的。所以，你知道的，我们规定了一些限制，不希望模型基本上会帮助人们犯下暴力行为或欺诈行为，或者以不同方式伤害他人。因此，在实践中针对今天的模型，以及我猜测下一代，甚至之后的一代，我认为虽然谈论存在风险可能在智力上更有趣，但实际上更需要花更多精力去消减的真正伤害，是那些可能会导致某人利用模型做出伤害他人的行为，以及我们今天看到的那种更加平凡的伤害，比如人们互相欺诈等。

So that I just don't want to shortchange that. I think we have a responsibility to make sure we do a good job on that. Yeah, it matters to become a you can handle both. Yeah. Okay, so as far as the open source goes, I'm actually curious if you think the impact of the open source or in PyTorch, open compute, these things has been bigger for the world than even the social media aspects of meta, because I like talk to people who use these services who think like it's plausible, because a big part of the internet runs on these things. It's an interesting question. I mean, I think almost half the world uses are. Yeah, that's a true. So I think it's hard to beat that, but no, I think I think open source is it's really powerful as a new way of building things. And yeah, I mean, it's possible. I mean, it's maybe one of these things where I don't know, like Dell Labs, right, where they it's like they were working on the transistor because they wanted to enable long distance calling. And they did.

所以我只是不想做得马马虎虎。我认为我们有责任确保我们在这方面做得很好。是的，成为一个能够应对两者的人很重要。是的。好的，就开源而言，我实际上很好奇你是否认为开源或PyTorch、开放计算等这些东西对世界的影响比Meat社交媒体方面还要大，因为我和那些使用这些服务的人交谈时他们认为这是可能的，因为互联网的很大一部分都在依赖这些东西。这是一个有趣的问题。我认为几乎有一半的世界人口使用它们。是的，这是真的。所以我认为很难超越这一点，但是，我认为开源作为一种新的构建事物的方式非常强大。是的，我是说，这可能是其中一个，我不知道，像戴尔实验室，他们就是因为想要实现长途通话而研究晶体管的。而他们成功了。

And it ended up being really profitable for them that they were able to enable long distance calling. And if you ask them five to 10 years out from that, what was the most useful thing that they invented, it's like, okay, well, we enabled long distance calling and now all these people are long distance calling. But if you ask 100 years later, maybe it's a different question. So I think that that's true of a lot of the things that we're building, right, reality labs, some of the AI stuff, some of the open source stuff, I think it's like the specific products evolve and to some degree come and go. But I think like the advances for humanity persist. And that's like a, I don't know, cool part of what we all get to do.

他们最终发现，他们能够实现长途通话对他们来说真的很有利可图。如果你问他们从那时起五到十年后，他们发明了什么最有用的东西，他们可能会说，好吧，我们实现了长途通话，现在所有这些人都在进行长途通话。但是如果你问100年之后，也许会得到不同的答案。因此，我认为我们正在开发的很多东西，比如现实实验室，一些人工智能技术，以及一些开源技术，具体产品可能会发展变化，甚至消失，但对人类的进步会持续存在。这是我们所从事的工作中的一个不错的部分。

By when will the llama models be trained on your own custom silicon? Soon, not not not llama for. The approach that we took is first we we basically built custom silicon that could handle inference for our ranking and recommendation type stuff. So reels, newsfeed ads. And that was consuming a lot of GPUs. But when we were able to move that to our own silicon, we now were able to use the more expensive and video GPUs only for training. So at some point, we will hopefully have silicon ourselves that we can be using for probably first training, some of the simpler things that eventually training these like really large models. But in the meantime, I'd say the program is going quite well. And we're just rolling it out methodically and have a long term roadmap for it.

什么时候您们将在自定义硅片上训练羊驼模型？很快了，不是不是不是羊驼。我们采取的方法首先是建立能够处理排名和推荐类别推理的自定义硅片。比如Reels、新闻广告。这些工作一直在消耗大量的GPU。但是当我们能够将这些工作迁移到自己的硅片上，我们就能够只用较昂贵的视频GPU来进行训练。因此，希望在某个时候我们可以拥有自己的硅片，可能首先用于训练一些更简单的模型，最终训练那些非常庞大的模型。但与此同时，我可以说这个项目进展得相当顺利。我们正在有条不紊地推进，并为其制定了长期的路线图。

Final question, this is totally out of left field. But if you were a mate CEO of Google+, could you have made it work Google+, oof. Well, I don't know. I don't know. That's a that's a very difficult, very difficult, counterfactual. Okay, then the real final question will be when Gemini was launched, did you was there any chance that somebody in the office uttered Karthika de Linda est? No, I think we're tamer now. Google, as a market. Yeah, I don't know. It's a good question.

最后一个问题，这完全是出乎意料的。但如果你是Google+的首席执行官，你能让它成功吗？Google+哎呀。嗯，我不知道。我不知道。这是一个非常困难的反事实问题。好的，那么真正的最后一个问题是，当Gemini推出时，你们办公室里有人说过Karthika de Linda est吗？不，我认为我们现在更加谨慎了。Google作为一个市场。是的，我不知道。这是一个很好的问题。

I don't know. The problem is there was no CEO of Google+, it was just like a division within a company. I think it's like, and you asked before about what are the kind of scariest commodities, but you asked about it in terms of dollars. And I actually think for most companies, it's, it's of this scale, at least it's focus, right? It's like when you're a startup, maybe you're more constrained on capital. You know, you just are working on one idea and you might not have all the resources.

我不知道。问题是Google+没有CEO，它只是公司内的一个部门。我觉得，你之前问的是关于最可怕的商品，但你在谈论的是以美元计算。我认为对于大多数公司来说，它至少在关注方面更重要，对吧？就像当你是一家创业公司时，可能会更受到资本的限制。你只是在专注于一个想法，可能并没有所有的资源。

I think you cross some threshold at some point where the nature of what you're doing, you're building multiple things and you're creating more value across them, but you become more constrained on what can you direct and to go well. And like, there's always the cases where something just random awesome happens in the organization. I don't even know about it. And those are, that's great. But like, but I think in general, the organization's capacity is largely limited by what like the CEO and the management team are able to kind of oversee and kind of manage.

我认为在某个时刻你会超越某个阈值，那时你所做的事情的性质会发生变化，你会建立多个事物，为它们创造更多价值，但你也会受到更多限制，无法掌控一切并使之顺利进行。就像总会有一些偶然发生的很棒的事情在组织中，甚至我都不知道。那是很棒的。但是总的来说，组织的能力在很大程度上受限于CEO和管理团队能够监督和管理的范围。

It's, I think that that's just been a big focus for us is like, all right, keep the, as I guess Ben Horowitz says, keep the main thing, right? And, and try to kind of stay focused on your key priorities. Yeah. All right. Awesome. That was excellent. Mark. Thanks so much. That was a lot of fun. Yeah. Really fun. Thanks, Raphne. Yeah. Absolutely. Hey, everybody. I hope you enjoyed that episode with Mark. As you can see, I'm now doing ads. So if you're interested in advertising on the podcast, go to the link in the description.

我认为我们一直在重点关注的是保持重点，就像本·霍洛维兹说的，保持主要事情。并且，试着保持专注在你的重要事务上。是的。好的。太棒了。马克，太棒了。非常感谢。这真的很有趣。是的。非常有趣。谢谢，拉夫尼。绝对是。嘿，大家。希望你们喜欢和马克的这一期。正如你们看到的，我现在在做广告。所以如果你有兴趣在播客上做广告，请点击描述中的链接。

Otherwise, as you know, the most helpful thing you can do is just share the podcast with people who you think might enjoy it, you know, your friends, group chats, Twitter, I guess, threads. Yeah. Hope you enjoyed. And I'll see you on the next one.

否则，就像你知道的那样，你可以做的最有帮助的事情就是与那些你认为可能会喜欢的人分享这个播客，你知道的，你的朋友，群聊，Twitter，我猜，还有一些其他社交平台。是的。希望你喜欢。我们下次见。