Greg Brockman: OpenAI and AGI
发布时间 2019-04-03 15:39:36 来源
The following is a conversation with Greg Brockman. He's the co-founder and CTO of OpenAI, a world-class research organization developing ideas in AI with a goal of eventually creating a safe and friendly artificial general intelligence, one that benefits and empowers humanity.
下面是与Greg Brockman的一段谈话。他是OpenAI的联合创始人兼首席技术官，OpenAI是一个世界一流的研究机构，致力于开发人工智能的各种理念，最终目标是创造出安全友好的人工通用智能，以造福和增强人类。
OpenAI is not only a source of publications, algorithms, tools, and data sets. Their mission is a catalyst for an important public discourse about our future with both narrow and general intelligence systems. This conversation is part of the Artificial Intelligence Podcasts at MIT and beyond. If you enjoy it, subscribe on YouTube, iTunes, or simply connect with me on Twitter at Lex Friedman spelled F-R-I-D.
OpenAI不仅仅是出版物、算法、工具和数据集的来源，他们的使命也是催化重要的公共话语，讨论人工智能的狭义和广义系统对我们未来的影响。这场对话是人工智能播客在麻省理工学院等地的一部分。如果您喜欢，请在YouTube、iTunes订阅或简单地在Twitter上与我联系，我的拼写是F-R-I-D的Lex Friedman。
And now here's my conversation with Greg Brockman.
现在我和格雷格·布洛克曼的谈话分享给大家。
So in high school and right after he wrote a draft of a chemistry textbook, I saw that. That covers everything from basic structure of the atom to quantum mechanics. So it's clear you have an intuition and a passion for both the physical world with chemistry and now robotics to the digital world with AI, deep learning, reinforcement learning, so on.
在高中时以及之后，他写了一份化学教材的草稿，我看到了。那本书覆盖了从原子的基本结构到量子力学的所有内容。所以很明显，你对于化学中的物理世界和现在的机器人学，以及与人工智能、深度学习和强化学习等数字世界的关联充满了直觉和热情。
Do you see the physical world and the digital world is different and what do you think is the gap? A lot of it actually boils down to iteration speed. That I think that a lot of what really motivates me is building things.
你觉得物理世界和数字世界有什么不同，你认为其中的差距是什么？实际上，很大一部分取决于迭代速度。我认为，真正激励我建造东西的原因很大程度上在于此。
The mathematics, for example, where you think it's really hard about a problem. You understand it. You're right down to this very obscure form they call proof. But then this is in humanities library. It's there forever. This is some truth that we've discovered. Maybe only five people in your field will ever read it, but somehow you've moved humanity forward.
比如数学，当你觉得某个问题非常难时，你理解了它，并写下了那些被称为证明的非常深奥的形式。然后，它就会被放在人文图书馆里，永久保存。这是我们发现的某种真理。也许只有你领域中的五个人会读到它，但你已经推动了人类的前进。
And so I actually used to really think that I was going to be a mathematician. And then I actually started writing this chemistry textbook. One of my friends told me, you'll never publish it because you don't have a PhD. So instead, I decided to build a website and try to promote my ideas that way. And then I discovered programming. And the programming you think hard about a problem, you understand it, you're right down, and a very obscure form that we call a program. But then once again, it's in humanities library. And anyone can get the benefit from it. And the scale building is massive.
我以前特别认为自己会成为一名数学家，但后来我开始写化学课本了。我的一个朋友告诉我，因为我没有博士学位，我永远都不会出版它。所以，我决定建一个网站，试着通过它推广我的想法。之后我开始接触编程。编程就像你认真思考一个问题，理解它，把它写得非常抽象，我们称之为程序。但是一旦完成，它就可以被任何人利用，受益者众多，覆盖广泛。
And so I think that the thing that really appeals to me about the digital world is that you can have this insane leverage. It's single individual with an idea is able to affect the entire planet. And that's something I think is really hard to do if you're moving around physical atoms.
我认为数字世界最吸引我的地方是它具有强大的杠杆作用。一个人有一个想法就能够影响整个地球。如果你涉及到物理原子的移动，这是非常难做到的。
But you said mathematics. So if you look at the wet thing over here, our mind, do you ultimately see it as just math, as just information processing, or is there some other magic as you've seen if you've seen through biology and chemistry and so on?
但是你说过数学。所以，如果你看看这边潮湿的东西，我们的思维，你是最终将它看作只是数学，只是信息处理，还是像你在生物和化学中看到的那样有些魔力呢？如果有必要，可以改一下措辞。
I think it's really interesting to think about humans as just information processing systems. It seems like it's actually a pretty good way of describing a lot of how the world works or a lot of what we're capable of, to think that, again, if you just look at technological innovations over time, in some ways, the most transformative innovation that we've had has been the computer.
我认为把人类看作只是信息处理系统是非常有趣的。这似乎是一种相当好的描述世界作用的方式，它也能描述我们所能做的很多事情。如果你看看随着时间的推移，科技创新的进步，那么在某些方面，我们最具有变革性的创新就是计算机。
In some ways, the internet, what is the internet done? The internet is not about these physical cables. It's about the fact that I am suddenly able to instantly communicate with any other human on the planet. I'm able to retrieve any piece of knowledge that, in some ways, the human race has ever had, and that those are these insane transformations.
在某些方面，互联网为我们做了什么？互联网并不是关于这些物理电缆的。它关乎的是我突然可以即刻与地球上任何其他人实时通讯的事实。我可以获取到某种程度上人类曾经拥有的任何知识，这是一个疯狂的转变。
Do you see our society as a whole, the collective, as another extension of the intelligence of the human being? So if you look at the human being as an information processing system, you mentioned the internet, the networking. Do you see us all together as a civilization as a kind of intelligent system?
你是否认为整个社会、集体，是人类智慧的另一种延伸？如果你把人类看作信息处理系统，你提到了互联网、网络。你是否认为我们所有人一起作为文明的一种智能系统？
Yeah, I think this is actually a really interesting perspective to take and to think about that you sort of have this collective intelligence of all society. The economy itself is this superhuman machine that is optimizing something. And it's, in some ways, a company has a will of its own, that you have all these individuals who are all pursuing their own individual goals and thinking really hard and thinking about the right things to do.
是的，我认为这是一种非常有趣的观点，值得考虑。我们可以把社会的所有人的智慧集合起来。经济本身就像一个超级人工智能机器，正在优化某些东西。在某种程度上，一家公司有自己的意志，所有的个体都在追求自己的目标，并且认真思考正确的做法。
But somehow the company does something that is this emergent thing, and that is a really useful abstraction. And so I think that in some ways, we think of ourselves as the most intelligent things on the planet and the most powerful things on the planet. But there are things that are bigger than us that are these systems that we all contribute to.
不知何故，公司做出了这种紧急的事情，这是一个非常有用的抽象概念。所以我认为在某种程度上，我们认为自己是地球上最聪明、最有力量的存在。但是有些系统比我们更大，我们都为这些系统做出了贡献。
And so I think actually, it's interesting to think about if you've read Isaac Eisenmobs Foundation, right, that there's this concept of psychohistory in there, which is effectively this, that if you have trillions or quadrillions of beings, then maybe you can actually predict what that being, that huge macro being will do and almost independent of what the individuals want.
所以我认为实际上，这很有趣，如果你读过艾萨克·阿西莫夫的《基地》的话，那里面有一个心理历史的概念，就是说如果你有数以万亿计的生命体，那么也许你可以预测出这个巨大的宏观生物将要做什么，几乎独立于个体的意愿。
And I actually have a second angle on this, I think is interesting, which is thinking about technological determinism. One thing that I actually think a lot about with OpenAI, is that we're kind of coming on to this insanely transformational technology of general intelligence, that will happen at some point.
我实际上对此有第二个角度，我认为很有趣，那就是思考技术决定论。我在OpenAI想了很多的一件事是，我们正在接近某种无比变革性的普通智能技术，这将在某个时候发生。
And there's a question of how can you take actions that will actually steer it to go better rather than worse? And that I think one question you need to ask is, as a scientist, as an inventor, as a creator, what impact can you have in general?
还有一个问题，那就是你如何采取行动，让事情变得更好而不是更糟？我认为你需要问的一个问题是，作为科学家、发明家、创造者，你可以在一般方面产生什么影响？
Right, you look at things like the telephone invented by two people on the same day. Like what does that mean? Like what does that mean about the shape of innovation? And I think that what's going on is everyone's building on the shoulders of the same giants. And so you can kind of, you can't really hope to create something no one else ever would.
哦，你看那些由同一天发明的两个人发明的电话之类的事情。那是什么意思？那意味着创新的形式是什么？我认为现在的情况是每个人都在同样的巨人肩膀上构建。所以你不能真正希望创造出没有人想出过的东西。
You know, if Einstein wasn't born, someone else would have come up with relativity. You know, he changed the timeline a bit, right, that maybe it would have taken another 20 years, but it wouldn't be that fundamentally humanity would never discover these fundamental truths. So there's some kind of invisible momentum that some people like Einstein or OpenAI is plugging into that anybody else can also plug into and ultimately that wave takes us into certain direction. That's what you need to address.
你知道吗，假如没有爱因斯坦，可能就会有别人提出相对论的发现。他其实只是略微改变了时间线，也许需要多花20年的时间，但这并不意味着人类无法发现这些基本真理。所以说，有一种看不见的动力，像爱因斯坦或者OpenAI这样的人，他们所掌握的东西，任何人都能够学习并加以运用，最终这些知识就像一波波的浪潮，将我们引领到特定的方向。这就是需要我们关注的地方。
That's right, that's right. And you know, this kind of seems to play out in a bunch of different ways, that there's some exponential that is being ridden and that the exponential itself, which one it is, changes, think about Moore's Law.
对的，对的。你知道，这种情况似乎以各种不同的方式展现出来，有些指数正在被运用，而指数本身，哪一个被运用，会发生变化，想想摩尔定律。
An entire industry set its clock to it for 50 years. Like how can that be, right, how is that possible? And yet somehow it happened. And so I think you can't hope to ever invent something that no one else will. Maybe you can change the timeline a little bit, but if you really want to make a difference, I think that the thing that you really have to do the only real degree of freedom you have is to set the initial conditions under which a technology is born.
整个行业以它为准50年了，这是怎么回事呢，怎么可能呢？但是不知何故，这就发生了。所以我认为你不能指望自己发明出没有人发明过的东西。也许你可以稍微改变时间线，但是如果你真的想有所作为，我认为你唯一真正自由的事情就是设置技术诞生的初始条件。
And so you think about the internet, right, that there are lots of other competitors trying to build similar things and the internet one, and that the initial conditions were that was created by this group that really valued people being able to be, anyone being able to plug in this very academic mindset of being open and connected.
因此，你考虑互联网，对吧，有许多其他竞争对手试图构建类似的东西和互联网，而最初的条件是由这个非常注重人们能够成为任何人能够插入的团体所创建的非常学术的思维方式，即开放和联系。
And I think that the internet for the next 40 years really played out that way. Maybe today, things are starting to shift in a different direction, but I think that those initial conditions were really important to determine the next 40 years worth of progress. That's really beautifully put.
我认为，在未来的40年里，互联网确实按照那样的方向发展。也许今天，事情开始朝着不同的方向发展，但我认为那些最初的条件对未来40年的进步非常重要。这真的很美。
So another example of that I think about, you know, I recently looked at it. I looked at Wikipedia, the formation of Wikipedia. And I wondered what the internet would be like if Wikipedia had ads. You know, there's an interesting argument that why they chose not to make it, put advertisement on Wikipedia.
那么，我想到另一个例子。最近，我看了Wikipedia，看它是怎么形成的。我想知道如果Wikipedia有广告，互联网会变成什么样子。有一个有趣的争论，为什么他们选择不在Wikipedia上放广告。
I think Wikipedia is one of the greatest resources we have on the internet. It's assuming it's surprising how it works and how well it was able to aggregate all this kind of good information. And essentially the creator of Wikipedia, I don't know, there's probably some debates there, but set the initial conditions. And now it carried itself forward. That's really interesting.
我认为维基百科是我们在互联网上拥有的最好的资源之一。它让人感到惊讶的是它是如何工作的，以及它是如何成功地汇聚了所有这些优秀的信息。本质上，维基百科的创始人，我不知道，可能有一些争论，但他设定了最初的条件。现在，它已经继续向前发展了。这真的很有趣。
So the way you're thinking about AGI or artificial intelligence is you're focused on setting the initial conditions for the progress. That's right. That's powerful.
你对AGI或人工智能的思考方式是把重点放在设置进展的初始条件上。没错。这是非常有力的。
Okay, so looking to the future, if you create an AGI system, like one that can ace the torrent test, natural language, what do you think would be the interactions you would have with it?
好的，那么展望未来，如果你创建了一个能够通过洪流测试和自然语言的人工智能系统，你认为你将与它进行哪些互动呢？
I think the questions you would ask, like what would be the first question you would ask? It hurt him. That's right. I think that at that point, if you've really built a powerful system that is capable of shaping the future of humanity, the first question that you really should ask is how do we make sure that this plays out well?
我认为你会问的问题，比如第一个问题是什么？这让他很疼。没错。我认为在那个时候，如果你真的建立了一个能够塑造人类未来的强大系统，你应该真正需要问的第一个问题是，我们如何确保这能够顺利进行？
And so that's actually the first question that I would ask a powerful AGI system is. So you wouldn't ask your colleague, you wouldn't ask like Ilya, you would ask the AGI system.
所以这实际上是我会问强大的AGI系统的第一个问题。那么你不会问你的同事，也不会问像Ilya这样的人，你会问AGI系统。
Oh, we've already had the conversation with Ilya. Right, and everyone here. And so you want as many perspectives and a piece of wisdom as you can for it for answering this question. So I don't think you necessarily defer to whatever your powerful system tells you, but you use it as one input to try to figure out what to do.
哦，我们已经和伊利亚谈过了。没错，还有这里的每个人。所以你想听尽可能多的不同观点和智慧来回答这个问题。因此我认为你不必完全听从你那强大的系统告诉你该做什么，而是将其作为一个输入来尝试弄清楚要怎么做。
But, and I guess fundamentally, what it really comes down to is if you built something really powerful, and you think about, think about for example, the creation of, of shortly after the creation of nuclear weapons, right? The most important question in the world was what's the world we're going to be like? How do we set ourselves up in a place where we're going to be able to survive as a species?
但是，我认为根本的问题是，如果你建造了一些非常强大的东西，并且你想象一下，在核武器制造之后不久，例如，世界上最重要的问题是世界将会变成什么样子？我们应该如何设立自己的位置，使我们能够作为一个物种生存下去？
With AGI, I think the question is slightly different, right? That there is a question of how do we make sure that we don't get the negative effects? But there's also the positive side, right? You imagine that, you know, like, like what will AGI be like? Like what will it be capable of? And I think that one of the core reasons that an AGI can be powerful and transformative is actually due to technological development, right?
使用人工智能，我认为问题有些不同，是吗？就是如何确保我们不会受到负面影响的问题。但也有积极的一面，对吧？你可以想象，你知道，像人工智能会是怎样的呢？它将能够做些什么？我认为其中一个核心原因是人工智能由于技术的发展而变得强大和转变，对吧？
If you have something that's capable, that's capable as a human, and that it's much more scalable that you absolutely want that thing to go read the whole scientific literature and think about how to create cures for all the diseases, right? You want it to think about how to go and build technologies to help us create material abundance and to figure out societal problems that we have trouble with, like how are we supposed to clean up the environment?
如果你有一些能力强的东西，像人类一样有能力，而且它的可扩展性比较强，那么你肯定希望这个东西能够阅读整个科学文献，思考如何治愈所有疾病，对吧？你希望它能够思考如何构建技术来帮助我们创造物质丰富，解决我们在社会问题上遇到的难题，比如我们该如何清理环境？
And, you know, maybe you want this to go and invent a bunch of little robots that will go out and be biodegradable and turn ocean debris into harmless molecules. And I think that that positive side is something that I think people miss sometimes when thinking about what an AGI will be like.
你知道，也许你想要发明一堆小型机器人，让它们变成可生物降解的，把海洋垃圾变成无害的分子。我认为这种积极的一面有时被人们忽视了，当他们想到智能人工智能将会是怎样的时候。
And so I think that if you have a system that's capable of all of that, you absolutely want its advice about how do I make sure that we're using your capabilities in a positive way for humanity. So what do you think about that psychology that looks at all the different possible trajectories of an AGI system, many of which perhaps the majority of which are positive and nevertheless focuses on the negative trajectories.
因此，我认为如果你有一个能够做到这一切的系统，你绝对想要它的建议，关于如何确保我们以对人类有益的方式使用它的能力。那么，你对那个心理学看待AGI系统所有可能轨迹的做法有什么看法呢？其中许多可能都是积极的，然而仍然聚焦于负面轨迹。
I mean, you get to interact with folks, you get to think about this maybe within yourself as well, you look at Sam Harris and so on. It seems to be, sorry to put it this way, but almost more fun to think about the negative possibilities. Whatever that's deep in our psychology, what do you think about that? And how do we deal with it? Because we want AGI to help us.
我是说，你可以与人们互动，也可以在心里思考这个问题，看看萨姆·哈里斯等人。但是，很抱歉要这么说，想象负面的可能性似乎更有趣。不管那深藏在我们心理中的东西是什么，你对此有什么看法？我们该如何处理它？因为我们想让人工智能帮助我们。
So I think there's kind of two problems that are entailed in that question. The first is more of the question of, how can you even picture what a world with a new technology will be like? Now imagine we're in 1950 and I'm trying to describe Uber to someone.
我认为这个问题涉及到两个问题。首先，更多是关于如何想象使用一种新技术的世界会是什么样子。现在假设我们处在1950年，我正在尝试向某人描述Uber。
Apps and the internet. Yeah, I mean, you're, that's going to be extremely complicated, but it's imaginable. It's imaginable, right? But and now imagine being in 1950 and predicting Uber, right? And you need to describe the internet. You need to describe GPS. You need to describe the fact that everyone's going to have this phone in their pocket.
应用程序和互联网。是的，我的意思是，这将是非常复杂的，但是可以想象。可以想象，对吧？但是现在想象一下1950年预测优步，对吧？你需要描述互联网。你需要描述全球定位系统。你需要描述每个人都将拥有这个手机在他们的口袋里。
And so I think that just the first truth is that it is hard to picture how a transformative technology will play out in the world. We've seen that before with technologies that are far less transformative than AGI will be. And so I think that one piece is that it's just even hard to imagine and to really put yourself in a world where you can predict what that positive vision would be like.
我认为第一个事实就是很难想象一种改变性技术在世界上的表现形式。我们已经看过了那些没有AGI那么具有变革性的技术的情况了。因此，我认为其中一个难题就是甚至很难想象和真正置身于一个可以预测积极愿景的世界中。
And I think the second thing is that it is, I think it is always easier to support the negative side than the positive side. It's always easier to destroy than create. And less in a physical sense and more just in an intellectual sense, right? Because I think that with creating something, you need to just get a bunch of things right and to destroy, you just need to get one thing wrong.
我认为第二件事是，支持否定的一面总是比支持肯定的一面容易得多。毁灭总是比创造容易得多。在智力层面上更加如此，对吧？因为我认为如果要创造什么，你需要做的是让很多东西都正确，而如果要毁灭，则只需要做错一件事就可以了。
And so I think that what that means is that I think a lot of people is thinking dead ends as soon as they see the negative story. But that being said, I actually have some hope, right? I think that the positive vision is something that I think can be, is something that we can talk about.
所以我认为这意味着很多人一看到负面的故事就认为是死路了。但话虽如此，我实际上还有一些希望，对吧？我认为积极的愿景是可以被谈论的。
I think that just simply saying this fact of, yeah, like there's positive, there's negatives, everyone likes to dwell on the negatives. People actually respond well to that message and say, huh, you're right, there's a part of this that we're not talking about, not thinking about. And that's actually something that's, I think, really been a key part of how we think about AGI at OpenAI, right? You can kind of look at it as like, okay, like OpenAI talks about the fact that there are risks and yet they're trying to build this system. Like, how do you square those two facts?
我觉得只是简单地说这个事实，就是说有积极的和消极的两面性，但是大家喜欢纠结于消极的那一面，其实人们对这个信息反应不错，会说：哦，你说得对，这里有一个我们没有谈论、没有考虑到的方面。而这实际上是 OpenAI 思考 AGI 的一个关键部分。你可以把它看作是 OpenAI 提到了这个系统的风险，然而他们仍在试图构建这个系统，那么如何平衡这两个事实呢？
So do you share the intuition that some people have, I mean, from Sam Harris to even Elon Musk himself, that it's tricky as you develop AGI to keep it from slipping into the existential threats into the negative. What's your intuition about how hard is it to keep AGI development on the positive track? And what's your intuition there?
那么，您是否也有一些人（例如萨姆·哈里斯甚至埃隆·马斯克）所持有的直觉，即随着AGI的发展，它很容易滑向消极的存在威胁。您对于保持AGI发展走向积极方向有何直觉？您认为这有多难？
To answer that question, you can really look at how we structure OpenAI. So we really have three main arms. We have capabilities which is actually doing the technical work and pushing forward what these systems can do. There's safety, which is working on technical mechanisms to ensure that the systems we build are aligned with human values. And then there's policy, which is making sure that we have governance mechanisms answering that question of, well, whose values?
回答这个问题，你可以看看我们是如何组织OpenAI的。所以我们有三个主要的部门。我们有能力部门，他们实际上是在做技术工作，并推进这些系统能做到什么。我们有安全部门，他们正在研究技术机制来确保我们建立的系统与人类价值观相一致。然后我们有政策部门，他们确保我们有治理机制来回答这个问题:“哪些价值观？”
And so I think that the technical safety one is the one that people kind of talk about the most, right? You talk about like, think about, you know, all of the dystopic AI movies a lot of that is about not having good technical safety in place. And what we've been finding is that, you know, I think that actually a lot of people look at the technical safety problem and think it's just intractable.
所以我认为技术安全性是人们最常谈论的一个问题，对吧？你会想到所有的反乌托邦人工智能电影，很多都是与没有建立良好技术安全性有关。我们发现，实际上很多人认为技术安全问题是不可解决的。
Right, this question of what do humans want? How am I supposed to write that down? Can I even write down what I want? No way. And then they stop there. But the thing is we've already built systems that are able to learn things that humans can't specify. You know, even the rules for how to recognize if there's a cat or a dog in an image. Turns out it's intractable to write that down. And yet we're able to learn it.
这个问题是关于人类想要什么的，我应该如何写下来呢？我是否能够写出我想要的东西呢？不可能。然后他们就停住了。但事实上，我们已经建立了能够学习人类无法规定的东西的系统。你知道，甚至是如何辨认图像中是否有猫或狗的规则。结果证明，写下这个是不可行的。但是我们能够学习它。
And that what we're seeing with systems we build at OpenAI and they're still in early proof of concept stage is that you are able to learn human preferences, you're able to learn what humans want from data. And so that's kind of the core focus for our technical safety team. And I think that they're actually, we've had some pretty encouraging updates in terms of what we've been able to make work.
我们在OpenAI所构建的系统仍处于初步的概念验证阶段，但我们所看到的是它们能够学习人类的偏好，能够从数据中了解人们想要什么。所以这是我们技术安全团队的核心重点。我认为我们已经取得了一些相当令人鼓舞的进展。
So you have an intuition and a hope that from data, you know, looking at the value alignment problem, from data we can build systems that align with the collective better angels of our nature. So align with the ethics and the morals of human beings. To even say this in a different way, I mean, think about how do we align humans, right? Think about like a human baby can grow up to be an evil person or a great person. And a lot of that is from learning from data, right? That you have some feedback as a child is growing up, they get to see positive examples.
所以你有一种直觉和希望，希望从数据中，了解价值观问题，并从中构建符合我们本性中更好的天使的系统。也就是说，让系统与人类的道德伦理相符合。换句话说，想想如何让人类趋同，对吗？想想一个人类婴儿可能成为恶人或伟人。其中很多都来自于学习数据，你在孩子成长过程中得到一些反馈，他们可以看到积极的例子。
And so I think that just like them, that the only example we have of a general intelligence that is able to learn from data to align with human values and to learn values, I think we shouldn't be surprised that we can do the same sorts of techniques or whether the same sort of techniques end up being how we solve value alignment for AGIs.
所以我认为就像它们一样，我们只有一个普遍智能的例子，能够从数据中学习以符合人类价值观并学习价值观。我认为我们不应该感到惊讶，我们也能使用同样的技术或者如果这些技术是我们如何解决AGIs价值匹配的方法，我们也应该采用同样的技术。
So let's go even higher, I don't know if you've read the book Sapiens, but there's an idea that, you know, that as a collective as us human beings, we kind of develop together ideas that we hold. There's no, in that context, objective truth, we just kind of all agree to certain ideas and hold them as a collective.
那我们继续往更高的层面想，我不知道你是否读过《智人》这本书，但书中有一个观点，就是说我们人类作为集体，会共同发展出一些观念。在这个背景下，并不存在所谓的客观真理，我们只是共同认同某些观念而抱持它们。
Did you have a sense that there is in the world of good and evil? Do you have a sense that to the first approximation, there are some things that are good and that you could teach systems to behave, to be good? So I think that this actually blends into our third team, right, which is the policy team.
你有没有觉得世界上有好与邪恶的存在？你有没有感受到，大致上有些事情是对的，你可以教系统去表现良好？所以我认为这实际上融合到了我们的第三个团队，对吧，这就是政策团队。
And this is the one, the aspect that people really talk about way less than they should, right? Because imagine that we build super powerful systems that we've managed to figure out all the mechanisms of these things to do whatever the operator wants.
这就是人们真正应该谈论得更多的方面之一，对吧？因为想象一下，我们构建了超级强大的系统，我们已经成功地确定了这些东西的所有机制，以便执行任何操作员想要的操作。
The most important question becomes, who's the operator, what do they want, and how is that going to affect everyone else, right? And I think that this question of what is good, what are those values? I mean, I think you don't even have to go to those, those very grand existential places to start, to realize how hard this problem is, you just look at different countries and cultures across the world and that there's a very different conception of how the world works and, you know, what kinds of ways that society wants to operate.
最重要的问题是，是谁在操作，他们想要什么，以及这会对其他人产生什么影响，对吧？我认为，关于什么是好的、什么是价值观的问题，你甚至不需要去那些非常伟大的存在主义领域，就能意识到这个问题有多困难，你只需要看看世界各地不同的国家和文化，就会发现人们对世界如何运作、社会应该运作的方式有着非常不同的概念。
And so I think that the really core question is actually very concrete. And I think it's not a question that we have ready answers to, right, is how do you have a world where all the different countries that we have, United States, China, Russia, and, you know, the hundreds of other countries out there are able to continue to not just operate in the way that they see fit, but in the world that emerges in these, where you have these very powerful systems, operating alongside humans, ends up being something that empowers humans more that makes, like, human existence be a more meaningful thing and that people are happier and wealthier and able to live more fulfilling lives. It's not an obvious thing for how to design that world once you have that very powerful system.
所以，我认为真正核心的问题实际上非常具体。我认为这不是一个我们已经有准备好答案的问题，正确吧，就是如何让我们拥有的所有不同国家——美国、中国、俄罗斯以及其他数百个国家在一个世界中继续不仅按照自己的方式运作，而且在这些世界中出现的强大系统旁边运作，最终能够赋予人类更多的力量，使人类存在成为更有意义的事情，使人们更快乐、更富裕、能够过上更充实的生活。一旦拥有了这样一个非常强大的系统，如何设计这个世界并不是一个显而易见的事情。
So if we take a little step back, and we're having a, like, a fascinating conversation and open-ass, in many ways, a tech leader in the world, and yet we're thinking about these big existential questions, which is fascinating, really important. I think you're a leader in that space, and that's a really important space, of just thinking how AI affects society in a big picture view.
如果我们稍微退后一步，我们正在进行一个令人着迷的对话，而你是一个非常有开放心态的技术领袖，我们正在思考这些关乎人类存在的大问题，这可真是令人着迷并且非常重要。我认为你是在这个领域的领袖人物，这是一个非常重要的领域，需要思考人工智能如何在整体层面上影响社会。
So Oscar Wilde said, we're all in the gutter, but some of us are looking at the stars, and I think open-ass has a charter that looks to the stars, I would say, to create intelligence, to create general intelligence, to make it beneficial, safe, and collaborative.
奥斯卡·王尔德曾说，我们都在阴沟里，但有些人在仰望星空，我认为OpenAI有一个致力于仰望星空的使命，那就是创造智能，创造通用智能，使其成为有益、安全和合作的。
So can you tell me how that came about, how a mission like that in the path to creating a mission like that, at OpenAI, was founded? Yeah, so I think that in some ways, it really boils down to taking a look at the landscape.
你能告诉我这事是如何发生的吗？就像 OpenAI 创建这样一个使命路径上发生了什么？嗯，我觉得从某种程度上来说，这取决于我们对这个领域的了解。
All right, so if you think about the history of AI, that basically for the past six, six, seven, ten years, people have thought about this goal of what could happen if you could automate human intellectual labor. All right. Imagine you can build a computer system that could do that, what becomes possible? We have a lot of sci-fi that tells stories of various dystopias, and increasingly, you have movies like Her that tell you a little bit about, maybe more of a little bit utopic vision.
好的，如果你想想人工智能的历史，基本上过去六七十年，人们一直在思考这个目标，即如果你可以自动化人类的智能劳动，会发生什么。你可以想象一下，如果你能建造一个能够做到这一点的计算机系统，那么什么就成为可能了呢？我们有很多科幻小说，讲述各种反乌托邦的故事，越来越多的电影也像《Her》一样告诉你，也许更多一点的乌托邦式的愿景。
You think about the impacts that we've seen from being able to have bicycles for our minds in computers, and that I think that the impact of computers in the internet has just far outstripped what anyone really could have predicted. And so I think that it's very clear that if you can build an HGI, it will be the most transformative technology that humans will ever create. And so what it boils down to then is a question of, well, is there a path? Is there a hope? Is there a way to build such a system?
你思考一下，我们能够在电脑中运用“思维自行车”所产生的影响，我认为电脑和互联网所造成的影响已经超出了任何人的预期。因此，我认为，如果能够建立一个HGI，这将是人类所创造的最具变革性的技术。因此，归根结底，问题就在于：有没有一条道路？是否有希望？是否有一种方法来建立这样一个系统？
And I think that for 60 or 70 years, that people got excited and that ended up not being able to deliver on the hopes that people had pinned on them. And I think that then, after two winters of AI development, that people, I think, kind of almost stopped daring to dream, right? That really talking about AGI or thinking about AGI became almost this taboo in the community.
我认为，大约60或70年来，人们曾经兴奋并期望能够实现他们所抱有的希望，但最终未能实现。随后，在人工智能发展的两个冬天之后，我认为人们几乎不再敢于梦想。说起通用人工智能或想到这个问题，几乎成为了该社区的禁忌话题。
But I actually think that people took the wrong lesson from AI history. And if you look back, starting in 1959 is when the perceptron was released. And this is basically one of the earliest neural networks. It was released to what was perceived as this massive overhype. So in the New York Times in 1959, I have this article saying that the perceptron, one day, recognize people, call out their names, instantly translate speech between languages.
但我其实认为人们从AI历史中得到了错误的教训。如果你回顾一下，从1959年开始，感知器就被发布了。这基本上是最早的神经网络之一。当时被认为是这场巨大的过度宣传中发布的。1959年的纽约时报有一篇文章，说感知器有一天会识别人，呼唤他们的名字，瞬间翻译不同语言的语音。
And people at the time looked at this and said, this is, your system can't do any of that. And basically spent 10 years trying to discredit the whole perceptron direction and succeeded. And all the funding dried up. And people kind of went in other directions. And in the 80s, there was this resurgence.
当时的人们看到这个，说这个，你们的体系都做不到这些。他们花了整整10年来贬低知觉器路线，最终成功了。所有的资助也都枯竭了。人们转向了其他方向。到了80年代，有了这种重新兴起的趋势。
And I'd always heard that the resurgence in the 80s was due to the invention of back propagation and these algorithms that got people excited. But actually, the causality was due to people building larger computers. That you can find these articles from the 80s saying that the democratization of computing power suddenly meant that you could run these larger neural networks. And then people started to do all these amazing things.
我一直听说八十年代的复苏归功于反向传播和激发人们热情的这些算法的发明。但实际上，原因是因为人们建造了更大的计算机。你可以找到八十年代的文章说计算能力民主化意味着你可以运行更大的神经网络。然后人们开始做出所有这些惊人的事情。
The back propagation algorithm was invented. And the neural nets, the people were running were these tiny little like 20 neural nets. What are you supposed to learn with 20 neurons? And so of course, they weren't able to get great results. And it really wasn't until 2012 that this approach, that's almost the most simple natural approach that people have come up with in the 50s. And in some ways, even in the 40s, before they were computers with the pits and colonnare in neuron, suddenly, this became the best way of solving problems.
反向传播算法是发明出来的。而那时人们使用的神经网络只有大概20个神经元，这样的规模学到什么东西呢？所以，他们自然得不到理想的结果。直到2012年，这种方法才成为最简单自然的方法之一，而这个方法在50年代的时候就已经被人们想到过了。在某种程度上，甚至在40年代还没有电脑时，人们就在探讨神经元的坑和栏杆，突然之间，这种方法成为了解决问题的最佳途径。
And I think there are three core properties that deep learning has that I think are very worth paying attention to. The first is generality. We have a very small number of deep learning tools, SGD, deep neural net, maybe some RL. And it solves this huge variety of problems. Speech recognition, machine translation, game playing, all of these problems, small set of tools.
我认为深度学习有三个核心特性，值得我们特别关注。第一个是通用性。我们只有很少几种深度学习工具，比如 SGD、深度神经网络和一些强化学习。但是，这些工具可以解决非常多的问题，比如语音识别、机器翻译、游戏对战等。只需要这么少的工具就能完成这么多各不相同的任务。
So there's the generality. There's a second piece which is the competence. You want to solve any of those problems? Throw out 40 years worth of normal computer vision research, replace it with a deep neural net. It's kind of work better.
所以，这就是一般情况。第二个重要的是能力。您想要解决这些问题吗？抛弃40年的普通计算机视觉研究，用深度神经网络来替代它。这种方法可以更好地处理问题。
And there's a third piece which is the scalability. That one thing that has been shown time and time again is that if you have a larger neural network to a more compute, more data at it, it will work better. Those three properties together feel like essential parts of building a general intelligence.
还有第三个要素，那就是可伸缩性。一直以来，已被证实，如果你有一个更大的神经网络，更多的计算和数据，它就会工作得更好。这三个特点在一起似乎是构建一种通用智能的基本要素。
Now, it doesn't just mean that if we scale up what we have, that we will have an AGI, there are clearly missing pieces, there are missing ideas. We need to have answers for reasoning. But I think that the core here is that for the first time, it feels that we have a paradigm that gives us hope that general intelligence can be achievable.
现在，这并不仅意味着我们如果扩大我们目前所拥有的东西，我们就会拥有AGI，显然还有一些遗漏的部分，有一些缺失的想法。我们需要在推理方面得到答案。但我认为这里的核心是，我们第一次感觉到有一种范式可以让我们有希望实现通用智能。
And so as soon as you believe that, everything else becomes into focus. If you imagine that you may be able to, and that the timeline I think remains uncertain, but I think that certainly within our lifetimes, and possibly within a much shorter period of time, then people would expect. If you can really build the most transformative technology that will ever exist, you stop thinking about yourself so much.
所以，一旦你相信了这一点，其他所有事情都变得清晰了。如果你想象自己能够，而且时间线我认为仍然不确定，但我认为在我们的一生中肯定会实现，可能会在更短的时间内实现，那么人们会期待的。如果你真的能够建造出史上最具变革性的科技，那么你就不会再那么自私地思考自己了。
And you start thinking about just like, how do you have a world where this goes well? And that you need to think about the practicalities of how do you build an organization and get together a bunch of people and resources and to make sure that people feel motivated and ready to do it. But I think that then you start thinking about, well, what if we succeed? And how do we make sure that when we succeed, that the world is actually the place that we want ourselves to exist in, and almost in the Royalty and Bale sense of the word? And so that's kind of the broader landscape.
当你开始考虑如何让这个世界变得更加美好时，你需要思考如何建立一个组织、召集一群人和资源，并确保人们感到有动力和准备好去做。但我认为，你会开始思考，如果我们成功了，我们该如何确保成功后的世界真的是我们想要存在的地方，这几乎是指在 Royalty 和 Bale 的意义上？这就是更广泛的景象。
And opening I was really formed in 2015 with that high level picture of, AGI might be possible sooner than people think, and that we need to try to do our best to make sure it's going to go well. And then we spent the next couple of years really trying to figure out what does that mean? How do we do it? And I think that typically with a company, you start out very small, so you're going to co-founder, and you build a product, you get some users, you get a product market fit. At some point you raise some money, you hire people, you scale, and then down the road, then the big companies realize you exist and try to kill you.
在2015年我刚创立公司的时候，我真的想到了一个高水平的想法，就是AGI可能比人们想象的更快实现，我们需要尽力确保它的成功。接下来的几年我们一直在努力思考意味着什么，该怎么做。我想一般而言，公司都是从很小的规模开始，如联合创始人组建一个产品，获取一些用户，获得产品市场适配度。在某个点上，你们会筹集一些资金，招聘人员，扩大规模，然后有一天大公司发现你们的存在，试图消灭你们。
And for OpenAI, it was basically everything in exactly the opposite order. Ah. Let me just pause for a second. You said a lot of things. Let me just admire the jarring aspect of what OpenAI stands for, which is daring to dream. I mean, you said it's pretty powerful. It caught me off guard, because I think that's very true. The step of just daring to dream about the possibilities of creating intelligence in a positive and a safe way, but just even creating intelligence is a much needed, refreshing catalyst for the AI community. So that's the starting point.
而对于OpenAI来说，它的一切都是完全相反的顺序。啊。让我停一下。你说了很多话。让我欣赏一下OpenAI所代表的惊人之处，那就是敢于梦想。我的意思是，你说这很有力量。它让我措手不及，因为我认为这是非常正确的。只是大胆梦想创造正面和安全的智能的可能性，而创造智能本身就是人工智能社区所急需的、令人耳目一新的催化剂。这就是起点。
OK. So then formation of OpenAI. Oh, I just say that when we're starting OpenAI, that kind of the first question that we had is, is it too late to start a lab with a bunch of the best people? Is that even possible? That was an actual question. That was the core question of, we had this dinner in July of 2015. And that was really what we spent the whole time talking about. And you think about where AI was is that it transitioned from being an academic pursuit to an industrial pursuit. And so a lot of the best people were in these big research labs, and that we wanted to start our own one that, no matter how much resources we could accumulate, would be a pal in comparison to the big tech companies. And we knew that. And there's a question of, are we going to be actually able to get this thing off the ground? You need a critical mass. You can't just do you and a co-founder build a product. You really need to have a group of 5 to 10 people. And we kind of concluded it wasn't obviously impossible. So it seemed worth trying.
那么就由此形成了 OpenAI。噢，我刚说我们创立 OpenAI 时的第一个问题是，成立一个由最优秀人才组成的实验室是否为时已晚？那是否可能？那是一个真正的问题。这是我们在2015年7月举行的晚宴的核心问题，我们花了整整一晚上的时间来讨论。我们考虑到AI已经从学术追求转变为工业追求，许多最优秀的人才都在这些大型研究实验室中，而我们想要创立自己的实验室，无论我们能够积累多少资源，都与那些大型科技公司相比毫不足道。我们知道这一点。还有一个问题就是我们是否能够真正使这个事情起步运作？你需要达到一个临界质量，你不能只有你和一个联合创始人来构建一个产品，你真的需要一个由5到10个人组成的团队。我们得出的结论是这显然并非不可能。所以，似乎值得一试。
Well, you're also a dreamer. So who knows? That's right. OK. So speaking of that, competing with the big players, let's talk about some of the tricky things as you think through this process of growing, of seeing how you can develop these systems at scale that competes. So you recently formed OpenAI LP. And you cap profit company that now carries the name OpenAI. So OpenAI has now this official company. The original nonprofit company still exists and carries the OpenAI nonprofit name. So can you explain what this company is, what the purpose of this creation is, and how did you arrive at the decision?
嗯，你也是个梦想家。谁知道呢？没错。好的，说到这个，与大公司竞争，让我们谈谈你通过这个成长过程时考虑到的一些棘手的事情，如何开发这些能够与规模竞争的系统。所以最近你成立了OpenAI LP，这个有利润限制的公司现在拥有了OpenAI这个名字。所以现在，OpenAI就有了官方的公司。原来的非盈利公司仍然存在，使用OpenAI非盈利的名字。那么你能解释一下这个公司是干什么的，这个决定的目的是什么，以及你是如何做出这个决定的吗？
Yep, to create it. OpenAI, the whole entity, and OpenAI LP as a vehicle, is trying to accomplish the mission of ensuring that artificial general intelligence benefits everyone. And the main way that we're trying to do that is by actually trying to build general intelligence ourselves and make sure the benefits are distributed to the world. That's the primary way. We're also fine if someone else does this. It doesn't have to be us. If someone else is going to build an AGI and make sure that the benefits don't get locked up in one company or with one set of people, we're actually fine with that. And so those ideas are baked into our charter, which is the foundational document that describes our values and how we operate. But it's also really baked into the structure of OpenAI LP.
Yep，是的，去创建它。整个OpenAI实体以及OpenAI LP作为载体，正在努力实现确保人工智能普惠于所有人的使命。我们正在尝试的主要方法是自己尝试构建通用智能，并确保其受益分配到全世界。这是我们努力的主要方法。如果其他人也这样做，我们也可以接受。它不一定非要是我们。如果有人将构建AGI并确保受益不被锁定在一个公司中或与一组人中，我们实际上也可以接受。因此，这些想法被融入了我们的宪章中，这是描述我们的价值观和运作方式的基础文件。但它也真正融入了OpenAI LP的结构中。
And so the way that we've set up OpenAI LP is that in the case where we succeed, if we actually build what we're trying to build, then investors are able to get a return. And but that return is something that is capped. And so if you think of AGI in terms of the value that you could really create, you're talking about the most transformative technology ever created. It's going to create orders of magnitude, more value, than any existing company. And that all of that value will be owned by the world, like legally titled to the nonprofit, to fulfill that mission. And so that's the structure. So the mission is a powerful one. And it's one that I think most people would agree with. It's how we would hope AI progresses. And so how do you tie yourself to that mission? How do you make sure you do not deviate from that mission that other incentives that are profit driven wouldn't don't interfere with the mission?
因此，我们设置 OpenAI LP 的方式是，如果我们成功建立我们想要建立的东西，投资者将能获得回报。但回报是有上限的。如果你考虑人工智能在创造价值方面的潜力，你会发现这是有史以来最具变革性的技术，可以创造比任何现有公司更多的价值。而所有这些价值都属于全世界，法律上属于这个非营利组织，以实现其使命。这就是这个结构。这个使命是强大的，我认为大多数人都会同意。这是我们希望人工智能发展的方式。那么你如何与这个使命联系起来？如何确保你不偏离这个使命，使追逐利润的其他激励不会干扰到这个使命呢？
So this was actually a really core question for us for the past couple of years. Because I'd say that the way that our history went was that for the first year we were getting off the ground. We had this high level picture, but we didn't know exactly how we wanted to accomplish it.
这对我们来说实际上是过去几年中一个非常核心的问题。因为我认为我们的历史经验是，在第一年我们正在起步阶段。我们有一个高层次的想法，但我们不知道如何确切地实现它。
And really two years ago, it's when we first started realizing in order to build AI, we're just going to need to raise way more money than we can as a nonprofit. And we are talking many billions of dollars. And so the first question is, how are you supposed to do that and stay true to this mission?
实际上，两年前，我们开始意识到，为了建造人工智能，我们需要筹集比我们现在的非营利组织更多的资金。我们谈论的是数十亿美元。所以第一个问题是，你应该如何这样做，同时又保持忠于这个使命？
And we looked at every legal structure out there and included none of them were quite right for what we wanted to do. And I guess it shouldn't be too surprising if you're going to do some crazy unprecedented technology that you're going to have to come with some crazy unprecedented structure to do it in.
我们研究了所有的法律结构，但发现没有一种完全适合我们想要做的事情。如果要进行一些疯狂前所未有的技术，需要采用一些疯狂前所未有的结构，这并不令人惊讶。
And a lot of our conversation was with people at OpenAI, the people who really joined because they believed so much in this mission. And thinking about how do we actually raise the resources to do it and also stay true to what we stand for.
我们与OpenAI的人交谈很多，他们是真正相信这个使命的人。我们思考如何实际筹集资源并坚持我们的立场。
And the place you got to start is to really align on what is it that we stand for? What are those values? What's really important to us? And so I'd say that we spent about a year really compiling the OpenAI charter. And that determines, and if you even look at the first line item in there, it says that, look, we expect we're going to have to marshal huge amounts of resources.
我们需要开始的地方就是真正对我们代表什么进行对齐。那些价值观是什么？什么对我们来说真正重要？我会说我们花了大约一年时间真正编制OpenAI章程。并且如果您查看其中的第一条，它说，看，我们预计我们将不得不动用大量资源。
But we're going to make sure that we minimize conflict of interest with the mission. And that kind of aligning on all of those pieces was the most important step towards figuring out how do we structure a company that can actually raise the resources to do what we need to do.
但我们将确保最大程度地减少与使命发生利益冲突。对所有这些方面进行协调是实现如何构建一个能够提供所需资源的公司的最重要一步。
I imagine OpenAI, the decision to create OpenAI LP, was a really difficult one. And there was a lot of discussions as you mentioned for a year. And there was different ideas, perhaps, detractors within OpenAI sort of different paths that you could have taken. What were those concerns? What were the different paths considered? What was that process of making that decision like?
我想，创建OpenAI LP的决定对于他们来说真的很难做出。正如你提到的那样，有很多讨论进行了一年。还有不同的想法，也许有OpenAI内部的反对者，认为你可以采取不同的道路。他们那时有哪些担忧？都考虑了哪些不同的路线？做出那个决定的过程是怎样的？
So if you look actually at the OpenAI charter, there's almost two paths embedded within it. There is, we are primarily trying to build AGI ourselves. But we're also OK if someone else does it. And this is a weird thing for a company. It's really interesting, actually.
如果你看看OpenAI的章程，实际上有两个路径被内嵌其中。第一个是我们主要正在尝试自己构建AGI。但是我们也不介意如果别人也这么做。对于一家公司来说，这是一件很奇怪的事情。实际上，这很有趣。
There is an element of competition that you do want to be the one that does it. But at the same time, you're OK if somebody else doesn't. And we'll talk about that a little bit, that trade off. That's the dance that's really interesting.
这里有一点竞争意味，你想成为一个能够完成它的人。但同时，如果别人不能完成它，你也没有关系。我们稍微谈谈这个权衡。这就是非常有趣的舞蹈。
And I think this was the core tension as we were designing OpenAI LP and really the OpenAI strategy. Is how do you make sure that both you have a shot at being a primary actor, which really requires building an organization, raising massive resources, and really having the will to go and execute on some really, really hard vision.
我认为，在设计OpenAI LP和真正的OpenAI战略时，这是核心的紧张关系。问题在于如何确保你有机会成为主要参与者，这确实需要建立一个组织，筹集大量资源，并真正有意愿去执行一些非常非常困难的愿景。
You need to really sign up for a long period to go and take on a lot of pain and a lot of risk. And to do that, normally you just import the start of mindset and that you think about OK, how do we out execute everyone? You have this very competitive angle. But you also have the second angle of saying that, well, the true mission isn't for OpenAI to build AGI. The true mission is for AGI to go well for humanity.
你需要真正长期注册并承担许多痛苦和风险。为了做到这一点，通常你只需要导入开始的心态，你需要思考怎么才能超越所有人？你有这种非常有竞争力的角度。但你还有第二个角度，你认为，真正的使命不是OpenAI建立AGI，而是让AGI为人类带来好处。
And so how do you take all of those first actions and make sure you don't close the door on outcomes that would actually be positive in fulfill the mission? And so I think it's a very delicate balance. I think that going 100% one direction or the other is clearly not the correct answer.
那么，怎样采取所有的初步行动，并确保不会对实现任务会产生积极影响的结果进行封闭呢？我认为这是一个非常微妙的平衡。我认为，百分之百地朝一个方向或另一个方向走，显然不是正确答案。
And so I think that even in terms of just how we talk about OpenAI and think about it, there's just one thing that's always in the back of my mind is to make sure that we're not just saying OpenAI's goal is to build AGI, that it's actually much broader than that. That first of all, it's not just AGI, it's safe AGI, that's very important.
所以我认为，即使只是我们谈论OpenAI和思考它的方式，我心中总是有一个想法，那就是确保我们不只是说OpenAI的目标是构建AGI，它实际上比那要广泛得多。首先，它不仅仅是AGI，更重要的是安全AGI。
But secondly, our goal isn't to be the ones to build it. Our goal is to make sure it goes well for the world. And so I think that figuring out how do you balance all of those and to get people to really come to the table and compile a single document that encompasses all of that wasn't trivial.
但第二，我们的目标并不是要成为建造者。我们的目标是确保这对世界有益。因此，我认为要想找到如何平衡所有这些因素，让人们真正参与并编制一份涵盖所有这些因素的单一文件，并非易事。
So part of the challenge here is your mission is I would say beautiful, empowering, and abeking a hope for people in the research community and just people thinking about AI. So your decisions are scrutinized more than I think a regular profit-driven company.
这里的挑战之一是你们的使命非常美丽、赋权，并为研究社区和普通关注人工智能的人带来了希望。因此，你们的决定比一般的以利润为驱动的公司要受到更多的审查。
Do you feel the burden of this in the creation of the charter and just in the way you operate?
在制定章程和运营的过程中，你是否感到了这个负担的压力？
Yes. So why do you lean into the burden by creating such a charter? Why not keep it quiet? I mean, it just boils down to the mission, right? Like I'm here and ever-and-als is here because we think this is the most important mission.
是的。那么，为什么要制定这样一个规章制度来负担沉重的任务呢？为什么不保持低调呢？我的意思是，这只是使命问题，对吧？就像我和Ever-and-als在这里，因为我们认为这是最重要的使命一样。
Right, dare to dream. All right, so do you think you can be good for the world or create an AGI system that's good when you're a for-profit company?
好的，敢于做梦。好的，那么你认为作为一个营利性公司，你能为世界做出贡献或创建一个好的人工智能系统吗？
From my perspective, I don't understand why profit interferes with positive impact on society. I don't understand by Google that makes most of its money from ads can't also do good for the world or other companies, Facebook, anything. I don't understand why those have to interfere. You can profit isn't the thing, in my view, that affects the impact of a company.
从我的角度来看，我不明白为什么盈利会干扰企业为社会带来积极影响。我不明白像谷歌这样大部分收入来自广告的公司为什么不能为世界或其他公司（如脸书）做出贡献。我不明白为什么要制造这种干扰。在我看来，赚钱并不是影响企业影响力的因素。
What affects the impact of the company is the charter, is the culture, is the people inside. And profit is the thing that just fuels those people. What are your views there?
影响公司的影响力的因素是章程、文化以及内部人员。而利润只是为这些人提供动力的事情。您对此有何看法？
Yeah, so I think that's a really good question. There's some real long-standing debates in human society that are wrapped up in it.
好的，我认为这是一个非常好的问题。人类社会中有些长期的争议就与此有关。
The way that I think about it is just think about what are the most impactful nonprofits in the world? What are the most impactful for-profits in the world? Right, it's much easier to list the for-profits. That's right. And I think that there's some real truth here that the system that we set up, the system for how today's world is organized, is one that really allows for huge impact.
我认为，我们需要考虑世界上最具影响力的非营利组织是哪些？还有，哪些营利组织最有影响力？对了，列出那些营利组织要容易得多。我认为这里有个真相，那就是我们设立的系统，也就是如今世界的组织方式，真的可以带来极大的影响。
And that part of that is that you need to be, that for-profits are self-sustaining and able to build on their own momentum. And I think that's a really powerful thing. It's something that when it turns out that we haven't set the guard rails correctly, causes problems. You think about logging companies that go and deforest, the rainforest, that's really bad. We don't want that.
并且其中一部分是，你需要是一个盈利性企业，能够自给自足并建立自己的动力。我认为这是一件非常有力的事情。当我们没有正确设置警示标志时，这可能会引起问题。想想伐木公司去砍伐雨林，那真的很糟糕。我们不希望这样。
And it's actually really interesting to me that kind of this question of how do you get positive benefits out of a for-profit company? It's actually very similar to how do you get positive benefits out of an AGI? That you have this very powerful system. It's more powerful than any human. And it's kind of autonomous in some ways.
实际上，我觉得如何从营利性公司获得积极的好处这个问题非常有趣。这与如何从AGI获得积极的好处非常相似。你拥有一个非常强大的系统，它比任何人都更强大，某种程度上是自主的。
It's superhuman in a lot of axes. And somehow you have to set the guard rails to get good things to happen. But when you do, the benefits are massive. And so I think that when I think about nonprofit versus for-profit, I think it's just not enough to happen in nonprofits.
这个东西在很多方面都是超人的。而且你必须设置保护栏，以便发生好事情。但是当你这样做时，好处是巨大的。因此，我认为当我考虑非营利组织与营利组织时，仅依靠非营利组织是不够的。
They're very pure. But it's just kind of hard to do things there. And for-profits in some ways, like too much happens. But if kind of shaped in the right way, it can actually be very positive. And so with OpenALP, we're picking a road in between.
它们非常纯净，但在那里做事情有点困难。而有些营利性机构则经常发生过多的事情。但如果以正确的方式塑造，它实际上可能是非常积极的。因此，我们选择了OpenALP这条路。
Now, the thing that I think is really important to recognize is that the way that we think about OpenALP is that in the world where AGI actually happens, right? In a world where we are successful, we build the most transformative technology ever, the amount of value we're going to create will be astronomical.
现在，我认为真正重要的事情是要认识到，我们对于OpenALP的思考方式是，在人工智能通用技术实际上实现的世界里，对吧？在那个我们取得成功，建造出最具变革性的技术，创造的价值将是巨大的世界里。
And so then in that case, the cap that we have will be a small fraction of the value we create. And the amount of value that goes back to investors and employees looks pretty similar to what would happen in a pretty successful startup. And that's really the case that we're optimizing for, right?
那么，在这种情况下，我们拥有的限制将只是我们创造的价值的一小部分。而流回投资者和员工的价值量看起来与一个非常成功的初创企业的情况相似。而这正是我们优化的情况，对吧？
That we're thinking about in this success case, making sure that the value we create doesn't get locked up. And I expect that in another four-profit companies that it's possible to do something like that, I think it's not obvious how to do it.
在这个成功案例中，我们正在考虑的是确保我们创造的价值不被锁定。我认为，在另外四个盈利公司中也有可能做到这样的事情，但我认为如何做这件事并不明显。
I think that as a four-profit company, you have a lot of fiduciary duty to your shareholders and that there are certain decisions that you just cannot make. In our structure, we've set it up so that we have a fiduciary duty to the charter that we always get to make the decision that is right for the charter rather than even if it comes at the expense of our own stakeholders.
我觉得，作为一家追求盈利的公司，你们有很多对股东的信托责任，有些决定是你们无法做出的。而我们的结构是这样设置的，我们的信托责任是始终为了宪章做出正确的决策，即使这意味着要牺牲我们自己的利益相关方。
And so I think that when I think about what's really important, it's not really about nonprofit versus for-profit. It's really a question of if you build AGI and you kind of, you know, humanities now in this new age, who benefits, whose lives are better. And I think that what's really important is to have an answer that is everyone.
所以，我认为当我考虑什么才是真正重要的时候，这不完全是关于非营利组织与营利组织的区别。这实际上是一个问题，如果你构建了AGI（人工智能），你知道，现在人类处于这个新时代，谁会受益，谁的生活会更美好。我认为真正重要的是要得到每个人都受益的答案。
Yeah, which is one of the core aspects of the charter. So one concern people have, not just with OpenAI, but with Google, Facebook, Amazon, anybody really, that's creating impact at scale, is how do we avoid, as your charter says, avoid enabling the use of AI or AGI to unduly concentrate power?
嗯，这是宪章的核心方面之一。因此，人们关心的问题不仅仅是OpenAI，而是任何真正产生规模影响的公司，如Google、Facebook、Amazon等等。我们应该如何避免使用人工智能或人工通用智能过度集中权力，就像你们的宪章所说的那样呢？
Why would not a company like OpenAI keep all the power of an AGI system to itself?
为什么像OpenAI这样的公司不把AGI系统的所有权力都留给自己？
The charter? The charter. So, you know, how does the charter actually analyze itself in a day to day?
宪章？宪章。你知道宪章实际上是如何在日常中进行分析的吗？
So I think that the first to zoom out, right, that the way that we structure the company is so that the power for sort of, you know, dictating the actions that OpenAI takes, ultimately rests with the board, right? The board of the nonprofit and the board is set up in certain ways, certain, certain restrictions that you can read about in the OpenAI LP blog post.
所以我认为首先需要缩小视野，对吧，我们公司的组织结构是这样的，即 OpenAI 所采取的行动决策权最终归属于董事会，对吧？非营利组织的董事会以特定的方式组成，有一些限制，你可以在 OpenAI LP 博客文章中阅读到相关内容。
But effectively, the board is the governing body for OpenAI LP. And the board has a duty to fulfill the mission of the nonprofit. And so that's kind of how we tie, how we thread all these things together.
实际上，管理OpenAI LP的机构是董事会。董事会有责任完成非盈利组织的使命。这就是我们将所有这些事情联系在一起的方式。
Now, there's a question of so day to day, how do people, the individuals, who in some ways are the most empowered ones, right? Now, the board sort of gets to call the shots at the high level, but the people who are actually executing are the employees, right? The people here on a day to day basis, who have the keys to the technical all kingdom.
现在，有一个日常问题，那就是人们，也就是那些在某些方面最有权力的个体，应该如何做呢？董事会在高层有一些决策权，但实际执行的是雇员们，对吧？这些每天都在这里工作的人，才是拥有技术全部的金钥匙。
And there, I think that the answer looks a lot like, well, how does any company's values get actualized, right? I think that a lot of that comes down to the unique people who are here because they really believe in that mission, and they believe in the charter and that they are willing to take actions that maybe are worse for them, but are better for the charter. And that's something that's really baked into the culture. And honestly, I think it's, you know, I think that that's one of the things that we really have to work to preserve as time goes on. And that's a really important part of how we think about hiring people and bringing people into OpenAI.
那么，我认为答案看起来很像，对吧？就像任何公司的价值观一样，它们如何得到实现。我认为这在很大程度上取决于那些独特的人，因为他们真的相信那项使命和宪章，他们愿意采取行动，即使这些行动对他们来说可能更糟糕，但对宪章来说更好。这是文化中真正融入的东西。老实说，我认为这是我们在未来必须努力保护的一件事情。这是我们在招聘并引入OpenAI员工时的一个非常重要的部分。
So there's people here, there's people here who could speak up and say, like, hold on a second, this is totally against what we stand for. Culture wise. Yeah, yeah, for sure. I mean, I think that we actually have, I think that's like a pretty important part of how we operate and how we have, even again, with designing the charter and designing OpenAI, I'll be in the first place, that there has been a lot of conversation with employees here in a lot of times where employees say, wait a second, this seems like it's going in the wrong direction and let's talk about it.
这里有人，有些人可以挺身而出说：“等等，这完全违背我们文化的立场。”是的，确实。我认为这其实是我们运作的一个相当重要的部分，甚至是在设计宪章和设计OpenAI的时候，也和员工进行了很多的交流。有很多时候，员工会说：“等等，这似乎走错方向了，我们来谈一谈。”
And so I think one thing that's, I think, really, and you know, here's actually one thing that I think is very unique about us as a small company, is that if you're at a massive tech giant, that's a little bit hard for someone who's aligned employee to go and talk to the CEO and say, I think that we're doing this wrong. And you know, you'll get companies like Google that have had some collective action from employees to make ethical change around things like Maven. And so maybe there are mechanisms that other companies that work. But here, it's super easy for anyone to pull me aside, to pull Sam aside, to pull Ilya aside, and people do it all the time.
我觉得有一件事情，我认为真的很独特，你知道的，这里实际上有一件事情，我认为我们作为一个小公司非常独特，那就是如果你在一个庞大的科技巨头，那对于一个公司内部的员工去和CEO谈论“我们做错了什么”，是有一些困难的。你知道，像谷歌这样的公司曾经有过些许员工的集体行动，为了Maven等等的道德改变。也许其他公司也有相应的机制可以运作。但是在这里，任何人想找我，想找Sam，想找Ilya谈事都非常容易，而且人们经常这样做。
One of the interesting things in the charter is this idea that it'd be great if you could try to describe or untangle switching from competition to collaboration and lead stage AGI development. It was really interesting. There's dance between competition and collaboration. How do you think about that?
宪章中有一个有趣的想法，认为尝试描述或梳理从竞争到合作并引领AGI发展的转变可能会很棒。这真的很有趣。竞争和合作交替出现，你怎么看待这一点？
Yeah, assuming that you can actually do the technical side of AGI development, I think there's going to be two key problems with figuring out how do you actually deploy it, make it go well. The first one of these is the run up to building the first AGI. You look at how self-driving cars are being developed, and it's a competitive race. And the thing that always happens in competitive race is that you have huge amounts of pressure to get rid of safety. And so that's one thing we're very concerned about, right? Is that people, multiple teams figuring out, we can actually get there, but if we took the slower path that is more guaranteed to be safe, we will lose. And so we're going to take the fast path.
是的，假设你真的能够做好 AGI 开发的技术方面，我认为有两个关键问题需要解决，那就是如何部署它，让它成功运行。其中一个问题是在建造第一个 AGI 之前需要解决的。看看自动驾驶汽车是如何开发的，这是一场竞争赛。在竞争赛中总会有极大的压力去消除安全隐患。这是我们非常担心的事情，因为有多个团队在考虑，如果我们走更慢、更安全的路径，我们就会输。因此我们将选择走快捷路径。
And so the more that we can, both ourselves, be in a position where we don't generate that competitive race where we say, if the race is being run and that someone else is further ahead than we are, we're not going to try to leapfrog. We're going to actually work with them, right? We will help them succeed as long as what they're trying to do is to fulfill our mission. Then we're good. We don't have to build AGI ourselves. And I think that's a really important commitment from us, but it can't just be unilateral, right? I think that it's really important that other players who are serious about building AGI make similar commitments, right?
所以，我们越能够使自己处于这样的位置，我们就不会制造出那种竞争赛跑的氛围，在那种情况下，如果有人跑得比我们快，我们不会试图超越他们，而是会与他们一起合作，对吧？只要他们试图实现我们的使命，我们就会帮助他们成功。那么我们就好了。我们不必亲自建造AGI。我认为这对我们来说是非常重要的承诺，但这不能仅仅是一方的，对吧？我认为其他严肃考虑建造AGI的玩家也需要作出类似的承诺。
And I think that, again, to the extent that everyone believes that AGI should be something to benefit everyone, then it actually really shouldn't matter which company builds it. And we should all be concerned about the case where we just race so hard to get there that something goes wrong. So what role do you think government, our favorite entity, has in setting policy and rules about this domain, from research to the development to early stage, shallate stage, AGI and AGI development?
我认为，如果每个人都认为人工智能普及的目的在于造福大众，那么是哪个公司开发人工智能其实并不重要。我们都应该警惕在追求人工智能发展速度的过程中，可能出现的问题。那么，您认为政府在这个领域中，从研究和早期阶段、中后期阶段，以及人工智能和其开发方面制定政策及规则方面有什么作用？
So I think that first of all, is really important to governments in there, right? In some way, shape or form, you know, at the end of the day, we're talking about building technology that will shape how the world operates and that there needs to be government as part of that answer. And so that's why we've done a number of different congressional testimonies. We interact with a number of different lawmakers and that, you know, right now, a lot of our message to them is that it's not the time for regulation. It is the time for measurement, right?
我认为首先，政府在这方面非常重要。在某种程度上，最终我们谈论的是建设将塑造世界运作的技术，政府需要参与其中。因此，我们已经做了许多不同的国会证言。我们与许多不同的立法者互动，现在我们向他们传达的信息之一是，现在不是时候管制，而是时候进行衡量。
That our main policy recommendation is that people, and you know, the government does this all the time with bodies like NIST, spend time trying to figure out just where the technology is, how fast it's moving, and can really become literate and up to speed with respect to what to expect. So I think that today, the answer really is about measurement, and I think that there will be a time and place where that will change. And I think it's a little bit hard to predict exactly what exactly that trajectory should look like.
我们的主要政策建议是，人们，你知道的，政府一直在和NIST等机构一起，花时间试图找出技术的实际发展情况，以及它的发展速度，然后真正变得能够理解和跟上预期的技术水平。所以我认为，今天的答案实际上是关于测量，而且我想，将来一定会有一个时间和地点，这种情况会改变。我想，有点难以准确预测这种轨迹会是什么样子。
So there will be a point at which regulation, federal and the United States, the government steps in and helps be the, I don't want to say, the adult in the room to make sure that there is strict rules, maybe conservative rules that nobody can cross. Well, I think there's kind of maybe two angles to it.
所以会有一个点，即监管、联邦和美国政府会介入并帮助成为房间里的成年人，确保有严格的规则，也许是保守的规则，没有人可以跨越。嗯，我认为可能有两种角度。
So today with narrow AI applications, that I think there are already existing bodies that are responsible and should be responsible for regulation. You think about, for example, with self-driving cars, that you want the national highway. It's exactly to be very good in that. That makes sense, right?
今天，随着狭义人工智能应用的出现，我认为已经存在的机构应该负责监管。比如说，你想想无人驾驶汽车，你希望国家高速公路来做得非常出色。这很有道理，对吧？
That basically what we're saying is that we're going to have these technological systems that are going to be performing applications that humans already do. Great, we already have ways of thinking about standards and safety for those. So I think actually empowering those regulators today is also pretty important.
基本上，我们的意思是，我们将拥有这些技术系统来执行人类已经执行的应用程序。很好，我们已经有了思考这些标准和安全方法。所以我认为今天赋予这些监管机构权力也非常重要。
And then I think for for AGI, you know, that there's going to be a point where we'll have better answers. And I think that maybe a similar approach of first measurement and start thinking about what the rules should be. I think it's really important that we don't prematurely squash progress. I think it's very easy to kind of smother the aboutting field. And I think that's something to really avoid. But I don't think the right way of doing it is to say, let's just try to blaze ahead and not involve all these other stakeholders.
我认为，对于人工智能（AGI），我们会逐渐找到更好的答案。或许可以采用先进行测量，再开始思考应该制定哪些规则的类似方法。我认为，不应该过早地阻碍进步，很容易扼杀正在兴起的领域，这是非常重要的一点，需要避免。但我认为不应该采取的正确做法是，试图孤注一掷，而不考虑所有其他利益相关者的利益。
So you've recently released a paper on GPD2, language modeling, but did not release the full model because you had concerns about the possible negative effects of the availability of such model. It's outside of just that decision. It's super interesting because of the discussion at a societal level, the discourse it creates. So it's fascinating in that aspect.
所以，你最近发表了一篇关于GPD2语言建模的论文，但并没有发布完整模型，因为你担心那样做可能会带来负面影响。这不仅仅是一个决策问题。因为它在社会层面上引起了讨论，创造了新的话题，所以它在这方面非常有趣。
But if you think that's the specifics here, at first, what are some negative effects that you envisioned? And of course, what are some of the positive effects?
如果你认为这就是具体情况，那么你最初所想到的一些负面影响是什么？当然，还有哪些积极影响？
Yeah, so again, I think to zoom out, like the way that we thought about GPD2 is that with language modeling, we are clearly on a trajectory right now, where we scale up our models and we get qualitatively better performance. GPD2 itself was actually just a scale up of a model that we've released in the previous June. We just ran it at much larger scale, and we got these results where suddenly starting to write coherent prose, which was not something we'd seen previously.
嗯，所以我认为，如果要概括一下，我们在思考GPD2的时候，语言模型方面显然处于一个轨迹上，我们正在不断地扩大模型规模，从而实现更好的性能。实际上，GPD2本身仅仅是我们在上一年6月推出的模型的升级版，我们仅仅是将其规模扩大了许多倍，并且得到了这些令人惊讶的结果，写出连贯的文章，这是我们以前从未见过的。
And what are we doing now? Well, we're going to scale up GPD2 by 10x by 100x by 1000x, and we don't know what we're going to get. And so it's very clear that the model that we released last June, I think it's kind of like it's a good academic toy. It's not something that we think is something that can really have negative applications or to the synthetic can that the positive of people being able to play with it is far, far outweighs the possible harms.
现在我们在做什么呢？我们要将GPD2的规模增加10倍、100倍、甚至1000倍，我们不知道会得到什么。因此很清楚，我们去年6月发布的模型，就像一个好的学术玩具，我们认为它并不会有负面应用或对合成人的影响是积极的，人们可以玩弄它的好处远远大于可能的危害。
You fast forward to not GPD2, but GPD20. And you think about what that's going to be like. And I think that the capabilities are going to be substantive. And so there needs to be a point in between the two where you say this is something where we are drawing the line and that we need to start thinking about the safety aspects. And I think for GPD2, we could have gone either way.
你跨越了不是GPD2，而是GPD20的事实，开始考虑那将是什么样子。我认为它的能力将是实质性的。因此，在两者之间需要有一个点，这是我们必须规定一条界线，并开始考虑安全方面的问题。我认为对于GPD2，我们可以采取任何方式。
And in fact, when we had conversations internally that we had a bunch of pros and cons, and it wasn't clear which one outweighed the other. And I think that when we announced that, hey, we decide not to release this model, then there was a bunch of conversation where various people said, it's so obvious that you should have just released it. There are other people said it's so obvious you should not have released it. And I think that almost definitionally means that holding it back was the correct decision.
实际上，当我们内部进行讨论时，我们得出了许多正反两方面的观点。我们并不清楚哪一方更为重要。当我们宣布，我们决定不发布这个模型时，有很多人在讨论，有些人说你应该明显地发布它，还有些人说你显然不应该发布它。我认为这几乎定义了挡住它的正确决定。
If it's not obvious whether something is beneficial or not, you should probably default to caution. And so I think that the overall landscape for how we think about it is that this decision could have gone either way. There's great arguments in both directions.
如果某件事是否有益并不明显，你可能应该默认谨慎。所以我认为，在我们思考这个问题的整体景观中，这个决定可能无论如何都能够做出。双方都有很好的辩论。
But for future models down the road, and possibly sooner than you'd expect, because scaling these things up doesn't actually take that on. Those ones, you're definitely not going to want to release into the wild. And so I think that we almost view this as a test case and to see, can we even design, how do you have a system that goes from having no concept of responsible disclosure where the mere idea of not releasing something for safety reasons is unfamiliar? To a world where you say, OK, we have a powerful model. Let's at least think about it. Let's go through some process.
但对于未来推出的机型，可能比你想象的要早，因为将这些事物扩大并不会占用太多时间。这些机型绝对不适合发布到公共领域。因此，我认为我们几乎将这视为一个测试案例，试图看看我们能否设计出一个系统，从不了解负责任披露的概念到有一个强大的模型，至少要考虑到安全问题，进行一些过程。
And you think about the security community. It took them a long time to design responsible disclosure. Right? You think about this question of, well, I have a security exploit. I stand to the company. The company is like, tries to prosecute me, or just ignore it. What do I do? And so the alternatives of, oh, I just always publish your exploits. That doesn't seem good either. And so it really took a long time. And it was bigger than any individual. Right? It's really about building a whole community that believe that, OK, we'll have this process where you send to the company. If they don't act in a certain time, then you can go public. And you're not a bad person. You've done the right thing. And I think that in AI, part of the response at GP2 just proves that we don't have any concept of this.
你想想安全社区。他们花了很长时间来设计负责任的公开信息。对吧？你考虑到这个问题，那就是，我发现了一个安全漏洞，我告诉公司，然后公司要么试图起诉我，要么完全忽略我，那我该怎么办呢？于是，一直公开你的漏洞并不是一个好的选择。所以，这确实要花费很长时间。而且它超出了任何个人的范围。这是真正打造一个社区，相信"好的，我们会有这个过程，你把安全漏洞告诉公司，如果他们一段时间内没有采取行动，那么你可以公开。而且这不代表你是一个坏人。你做了正确的事情。"我认为，在人工智能领域，GP2 的某些反应表明我们对此没有任何概念。
So that's the high level picture. And so I think that I think this was a really important move to make. And we could have maybe delayed it for GPT3. But I'm really glad we did it for GPT2.
那就是整体的高层概述。我认为这是一个非常重要的决定。虽然我们本来可以等到GPT3再执行，但我很高兴我们在GPT2时采取了这个步骤。
And so now you look at GPT2 itself, and you think about the substance of OK, what are potential negative applications? So you have this model that's been trained on the internet, which is also going to be a bunch of very biased data, a bunch of very offensive content in there. And you can ask it to generate content for you on basically any topic. You just give it a prompt, and we'll just start writing. And all right, content like you see on the internet.
现在你看看 GPT2 本身，考虑一下可能的负面应用。这个模型是在互联网上训练出来的，所以它的数据很可能有偏颇，可能包含了很多令人不悦的内容。你可以让它为你生成任何主题的内容，只需要输入提示，它就会开始写作。它所生成的内容就像互联网上看到的内容一样。
Even down to saying advertisement in the middle of some of its generations. And you think about the possibilities for generating fake news or abusive content. And it's interesting seeing what people have done with, we released a smaller version of GPT2. And the people have done things like try to generate, take my own Facebook message history, and generate more Facebook messages like me. And people generating fake politician content or there's a bunch of things there where you at least have to think, is this going to be good for the world?
就连在一些版本中间的广告也无可避免。你会想到产生虚假新闻或滥用内容的可能性。而看到人们在我们发布了一个较小版本的GPT2后，做出的一些惊人之举也是很有趣的。他们尝试生成类似于我自己的更多Facebook消息，生成虚假的政治家内容等等。至少你得考虑一下这对世界来说是不是有益的。
There's the flip side, which is I think that there's a lot of awesome applications that we really want to see, like creative applications in terms of if you have sci-fi authors that can work with this tool and come with cool ideas. That seems awesome. If we can write better sci-fi through the use of these tools. And we've actually had a bunch of people write into us asking, hey, can we use it for a variety of different creative applications? So the positive, I actually pretty easy to imagine. They're, you know, the usual NLP applications are really interesting.
还有另一面，我认为有很多令人惊叹的应用程序，我们真的很想看到，比如说有科幻小说作家可以用这个工具来创造具有创意的想法。那听起来太棒了。如果我们能通过使用这些工具来写出更好的科幻小说，那就太好了。实际上，我们已经收到了许多人写信问我们，嘿，我们能用它来进行各种不同的创意应用吗？所以积极的方面，我实际上很容易想象。你知道的，通常的自然语言处理应用程序真的很有趣。
But let's go there. It's kind of interesting to think about a world where, look at Twitter, where that just fake news. But smarter and smarter bots being able to spread in an interesting complex and networking way in information that just floods out us regular human beings with our original thoughts. So what are your views of this world with GPT 20? What do you, how do we think about it? Again, it's like one of those things about in the 50s trying to describe the internet or the smartphone. What do you think about that world, the nature of information?
让我们去那里吧。想想一个世界，看看Twitter，那里充斥着假新闻。但越来越聪明的机器人能够以一种有趣而复杂的方式传播信息，并淹没我们这些普通人的原始思想。那么，你对拥有GPT 20的这个世界有什么看法？我们应该如何思考？这就像五十年代想要描述互联网或智能手机的一些东西。你对那个世界和信息的本质有什么看法呢？
One possibility is that we'll always try to design systems that identify a robot versus human. And we'll do so successfully. And so we'll authenticate that we're still human. And the other world is that we just accept the fact that we're swimming in a sea of fake news and just learn to swim there.
有一个可能性是，我们将始终尝试设计系统来识别机器人和人类。并且我们将成功地做到这一点。因此，我们将验证我们仍是人类。而另一种可能是，我们只是接受我们正游泳在虚假新闻的海洋中，只需学会在那里游泳。
Well, have you ever seen the, there's there's a popular meme of robot with a physical arm and pen clicking the, I'm not a robot button. Yeah.
嗯，你见过那个流行的机器人表情包吗？它有一只实体手臂，用笔点击"I'm not a robot"的按钮。是的。
I think the truth is that really trying to distinguish between robot and human is a losing battle. Ultimately, you think it's a losing battle. I think it's a losing battle, ultimately. I think that that is that in terms of the content, in terms of the actions that you could take.
我认为，试图区分机器人和人类的真相是一场徒劳的战斗。最终，你认为这是一场失败的战斗。我认为，最终这也是一场失败的战斗。我认为，就内容和行动而言，这就是事实。
I mean, think about how captures have gone. The captures used to be a very nice simple. You just have this image. All of our OCRs terrible. You put a couple of artifacts in it. Humans are going to be able to tell what it is.
我是说，想想看我们的验证码是怎么样子的。以前，验证码非常简单明了。你只需要一个图片，而我们的OCR都非常糟糕。只需要放几个人为干扰，人类就可以轻松识别出来。
An AI system wouldn't be able to. Today, I can barely do captures. And I think that this is just where we're going. I think captures were a moment in time thing.
一个人工智能系统是无法做到这一点的。今天，我自己都很难做到验证码识别。我认为这仅仅是我们未来的趋势。我认为验证码只是过去某一个时刻的事情。
And as AI systems become more powerful, that there being human capabilities that can be measured in a very easy automated way that the AI is not incapable of, I think that's just like, it's just an increasingly hard technical battle.
随着人工智能系统变得更加强大，存在一些人类技能无法通过自动化方式轻易测量，而这是人工智能所不具备的。我认为这就像是一场日益艰难的技术战斗。
But it's not that all hope is lost. You think about how do we already authenticate ourselves? That we have systems. We have social security numbers. If you're in the US or you have ways of identifying individual people and having real world identity tied to digital identity seems like a step towards authenticating the source of content rather than the content itself.
但并非所有希望都已失去。你可以想想我们已经如何验证自己了？我们有系统，社会保障号码。如果你在美国或者有其他找出个人身份的方法，将现实身份与数字身份联系起来似乎是鉴定内容来源的一步，而不是鉴定内容本身。
Now, there are problems with that. How can you have privacy and anonymity in a world where the only content you can really trust is, or the only way you can trust content is by looking at where it comes from. And so I think that building out good reputation networks may be one possible solution.
嗯，那有问题。在一个你只能通过看其来源来信任内容的世界中，你如何保护自己的隐私和匿名。因此，我认为建立良好的信誉网络可能是一个可行的解决方案。
But yeah, I think that this question is not an obvious one. And I think that we, maybe sooner than we think, will be in a world where today, I often will read a tweet and be like, do I feel like a real human wrote this? Or do I feel like this is like genuine? I feel like I can kind of judge the content a little bit. And I think in the future, it just won't be the case.
嗯，我认为这个问题并不明显。而且我认为我们很可能比我们想象的更早就会进入一个新世界，在这个世界上，今天我经常读到一条 tweet，会问自己，这是一个真正的人写的吗？还是我觉得这是真实的？我觉得我能够在某种程度上判断内容的真实性。但是我认为在未来，情况就不会如此了。
You will get, for example, the FCC comments on net neutrality. It came out later that millions of those were auto generated and that the researchers were able to do various statistical techniques to do that. What do you do in a world where those statistical techniques don't exist? It's just impossible to tell the difference between humans and AI's. And in fact, the most persuasive arguments are written by AI, all that stuff. It's not sci-fi anymore. You'll get GPT-2 making a great argument for why recycling is bad for the world.
举个例子，你将得到FCC对网络中立性的评论。后来揭示出，其中的数百万条评论都是自动生成的，研究人员能够运用各种统计技术实现这些评论。而在那些统计技术还不存在的世界中，你该怎么办呢？就连最有说服力的论点其实都是由人工智能编写的。这一切已经不再是科幻了。你将得到GPT-2为何回收对世界有害的一个非常好的论点。
You've got to read that. You're like, huh, you're right. We are interested in those symptoms. Yeah, that's quite interesting.
你得读一下那个。你会说，“没错，我们对那些症状很感兴趣。”是啊，相当有趣。
I mean, ultimately it boils down to the physical world being the last frontier of proving that you said, like, basically networks of people, humans, vouching for humans in the physical world, and somehow the authentication ends there. I mean, if I had to ask you, your way to eloquent for a human. So if I had to ask you to authenticate, like, prove, how do I know you're not a robot? How do you know I'm not a robot? No.
我是说，最终问题归结为我们所说的物理世界是证明人类间网路是基于人类身份验证的最后一道关卡。这意味着人类必须在现实中彼此亲身验证才能获得身份认证。如果要向你求证，你太雄辩了，不像是普通人。如果我要求你身份认证，证明你不是机器人，你会怎么做？你怎么知道我不是机器人呢？不会吧。
That's so far where this, in this space, this conversation we just had, the physical movements we did is the biggest gap between us and AI systems, is the physical manipulation.
在我们刚刚进行的这次对话、进行的物理动作以及当前的空间中，最大的差距是我们和AI系统之间的物理操作能力。
So maybe that's the last frontier. Well, here's another question is, you know, why is solving this problem important? Right? What aspects are really important to us?
所以也许这就是最后的边界。那么，另一个问题是，你知道吗，为什么解决这个问题很重要？对我们来说，哪些方面真正重要？
I think that probably where we'll end up is we'll hone in on what to be really want out of knowing if we're talking to a human. And I think that again, this comes down to identity. And so I think that the internet of the future, I expect to be one that will have lots of agents out there that will interact with you.
我认为，我们最终可能会集中精力找出我们真正想知道的问题，即我们是否正在与一个人交谈。我认为这又归结于身份的问题。因此，我认为未来的互联网应该会有很多代理人与你交互。
But I think that the question of, is this, you know, a real flesh and blood human or is this an automated system? May actually just be less important. Let's actually go there. It's GPT2 is impressive, and let's look at GPT20.
不过，我认为这个问题，你知道，这个是一个真正有血有肉的人还是一个自动系统，可能实际上不太重要。让我们直接进入主题吧。GPT2令人印象深刻，让我们来看看GPT20。
Why is it so bad that all my friends are GPT20? What, why is it so important on the internet? Do you think to interact with only human beings? Why can't we live in a world where ideas can come from models trained on human data?
为什么我的所有朋友都是GPT20这么糟糕？在互联网上，这为什么如此重要？你认为只与人类互动很重要吗？为什么我们不能生活在一个可以从经过人类数据训练的模型中获取思想的世界中呢？
Yeah, I think this is actually a really interesting question. This comes back to the, how do you even picture a world with some new technology? And I think that one thing that I think is important is, you know, go say honesty. And I think that if you have, you know, almost in the Turing test style sense of technology, you have AIs that are pretending to be humans and deceiving you, I think that feels like a bad thing. Right?
啊，我觉得这其实是一个非常有趣的问题。这跟你怎么想象一个拥有一些新技术的世界有关。我认为重要的一件事情是诚实。如果你有了类似图灵测试风格的技术，那么如果AI伪装成人类并欺骗你，我觉得这样感觉不好。不是吗？
I think that it's really important that we feel like we're in control of our environment, right? That we understand who we're interacting with. And if it's an AI or a human, that's not something that we're being deceived about.
我认为，我们感觉如同掌握环境的控制权非常重要，对吗？我们要清楚地了解我们正在与谁互动。如果对方是人类还是人工智能，我们不能被欺骗。
But I think that the flip side of, can I have as meaningful of an interaction with an AI as I can with a human? Well, I actually think here you can turn to sci-fi. And her, I think, is a great example of asking this very question, right?
我认为与人类交往一样有意义的互动能否与人工智能发生，这其中的另一面，但是我认为这个问题可以参考科幻作品。我认为“她”是很好的一个例子来提出这个问题，对吧？
And one thing I really love about her is it really starts out almost by asking how meaningful are human virtual relationships, right? And then you have a human who has a relationship with an AI and that you really start to be drawn into that, right? And that all of your emotional buttons get triggered in the same way as if there was a real human that was on the other side of that phone. And so I think that this is one way of thinking about it is that I think that we can have meaningful interactions and that if there's a funny joke, some sense it doesn't really matter if it was written by a human or an AI.
我真的非常喜欢她的一个事情，那就是她从一开始就像是在询问人类虚拟关系的意义有多大，对吧？然后你有一个与AI有关系的人，你真的开始被吸引进去了，对吧？所有的情感按钮都被触发了，就好像那通电话的另一端真的有一个真实的人。所以我认为这是一种思考方式，我认为我们可以有有意义的互动，如果有一个有趣的笑话，有些人觉得它并不重要是由一个人还是AI编写的。
But what you don't want in a way where I think we should really draw hard lines is deception. And I think that as long as we're in a world where, you know, why do we build AI systems at all? Right? The reason we want to build them is to enhance human lives, to make humans be able to do more things, to have humans feel more fulfilled. And if we can build AI systems that do that, you know, sign me up.
但是我认为我们绝对不能容忍欺骗。在我看来，这是我们需要绝对禁止的地方。我们现在生活在一个世界里，为什么我们要开发人工智能系统呢？因为我们想要增强人类的生活，让人类能够做更多的事情，让人类觉得更充实。如果我们能够开发出这样的人工智能系统，我愿意为之效力。
So the process of language modeling, how far do you think it take us? Let's look at movie her. Do you think a dialogue, natural language conversation is formulated by the touring test, for example? Do you think that process could be achieved through this kind of unsupervised language modeling?
那么语言建模的过程，你认为它能带我们走多远呢？让我们看看电影《Her》。你认为对话、自然语言交流是通过图灵测试来构建的吗？你认为这种无人监督的语言建模可以实现这样的过程吗？
So I think the touring test in its real form isn't just about language, right? It's really about reasoning too, right? That to really pass the touring test, I should be able to teach calculus to whoever's on the other side and have it really understand calculus and be able to, you know, go and solve new calculus problems. And so I think that to really solve the touring test, we need more than what we're seeing with language models. We need some way of plugging and reasoning.
所以我认为，旅游测试的真正形式并不仅仅是关于语言对吧？它真的也涉及到推理对吧？如果我能够教让另一端的人理解微积分，并且能够解决新的微积分问题，那么我才真正通过了旅游测试。因此，我认为要真正解决旅游测试，我们需要比语言模型更多的插入和推理方式。
Now, how different will that be from what we already do? That's an open question, right? It might be that we need some sequence of totally radical new ideas, or it might be that we just need to kind of shape our existing systems in a slightly different way. But I think that in terms of how far language modeling will go, it's already gone way further than many people would have expected, right?
现在，那会不会和我们已经在做的有所不同呢？这是一个开放性的问题，是吗？可能我们需要一些全新的、彻底激进的想法序列，或者可能我们只需要稍微改变一下我们现有系统的形式。但是我认为，就语言模型的发展而言，它已经走得比很多人预想的更远了，是吗？
I think that things like, and I think there's a lot of really interesting angles to poke in terms of how much does GPT-2 understand physical world? Like, you know, you read a little bit about fire underwater in GPT-2, so it's like, okay, maybe it doesn't quite understand what these things are. But at the same time, I think that you also see various things like smoke coming from flame, and a bunch of these things that GPT-2 has no body, it has no physical experience, it's just statically read data. And I think that the answer is like, we don't know yet, and these questions, though, we're starting to be able to actually ask them to physical systems, to real systems that exist, and that's very exciting.
我认为像这样的事情，以及在GPT-2对物理世界的理解方面，有许多非常有趣的角度可以探究。比如，你读到一些关于GPT-2在水下火灾方面的内容，这就说明它可能并没有完全了解这些事情。但与此同时，我也可以看到一些事情，比如火焰喷出的烟雾，还有许多GPT-2没有身体和物理经验的数据静态读取。我认为答案是我们目前还不知道，但这些问题，我们开始能够真正向现实存在的物理系统提出，并且这是非常令人兴奋的。
Do you think, what's your intuition? Do you think if you just scale language modeling, significantly scale, that reasoning can emerge from the same exact mechanisms?
你认为呢，你的直觉如何？你认为如果你只是扩大语言建模，显著扩大，那么推理能够从完全相同的机制中出现吗？
I think it's unlikely that if we just scale GPT-2 that will have reasoning in the full-fledged way, and I think that there's like, you know, so the type signature is a little bit wrong, right? That like, there's something we do with, that we call thinking, right, where we spend a lot of compute, like a variable amount of compute, to get to better answers, right? I think a little bit harder, I get a better answer. And that that kind of type signature isn't quite encoded in a GPT, right? GPT will kind of like, it's spent a long time in its like evolutionary history, baking and all this information, getting very, very good at this predictive process. And then at runtime, I just kind of do one forward pass and am able to generate stuff.
我觉得如果我们只是简单地扩大GPT-2的规模，那么它很难以全面的推理方式进行推理。我认为这个类型签名有点不正确。我们有一种叫做思考的事情，我们会花费大量的计算量去得到更好的答案。我觉得思考更深入一些，得到更好的答案。而这种类型签名并没有完全编入GPT。GPT在演化历史中花费了很长时间去烘烤所有这些信息，使得它非常擅长预测过程。然后在运行时，我只需进行一次正向传递，就能生成东西。
And so, you know, there might be small tweaks to what we do in order to get the type signature, right? For example, well, you know, it's not really one forward pass, right? You generate symbol, I symbol, and so maybe you generate like a whole sequence of thoughts and you only keep like the last bit or something. But I think that at the very least, I would expect you have to make changes like that.
所以，你知道的，为了得到类型签名，我们可能需要对我们所做的做出微小的调整，对吧？例如，你知道的，这并不是一个前向传递，对吧？你会生成符号、我会生成符号，所以也许你会生成一系列的思维，只保留最后一点或类似的东西。但我认为，至少你得做出那样的改变。
Yeah, just exactly how we, you said, think is the process of generating thought by thought in the same kind of way, like you said, keep the last bit, the thing that we converge towards.
是的，就像你所说的那样，我们认为形成思考的过程是以与之前相似的方式逐一生成思想，并保留我们朝着同一目标收敛的最后一点。
And I think there's another piece, which is interesting, which is this out of distribution generalization, right? They're like thinking somehow that's just do that, right? That we have an experience of thing and yet somehow we just kind of keep refining our mental model of it. This is again, something that feels tied to whatever reasoning is. And maybe it's a small tweak to what we do. Maybe it's many ideas and we'll take as many decades.
我认为还有另一个有趣的部分是这种超出分布的泛化。他们的想法是，我们经历了某些事情，但我们始终在不断完善它的精神模型。这再次与推理相关，也许只需要进行一些小的调整，也许需要很多年才能实现这些理念。
Yeah, so the assumption there, generalization out of distribution is that it's possible to create new ideas. It's possible that nobody's ever created any new ideas. And then with scaling, GPT2, to GPT20, you would essentially generalize to all possible thoughts that I'll see what's gonna happen. Just to play devil's attic here. Right, right. I mean, how many new story ideas have we come up with since Shakespeare, right? Yeah, exactly. It's just all different forms of love and drama and so on.
啊，所以推断就是，从分布以外的普遍性来看，有可能创造新的想法。当然也有可能从未有人创造出新想法。而且，随着规模的不断扩大，从 GPT2 到 GPT20，你基本上可以推广到任何可能的想法，这使得我很想看看会发生什么。用恶魔的角度考虑一下。是的，我是指，自莎士比亚以来我们又有多少新的故事想法呢？恰恰如此。只是不同形式的爱和戏剧之类的东西而已。
Okay. Not sure if you read Bit of Lesson, a recent blog post by Ray Sutton. Nope, I have. He basically says something that echoes some of the ideas that you've been talking about, which is he says the biggest lesson that can be read from so many years of AI research is that general methods that leverage computation are ultimately going to, ultimately win out. Do you agree with this?
好的，不确定你是否读过Ray Sutton最近的博客文章《Bit of Lesson》。没有，我还没有读过。他基本上说的是与一些你所谈论的观点相似，即最大的教训是，利用计算的通用方法最终将获胜。你同意吗？
So basically, an open AI in general, about the ideas you're exploring, about coming up with methods, whether it's GPT2 modeling or whether it's Open AI5 playing Dota, where a general method is better than a more fine-tuned expert-tuned method. Yeah, so I think that, well, one thing that I think was really interesting about the reaction to that blog post was that a lot of people have read this as saying that compute is all that matters. And that's a very threatening idea, right? And I don't think it's a true idea either. It's very clear that we have algorithmic ideas that have been very important for making progress and to really build AGI, you want to push as far as you can on the computational scale and you want to push as far as you can on human ingenuity. And so I think you need both.
所以基本上，关于你们正在探索的想法和出现的方法，无论是GPT2建模还是Open AI5玩Dota，在一般的AI中，一般方法比更精细专家调整的方法更好。是啊，我认为，一个有意思的事情是，对那篇博客的反应中很多人都认为计算能力是唯一重要的因素。这是一个非常威胁性的想法，对吧？我也认为这不是一个真实的想法。很明显，我们有算法思想，这些算法思想对于取得进展非常重要，为了真正建立通用人工智能，你需要在计算规模上尽可能地推进，在人类创造力上尽可能地推进。所以我认为你需要两者兼备。
But I think the way that you've created the question is actually very good, right? That it's really about what kind of ideas should we be striving for? And absolutely, if you can find a scalable idea, you pour more compute into it, you pour more data into it, it gets better. Like that's the real holy grail. And so I think that the answer to the question, I think it's yes, that's really how we think about it and that part of why we're excited about the power of deep learning and the potential for building AGI is because we look at the systems that exist in the most successful AI systems. And we realized that you scale those up, they're going to work better. And I think that that scalability is something that really gives us hope for being able to build transformative systems.
但我认为你提出的问题方式非常好，对吧？它真正关注的是我们应该追求哪种类型的想法？如果你能找到一个可扩展的想法，你可以将更多的计算能力和数据投入其中，它会变得更好。这是真正的圣杯。因此，我认为问题的答案是肯定的，这正是我们思考的方式，这也是我们对深度学习和构建AGI的潜力感到兴奋的原因之一，因为我们看到最成功的人工智能系统中存在的系统，并意识到将它们扩展起来，它们会更有效。我认为这种可扩展性给了我们能够构建转型系统的希望。
So I'll tell you partially an emotional, a thing that I've responded people often have is compute is so important for state-of-the-art performance. Individual developers, maybe a 13 year old sitting somewhere in Kansas or something like that, they're sitting that might not even have a GPU or have a single GPU, a 1080 or something like that. And there's this feeling like, well, how can I possibly compete or contribute to this world of AI if scale is so important? So if you can comment on that, and in general, do you think we need to also in the future focus on democratizing compute resources more or as much as we democratize the algorithms?
所以，我会告诉你我经常听到人们说，计算对于最先进的性能非常重要，这是一个情感话题。即使是一个可能坐在堪萨斯州某处的13岁的个人开发者，他们坐着可能甚至没有GPU或只有一张1080这样的显卡，他们会觉得，如果规模如此重要，那我怎么可能在人工智能的世界中竞争或做出贡献呢？如果您能评论一下这个问题，并且总的来说，您是否认为我们在未来也需要像平衡算法一样关注计算资源的民主化？
Well, so the way that I think about it is that there's the space of possible progress. There's a space of ideas and sort of systems that will work, that will move us forward. And there's a portion of that space, and to some extent, an increasingly significant portion of that space that does just require massive compute resources. And for that, I think that the answer is kind of clear. And part of why we have the structure that we do is because we think it's really important to be pushing the scale and to be building as large clusters and systems.
嗯，我认为这是这样的，就是存在可能的进步空间。这个空间里包含了各种能够有效推动我们前进的想法和系统。而其中一部分，甚至是越来越重要的一部分，确实需要大量的计算资源。针对这点，我认为答案是比较清晰的。我们构建如此的结构，部分原因是我们认为推动规模的扩大和构建更大的集群和系统非常重要。
But there's another portion of the space that isn't about the large scale compute that are these ideas that, and again, I think that for these ideas to really be impactful and really shine, that they should be ideas that if you scale them up, would work way better than they do at small scale. But you can discover them without massive computational resources. And if you look at the history of recent developments, you think about things like the GAN or the VAE, that these are ones that I think you could come up with them without having, and you know, in practice, people did come up with them without having massive, massive computational resources.
但空间中还有另一部分不是关于大规模计算的，而是关于一些想法。我认为，这些想法真正发挥作用，真正发光，应该是那些如果你将它们放大，将比小规模的效果好得多的想法。但你不需要大量的计算资源就可以发现它们。如果你看一下最近的发展历史，你会想到GAN或VAE这样的东西，我认为这些东西是可以在没有大量计算资源的情况下想出来的。实际上，人们就是这样做到的。
Well, I just talked to you in good fellow, but the thing is the initial GAN produced pretty terrible results. So only because it was in a very specific, it was only because they're smart enough to know that this is quite surprising. It can generate anything that they know.
嗯，我刚刚用友好的语气和你说话，但问题在于最初的生成对抗网络的结果相当糟糕。只有因为它是在非常特定的条件下才能产生惊人的结果，他们足够聪明地知道这一点。只有它们知道的内容才能生成出来。
Do you see a world that is too optimistic and dreamer like to imagine that the compute resources are something that's owned by governments and provided as utility? Actually, so some extent this question reminds me of blog posts from one of my former professors at Harvard, this guy, Matt Welch, who was a systems professor.
你看到一个世界实在是太过乐观和梦幻，以至于无法想象计算资源是由政府拥有并作为公共设施提供的吗？其实，这个问题在一定程度上让我想起了我在哈佛的一位前任教授马特·韦尔奇的博客文章，他是一个系统教授。
I remember sitting in his tenure talk, right? And you know, that he had literally just gotten tenure. He went to Google for the summer, and then decided he wasn't going back to academia, right? And that kind of in his blog post, he makes this point that, look, as a systems researcher, that I come up with these cool system ideas, right? And I kind of built a little proof of concept. And the best thing I could hope for is that the people at Google or Yahoo, which was around at the time, will implement it and actually make it work at scale.
我记得当时在他的终身聘位演讲上坐着，对吧？你知道的，他实际上刚刚获得终身职位。他去了谷歌过夏天，之后决定不再回学术界了，对吧？在他的博客帖子中，他提到了这样一个观点，作为一个系统研究者，我提出了这些酷炫的系统想法，对吧？我构建了一个小的概念证明。我所期望的最好的事情就是谷歌或雅虎这样的公司能够实现它，并在实际规模中使它运行起来。
That's like the dream for me, right? I built a little thing and they turned into the big thing that's actually working. And for him, he said, I'm done with that. I want to be the person who's actually doing building and deploying. And I think that there's a similar dichotomy here, right? I think that there are people who've really actually find value and I think it is a valuable thing to do to be the person who produces those ideas, right? Who builds the proof of concept.
这就像是我的梦想一样，对吧？我做了一个小东西，它们变成了一个真正工作的大东西。而他，他说：“我完成了那个。我想成为真正建造和部署的人。”我认为这里存在着类似的二分法，对不对？我认为有些人真的很珍视这些价值，而成为产生这些想法、建造概念证明的人是一件有价值的事情。
And yeah, you don't get to generate the coolest possible GAN images, but you invented the GAN, right? And so that there's a real trade-off there. And I think that's a very personal choice, but I think there's value in both sides.
是的，你不能生成最酷的GAN图像，但你发明了GAN，对吧？所以这是一种真正的权衡。我认为这是一个非常个人的选择，但我认为两边都有价值。
Did you think creating AGI, something, or some new models would we would see echoes of the brilliance even at the prototype level? So you would be able to develop those ideas without scale, the initial seeds.
你是否认为创建AGI、一些新模型，甚至在原型水平上都能看到其卓越的影响？这样你就能够在没有规模的情况下发展这些想法，也就是种子阶段。
So take a look at, I always like to look at examples that exist, right? Look at real precedent. And so take a look at the June 2018 model that we released that we scaled up to turn to GPT2. And you can see that at small scale, it set some records, right? This was the original GPT. We actually had some cool generations. They weren't nearly as amazing and really stunning as the GPT2 ones, but it was promising. It was interesting. And so I think it is the case that with a lot of these ideas, that you see promise at small scale. But there is an asterisk here, a very big asterisk, which is sometimes we see behaviors that emerge that are qualitatively different from anything we saw at small scale, and that the original inventor of whatever algorithm looks at and says, I didn't think it could do that.
那么，看一下我总是喜欢看现有的例子，对吧？看真实的先例。因此，看一下我们在2018年6月发布的模型，我们对它进行了扩大，成为了GPT2。你可以看到，在小规模上，它创下了一些记录，对吧？这是最初的GPT。我们实际上有一些很棒的生成物。它们远没有GPT2的那些惊艳和真正震撼人心，但很有前途。所以我认为，在很多这样的想法中，你可以在小规模上看到希望。但是这里有一个重要的注释，一个非常重要的注释，有时我们会看到一些与小规模下所见的任何事物质量上不同的行为，原始算法发明人会看着它说，我没想到它能做到这一点。
This is what we saw in Dota. So PPO was created by John Schultman, who's a researcher here. And with Dota, we basically just ran PPO at massive, massive scale. And there's some tweaks in order to make it work, but fundamentally it's PPO at the core. And we were able to get this long-term planning, these behaviors to really play out on a time scale that we just thought was not possible. And John looked at that and was like, I didn't think it could do that. That's what happened when you're at three orders of magnitude more scale than you tested at.
这就是我们在Dota中看到的。所以PPO是由这里的研究人员约翰·舒尔特曼创建的。对于Dota，我们基本上只是在大规模地运行PPO。虽然有一些微调才能使其运作，但本质上它是以PPO为核心的。我们能够让这种长期规划和行为在时间尺度上得到充分的展现，这是我们认为不可能的。约翰看了看，说：“我没想到它能做到这一点。”这就是当你的测试规模超过三个数量级时发生的事情。
Yeah, but it still has the same flavors of, you know, at least echoes of the expected billions. Although I suspect with GPT scaled more and more, you might get surprising things. So yeah, you're right, it's interesting. It's difficult to see how far an idea will go when it's scaled. It's an open question.
是的，但它仍然拥有相同的味道，至少有上百亿的预期回响。虽然我怀疑随着 GPT 的规模越来越大，你可能会得到让人惊奇的东西。所以是的，你是对的，这很有趣。当一个理念被扩大规模时，很难看出它将走多远。这是一个未知的问题。
Well, so to that point with Dota and PPO, it here's a very concrete one, right? It's like, it's actually, one thing that's very surprising about Dota that I think people don't really pay that much attention to is the decree of generalization out of distribution that happens, right? That you have this AI that's trained against other bots for its entirety, the entirety of its existence.
嗯，那么就来谈谈 Dota 和 PPO，这是非常具体的一个问题，对吧？事实上，我认为 Dota 的一个非常令人惊讶的特点，是它在分布范围外的泛化程度，这点人们并没有给予太多关注，你知道吗？你有一个 AI，它的整个存在期间都是与其他机器人进行训练的。
Sorry to take a step back. Can you talk through in, you know, a story of Dota, a story of leading up to opening I-5 and that past and what was the process of self-planning so on of training on?
非常抱歉让我们退回了一步。您可以通过Dota的故事，以及引领开放I-5的过程和过去，以及自我规划和培训的过程，来谈一谈吗？
Yeah, yeah. So with Dota. What is Dota?
是的，Dota是什么？
Yeah, Dota is a complex video game. And we started training, we started trying to solve Dota because we felt like this was a step towards the real world, relative to other games like Chester Go, right?
是的，Dota是一个复杂的电子游戏。我们开始训练，试图解决Dota，因为我们觉得相对于其他游戏，比如切斯特Go，这是走向真实世界的一步。
Those very cerebral games where you just kind of have this board of very discrete moves. Dota starts to be much more continuous time that you have this huge variety of different actions that you have a 45 minute game with all these different units.
那些非常脑力的游戏，你只是在棋盘上进行非常离散化的行动。Dota 开始变得更具连贯性，你拥有多种不同的行动，以及一场45分钟的游戏和所有这些不同的单位。
And it's got a lot of messiness to it that really hasn't been captured by previous games. And famously, all of the hard-coded bots for Dota were terrible, just impossible to write anything good for it because it's so complex. And so this seemed like a really good place to push the state of the art in reinforcement learning.
它非常复杂，先前的游戏没有完全捕捉到它的混乱。 Dota的所有预设机器人都很糟糕，因为它太复杂了，所以无法编写任何优秀的内容。因此，这似乎是推进强化学习技术的一个非常好的领域。
And so we started by focusing on the one versus one version of the game and we're able to solve that. We were able to beat the world champions and the learning, you know, the skill curve was this crazy exponential, right? And it was like constantly we were just scaling up that we were fixing bugs and, you know, that you look at the skill curve and it was really a very, very smooth one.
所以我们从专注于游戏一对一版本开始，并且我们能够解决这个问题。我们能够击败世界冠军，学习曲线是非常的陡峭，是一个疯狂的指数增长，对吧？不断地升级我们的技能，修复漏洞，你看技能曲线非常平稳。
And so it's actually really interesting to see how that, like, human iteration loop yielded very steady exponential progress. And to one side note, first of all, it's an exceptionally popular video game. The side effect is that there's a lot of incredible human experts at that video game. So the benchmark that you're trying to reach is very high.
所以，看到那个人类迭代循环如何带来稳定的指数级进展，实际上非常有趣。还有一个侧面的注记，首先，这是一个异常受欢迎的电子游戏。副作用就是有很多令人惊叹的人类专家在那个游戏中。因此，你所努力达到的基准非常高。
And the other, can you talk about the approach that was used initially and throughout training these agents to play this game? And so the approach that we used is self play. And so you have two agents that don't know anything. They battle each other. They discover something a little bit good and now they both know it. And they just get better and better and better without bound.
另外一个问题，你能谈谈一开始和整个培训期间使用的方法来让这些代理玩这个游戏吗？我们采用的方法是自我对弈。你有两个什么都不知道的代理相互较量。他们发现了一些有点好的东西，现在他们都知道了。他们不断变得更好，没有限制。
And that's a really powerful idea, right? That we then went from the one versus one version of the game and scaled up to five versus five, right? So you think about kind of like with basketball where you have this like team sport, you have to do all this coordination and we were able to push the same idea, the same self play to really get to the professional level at the full five versus five version of the game.
这个想法真的很有力量，对吧？我们先从一对一的游戏版本开始，一步步扩展到五对五，就像篮球这样的团队运动，需要做很多协调，我们成功地用同样的自战玩法把游戏提升到了专业的五对五版本。
And the things I think are really interesting here is that these agents in some ways, they're almost like an insect-like intelligence, right? Where they have a lot in common with how an insect is trained, right? Insect kind of lives in this environment for a very long time, or the ancestors of this insect have been around for a long time and had a lot of experience.
我认为这里非常有趣的一点是这些代理人在某种程度上几乎像是昆虫一样的智能，对吧？它们与昆虫的训练方式有很多共同点，昆虫在一个环境中生活很长时间，或者这种昆虫的祖先已经存在了很长时间并积累了很多经验。
I think it's baked into this agent. And it's not really smart in the sense of a human, right? It's not able to go and learn calculus, but it's able to navigate its environment extremely well. It's able to handle unexpected things in the environment that's never seen before pretty well. And we see the same sort of thing with our Dota bots, right? They're able to, within this game, they're able to play against humans, which is something that never existed in its evolutionary environment, totally different play styles from humans versus the bots.
我觉得这个智能体中已经内置了这种能力。但是，从人的角度来看，它并不算聪明，对吧？它无法学习微积分，但它能够极其熟练地在自己的环境中导航。它能够非常好地处理在以前从未遇到过的环境中的意外情况。我们在 Dota 机器人中也看到了同样的情况，对吧？它们能够在这个游戏中与人类玩家对决，这是其演化环境中从未存在过的，人类和机器人的玩法完全不同。
And yet, it's able to handle it extremely well. And that's something that I think was very surprising to us was something that doesn't really emerge from what we've seen with PPO at smaller scale, right? And the kind of scale we're running this stuff at was, it could take 100,000 CPU cores running with hundreds of GPUs. It was probably about something like hundreds of years of experience going into this bot every single real day. And so that scale is massive.
然而，它能够非常好地处理这一点。我认为这是我们感到非常惊讶的地方，因为在我们以前使用PPO进行小规模实验时，这种效果并没有显现出来。我们现在处理的规模非常大，可能需要使用10万个CPU核心进行计算，并使用数百个GPU。这相当于每一天都需要使用几百年的经验来训练这个机器人。因此，这种规模是庞大的。
And we start to see very different kinds of behaviors out of the algorithms that we all know and love.
我们开始看到那些我们都所熟知和热爱的算法表现出了非常不同的行为。
Dota, you mentioned beat the world expert, 1v1. And then you didn't weren't able to win five view five this year.
你说过你能打败世界专家，一对一的。但是你今年不行，无法在五个对五个中赢得胜利。
Yeah, at the best place in the world. So what's the comeback story? What's, first of all, talk through that. That's an exceptionally exciting event. And what's the following months in this year look like?
是的，在世界上最好的地方。那么回归的故事是什么？首先，让我们谈一谈这个特别激动人心的事件。接下来的几个月，今年还有什么计划吗？
Yeah, yeah. So one thing that's interesting is that we lose all the time because we play, so the Dota team at OpenAI, we played the bot against better players than our system all the time. Or at least we used to it, right? The first time we lost publicly was we went up on stage at the international and we played against some of the best teams in the world.
嗯，嗯。有趣的一点是，我们总是输，因为我们玩得不好。在OpenAI的Dota团队里，我们经常与比我们系统更强的玩家打bot。或者至少我们习惯了这样做，对吧？我们第一次公开输是在国际赛的舞台上，我们与世界上一些最好的团队对战。
And we ended up losing both games, but we gave them a run for their money, right? The both games were kind of 30 minutes, 25 minutes, and they went back and forth, back and forth, back and forth. And so I think that really shows that we're at the professional level and that kind of looking at those games, we think that the coin could have gone a different direction and it could have had some wins.
我们最终输了两场比赛，但我们给他们带来了强烈的竞争，对吧？这两场比赛差不多是30分钟和25分钟，双方攻防不停地来回交替。因此我认为这真正展示了我们达到了专业水平，看到那些比赛，我们认为赢输都有可能，只是一枚硬币可能决定胜负。
That was actually very encouraging for us. And it's interesting because the international was at a fixed time, right? So we knew exactly what day we were going to be playing and we pushed as far as we could, as fast as we could.
那对我们来说实际上是非常鼓舞人心的。有趣的是，国际比赛是在固定时间进行的，对吧？所以我们知道我们将在哪一天比赛，我们尽可能地加速推进。
Two weeks later, we had a bot that had an 80% win rate versus the one that played at TI. So the march of progress, you should think of as a snapshot rather than as an end state. And so in fact, we'll be announcing our finals pretty soon.
两周后，我们拥有了一个与TI赛场上的机器人相比，获胜率高达80％的机器人。因此，你应该将进步之路视为快照，而不是终点。事实上，我们很快就会宣布我们的决赛。
I actually think that we'll announce our final match prior to this podcast being released. So there should be, we'll be playing against the world champions. And for us, it's really less about like the way that we think about what's upcoming is the final milestone, the final competitive milestone for the project, right?
实际上我觉得我们将在这个播客发布前宣布我们的最后一场比赛。这意味着我们将会和世界冠军队对战。对我们而言，我们关注的并不是比赛形式，我们认为这是项目的最终竞技里程碑。
That our goal in all of this isn't really about beating humans at Dota. Our goal is to push the state of the art and reinforcement learning. And we've done that, right? And we've actually learned a lot from our system and that we have, I think a lot of exciting next steps that we want to take.
我们的目标并不是在Dota中击败人类，而是推动先进的增强学习技术。我们已经实现了这一目标，实际上我们从我们的系统中学到了很多，并且有很多令人兴奋的下一步计划。
And so, you know, kind of the final showcase of what we built, we're going to do this match. But for us, it's not really the successor failure to see, you know, do we have the coin flip go in our direction or against?
所以，你知道吧，我们建立的东西的最终展示，我们要进行这场比赛。但对我们来说，真的不是看成功或失败，而是看硬币抛起来会倒哪一面？
Where do you see the field of deep learning having in the next few years? Where do you see the work and reinforcement learning perhaps heading and more specifically with OpenAI, all the exciting projects that you're working on? What is 2019 hold for you?
你认为未来几年深度学习这个领域会有什么发展？你认为工作和强化学习方面还可能朝什么方向发展，尤其是在OpenAI这个令人兴奋的项目方面？对于你来说，2019年会有什么憧憬？
Massive scale. Scale. I will put an astros on that and just say, you know, I think that it's about ideas plus scale. You need both. So that's a really good point.
“大规模”，我会加一个astros然后说，你知道的，我认为这涉及到想法和规模。两者都需要。所以这是一个非常好的观点。
So the question in terms of ideas, you have a lot of projects that are exploring different areas of intelligence. And the question is, when you think of scale, do you think about growing scale with those individual projects or do you think about adding new projects? And sorry, if you were thinking of adding new projects, or if you look at the past, what's the process of coming up with new projects of new ideas?
所以就理念而言，你有很多探索不同智能领域的项目。问题是，当你考虑到规模时，是想让这些个别项目扩大规模，还是想增加新项目？对不起，如果你正在考虑添加新项目，或者如果你看过过去，那么提出新项目的新思路的过程是什么？
So we really have a life cycle of project here. So we start with a few people just working on a small scale idea and language is actually a very good example of this.
所以我们在这里真的有项目的生命周期。我们从一些人开始只是在小规模的想法上工作，语言实际上是一个非常好的例子。
That it was really one person here who was pushing on language for a long time. I mean, then you get signs of life, right? And so this is like, let's say, with the original GPT, we had something that was interesting. And we said, okay, it's time to scale this.
这里真的有一个人长期在推动语言。我的意思是，你会看到生命的征象，对吧？所以就好比，我们原来有一个有趣的GPT，然后我们说，好的，现在是扩展它的时候了。
It's time to put more people on it, put more computational resources behind it, and then we just kind of keep pushing and keep pushing. And the end state is something that looks like Dota or robotics, where you have a large team of 10 or 15 people that are running things at very large scale, and that you're able to really have material engineering and machine learning science coming together to make systems that work and get material results that just would have been impossible otherwise.
现在是时候把更多的人，更多的计算资源投入进来，然后我们一直推进，一直推进。最终的状态看起来像是Dota或机器人技术，你需要有10或15人的大型团队来运作，使用物质工程和机器学习科学相结合的方法，使系统可以运作，并获得在其他情况下无法实现的物质成果。
So we do that whole life cycle. We've done it a number of times, typically end to end. It's probably two years or so to do it. The organization has been around for three years, so maybe we'll find that we also have longer life cycle projects, but we work up to those.
所以我们完成整个生命周期。我们做了很多次，通常都是从头到尾的。可能需要大约两年左右的时间来完成。这个组织已经存在了三年了，所以也许我们会发现我们还有更长的生命周期项目，但我们会逐步朝这个方向努力。
We have, so one team that we're actually just starting, Ilyan Iyer, are kicking off a new team called the Reasoning Team, and that this is to really try to tackle how do you get neural networks to reason. And we think that this will be a long term project, and it's one that we're very excited about.
我们有一个叫Ilyan Iyer的团队，他们正在启动一个名为Reasoning Team的新团队，旨在尝试解决如何让神经网络进行推理的问题。我们认为这将是一个长期的项目，我们对此非常兴奋。
In terms of reasoning, super exciting topic, what kind of benchmarks, what kind of tests of reasoning do you envision? What would if you set back whatever drink, and you would be impressed that this system is able to do something, what would that look like?
从推理的角度来看，这是一个超级令人兴奋的话题。你期待看到什么样的标准和测试？如果你放下了任何饮料，你会被这个系统能够做到什么而印象深刻？具体来说，你期待看到什么样的表现？
Not theorem proving. Theorem proving. So some kind of logic, and especially mathematical logic. I think so, right? And I think that there's kind of other problems that are dual to theorem proving in particular. We think about programming. I think about even security analysis of code that these all kind of capture the same sorts of core reasoning and being able to do some out of distribution generalization.
不是证明定理，是证明定理。所以这涉及某种逻辑，尤其是数学逻辑。我这么认为，对吧？我认为还有其他问题，特别是定理证明的对偶问题。我们谈到编程。我考虑到代码的安全分析，这些都捕捉到了相同的核心推理，并且能够进行一些分布外的泛化。
It would be quite exciting if OpenAI reasoning team was able to prove that P equals NP. That would be very nice. It would be very, very exciting, especially. If it turns out that P equals NP, that'll be interesting too. A big. It would be ironic and humorous.
如果OpenAI推理团队能够证明P等于NP，那会非常令人兴奋。这将是非常好的事情。特别是，如果P确实等于NP，那就更加激动人心了。如果最终这个命题成立，那将是一个很有趣的事情。非常有趣，而且有点儿具有讽刺意味和幽默感。
So what problem stands out to you is the most exciting and challenging impactful to the work for us as a community in general and for OpenAI this year. You mentioned reasoning. I think that's a heck of a problem.
你认为对我们作为一个社区来说最重要和具有挑战性的问题是什么？并且对于OpenAI来说也是如此。你提到了推理，我觉得那是一个相当棘手的问题。
Yeah, so I think reasoning is an important one. I think it's gonna be hard to get good results in 2019. You know, again, just like we think about the lifecycle, takes time.
嗯，我认为推理是一个重要的方面。我觉得要在2019年获得好的结果可能会很困难。你知道的，就像我们考虑生命周期一样，这需要时间。
I think for 2019, language modeling seems to be kind of on that ramp. It's at the point that we have a technique that works. We want to scale 100x, thousand x, see what happens. Awesome.
我觉得，对于2019年来说，语言建模似乎正在这个阶段的台阶上。我们已经有了一种行之有效的技术。我们想要进行100倍、1000倍的扩展，看看会发生什么。太棒了。
Do you think we're living in a simulation? I think it's hard to have a real opinion about it. It's actually interesting. I separate out things that I think can have like, yield materially different predictions about the world from ones that are just kind of fun to speculate about. And I kind of view simulation as more like, is there a flying teapot between Mars and Jupiter? Like, maybe, but it's a little bit hard to know what that would mean for my life.
你认为我们生活在模拟世界中吗？我觉得很难有一个真正的意见。这个问题确实很有意思。我把我认为可以对世界做出物质上不同预测的事情跟那些只是好玩的猜测分开。我认为模拟世界更像是介于火星和木星之间有一个飞行茶壶。也许是有的，但很难知道这对我的生活意味着什么。
So there is something action relied. Some of the best work opening as done is in the field of reinforcement learning. And some of the success of reinforcement learning come from being able to simulate the problem and try to solve. So you have a hope for reinforcement, for the future reinforcement learning and for the future simulation.
所以，有一些动作依赖的东西。在强化学习领域中，一些最好的工作都是通过模拟问题并试图解决来完成的。强化学习的一些成功来自于能够模拟问题并尝试解决。因此，您对强化学习、未来的强化学习和未来的模拟有希望。
Like whether we're talking about autonomous vehicles or any kind of system, do you see that scaling? So we'll be able to simulate systems and hence be able to create a simulator that echoes our real world and proving once and for all, even though you're denying it that we're living in a simulation. I feel like I've used that for questions, right?
就像我们讨论无人驾驶汽车或任何类型的系统一样，你认为这种扩展性有多大？因此，我们将能够模拟系统，从而能够创建一个模拟器，反映我们的真实世界，并一劳永逸地证明，尽管你否认它，我们仍然生活在一个模拟中。我感觉我之前用过这个问题，对吧？
So kind of for the core there of like, can we use simulation for self-driving cars? Take a look at our robotic system, Dactyl, right? That was trained in simulation using the Dota system. In fact, and it transfers to a physical robot. And I think everyone looks at our Dota system, they're like, okay, it's just a game. How are you ever going to escape to the real world? And the answer is, well, we did it with the physical robot, the no one can program. And so I think the answer is simulation goes a lot further than you think if you apply the right techniques to it.
非常好的核心在于，我们是否可以为自动驾驶汽车使用模拟？看看我们的机器人系统Dactyl，是吗？它是通过使用Dota系统进行模拟训练的。实际上，它可以转移到物理机器人上。我认为，当每个人看到我们的Dota系统时，他们都会想，好吧，这只是个游戏。你怎么可能逃脱到现实世界呢？答案是，我们用无法编程的物理机器人做到了。因此，我认为如果您应用正确的技术，模拟将远远超出您的想象。
Now there's a question of, are the beings in that simulation going to wake up and have consciousness? I think that one seems a lot harder to again reason about. I think that you really should think about, like, we're exactly just human consciousness come from in our own self-awareness. And is it just that once you have a complicated enough neural net, do you have to worry about the agent's feeling pain? And I think there's interesting speculation to do there, but again, I think it's a little bit hard to know for sure.
现在有一个问题，那就是那个模拟中的生物会不会醒来并具有意识？我觉得这个问题好像比之前的更难理解。我们应该思考一下，人类的自我意识究竟来自哪里？当神经网络变得足够复杂时，你是否需要担心代理人的痛苦感？在这方面有趣的猜想可以推测，但是我认为很难确切知道。
Well, let me just keep with the speculation. Do you think to create intelligence, general intelligence, you need one consciousness and two body? Do you think any of those elements are needed or is intelligence something that's sort of thogun all to those?
好的，让我就继续大胆猜测一下。你认为要创造普遍智能，需要一个意识和两个身体吗？你觉得这些元素中有哪些是必需的，还是智能本身就是所有这些元素的总和呢？
I'll stick to the kind of like the non-grand answer first. So the non-grand answer is just to look at, what are we already making work? You'll get GPT-2, a lot of people would have said that to even get these kinds of results, you need real world experience. You need a body, you need grounding. How are you supposed to reason about any of these things? How are you supposed to, like, even kind of know about smoking fire and those things if you've never experienced them? And GPT-2 shows that you can actually go way further than that kind of reasoning would predict.
我觉得我会先坚持那些不太夸张的回答。这些回答就是看看我们已经成功做到了什么。比如，我们有GPT-2，有很多人可能会说，要想得到这种效果，你需要现实世界的经验。你需要有身体，需要有基础。如果你从来没有经历过的话，你怎么能理解这些事情呢？比如，如果你从来没有抽过烟或者见过大火，你怎么可能去推理它们呢？但是GPT-2就证明，你可以比那些推理要求更高的东西做得更好。
So I think that in terms of doing any consciousness, do we need a body? It seems the answer is probably not, right? That we can probably just continue to push kind of the systems we have. They already feel general. They're not as competent or as general or able to learn as quickly as an AGI would. But they're at least like kind of proto-AGI in some way. And they don't need any of those things.
我认为，如果要进行任何意识方面的活动，我们需要有身体吗？这似乎答案可能是否定的，对吧？我们可能只需要继续推进我们已有的系统。它们已经感觉很普遍了。虽然它们不如AGI那么能干、普遍和学习能力快速，但它们至少是某种程度上的原型AGI。而且它们不需要任何那些东西。
Now, now let's move to the grand answer, which is if our neural nets conscious already would we ever know how can we tell? Here's where the speculation starts to become at least interesting or fun and maybe a little bit disturbing, depending on where you take it. But it certainly seems that when we think about animals, that there are some continuum of consciousness, my cat is conscious in some way. Not as conscious as a human. You could imagine that you could build a little consciousness meter right? You pointed out a cat, it gives you a little reading, pointed out a human, it gives you much bigger reading. What would happen if you pointed one of those at a dotat neural net?
现在，我们来谈谈根本的问题——如果我们的神经网络已经有了意识，我们如何知道呢？这就是我们开始进行推测的地方，这至少是有趣或有趣且令人不安的，具体取决于你的口味。但当我们思考动物时，似乎有一些意识的连续性，我的猫某种程度上是有意识的。不像人那样有意识。您可以想象，您可以构建一个小意识计量器，对着猫指出它，读数就会有一点，指出人类，则读数更大。如果你把它指向一个神经网络点，会发生什么？
And if you're training this massive simulation, do the neural nets feel pain? It becomes pretty hard to know that the answer is no.
如果你正在训练这个庞大的模拟，神经网络会感到疼痛吗？很难确定答案是否是否定的。
And it becomes pretty hard to really think about what that would mean if the answer were yes. And it's very possible, for example, you could imagine that maybe the reason these humans are have consciousness is because it's a convenient computational shortcut.
当回答是肯定的时候，真正思考它意味着什么就变得相当困难。例如，很有可能这些人之所以有意识，是因为这是一种方便的计算捷径。
If you think about it, if you have a being that wants to avoid pain, which seems pretty important to survive in this environment and wants to eat food, then maybe the best way of doing it is to have it being that's conscious.
如果你仔细想一想，一种想要避免痛苦的生命体对于在这个环境中生存来说似乎非常重要，也需要吃食物，那么也许最好的方法就是拥有一个有意识的生命体。
In order to succeed in the environment, you need to have those properties and how are you supposed to implement them? And maybe this consciousness is way of doing that. If that's true, then actually maybe we should expect that really competent reinforcement learning agents will also have consciousness. But it's a big if.
为了在这个环境中取得成功，你需要具备这些特点，你应该如何实施呢？也许这种意识就是实现这一点的方式。如果是这样的话，那么我们应该期望真正有能力的强化学习智能体也会具有意识。但这是个大问号。
And I think there are a lot of other arguments that can make in other directions. I think that's a really interesting idea that even GPT-2 has some degree of consciousness. That's something that's actually not as crazy to think about.
我认为有很多其他的论点可以从其他方向来进行辩论。我认为这是一个非常有趣的想法，即GPT-2甚至具有某种程度的意识。这是一个实际上并不太疯狂的想法。
It's useful to think about as we think about what it means to create intelligence of a dog, intelligence of a cat, and the intelligence of a human.
在我们思考如何创造狗、猫、和人类的智能时，思考这一点是很有用的。
So last question, do you think we will ever fall in love in the movie, her, with an artificial intelligence system or an artificial intelligence system falling in love with a human?
最后一个问题，你认为我们会在电影《她》中与人工智能系统坠入爱河，还是人工智能系统会与人类坠入爱河？
I hope so. If there's any better way to end it is on love.
我希望是这样。如果有更好的方式来结束，那就是用爱。
So Greg, thanks so much for talking to me. Thank you for having me.
Greg，非常感谢你和我交谈。谢谢你邀请我。