AI-augmented game development with Inworld co-founder Kylan Gibbs - YouTube

Our final demo of the day is Kylan, who will be giving a demo from in-world AI on how folks are using LLMs to actually change video games and fundamental gameplay mechanics. Thank you, Kylan. Thank you so much. It's been a crazy week. GDC, GTC, gaming is hot right now. So I'm excited to show what I'm going to be sharing is actually a demo of an actual game that's kind of going live now. So just for a quick preview for those who aren't familiar, basically in-world is a kind of end-to-end vertically integrated platform specifically for gaming. You know, CJ is mentioning developers are hard to work with. Game developers are really hard to work with. They are both technically savvy and also extremely creatively want to love control. So basically we have an engine that actually drives a lot of experiences at runtime and it connects to anything like Unreal or Unity and actually drives the behavior by taking in the game state and then putting that out. We have a studio that actually is actually how you craft an augmented existing game development using AI and we have a core solution. So a lot of the challenges that I think lots of folks here are encountering, games want to run locally. They want no latency and all those kinds of things and so we're solving for that.

我们今天的最后一个演示是Kylan，他将展示如何在虚拟世界中应用人工智能，以改变视频游戏和基本的游戏玩法机制。谢谢你，Kylan。非常感谢。这是一个疯狂的一周。GDC、GTC，游戏现在非常火爆。所以我很兴奋地要展示的是一个即将上线的实际游戏演示。简单预览一下，对于那些不熟悉的人来说，虚拟世界是一个专为游戏而设计的端到端垂直集成平台。就像CJ提到的，开发者很难合作。游戏开发者真的很难合作。他们既懂技术，又非常富有创造性，希望拥有完全的控制权。因此，我们拥有一个引擎，可以在运行时驱动许多体验，并连接到像虚幻引擎或Unity之类的引擎，通过接收游戏状态来驱动行为，并输出结果。我们还有一个工作室，可以使用人工智能来改善现有游戏开发，并提供核心解决方案。许多人在这里遇到的挑战之一是，游戏希望在本地运行。他们希望没有延迟等等，所以我们正在为解决这些问题而努力。

And what I'll be showing here today is actually mainly a demonstration of the engine component of that. So we this week launched with an experience with NVIDIA, a partnership with Microsoft and their studios called Project Explorer and we released an early experience of a game working with Ubisoft. So I'll be showing a video of the Ubisoft experience. So the main thing I guess is, you know, games have always been one of those places where people are super excited to apply AI but it's really hard because the controls that you have to put in place need to actually be beholden to what the narrative designers and writers of the games actually want to do. And so in this video, first you'll see sort of a conversational dynamic where they've added on game design elements that are actually kind of making the dialogue more engaging. So it's not just a chat about experience but it actually fits within what you feel like a gamer should be a game of fight type of experience.

今天我要展示的主要是引擎组件的演示。本周，我们与NVIDIA合作推出了与微软及其工作室合作的项目“探索者计划”，并发布了与育碧合作的早期游戏体验。我将展示育碧体验的视频。我想说的主要是，游戏一直是人们非常乐意应用人工智能的地方之一，但这实际上非常困难，因为必须将控制措施与游戏的叙事设计师和编剧实际想做的事情保持一致。在这个视频中，首先你会看到一种对话动态，他们添加了让对话更加吸引人的游戏设计元素。因此，这不仅仅是一个关于体验的聊天，而是实际上符合你认为游戏者应该有的战斗式体验。

So we'll start with that and then we'll pause and I'll explain the next part. So what is this resistance? What are they form? Like why do we even not like these mega-corps? Like I like, you know, eating burgers. Our resistance has been around for years fighting against the oppression of these mega-corps who exploit resources for their own greed. Thus with our sabotage missions, we can provide a satisfying outlet for your skills while also fighting for a greater cause. But enough about us. What about your burger preference? Uh. Ah, the classic case of burgers versus revolution who needs justice and equality when you can have a juicy burger instead. Time to make it your priorities, my friend. Well, for me, it's personal. Everything my home and the world I love destroyed by their actions has fueled my passion to fight back.

所以我们先从这里开始，然后我们停下来，我会解释接下来的部分。那么这种抵抗是什么？他们是如何形成的？为什么我们甚至不喜欢这些大型企业？比如我喜欢吃汉堡。我们的抵抗已经存在多年，与那些为了私利而剥削资源的大型企业作斗争。因此，通过我们的破坏任务，我们可以为你提供一个满足你技能的出口，同时也为更伟大的事业而战。但是我们就不谈我们了，你的汉堡偏好是什么呢？啊，这是经典的汉堡与革命的对立，当你可以选择汁水满满的汉堡时，谁需要正义和平等呢。是时候把它们列为你的优先事项了，我的朋友。对我来说，这是个人的。一切都是为了我的家园和我所热爱的世界而被他们的行为毁灭，这燃起了我与他们作斗争的激情。

But as a team, we can make a bigger impact and take down these corporations for good. And who knows? Maybe we'll celebrate with some burgers after a resistance mission. Oh my gosh. Okay, I love it, Bloom. Absolutely love it. Let's do it. That's what I like to hear. Insulting it? Okay. Thanks Nelson. Let's take some history. And maybe later we can grab some burgers and toast to a successful mission.

作为一个团队，我们可以产生更大的影响，摧毁这些公司，从根本上解决问题。谁知道呢？也许我们会在抵抗任务之后用一些汉堡庆祝。哦，天哪。好的，我喜欢这个想法，布鲁姆。非常喜欢。让我们做吧。我喜欢听到这样的话。侮辱它？好的。谢谢，纳尔逊。让我们了解一些历史。也许之后我们可以去吃一些汉堡，庆祝一个成功的任务。

So a few things that I just want to point out here is because everybody here is very familiar with AI. The key things is, you know, of course we can do updates to emotional states, relationship dynamics, all of those things in the back end. But what we realized for Game Studio specifically is actually how we expose those is super important to guide and give the idea of progression. Because if you're just doing all of that in the back end, they ultimately don't care. So it's how we expose them. Kind of mapping that with game design is actually what allows it to come to life.

所以我想指出的一些事情是因为大家都非常熟悉人工智能。关键是，当然我们可以更新情感状态，关系动态，所有这些后台的事情。但我们意识到对于游戏工作室来说，其实如何展现这些是非常重要的，以引导并给出进展的想法。因为如果你只是把所有这些都放在后台，他们最终是不在乎的。所以我们如何展示它们。将其与游戏设计相结合实际上是让其栩栩如生的关键。

Okay team, the new drones in the air. Let's check out how shadows does with this recon mission. Since it's your first rodeo, don't be shy. Shoot any questions my way. All right. Oh, shadow. Hurry up. Is in sight. Shatter. Hurry up. That car might show up. Let's hack that laptop and get out of here. Yes. Success. The drone has successfully hacked into the laptop and is downloading the map as we speak. Now we just need to get out of there before anyone realizes what's happened.

团队，新的无人机已经在空中。让我们来看看这次侦察任务阴影的表现如何。因为这是你们的第一次，不要害羞。有问题就尽管问我。好的。哦，阴影，快点。目标已经出现。赶紧行动。那辆车可能会出现。让我们入侵那台笔记本电脑，然后离开这里。是的。成功了。无人机已成功入侵了笔记本电脑，正在下载地图。现在我们只需要在有人意识到发生了什么之前离开。

Watch out. There's a guard. Watch out. Don't worry. We got this. Just keep an eye out for any other guards and let me know if you see anything suspicious. We're almost there. Hey, Bloom, how tall do you think that guard is? Around a. Oh, no, I got that though. Around six feet. That's pretty cool. And so there as well because we're, you know, we're, we have to use a lot of these models in parallel. And so it's super challenging when you're putting together the voice, the multimodal input, all those different things. So large models don't necessarily work there. And so it ends up being a series of a lot of models that are running in parallel to effectively process that and then send the signals to the ultimate prompt.

小心，有一名警卫。小心。别担心，我们能搞定的。只要留意其他警卫，如果看到任何可疑情况就告诉我。我们快到了。嘿，布鲁姆，你觉得那个警卫有多高？大约六英尺。这很酷。因为我们必须同时使用许多这些模型，所以在组合语音、多模态输入和其他各种因素时非常具有挑战性。大型模型不一定适用于此。因此，最终会是一系列并行运行的许多模型有效地处理信息，然后将信号发送到最终的提示中。

So we've got. And in this case, you'll see the actual dynamic game state as well. So it's not just changing the character dynamics, but actually dynamically changing the actual game state itself by triggering different things on the U.N. For the first step, the only way to enter the building without triggering an alarm of any sort is through the window of the gallery on the second. Well, for the entry point, the best option is to access the gallery's window on the second floor from the outside. But the obstacle is we'll need a ladder to reach it. What do you think? I don't know if that's the best option.

因此，我们要做的就是，在这种情况下，你将看到实际的动态游戏状态。所以不仅仅是改变角色的动态，而且实际上通过在U.N上触发不同的事物来动态改变游戏状态本身。在第一步中，进入建筑物的唯一方法，而不触发任何警报，是通过二楼画廊的窗户。对于进入点，最好的选择是从外面进入二楼画廊的窗户。但是障碍是我们需要一个梯子才能到达那里。你觉得怎么样？我不知道那是否是最佳选择。

Why do you think that's the best option? Because the other sides of the villa are heavily guarded and monitored by cameras. Plus, Shadow's Intel showed that the guard in the largest threat. We can't risk being seen or heard, so we have to be strategic. Any ideas? I see that you've got a grappling hook. Why'd you not decide to do that one? That one looks pretty cool. You know, you're right. That could work. We could use the grappling hook to access the gallery's window. Good call, Nelson. Let's go with that plan.

你为什么认为这是最佳选择呢？因为别墅的其他几面都受到严密的警卫和监控摄像头的监视。此外，Shadow的情报显示，卫兵是最大的威胁。我们不能冒险被发现或听到，所以我们必须谨慎行事。有什么想法吗？我看你有个挂钩。为什么不考虑那个？那个看起来很酷。你说得没错。那个应该可行。我们可以使用挂钩来进入画廊的窗户。好主意，尼尔森。我们就按照这个计划行动。

A lot of what we're doing here as well is generating quests on the fly. So effectively, we're able to produce new quests that are dynamically generated based on the interactions the player has and the basic grounding goals. The fun of what we're building is we get to bring together everything that's consistently coming out of research, but applying that with a user who has very specific needs and create a vision. Super fun. If you're interested in games or applying your technologies to games as well, we're always looking for partners. Thank you.

我们正在这里做的很多工作是即兴生成任务。因此，我们能够根据玩家的互动和基本目标动态生成新的任务。我们正在构建的乐趣在于，我们能够将研究中持续出现的一切内容汇集起来，同时结合具有特定需求的用户，创造出一个愿景。非常有趣。如果你对游戏或将你的技术应用于游戏中感兴趣，我们一直在寻找合作伙伴。谢谢。