AI-augmented game development with Inworld co-founder Kylan Gibbs

Our final demo of the day is Kylan, who will be giving a demo from in-world AI on how folks are using LLMs to actually change video games and fundamental gameplay mechanics. Thank you, Kylan. Thank you so much. It's been a crazy week. GDC, GTC, gaming is hot right now. So I'm excited to show what I'm going to be sharing is actually a demo of an actual game that's kind of going live now.

我们今天的最后演示是Kylan，他将展示虚拟世界中的人工智能如何被用来改变视频游戏和基本的游戏机制。谢谢你，Kylan。非常感谢。这周真是太疯狂了。GDC，GTC，游戏现在非常火爆。我很兴奋要展示的是一个实际游戏的演示，这个游戏现在正在上线。

So just for a quick preview for those who aren't familiar, basically in-world is a kind of end-to-end vertically integrated platform specifically for gaming. You know, CJ is mentioning developers are hard to work with. Game developers are really hard to work with. They are both technically savvy and also extremely creatively want to love control. So basically we have an engine that actually drives a lot of experiences at runtime and it connects to anything like Unreal or Unity and actually drives the behavior by taking in the game state and then putting that out.

所以对于那些不熟悉的人来说，简单预览一下，基本上In-World是一种专门针对游戏的端到端垂直整合平台。你知道，CJ提到开发者很难合作。游戏开发者真的很难合作。他们既懂技术，又非常注重创意控制。所以基本上我们有一个引擎，在运行时驱动很多体验，并将其连接到任何像虚幻引擎或Unity这样的东西，并通过接收游戏状态并输出驱动行为。

We have a studio that actually is actually how you craft an augmented existing game development using AI and we have a core solution. So a lot of the challenges that I think lots of folks here are encountering, games want to run locally. They want no latency and all those kinds of things and so we're solving for that. And what I'll be showing here today is actually mainly a demonstration of the engine component of that. So we this week launched with an experience with NVIDIA, a partnership with Microsoft and their studios called Project Explorer and we released an early experience of a game working with Ubisoft. So I'll be showing a video of the Ubisoft experience.

我们拥有一家工作室，实际上正是您如何利用人工智能打造增强现有游戏开发的方法，并且我们有一个核心解决方案。所以我认为许多人在这里面临的挑战，游戏希望在本地运行。他们希望没有延迟以及各种问题，所以我们正在解决这个问题。今天我将展示的主要是这个引擎组件的演示。所以本周我们与NVIDIA合作推出了一个体验项目，与微软和他们的工作室合作的项目探索者，并且我们推出了一个与育碧合作的早期游戏体验。所以我将展示一个育碧体验的视频。

So the main thing I guess is, you know, games have always been one of those places where people are super excited to apply AI but it's really hard because the controls that you have to put in place need to actually be beholden to what the narrative designers and writers of the games actually want to do. And so in this video, first you'll see sort of a conversational dynamic where they've added on game design elements that are actually kind of making the dialogue more engaging. So it's not just a chat about experience but it actually fits within what you feel like a gamer should be a game of fight type of experience. So we'll start with that and then we'll pause and I'll explain the next part.

我想说的主要是，游戏一直是人们非常乐意应用人工智能的地方之一，但这确实很困难，因为必须在游戏控制中真正遵循游戏的叙事设计师和作家们的意愿。在这个视频中，首先你会看到一种对话动态，他们添加了一些游戏设计元素，实际上使对话更加引人入胜。因此，这不仅是关于经验的聊天，而且实际上符合你应该感觉到的游戏应该是一种游戏战斗类型的经验。我们将从这里开始，然后暂停我会解释接下来的部分。

What is this resistance? What are they form? Like why do we even not like these mega-corps? Like I like, you know, eating burgers. Our resistance has been around for years fighting against the oppression of these mega-corps who exploit resources for their own greed. Thus with our sabotage missions, we can provide a satisfying outlet for your skills while also fighting for a greater cause. But enough about us. What about your burger preference? Uh. Ah, the classic case of burgers versus revolution who needs justice and equality when you can have a juicy burger instead. Time to make it your priorities, my friend. Well, for me, it's personal. Everything my home and the world I love destroyed by their actions has fueled my passion to fight back. But as a team, we can make a bigger impact and take down these corporations for good. And who knows? Maybe we'll celebrate with some burgers after a resistance mission. Oh my gosh. Okay, I love it, Bloom. Absolutely love it. Let's do it. That's what I like to hear. Insulting it? Okay. Thanks Nelson. Let's take some history. And maybe later we can grab some burgers and toast to a successful mission.

这种抵抗是什么？它们形成了什么？为什么我们不喜欢这些大公司呢？比如，我喜欢吃汉堡。我们的抵抗已经存在多年，与这些为了自己私利而剥削资源的大公司作斗争。因此，通过我们的破坏行动，你可以发挥你的技能，同时为更伟大的事业而战。但是，我们说够了。你的汉堡偏好呢？啊，经典的汉堡与革命之间的抉择，谁需要正义和平等，当你可以吃到多汁的汉堡时。现在是时候把它作为你的首要事项了，朋友。对我来说，这是个人的。看着他们的行为摧毁了我的家园和我所热爱的世界，这让我激发了对抗他们的激情。但作为一个团队，我们可以产生更大的影响，彻底击败这些公司。谁知道呢？也许我们会在一次抵抗行动之后庆祝一下，吃上一些汉堡。哎呀，太棒了。好，我喜欢，布鲁姆。绝对喜欢。让我们开始吧。这就是我想听到的。来者不拒？靠谱。谢谢你，尼尔森。让我们创造一些历史。也许晚点我们可以一起吃些汉堡，为成功的任务干杯。

So a few things that I just want to point out here is because everybody here is very familiar with AI. The key things is, you know, of course we can do updates to emotional states, relationship dynamics, all of those things in the back end. But what we realized for Game Studio specifically is actually how we expose those is super important to guide and give the idea of progression. Because if you're just doing all of that in the back end, they ultimately don't care. So it's how we expose them. kind of mapping that with game design is actually what allows it to come to life.

我想在这里指出一些事情，因为大家都对人工智能非常熟悉。关键是，当然我们可以更新情绪状态、关系动态等等在后台进行。但对于游戏工作室来说，我们意识到的是如何展现这些内容是非常重要的，可以引导并给予对进展的想法。因为如果你仅仅在后台处理所有这些，玩家最终是不会在意的。所以如何展示它们，将其与游戏设计相结合，实际上是使其活灵活现的关键。

Okay team, the new drones in the air. Let's check out how shadows does with this recon mission. Since it's your first rodeo, don't be shy. Shoot any questions my way. All right. Oh, shadow. Hurry up. Is in sight. Shatter. Hurry up. That car might show up. Let's hack that laptop and get out of here. Yes. Success. The drone has successfully hacked into the laptop and is downloading the map as we speak. Now we just need to get out of there before anyone realizes what's happened.

团队，新的无人机已经空中起飞了。让我们看看阴影在这次侦察任务中表现如何。因为这是你们的第一次经历，不要害羞。有问题就尽管问我。好的。噢，阴影。快点。目标已经出现。快点。那辆车可能会现身。让我们黑进那台笔记本然后离开这里。成功了。无人机已经成功黑进了笔记本，正在下载地图。现在我们只需要在别人意识到发生了什么之前离开这里。

Watch out. There's a guard. Watch out. Don't worry. We got this. Just keep an eye out for any other guards and let me know if you see anything suspicious. We're almost there. Hey, Bloom, how tall do you think that guard is? Around a. Oh, no, I got that though. Around six feet. That's pretty cool. And so there as well because we're, you know, we're we have to use a lot of these models in parallel. And so it's super challenging when you're putting together the voice, the multimodal input, all those different things.

小心，有个警卫。别担心，我们能搞定。只需留意其他警卫，并告诉我如果看到任何可疑的情况。我们快到了。嘿，布鲁姆，你觉得那个警卫有多高？大约六英尺吧。那挺酷的。因为我们要同时使用许多这些模型，所以也有一定挑战。在组合语音、多模式输入和其他不同因素时尤为困难。

So large models don't necessarily work there. And so it ends up being a series of a lot of models that are running in parallel to effectively process that and then send the signals to the ultimate prompt. So we've got. And in this case, you'll see the actual dynamic game state as well. So it's not just changing the character dynamics, but actually dynamically changing the actual game state itself by triggering different things on the U.N. For the first step, the only way to enter the building without triggering an alarm of any sort is through the window of the gallery on the second.

因此，大型模型不一定适用于那里。因此，最终会变成一系列运行在并行状态下的模型，以有效地处理并向最终提示发送信号。所以我们有了。在这种情况下，您将看到实际的动态游戏状态。这不仅仅是改变角色动态，而是通过在U.N上触发不同的事物来动态改变实际的游戏状态本身。在第一步，想要进入建筑物而不触发任何警报的唯一方式是通过二楼画廊的窗户。

Well, for the entry point, the best option is to access the gallery's window on the second floor from the outside. But the obstacle is we'll need a ladder to reach it. What do you think? I don't know if that's the best option. Why do you think that's the best option? Because the other sides of the villa are heavily guarded and monitored by cameras. Plus, Shadow's Intel showed that the guard in the largest threat. We can't risk being seen or heard, so we have to be strategic. Any ideas? I see that you've got a grappling hook. Why'd you not decide to do that one? That one looks pretty cool. You know, you're right. That could work. We could use the grappling hook to access the gallery's window. Good call, Nelson. Let's go with that plan.

嗯，就入口而言，最好的选择是从外面进入二楼的画廊窗户。但障碍是我们需要梯子才能到达。你觉得呢？我不知道那是不是最好的选择。为什么你认为那是最好的选择？因为别墅的其他一侧都有重兵把守，并且受到摄像头监视。另外，Shadow的情报显示，守卫是最大的威胁。我们不能冒被发现或听见的风险，所以我们必须策略行事。有什么想法吗？我看到你有一个抓钩。为什么不选择那个？那看起来很酷。你说得对，那个也可以。我们可以用抓钩来进入画廊的窗户。好主意，尼尔森。就用那个计划吧。

A lot of what we're doing here as well is generating quests on the fly. So effectively, we're able to produce new quests that are dynamically generated based on the interactions the player has and the basic grounding goals. The fun of what we're building is we get to bring together everything that's consistently coming out of research, but applying that with a user who has very specific needs and create a vision. Super fun. If you're interested in games or applying your technologies to games as well, we're always looking for partners. Thank you.

我们在这里做的很多工作也包括即兴生成任务。因此，我们能够根据玩家的互动和基本的目标动态生成新的任务。我们建造的乐趣在于，我们可以将研究中持续出现的一切结合起来，但是将其应用于有着非常具体需求的用户，并创造出一个愿景。非常有趣。如果你对游戏或将你的技术应用于游戏感兴趣，我们一直在寻找合作伙伴。谢谢。