AI in gaming dominated GDC 2024, and some of it actually won this skeptic over

I'm not eager for our AI future, but some uses for it did impress me at GDC.

Traveling from meeting to meeting or wandering the vast expo halls, I noticed a handful of reoccurring themes at GDC 2024 in San Francisco. On the one hand, were the developers attending the conference and their focuses (I'll write my thoughts on that soon), but on the other hand was artificial intelligence. Yes, that buzzword is something that you can't walk two steps without hearing. I'm tentative in my thinking toward AI, recognizing both its potential for greatness and its potential for extreme abuse.

At GDC, though, I did see some visions for AI in gaming that captured my attention. There are companies investing in healthy AI that aim to either aid creators or open up new possibilities for creators, and I had the privilege of seeing some of those potential futures, as well as talking to some of the people behind them. AI still poses a lot of unanswered questions and is rightfully restrained by controversy (at this point, we're still too reliant on corporations being responsible), but it was refreshing to see the other side of that equation.

AI at GDC: Major moves from NVIDIA

I see so much potential in NVIDIA's Covert Protocol demo. (Image credit: NVIDIA)

First, let's address the massive AI elephant in the room: NVIDIA. The globally popular company most known for its RTX gaming graphics cards is also one of the world's leaders in AI investments and has risen to the upper echelons of the most valuable companies thanks largely to those investments. I met with NVIDIA at GDC 2024, and the company had a lot to show me, including using AI to animate character faces when talking, translate player speech into games, upscale and improve performance in games, and even completely change a game's visual style through filters or mods.

I won't go into detail on all of that, though, because I already did. In this article I'll be focusing more on the other AI products, concepts, and tools I saw at GDC. If you want to learn more about NVIDIA's contributions, please go read my full, in-depth article on RTX, DLSS, AI, and everything else NVIDIA showed me at GDC.

AI at GDC: Inworld with Xbox, Ubisoft, and more

Xbox and Inworld partnered to prototype a text-based RPG built on AI, and I had a lot of fun with the demo.

(Image credit: Windows Central | Zachary Boddy)

NVIDIA's Covert Protocol demo, which showed how NPCs powered by AI brains can create a deep, interactive detective-style game, was done in partnership with an AI company called Inworld. It's a company with three main areas of focus at the moment: narrative, characters, and worlds. In each category, Inworld is working with different companies to build out an AI engine that can power brand-new gaming experiences, and I had a chance to check out its booth at GDC and speak to some of the team behind it.

Starting with narrative, Inworld actually partnered with Xbox to create the Narrative Graph, which can be used by writers to plot out a dynamic, flexible narrative structure for games. Inworld demoed this by letting me play a TTRPG-esque text-based game. Similar to something like Dungeons & Dragons, there was a narrator who set the scene, guided player action, and provided in-game lore and information. My job was to respond to that information in any way I saw fit and push the story forward.

With Narrative Graph, game writers decide on story beats, using lore and dialogue points, and create character profiles to provide direction, nuance, and context to the AI engine. Character arcs, branching paths born of major decisions, and crucial story progression elements can all be plotted out in advance, ensuring that the AI engine has both the information and guidance necessary to stay within the realm of the game and prevent hallucinated answers.

Honestly, the demo was a lot of fun. The AI narrator responded well to all my queries and actions; when I was unsure of what to do, I asked for more information on my objective or the world explored the area or inquired about my available tools and abilities. In true TTRPG fashion, I explored a sci-fi fortress in search of a valuable artifact, using the ventilation system and my own stealthy skills to bypass security and slip out nearly unseen. It was dynamic and felt deeply personal because I truly made everything happen. I had to be flexible, but the result was a short but excellent roleplaying experience.

I spoke with Inworld afterward about the balance between player agency (including allowing room for players to goof off and mess with the AI) and the need for story progression, as well as how players always need a way to get back on track. We also discussed how the use of AI makes games like this considerably more malleable since I could've explored the surrounding area more instead of pursuing my objective or spoke more at length about the world. Overall, it opens up a ton of possibilities for RPGs and still requires the work of writers to create the world and characters.

Next, Inworld is considering how to use AI to create more believable and dynamic characters. It's in this category that NVIDIA lays with its Covert Protocol demo, which explores how the combination of Inworld's AI brains and NVIDIA's AI micromodules can create a unique detective gaming experience you can't find anywhere else. Here, writers can build character profiles, outlining background, in-game knowledge, motivations and objectives, personality traits, emotional tendencies and habits, and personal abilities.

It's an in-depth process that requires an intimate understanding of the character beyond writing simple lines of dialogue, but the result is an AI-powered brain that's able to generate in-world responses based on that criteria. It's fascinating in motion, and NVIDIA's tech helps animate facial expressions and open up a direct dialogue between the player and NPC via speech. The result? You can have an actual conversation with video game NPCs, including casual discussions, negotiations, and even full-blown interrogations.

You can read more about the Inworld Engine on the Inworld website. The company also partnered with Ubisoft to explore different implementations of this technology. Ubisoft's NEO NPCs are similar to NVIDIA's Covert Protocol NPCs in that they use the Inworld Engine as their brain but have deeper hooks into the game. In Ubisoft's demo, you see how its NPCs are actively aware of the in-game environments, ongoing events, and even the players themselves. These AI-powered NPCs are able to respond, comment on, or even engage with these elements as needed to progress the game or deepen the player experience. Of course, as you might've guessed, that ties into the next section: the worlds.

While these AI characters still require writers behind them (and are actually more effort than simply writing lines of dialogue), AI does make it easier to choose not to employ voice or motion capture actors. That's the trade-off for these kinds of unique gaming experiences — more narrative freedom but less emotional and impactful performances. Many games won't and shouldn't use these kinds of AI characters because many stories greatly benefit from the skills of human actors, and I don't believe AI should be used to attempt to blanket replace people in this way.

Unity is one of many companies exploring how AI can make game development easier. (Image credit: Unity)

Inworld is developing technologies that allow video game writers to create dynamic narratives and characters that embrace the chaos and creativity of players without sacrificing the believability of the in-game universe, but it can go further than that. These AI brains can actually be aware of the world in which they're placed, meaning those NPCs can watch the same video as you and comment on them in real-time or talk with you to lay out a mission plan that will actually be carried out, or recognize changes in the environments, new events, or player behavior that can elicit new reactions or story progression.

You can read more on Ubisoft's website about how the company's NEO NPCs specifically focus on how AI characters can contribute to this, but it's not just limited to that. Players may interact with something in their environment that may dynamically engage AI systems, creating the sort of evolving gameplay experiences we're typically only used to when our Dungeons & Dragons DM is particularly adept at spinning narrative webs. It's certainly not perfect, especially considering how dependent it is on writers very carefully crafting the detailed outlines through which the AI works, but it's certainly more dynamic than most narrative games can ever hope to be.

If you want to experience some of these things yourself, you can actually begin experimenting with Inworld today if you sign up on the Inworld website. Of course, Inworld is hardly the only company exploring these AI concepts. While Inworld partners with companies like NVIDIA, Microsoft, and Ubisoft, another AI company known as Convai partnered with Unity to show off its own implementation of AI-powered NPCs in games. There's a lot of movement in this field, for sure.

AI at GDC: Creating helpful development tools

AI doesn't just have the potential to open up new possibilities for gaming experiences; it can also make developing existing games easier, especially for indie developers with more limited resources. Inworld, for example, has tools to make it easier for developers to prototype and conceptualize game design ideas before committing to full development, so creators can iron out plans before investing significant time or resources. Inworld also offers a variety of documentation and tools for AI game development, infrastructure (including for enterprises), and more.

Elsewhere, GDC was packed with various summits discussing the potential uses of AI in video game development. Unity demoed some of these tools within its game engine, for example, while many speakers held lectures on how developers can take advantage of AI and machine learning for generating large levels, prototyping and game design, creating more believable NPCs and AI opponents, and even enhance performance or hunt down bugs.

The goal of this kind of AI usage is not to replace any part of the creative process but to streamline it for developers, and I liked a lot of what I saw. It might be especially useful for indie developers with game optimization, NPC behavior, and even the addition of enhanced localization or accessibility features, like assisting with facial animations when porting a game to another language or building safer online spaces.

While AI-influenced narratives and NPCs can be fascinating, this is honestly where I believe AI can have the biggest positive impact on video game development. It's an extremely complicated field, and smaller teams may not have the expertise or time to master every part of the development process; AI can fill in those gaps and give talented developers more time to focus on the creative elements, like gameplay, worldbuilding, and narrative. AI should be a tool, not a replacement.

AI at GDC: Between greatness and catastrophe

One day, a CP2077 sequel may let you chat with any NPC, inquiring about Night City, recent and current events,

opinions on local politics, and even advice and directions. (Image credit: CD Projekt RED)

Ah, artificial intelligence. The ambiguous blanket term for software that evolves and learns over time. The term "AI" has reached a boiling point in recent years with the ascension of companies like OpenAI, major investments from Google with Gemini and Microsoft with Copilot, and the growing number of AI tools, large language models, and unique implementations. AI is evolving so rapidly that governments aren't able to keep up with regulations, and the potential of AI is still so untapped that the potential risks are also largely unknown.

There's a fine line between greatness and catastrophe. AI has been a focal point for countless conversations and debates between casual internet dwellers and experts alike, and the controversy isn't liable to die down any time soon. In an ideal world, AI would make our jobs easier, freeing us from the mundane and giving humans more opportunities to embrace creativity and leisure; the adverse is one in which companies take every opportunity to squeeze as much profit out of AI as possible, regardless of how it affects everyday people and inspiring creatives.

I want to be optimistic, but I must be cautious about it — whatever your economic or political stances, companies shouldn't be trusted unreservedly because profits will always matter more than people. However, AI does have a lot of potential to create new game experiences that otherwise would be impossible, and can also enhance games during the development process as a tool for developers to use (especially those who may not have the resources or experience to do certain things manually).

After GDC 2024, I am intrigued to see what companies accomplish with AI and what kind of gaming experiences we'll see come out of these endeavors in the coming years. Soon, some of the best Xbox games may even use the AI I saw at GDC. I just hope that AI doesn't become yet another way for companies to drain the passion and creativity out of our games in the pursuit of money because I know we're all getting really tired of that.

Source