Microsoft Xbox Unveils an Impractical AI Model for ‘Gameplay Conceptualization’

Yesterday, Microsoft Xbox unveiled a new tool called Muse, which they describe as a “generative AI model designed for gameplay ideation.” This came along with an article on Nature.com and a blog post that features a YouTube video. If you’re wondering what “gameplay ideation” actually means, Microsoft explains it as a way to generate “game visuals, controller actions, or both.” Despite the fancy terminology, the reality is that its practical applications are quite limited. It’s not about skipping the actual game development process.

Still, there are some fascinating details to explore. The model was trained using H100 GPUs, and it took around a million training updates to stretch just one second of real gameplay into an additional nine seconds of simulated gameplay that’s both responsive and engine-accurate. Most of the training data came from existing multiplayer gameplay sessions, which is intriguing in itself.

Instead of running on a single PC, Muse had to be trained on a network of 100 Nvidia H100 GPUs. This setup is significantly more costly and power-hungry, yet it only yields an output resolution of 300×180 pixels for roughly nine more seconds of additional gameplay.

Amidst all this, the Muse model demonstrated a particularly interesting capability: it could duplicate existing props and enemies within a game environment, allowing them to function as they originally would. However, when you think about the massive hardware costs, power usage, and extensive AI training required for this outcome, it seems almost redundant given that developers already have tools to easily create and manage game elements like enemies or props.

While Muse shows promise by maintaining object permanence and accurately replicating game behaviors, its potential applications feel somewhat extravagant compared to the tried-and-true methods already utilized in video game development.

Looking ahead, Muse might evolve to achieve more impressive results. However, as it stands, it appears to join a long list of projects aiming to replicate gameplay through AI alone. Despite some level of engine accuracy and object permanence, it’s a less-than-ideal method for developing, testing, or playing video games. After spending hours diving into the material, it’s hard to see why developers would opt for this approach over conventional methods.