Home > News

Microsoft’s Muse AI Generates Gameplay By Itself

By Crystal Crowder
Xbox controller with games.

AI seems to be everywhere, and now it can even mimic game play, at least to a point. Microsoft’s Muse AI is changing the face of game development, though it’s quick to point out that Muse will never replace game developers.

First of Its Kind WHAM

Microsoft has introduced Muse AI, which is the first World and Human Action Model, or WHAM for short. The AI models a video game, allowing the model to generate visuals, along with simulating controller actions. Essentially, it studies a game environment, predicts what will happen next, and plays within that environment.

Muse AI was created by two teams – Teachable AI Experiences and Microsoft Research Game Intelligence. It was trained using actual Xbox gameplay from Ninja Theory’s Bleeding Edge. With over one billion images and game controller actions, equating to roughly seven years of continuous gameplay, Muse has ample data to play the game.

Person playing an Xbox game.
Image source: Unsplash

While the game itself didn’t exactly go over well, it will forever live on as the start of a new AI model.

To further develop this WHAM, Microsoft’s going open source and making sample data, weights, and even the WHAM Demonstrator executable, available for other researchers to explore. For those interested in the detailed specs, Nature published an in-depth study showing the potential and shortcomings of Muse AI.

Muse works by learning the basics of a game world, then begins playing like an actual human. The AI model generates visuals based on predictions of what it believes would logically happen next within the game.

Originally, Microsoft only had Muse AI generating a second or two of gameplay at a time. It’s since successfully managed two minutes. It’s still fairly limited, though growth potential is there. For instance, it only generates 300 x 180 resolution, which isn’t exactly impressive if you’re a gamer.

New Opportunities for Creatives

This may have seemed at first like a replacement for game developers, but Microsoft has designed Muse AI and the WHAM Demonstrator to aid them. For instance, creatives could upload images of specific areas or scenarios and use the AI model to generate what could happen next.

They can even use it to simulate gameplay to better understand what players could do. This allows them to better tweak games before release for improved gameplay. The WHAM Demonstrator lets you make adjustments as you go, such as choosing a different controller action or adding more images to better guide the AI.

The true potential of Muse has yet to be seen. For now, it’s still in its early stages but could prove to be an invaluable tool during the initial stages of a game’s development.

Another idea is to use Muse to aid in restoring older games and bringing them to newer consoles and devices, much like using game emulators. It would speed up the process, giving new generations access to the arcade and classic games many of us grew up on.

Arcade with classic games.
Image source: Unsplash

Could the Microsoft Muse AI create AI-generated games in the future? It’s possible. But for now, it’s still just a tool that could complement current game development. After all, it would need to advance far beyond the current time and resolution constraints to generate anything resembling a full game.

While you’re waiting for AI games on Xbox, learn how to share your games with others. Or, if you’re tired of Xbox, check out these Xbox Game Pass alternatives.

Image credit: Pexels

Subscribe to our newsletter!

Our latest tutorials delivered straight to your inbox

Crystal Crowder Avatar
Crystal Crowder
Crystal Crowder has spent over 15 years working in the tech industry, first as an IT technician and then as a writer. She works to help teach others how to get the most from their devices, systems, and apps. She stays on top of the latest trends and is always finding solutions to common tech problems.

Read next

Octopuses possess roughly 500 million neurons distributed across their body, with two-thirds located in their arms rather than their central brain, meaning each arm can taste, problem-solve, and react to stimuli independently of whatever the octopus is otherwise paying attention to.
Explore the historic Roman bridge in lush Salamanca, Spain captured beautifully in daylight.
The Roman aqueduct at Segovia, built around the first century AD without mortar, still carried water into the 1970s, its 167 granite arches held together by nothing but the precise weight distribution of stones cut to fit each other within fractions of a millimeter.
When the SS Great Eastern laid the first working transatlantic telegraph cable in 1866, a message that had taken ten days by steamship suddenly crossed the ocean in minutes, and the financial markets of London and New York were forced, within a single trading week, to invent the modern concept of synchronised global price.
From below of blue starry sky over radio telescope and trees with leaves
The Big Ear telescope was scanning at 1420.4056 megahertz on the night of 15 August 1977, the exact frequency at which hydrogen atoms vibrate across the universe, because Giuseppe Cocconi and Philip Morrison had argued years earlier that any species trying to be found would broadcast on that channel — and then, for 72 seconds, something did.
In 2016, archaeologists dated two rings of snapped stalagmites in France’s Bruniquel Cave to 176,500 years ago, evidence that Neanderthals had walked 336 metres into darkness with fire and built architecture deep underground long before modern humans reached Europe
Otto von Bismarck was 74 when Germany adopted the world’s first national old-age social insurance program in 1889, setting the pension age at 70 after years of fighting socialists with bans, laws, and a promise few workers would live long enough to use
When cosmonaut Valeri Polyakov stepped out of his Soyuz capsule in March 1995 after 437 consecutive days aboard Mir, doctors recorded him at several centimetres above his pre-flight height, and his spine had become so unaccustomed to gravity that the recovery team carried him to a chair rather than risk the compression of letting him walk.
When Bell Labs engineer Karl Jansky pointed a rotating antenna at the sky in 1932 looking for sources of transatlantic radio static, he kept picking up a faint hiss that peaked every 23 hours and 56 minutes, and he eventually realized he had become the first human to hear the center of the Milky Way.