The technology behind ChatGPT is capable of doing much more than just “talking”. Linxi “Jim” Fang, an AI researcher at Nvidia, and colleagues have come up with a way to use the powerful GPT-4 language model – the “brain” of ChatGPT and a growing number of other apps and services – in Minecraft.
The Nvidia team, which also included Anima Anandkumar, the company’s director of machine learning and a professor at Caltech, created a Minecraft bot called Voyager that uses GPT-4 to solve problems in the game. Here’s how it works:
- Task Generation: The GPT-4 language model generates tasks that help the agent explore the game.
- Bot Skill Improvement: The generated code helps the bot to improve his skills in the game.
- Direct game state reading: Voyager does not play the game as a human, but he can read the game state directly through the API.
- Determining Goals: With GPT-4, the bot can suggest a goal based on the state of the game.
- Code Generation: GPT-4 generates the code needed to complete a character’s goal.
The process is iterative, and over time Voyager builds a library of code to build more complex things and explore new features of the game. Compared to other AI agents, Voyager gains more than three times as many items, explores twice as far, and crafts tools 15 times faster.
The use of language models in projects like Voyager can lead to the automation of routine computer tasks. This approach has huge potential for economic impact and automation opportunities. Microsoft is also training AI programs to play Minecraft and is implementing machine learning and APIs in Windows 11 Copilot to automate tasks, notes NIX Solutions.
Using GPT-4 in Minecraft opens up new horizons for language models. The future of large language models promises opportunities for exploration and automation in areas ranging from games to operating systems.