Background

All Mines is a video-calling AR game created for Meta's "Spark Playtime" hackathon (in which it placed 4th among hundreds of entries). The game uses some of Instagram's popular AR features like face tracking. However it also uses networking features to allow players to interact with each other through video calling.

Game Design

Multiplayer AR gaming is largely unexplored territory (especially when it's done via video call!). So, this provided a great opportunity to experiment and see what unique challenges and opportunities there are for game design in this new medium. This gave birth to "All Mines": a 3-player game, in which each player controls a miner character within a shared play-space. In this game, players race to be the first to mine 10 rocks. Rocks spawn throughout the play area at semi-random times and locations. However, players can only collect a rock if one of its colours matches the player's assigned colour. Each rock only has two of the three possible colours. So this means that players will be forced into one-on-one confrontation with each other multiple times throughout the game. This makes it difficult for players to gang up on each other unless one player is dominating the others. To further encourage player interaction, players can hit each other with one of two random debuffs:

1. temporarily freeze another player's movement
2. temporarily reverse another player's movement controls

Red player sends debuff to blue player

When players debuff each other, there is a cooldown period before they can debuff the same player again. This encourages players to use their debuffs strategically.

Unlike most traditional online games, playing a game via video call allows you to see your fellow players. This means that you can hear AND see everyone's reactions to what's happening, potentially creating fun social moments. The debuff mechanic is built to exploit this by making sure that when a player gets debuffed, they know who did it to them!

Implementation

The implementation has multiple stages ranging from concept prototyping to network programming.

Prototyping

Building and testing multiplayer experiences can be a time consuming process. This is because of the need to have people on hand to meaningfully test functionality. This is especially the case with video-calling AR because our user controls are linked to a user's head movements and facial expressions. Therefore, to quickly validate and test our concept, we build a prototype in the Unity game engine where we can test the game mechanics with AI players.

Single-player non-AR prototype built with the Unity game engine

Mechanical Implementation

After validating the game concept, we bring in our custom 3D assets and implement the following game mechanics:

• Use the player's head rotation as a joystick to move their character
• Rock spawning and score tracking
• Let players send a debuff by closing one of their eyes for a few seconds

Controlling character movement using head rotation

Networking Overview

Usually when users open the same AR experience on an Instagram video call: they're opening their own "copy" of the experience. By default: these copies behave independently and can't influence each others' behaviour. However, our game only works if these copies can interact with each other. So, we add networking functionality so that all the copies can send messages to each other behind the scenes. These messages share information about the game state and ensure that the same things are happening in every copy of the game.

Without these messages:

• Players would only see their own miner character moving on their screen while other characters stand still
• Players wouldn't be able to debuff each other
• We wouldn't be able to ensure that rocks spawn in the same place, at the same time, for all players
• There would be no game...

UI/UX Considerations

Figuring out good UI/UX for this game was particularly challenging because in a new medium like this, there aren't any best practices. Players also only have a small portion of the screen to themselves in a video-call. This means that our UI needs to be able to communicate clearly without taking up too much space. All Mines is also a chaotic game, so we have to think of the best way to communicate information, even when there is a lot happening on-screen.

We try to put UI elements in places where it makes the most sense for players to see them. In the end, we put UI elements in the following places:

• Player's head - we place a coloured hat here. This makes it clear which colour is assigned to the player
• Player's face - contains UI with info about the player's debuff ability. These are placed here because debuffs are triggered by a player's eyes. The UI communicates when the debuff ability is ready to be used and when it is on cooldown. The colours of the UI also indicate which player an eye will target
• Player's character - the character's hat matches the colour of its player. When a player has been debuffed, an icon appears above their character indicating which debuff was applied and which player sent it.
• Top of player's screen - this is where a player's score is displayed. It's positioned here to ensure that the score is always highly visible.