Not your regular VR
Zombie Apocalypse Game !
Tired of playing games built for the Virtual Reality that require you to keep moving your head to perform almost every functionality, imagine how amazing things would be if you could actually throw stuff at Zombies, look no further. Introducing to you a different type of interactive Zombie Apocalypse VR game designed for the Android Platform that allows you to bring your superpower equipped hands into play, from now you can stop surviving and start pulverizing.
This game was created is the end result of an academic project at the University of Central Florida (Orlando) which was an integral part of the coursework (CAP6121 – 3D User Interfaces for Games and Virtual Reality) along with my project partner, Tanuj Palghamol. The main ingredients for this project were Unity 5, OpenCV for Android Unity Package, Google Cardboard VR SDK, Google Cardboard (or any head mounted device designed for VR) and last but not the least your Android smartphone (with Android Lollipop 5.0 and higher devices with OpenGLES 3.0 support and multicore rendering).
The Idea
The key objective was to eliminate the use of hand-held devices dedicated to implement game functionalities and push our focus towards using our hands instead. Since majority of the devices utilize the click button (Cardboard VR), touch pad (Gear VR), hand held controllers (Oculus Rift) or other wired/wireless input devices, we were able to create VR gaming experience using Computer Vision based Hand Detection with help of the Android phone’s in-built camera.
How it works?
With the help of OpenCV for Unity package, the extraction of the hand region in real time is performed on the input received from the phone’s main camera that undergoes the following processes: -
-
Color Space Conversion - The current frames from camera are converted from RGB to either HSV or YCbCr.​
-
Skin Thresholding - Based on the selected color space, minimum and maximum skin value thresholds can be set manually in order to extract desired skin color regions in the current environment.
-
Hand Location and Tagging - Based on the size of the region (hand) and its location in the detected frame, the hand is tagged as either left or right hand.
The Game Design
The game starts with the first player (i.e. You) is dropped in the middle of a Zombie infested swamp and in order to survive you need to destroy them with either a fire ball, ice ball or even the combination of the two which are activated by the presence of either right hand, left hand or even both hands in front of the phone’s main camera.
Majority of the assets used in the game were primitives provided by Unity itself (Zombie model which can be found in the Unity Asset Store).
The Gameplay
As mentioned earlier, the presence of hands carries out certain actions within the game. The functionalities are listed below: -
-
Fire Ball - Presence of the Right Hand creates a Fire ball and it is released when the hand is brought down, which results in the Zombie being burnt.
-
Ice Ball - Presence of the Left Hand creates an Ice Ball and it is released when the hand is brought down, freezing the Zombie at its spot after being trapped in between icicles.
-
Energy Blast - Presence of Both Hands creates an Energy Blast that throws Zombies away from the player, this comes in handy when you sense that things are getting a little out of hand and you need some breathing space.
​
In-Game Features
On launching the game, you will enter the video processing configuration scene. This scene is dedicated to ensure that the game runs smoothly by setting up the right thresholds for hand detection given the current surroundings.
Screenshot of the Configuration Screen
The Configuration Screen can be divided into 6 sections: -
-
Video output window - The Android camera feed is displayed depending on the options (color mode) set by the user.
-
Image Processing and Resolution - This contains the set of parameters that can be adjusted for each frame being processed.
-
Color Mode - It contains 3 modes: RGB2HSV_FULL, BGR2HSV_FULL, and RGB2YCBCR which converts the camera feed to the set color spaces either HSV or YCbCr.
-
Display Tex - This allows the user to view which processing stage is displayed in the video output window. The stages are: RGB, conversion to either the HSV or YCbCr color space, Thresholding (depending on the position of the sliders) and presence of Hands (red for right hand and green for left hand).
-
Camera Res - This enables us to set the video capture resolution from Full HD to QCIF(160x120).
-
Hist. Eq. - It contains histogram equalization options mainly None, Histogram Equalization and Adaptive Histogram Equalization.
-
Clip and Size - Sliders set the 'clip' and 'size' values for the Adaptive Histogram Equalization.
-
Area Thresh - This slider sets the minimum area for the extracted contours to be treated as hand regions.
-
-
Rendering Quality Options - Allows the user to set graphics quality for the game. The settings are the in-built graphics quality settings offered by Unity 5. The Expensive Processes toggle allows the use of Anti-aliasing in the game. Useful in case, the game rendering is slow on the mobile device.
-
Load/Save & Start Game Buttons - The Load/Save buttons save all the menu options and slider values, so that each scene can load the settings even after restarting the application and the Start button loads the game scene.
-
VR Mode Toggle and Show Processing in game - The 'Show Processing in game' toggle allows the user to see the processed output in game so that the user can view his/her hands during game-play. The VR mode toggle enables or disables the VR Mode in game.
-
Thresholding Sliders - The sliders on the left i.e. HL, SL, VL represent the lower threshold bound and the ones on the right, HH, SH, VH represent the higher threshold bound for the skin detection. The output can be seen by selecting the Threshold option from the Display Tex menu.
View of Configuration Window once both hands are detected
In-Game Screenshots
Fire Ball in Action (Right Hand detected here)
Ice Ball in Action (Left Hand detected here)
Deploying the Energy Blast (Both hands detected here)