Team1's Project for ECE180DA + DB

Chester Hulse, Erica Xie, Shalin Shah, Wesley Sakutori

We are implementing a virtual version of Hole in the Wall with added twists, including gesture and voice recognitiion for powerups, a virtual wall, and a multiplayer mode to play with friends.

Dependencies

See requirements.txt

Installation for Users

Make sure to have anaconda or miniconda installed before running the setup script. Clone our repo. Download and change key.py to your passwords + Raspberry Pi login information. Put in the main directory.

If you're on a PC:

./setup_win.sh
conda activate hitw
pip install pipwin 
pipwin install pyaudio

If you're on a MAC:

chmod +x setup.sh 
./setup_mac.sh
conda activate hitw
pip install pyaudio

On the Raspberry Pi (optional):

cd GestureRecognition 
chmod +x setup.sh
./setup.sh

Running the Game

On your laptop:

conda activate hitw
python game.py

User Manual

Read our User Manual for more details on how to play the game and some tips for troubleshooting!

Source Code Descriptions:

File	Description
GestureRecognition	Contains gesture recognition code, including IMU libraries and gesture data. More Details inside the folder
graphics	Includes images necesary for gameplay and user interface. - Sources: In-class OpenCV labs - Decisions: Use OpenCV for graphics for now, might migrate to pygame in the future - Bugs: None - TODO: Transition to pygame
pose	Folder containing modules used by our OpenCV DNN model for pose recognition. More Details inside the folder
ContourDetection.py	Contour detection code to check if someone is within the "hole". - Sources: OpenCV, OpenPose output points - Decisions: All joint points are equally weighted for this decision - Bugs: None - TODO: None
game.py	Main code running the overall game. - Sources: All sub-module files and in-class labs - Decisions: Singleplayer and Multiplayer implemented, try and run everything in the background and organize events into callbacks. Send MQTT packets and runs gesture recognition on the RPI. - Bugs: A player leaving a game sometimes freezes the program. - TODO: Move to pygame for music and a better UI.
PoseEstimation.py	Code using the OpenPose library to recognize a player's position on the screen. - Sources: OpenPose models and datasets - Decisions: prioritize pose detection speed with model choice - Bugs: None - TODO: GPU (NVIDIA CUDA) support, improve speed
README.md	This file with usage and documentation.
requirements.txt	Dependency information.
setup.py	Script to set up the game.
voice.py	Voice recognition code to activate powerups. - Sources: In-class labs - Decisions: Use microphone built into laptop. Use google voice with no limit on phrase length. - Bugs: Slow and sometimes hangs the program when closing. - TODO: Using Google Voice for now but may upgrade to more consistent recognizer in the future.