MotionPro. Team 2. Delphine Mweze, Elizabeth Cole, Jinbang Fu, May Oo. Advisor: Professor Bardin. Midway Design Review

Transcription

1 MotionPro Team 2 Delphine Mweze, Elizabeth Cole, Jinbang Fu, May Oo Advisor: Professor Bardin Midway Design Review 1

2 Project Review A projected game that can be played on any flat surface A step towards more interactive gaming Our project: the machine detects motion of a finger to control the game w_tracks_childrens_toys/022_lumo_play_interactive_floor_children_room.jpg 2

3 Address PDR Concerns Met with Professor Holcomb in October Clarified our system design and revised deliverables Switched process from edge detection to background subtraction Project significance Demonstrating real-time system Societal impact Dance game, YMCA, gym class Hologram video gaming Educational archaeology game at museum Alternatives IR filter Touch screen 3

4 High Level Block Diagram Raspberry Pi Camera (Background Subtraction) (Hand Motion) Raspbian OS Speaker Image Processing Coordinate Detection Class Interfaces + Ultrasonic Sensors (Hand Location) Main() Game Program Projector Power Supply 4

5 Reasons For Ultrasonic And Camera Ultrasonic Sensors Camera/Image Processing Pro: Can give an exact distance Pro: Fast enough to detect (in cm) user while playing game Pro: Detects obstacles; not affected by game projection Pro: More suited for motion detection Con: Too slow to detect user while playing the game Con: Not good at detecting still hand Con: Susceptible to noise Con: Can t give exact distance 5

6 Promised MDR Deliverables May Functioning background subtraction Elizabeth Detection of hand position Delphine Fu Programmed snake game Interface simulating actual game inputs and outputs 6

7 Background Subtraction: May MacBook Pro Camera MDR Deliverable: Functioning background subtraction using laptop Purpose: User s finger motion needs to be detected by pi camera in order to determine the direction of the fingers as a next step Status: Completely functioning Pi Camera 7

8 Background Subtraction: May Study Solution Average human reaction time: 0.25 seconds to a visual stimulus* Theoretically, estimated fps can be as low as 4 fps Fastest reaction time: seconds Use OpenCV and C++ to run background subtraction Use MacBook Pro camera to test the result Goal At least 10 fps ~ 0.1 seconds each frame CMake Use CMake to build a cross platform OpenCV Add opencv libraries CodeBlocks (IDE) Xcode (IDE) on Raspberry Pi * 8

9 Experiment Using Different Resolution: May 9

10 Background Subtraction: May Background Frame Video Detect Motion Frames Threshold Display Foreground Frame Choosing Background Subtraction parameter: Threshold variable: is chosen based on the standard deviation of gaussian distribution High threshold variable 10

11 Hand Coordinate Detection: Elizabeth MDR Deliverable: Detection of position of a hand using ultrasonic sensors Purpose: User needs to press menu buttons to start the game and to change modes Status: Completely functioning Demonstration at end 11

12 Visual Diagram 12

13 Timing Diagram 10 μs 40 khz 200μS <5.682 ms Total: <5.703 ms 13

14 Multiple Receivers Set a common Trigger signal Multiple Echo pins Sensor with smallest recorded distance indicates the sensor the obstacle is closest to 14

15 Snake Game: Fu Function of the Game 15

16 Snake Game: Fu Process of the Game 16

17 Snake Game: Fu Make it better Add more pictures Fruit Snake Improve the algorithm Decide which wall the snake may hit before the death of the snake Adjust the resolution of the game to make it portable 17

18 Game Interface: Delphine Goal: To build a well functioning interface to facilitate subsystem communication. Desired Specs: Negligble delay < 1 second Fast Processing Effective multithreading and synchronization Game inputs written in C++ and python. Current cross communication done using xml-rpc 18

19 Block Diagram Apache XML-RPC Library - HTTP used as Transport Layer rary/cc738291(v=ws.10).aspx 19

20 Block Diagram Snake Game Prototype (Java Client) Coordinates (Python Server) Apache XML-RPC Library EventSimulator Thread (robot) Main Thread (run snake) Camera Data (C++ Server) 20

21 Alternative Class Implementation Google Protocol Buffers: Enables user defined message types between languages Sophisticated algorithm in less code to reduce processing latency. Use of a communication model that allows the server to automatically send inputs when done with processing 21

22 CDR Deliverables May Background subtraction on Raspberry Pi at least 15 fps Translate the detected motion into right, left, up, down directions Fu Power supply design Circuit design Elizabeth Integrate ultrasonic data with rest of system Printed Circuit Board (PCB) layout Delphine Containers for synchronized data Integrated System 22

23 Schedule 12/14 1/18 2/1 2/15 3/1 3/15 3/29 4/12 4/26 May: Background subtraction on Raspberry Pi Elizabeth: Integrate ultrasonic data with Delphine s interface Fu: Power Supply Design Delphine: Determine direction from May s data May: Housing Elizabeth: PCB Layout Fu: Circuit Design Delphine: Containers for synchronized data All: Complete integrated system 23

24 Demos Deliverable #1 (May) Background subtraction using laptop Deliverable #2 (Elizabeth) Detecting hand position using multiple ultrasonic sensors Deliverable #3 (Fu) Showing the Snake game programmed from scratch Deliverable #4 (Delphine) Demonstrating game inputs/outputs 24

25 GameOver 25