Interactive Tic Tac Toe

Interactive Tic Tac Toe Stefan Bennie Botha Thesis presented in fulfilment of the requirements for the degree of Honours of Computer Science at the University of the Western Cape Supervisor: Mehrdad Ghaziasgar Co-supervisor: Reg Dodds April 2016

ii

Declaration I, Stefan Bennie Botha, declare that this thesis Interactive Tic Tac Toe is my own work, that it has not been submitted before for any degree or assessment at any other university, and that all the sources I have used or quoted have been indicated and acknowledged by means of complete references. Signature:........................ Date:........................ Stefan Bennie Botha. iii

iv

Abstract The game tic tac toe had an early variant which began in the first century in the Roman Empire. During that time it was called Terni Lapilli where the players only had three pieces and had to move around the empty spaces to play. The actual game of tic tac toe could be traced back to ancient Egypt. The game was also known during that time as Three Men s Morris. The first reference to Noughts and crosses was made in 1864 in a British novel called Can You Forgive Her. For many years the game was referred to as noughts and crosses but was changed in the 20th century by the United States to tic tac toe. This paper will consist of a user requirements document as well as a requirements analysis document. The pupose of this paper is to create a program by which the user can play a game of tic-tac-toe using a sheet of paper as well a pen. The program will pick up the changes on the sheet of paper and superimpose its move on the screen showing the user which move was made. Various requirements will be handled in this paper with the help of one image to explain effectively what is happening. This project is designed purely for the sake of interest sake and thus has no real need at this current point. v

vi

Key words Support Vector Machine Visual Programming Graphical User Interface Requirements Analysis Document User Requirements Document Adaptive Gaussian Thresholding Hough-Lines Histograms Erosion Dilation vii

viii

Acknowledgment I would like to thank my supervisor Mr. G Ghazi as well as my co-supervisor Mr. R Dodds for their assistance with designing my project. Their help and support was really appreciated immensely and I would not be where I am today without their help. ix

x

Contents Declaration................................. Abstract................................... Key words.................................. Acknowledgment.............................. iii v vii ix List of Figures............................... xiii Glossary................................... xv 1. User Requirements Document................... 1 1.1 Introduction............................. 1 1.2 User s view of the problem..................... 1 1.2.1 Requirements expected from program............... 2 1.2.2 Requirements not expected from the solution.......... 2 2. Requirements Analysis Document................. 5 2.1 Introduction............................. 5 2.1.1 Designer s perspective....................... 5 2.1.2 Existing solutions.......................... 6 2.1.2.1 First Solution............................ 6 2.1.2.2 Second Solution........................... 7 2.1.2.3 Third Solution........................... 7 2.1.3 Best solution............................ 8 2.1.4 Testing............................... 8 3. Design Phase............................ 11 3.1 User Interface............................ 11 4. High-Level Design......................... 17 4.1 Introduction............................. 17 4.1.1 Input................................ 18 4.1.2 Processing.............................. 18 4.1.3 Machine Learning......................... 18 4.1.4 Output............................... 18 xi

5. Low-Level Design.......................... 21 5.1 Introduction............................. 21 5.1.1 Input................................ 21 5.1.2 Processing.............................. 22 5.1.2.1 Locating the grid.......................... 23 5.1.2.2 Detecting the symbols....................... 26 5.1.2.3 Strategy of the game........................ 26 5.1.3 Machine Learning......................... 27 5.1.4 Output............................... 27 Bibliography................................ 29 xii

List of Figures 1.1 Interactive Tic Tac Toe Representation.............. 3 3.1 The message prompt........................ 12 3.2 The start grid when the game started................ 13 3.3 The high-level design of the user interacting with the user interface. 14 4.1 High Level Design Components.................. 17 4.2 High Level Design Cycle...................... 19 5.1 Low Level Design Components................... 21 5.2 Gray-Scale equivalent of the RGB grid.............. 22 5.3 Adaptive Gaussian Threshold equivalent of the gray-scale grid. 24 5.4 Histogram displaying frequency of black pixels against each column 25 5.5 Hough lines applied to the Adaptive Gaussian Threshold image. 25 5.6 Possible output of the program after the user has made a move. 28 xiii

xiv

Glossary SVM Support Vector Machine. Used to classify objects. RAD Requirements Analysis Document. URD User Requirements Document. CSV Comma separated variables. OpenCV Library used in Python for image processing. RASAC Random Sample Consensus. Custom Library designed and used in research paper. SURF Speeded Up Robust Features. Custom created method used in research paper. Hough-Lines A method used in order to find lines within a given image. Adaptive Gaussian Threshold A method used on a image to eliminate any unwanted pixels. Turns a image into a binary image where 0 is white and 1 is black. Dilation A method used to enrich a image by adding more pixels of the same kind. It takes the highest value pixel in a certain window and converts the rest into it. Erosion The opposite of Erosion. It depletes the image by removing pixel by taking the lowest pixel value as reference. Superimposed image A image is added into another image. xv

xvi

Chapter 1 User Requirements Document 1.1 Introduction The user requirements document (URD) serves as a description of the user. It explains how the user sees the project by explaining it in a non- technical form. The URD will cover topics such as: the user s view of the problem, what is expected in the software solution and what is not expected. The first topic that will be discussed will be the user s view of the problem. This section will explain what the user wants to be implemented. Topics such as the requirements expected from the project will be discussed as well as what is not required. The URD will serve as the main source of information regarding how to build/construct the project to meet the exact specifications of the user. 1.2 User s view of the problem A interactive tic-tac-toe program is needed where the player will be able to play against the computer by using a piece of paper and a web camera. The camera will be mounted facing the sheet of paper at all times in order to prevent confusing the program by introducing different angles. The player draws a grid on the paper which will serve as the game board for the current match. When the player makes their move on the paper, the program will pick up this change on the board and make its move by superimposing a image representing its move towards the user on the screen. The game will end when one of the players is considered a winner or when it is considered as a draw when all entries are filled with symbols. 1

2 1.2.1 Requirements expected from program This section will explain in bullet form a few expected requirements regarding what the program should do.the requirements regarding this project/program will be listed below: This game has to be able to run on most modern computers and also on any Windows or Linux system supporting the Python environment. The game should be user friendly. The game should be easy to set up. The external peripherals should not be expensive. It should be flexible and be able to work on any similar web cam to the one currently in use. 1.2.2 Requirements not expected from the solution This section will explain in bullet form a few expected requirements regarding what the program should not do.what is not expected from the solution will follow below: The program should not be able to detect any other game on paper that is not related to tic-tac-toe. The game should not be slow and tedious to play. The program should not be complex to use. The game should never crash at any moment. The program is not expected to track games with the camera at a angle to the paper. The program is not expected to work under poor light conditions.

3 The figure below explains visually how the game will be played. Starting from the left side the computer will display the tic-tac-toe grid on the screen. Following the arrow the web cam will proceed to pick up the grid or any changes made withing the grid. The next arrow represents the move the player made on the sheet of paper. Lastly the change is picked up by the program and its move is made by superimposing a image representing their symbol on the screen. This cycle is repeating until a winner emerges or a draw occurs. Figure 1.1: Interactive Tic Tac Toe Representation

4

Chapter 2 Requirements Analysis Document 2.1 Introduction The requirement analysis document (RAD) explains how the program will work in more detail. In this document there will be various sections and subsections explaining what this program will consist of. This document will consist of a designers perspective, the identifying of existing solutions, deciding the best current solution and various ways to test the solution. The designer s will explain the requirements needed regarding how this program will be implemented. 2.1.1 Designer s perspective This section will discuss what needs to be done from the designer s perspective. This section also consists of possible attributes which will be considered when designing the program. The program will consist of a web camera picking up a drawn tic tac toe board on a clean piece of paper. The program should detect the board on the sheet of paper and notify the player that the game has begun. The game will continue to be played until all the white space inside the board is filled by a symbol or when three identical symbols are detected in a vertical, horizontal or diagonal line. The following bullet points represented below will provide more detail concerning this project. A mounted hi-definition web camera will be used,facing the sheet of paper, to ensure that a clear picture is provided to the user. A normal desktop computer will be used in order to create this program. 5

6 This program/game will be played strictly indoors to limit any changes in light affecting the program. The program will be written in the Python programming language using the OpenCV library. The OpenCV library will handle the image processing needed by this project. The board and the symbols will be converted into grayscale to eliminate the factor of which colours are used for the symbol or board. A Support Vector Machine (SVM) will be used to train the program on how to recognize different symbols as well as picking up the grid. The information given above demands the player to play in a controlled environment to ensure that the program will not crash or exit unexpectedly. 2.1.2 Existing solutions This section will cover different possible solutions which could be implemented in order to solve the problem. In each subsection, a different solution will be described in high-level which could provide some oversight as to what needs to be done. 2.1.2.1 First Solution A possible solution was created by Maguire and Saltzman (Joe Maguire, 2014) where by they implemented this project successfully by doing the following. They started by breaking down their algorithm into three parts: Acquiring the homography,in other words the board; detecting if the player moved and updating the projected board. To acquire the board they processes the current frame using the Speeded Up Robust Features (SURF) algorithm. They wanted to take advantage of the fact that the board consisted of four lines initially, but it turned out to be picking up too much noise as the camera was moving further away from the board. The SURF algorithm worked exceptionally well since it was able to pick up key points, which are the edges of the board very well when a marker was used to draw the board. To discard false key points they used the Random

7 Sample Consensus (RASAC) feature. In order to pick up the symbols they used a OpenCV function called matchtemplate which was used after they took the nine subsections of the board, each corresponding where the player could potentially draw a symbol. To update the board they created an artificial intelligence agent called MinMax which detects any movement on the board between 5 10 frames per second. This method turned out to be robust since the camera had to mounted close to the board but it still did the job. In other words template matching was used to locate the grid effectively. 2.1.2.2 Second Solution Another solution could be to read in images instead of a real-time feed from the web camera. The pictures would then eliminate the factor of the player s hand coming into play. The program can be considered as much simpler to create and also eliminates a fair amount of problems that can occur. The picture can then be converted into grayscale which eliminates the variety of colours the player might use. After the conversion, a histogram can be used to detect any non-white pixels on the board which could potentially be the lines of the board. The use of morphological operations can also be applied to reduce the amount of noise found in the images. This solution will be programmed in Python using the OpenCV library. A artificial intelligence agent will need to be created which will play against the user. The program will also require some training to detect what each symbol as well as what the board will look like. A decision will then be made by the program whether the game has started according to what player has drawn on the page. 2.1.2.3 Third Solution This solution consists of using real-time Haar-cascades and other morphological operations. All tic tac toe boards have similar features which means by using Haar-cascades the same Haar-features will be used to detect every

8 board. By using morphological operation, the image can be cleaned up by removing any noise captured by the camera. This will, however demand more performance in order to be executed. The reason this would require more performance would be that thousands of Haar-Cascades will be applied to every real-time image. Other sections of this solution will be considered similar to the second solution mentioned above meaning that this solution will also require some training to be done as well as an artificial intelligence agent to be created. This solution will also be created in Python since it is an flexible and easy to understand programming language. 2.1.3 Best solution This section will discuss which solution is considered to be the most effective and easiest to implement. This section will also discuss some reasons as to why this solution is considered. All the above solutions are very promising to yield results, but the second solution seems like an effective solution to implement. The reason why this solution was chosen was not because it is easier to implement but it serves as an good starting point to serve a basic program towards the user. This solution, however could evolve in the future to meet all of the requirements set out by the user which allows this to serve as a prototype in the meantime. The fact that the image is converted into grayscale would also make the computations easier to calculate since the array elements for each picture is considerably less. Another reason why the second solution was chosen was because the Haar-Cascade is used for detecting faces which is not needed since this project does not detect faces but rather tic tac toe boards. 2.1.4 Testing This section will touch briefly on how one can test this program to make sure it is working effectively. The steps to testing will be listed in bullet form and will follow below. Compile the code and check if any errors occur.

9 Test if the board and symbols are recognized by the program. Test the web cam by taking example pictures and displaying them back to the screen. Test the tic tac toe algorithm. The above steps are general testing steps to ensure that the code is in working order and that it is ready to run as the program.

10

Chapter 3 Design Phase 3.1 User Interface This section focuses on the design and description of the user interface of this program. The user interface is simple using text prompts.the high-level as well as the low level design are also discussed. The high level design includes how the components fits together. The low-level design elaborates on the components mentioned in the high-level design by introducing various methods on how each component could be solved. Each section includes pictures to give a visual representation to assist in explaining each section below. The program starts by presenting a message prompt to the user with the instructions on how to start the program, whereas the separate window,which will be display later, will show the actual tic-tac-toe grid and any changes made as the game progresses. The pop-up message has an ok and a cancel button. The program responds to the space-bar or clicking on the ok button will also suffice throughout the session of each game. The space-bar will rather be used throughout the explanation, since the ok button is highlighted by default. The ok command mentioned will be used to further progress the game. Each time the player/user has made a change towards the paper the space-bar key triggers an event in the program which will let the web-camera take a updated picture towards the grid on the paper. In order to understand how this works, a step-by-step explanation follows. Assuming that the web-camera is mounted and the user is ready to play, the user may continue by running the program script on their computer. Upon opening this program a message prompt will be displayed towards the user explaining in brief how this program works, who created the program and which 11

12 keys will be used to play.the user can proceed by pressing the space-bar key to start the game, assuming the grid is drawn on the paper and positioned underneath the mounted web-camera. After the space-bar is pressed, the program executes and displays the empty tic-tac-toe grid in a separate window. At this point the program is waiting for the user to proceed by displaying a new message prompt stating that it is the user s turn to make a move by marking the grid with its move. After the user has moved the space-bar must be pressed. This allows the program to continue executing and display the user s move and its response. Moves are made in turns by the user and the program. The cycle ends when the program announces a win,lose or a draw. The following figure 3.1 shows what the pop-up box looks like to notify the user what to do to start the game. Figure 3.1: The message prompt

13 The following figure 3.2 is a representation of how the grid window will look when the game has started. Figure 3.2: The start grid when the game started.

14 The following figure 3.3 below is a high-level flow chart of the user interacting with the user interface. Following the arrows from the start node to the end describes the interactions with the system. The system reacts to live buttons and ignores other user actions. Figure 3.3: interface. The high-level design of the user interacting with the user

15

16

Chapter 4 High-Level Design 4.1 Introduction This section attempts to explain how the tic-tac-toe program has been broken down into its high-level components. This section covers each of these components in a high-level view to give a rough idea of how components fit together. The process is described in figure 3.4 The figure 4.1 below describes the connection of these high level components. These components include input, processing, machine learning and output. Each section and subsection is next explained. Figure 4.1: High Level Design Components 17

18 4.1.1 Input This module is merely the input that the program requires in order to run. The input comes from the camera and the user. The camera consists of multiple images after the user pressed the space-key to trigger the program to capture the next image for process. The user inputs are made through the keyboard but the program is concerned only with the camera images that it manipulates. 4.1.2 Processing This component has two parts (1) locating the grid and (2) locating the user s symbols within the grid. Various techniques have been tested and applied in this component to allow the program to successfully capture the grid frame and its sub-frames where the user s signs can be placed within the grid. 4.1.3 Machine Learning This component could also be considered as a classifier since a support vector machine classifies whether the given grid is a valid tic-tac-toe grid or not. The same holds for the symbols used for game-play. A support vector machine is a machine learning algorithm that can decide whether a certain input can be classified. This algorithm runs by making decisions based on the data to resolve whether this input is a tic-tac-toe grid or not. The same is done for both of the input symbols X and O. 4.1.4 Output The output module simply outputs a image of the grid. After the processing and machine learning components are finished the user is presented with an image as output. This image will show the user that the grid was located successfully and that changes made to the grid are valid. After the image is shown and the game is not finished, then the cycle will start again. The output also contains the pop-up message box informing the player what to do next.

19 The figure 4.2 presents the high level components as a cycle. Each output image will act as the new input to be processed during further game-play. Figure 4.2: High Level Design Cycle

20

Chapter 5 Low-Level Design 5.1 Introduction This section explains on the topics discussed in the high-level design in more depth. Each component is described into more technical detail. The figure 3.6 below displays all the components that are required in the lowlevel design. Figure 5.1: Low Level Design Components 5.1.1 Input This section makes use of images as input. A image is a binary representation of pixels which are used to display a combination of colours which transforms into a picture to the user. Millions of pixels form a picture which is recognized by the user. The image will be captured from the web-cam which is digitized 21

22 and manipulated by the tic-tac-toe program. In fact this image manipulation technique is used to create the idea of the user playing against an artificial intelligence unit. Each pixel exists of three variables, red,blue and green. The variable consists of a number which represents the intensity of the light, which in turn mixes with the other two colours to form a new colour to be presented to the user. As the intensities of each colour in each pixel changes such will the colour of the final pixel change. This program converts the image into its black and white equivalent known as gray-scale. This will simplify the manipulation process since each pixel can now be represented as one variable instead of using three. The gray-scale image is used as the program enters its next phase/component. The figure 3.7 below is similar to the colour picture provided but converted into its gray-scale equivalent. My finger is thus also in gray-scale as proof. Figure 5.2: Gray-Scale equivalent of the RGB grid 5.1.2 Processing This section attempts to further explain into more detail regarding the location of the grid as well as the symbols in a more low-level orientation.

23 5.1.2.1 Locating the grid From the previous component a image was given as input which is manipulated further during this stage. The image obtained is converted to gray-scale meaning there are still quite the amount of noise when it comes to locating the grid drawn on the sheet of paper. Noise in this context means unwanted pixels which makes the finding of the grid more difficult. In order to reduce the noise we can attempt to smooth out the image with a process known as Adaptive Gaussian Thresholding. One can simplify the image by turning it into an binary equivalent where 0 s will represent the white sheet of paper and the 1 s the lines on the board. Thus Adaptive Gaussian Thresholding can be seen as the process by which these types of black and white images are generated from grey images (Abreu, 2013). In other words replace all gray pixels with either black or white depending on the threshold. The threshold will be the separator to help decide whether the current pixel should be changed to a white or black pixel. This method will achieve more success compared to using morphological operators like erosion and dilation. Dilation is made by applying the max operator to all pixels in the neighborhood, while in erosion the min operator is applied. (Abreu, 2013) This means that a window will pass over a subset of the complete image and when a pixel of high value is found, all other pixels will be changed into that pixel shade. The effect of applying erosion and dilation will either enrich the image by adding more pixels or deplete it by eliminating pixels. Thus the image edges will be thicker since the edges consist of more pixels than before. This method will help in locating the board but it will not remove the noise found in the original image which makes this a useful method to apply but is still unnecessary since it will not simplify the given image.

24 The following figure 3.8 is an example of how the gray-scale image would look like when Adaptive Gaussian Threshold is applied. Figure 5.3: Adaptive Gaussian Threshold equivalent of the gray-scale grid After the smoothing of the image is completed we can start finding the lines that make up the grid. In order to find the grid-lines, a method known as Hough-lines can be used. When the program located something of interest, in this case the grid lines, it is considered as a contour. A contour feature f j can be defined as set of real values f j = f 0j, f ij,..., f nj (one for each contour pixel p i ) that represents features that can be extracted by a feature function F j : p i f ij, where j identifies the feature, i defines the pixel p i of the object contour and f ij R is the value of feature f j for the pixel p i. (Pedronette and da S. Torres, 2010) What this means is that we will have two values, a θ and a pixel value in question. A vector will be drawn from the origin towards that pixel with θ changing the whole time. θ is measured in radians. The radians will keep increasing until a vector from the origin is found that goes through that pixel. This vector is known as a contour feature which can be used to described where the vector is found by looking at the radians and the origin from that specific pixel. Another way to locate the grid is by using a histogram. Since the image consists of only a 1 value for white and a 0 value for black we can locate these black pixels on the image and project the frequency of black pixels on each

25 column onto a histogram. The figure 3.9 displays the frequency of the black pixels in the y-axis and the column number in the x-axis. Figure 5.4: column Histogram displaying frequency of black pixels against each The following figure 3.10 is the result after applying Hough lines onto the Adaptive Gaussian Threshold image. The threshold needed to be adjusted in order to detect all lines located on the image. Figure 5.5: Hough lines applied to the Adaptive Gaussian Threshold image After the grid is located we could potentially crop out the board to make it easier to work with at the next stage of processing.

26 5.1.2.2 Detecting the symbols After the grid lines are found,we can assume that the lines were detected successfully, we move onto the squares in which the user draws the symbol to play the game. The contour lines found in the previous section has to intersect at some point which will create a corner of an approximate right angle. Since a square is known for having four lines, we need to detect four lines which connect to one another. In other words four contour lines are needed which intersect with each other at some point which makes up the characteristics of a square. By using the Hough lines method located in programs such as Python for example we can extract possible squares which are found within the image. Sorting through these potential squares we are able to extract possible candidates for which could include possible positions to draw a symbol on the grid. 5.1.2.3 Strategy of the game Tic-Tac-Toe is straight-forward easy to understand game which can be played by children and does not require any insight to understand at all. Making the first move in this game does give the player the upper-hand since it can interfere with a strategy the opponent player wanted to execute. The most common first move players tend to make is to draw their symbol in the middle of the grid. This forces the opponent to make a move around the edges which increases the difficulty since the opponents needs to adapt to playing around the edges to win. The program will represent the grid as a 3X3 matrix which will eliminate any complexity since it avoids working with the image. As the changes are made on the sheet of paper the 3X3 grid will be updated and vice-versa, meaning the move made upon the grid will be reflected with a superimposed image on the sheet of paper. Thus the actual game can be reduced to a mere 3X3 matrix manipulation object to represent the actual game being played by the player on the sheet of paper.

27 5.1.3 Machine Learning In order for the program to recognize what is drawn on the sheet of paper actually is a tic-tac-toe grid, a mathematical models known as support vector machines will be used. Support vector machines construct decision functions that are linear combinations of kernel evaluations on the training set. (Steinwart, 2003). In other words by drawing what is called a regression line one is able to apply weights to different dimensions of a problem. This can help create decisions mathematically based on the input given, in this case the squares holding the symbols and the grid we want to examine. The support vector machine will train on given data, which includes various different forms of tic-tac-toe grids and symbols as well as non symbols in order to help the program to learn by examining features. As output the program will inform the user whether the drawing on the sheet of paper is in fact a tic-tac-toe grid or not. This component will act as a classifier to ensure that the image being modified actually is an grid along with the correct symbols. 5.1.4 Output The output for this program will be similar to the input component. The only difference being that the image given as input will be modified and displayed back towards the user with various changes made on the image. Firstly after the other components are completed, the user will be able to see the grid-lines along with the squares/boxes that the user can make a move inside. The image will also contain a superimposed image of a symbol which will represent the programs move it made on the grid. Secondly after the user made their move on the sheet the entire cycle will repeat and continue until the game ends. During these transitions a pop-up message will be displayed to guide them the user what to do next.

28 The following figure 3.11 shows a superimposed image representing the move made by the program. This output image will then be reused in the cycle and act as input again for the next move to be made by the player. Figure 5.6: Possible output of the program after the user has made a move

Bibliography Abreu, A. (2013). Introduction to opencv. Crowle, K. and Siegler, R. S. (1993). Flexible strategy use in young childrens tic-tac-toe. Joe Maguire, D. S. (2014). Augmented reality tic-tac-toe. Technical report, Department of Electrical Engineering. Pedronette, D. C. G. and da S. Torres, R. (2010). Shape retrieval using contour features and distance optimization. Steinwart, I. (2003). Sparseness of support vector machines. 29