A Hybrid Model of Painting: Pictorial Representation of Visuospatial Attention through an Eye Tracking Research S.A. Al-Maqtari, R.O. Basaree, and R. Legino Abstract A hybrid pictorial representation of visuospatial attention in performance art becomes a crucial issue in order to explore a new drawing technique development. This drawing technique development is straight related to the eye tracking research and biometric process that involved visual data analyses of eye movements, public behavior, and space- based visual attention features, which is the first study in a visual art field. Multidisciplinary approach will use in this study through combining between human eye movements analyses toward Paulikevith s dancing and the traditional techniques of drawing. In post-modernism, visible and invisible theory, the challenge is to represent quality content of invisible concepts such as attention, sound, motion, and time based on a clear interpretation of specific properties of the form. In this paper, we propose the hybrid model of Visuospatial Attention form (VAF) based on the outcomes of combining among spotlight attention model and pictorial representation through Aspect- Recognition theory. We will conduct the experiment using Tobii T60 remote eye tracking hardware and Tobii Studio software set to collect and analyze the participants eye movements when watching performance movie, Tajwal. The expected outcomes will be 2D and 3D installation artworks that represent the form s qualities of visuospatial attention in visual art which could be as an alternative concept of attention. Keywords Eye Tracking Research, Hybrid Painting, Pictorial Representation, Visual Attention. I. INTRODUCTION n the recent years, there has been an increasing interest in Ithe pictorial representation and depiction through the recognition- based hybrids theories as a crucial subject in the contemporary visual perception debates and depiction studies [1]-[7]. Cognition theories attempt to ground a hybrid theory of depiction that represents pictorial reference by mental perspective of properties from which the picture represents its objects. Lopes s theory of depiction the aspect- recognition theory is a well-known hybrid theory that embraces pictorial reference based on a denotative symbol systems, [5]-[7]. Today, visual art researchers focused on the visual representation of invisible study in order to get deep understanding of various complex visual perception issues in installation art, such as the attempts of exploring a space experience through representing the shape of invisible objects such as sonic in terms of technical practice, [8], or the attempts of representing of public experience in installation practices based on the theoretical and critical investigating, [9]-[11]. Indeed, attention is one of an emotional invisible concept that commonly has been utilized in visual art studies, however using a multidisciplinary approach in order to provide creative research with novel ideas are highly recommended in contemporary art research methodologies [12]-[13]. Particularly, the experimental studies based eye tracking research that capture and analyze the visuospatial attention in art is too rare, it can only found in Neuroscience, Psychology, Psycholinguistics, Ophthalmology, Usability, Humancomputer interaction, and Package Design and marketing researches for biomedical and industrial design purposes, [14]- [17]. A. The Problem Statement One major drawback of the recent visual representation studies, is that fails to take public attention into account which is the main property in visual representation theory. Although the recent using of multidisciplinary approaches for visual representation, some studies focused only on the standpoint of the art studio while exploring invisible concepts in outdoor spaces, fails to define the sound form properties like the study by, did not use specific models of attention in public visual behavior studies and used subjective inducting to archive their purposes. In addition, attention based on eye tracking research in art is too rare, it can only found in Neuroscience, Psychology, Psycholinguistics, Ophthalmology, Usability, Human-computer interaction, and Package Design and marketing researches for scientific reasoning and industrial design purposes. B. Aim This study aims to shape the form of visuospatial attention toward Paulikevitch s performance artwork see fig1, based on the study s model (VAF) using the combining of eye movements analyses with the traditional drawing techniques. C. Objectives We will achieve our aim through the following objectives: a) To identify the attention s features and the hybrid pictorial representation s properties in order to develop (VAF) model based on reviewing Attention and Pictorial theories. ISBN: 978-1-61804-218-7 97
b) To record the fixations using an eye tracker Tobii T60 in order to visualize attention through using the analyses methods of Tobii Studio software. c) To shape (VAF) using installation artworks based on the combining between the finding of a, b, and traditional techniques of drawing. D. Research Question We address the following questions: a) What are the components of (VAF) model based on the combining between Spotlight attention, and the aspect- recognition theories? b) How can detect public s attention through the fixations data in dynamic environment, and visualize its features using eye movement s analyses methods? c) How can shape the experience of public attention in performance art based on the outcomes of (VAF) model and traditional drawing? a) Literature review/ research: to identify (VAF) model based on attention spotlight theory and aspectrecognition theory of pictorial representation through Literature review. b) Visualization software / research: to record and visualize the gaze data based on two methods, Statistic and Visualization. a. Statistic: this method is based on area of interest (AOI) tool. The researcher will use the following metrics to provide gaze data, fixation count, fixation length, time of the first duration, fixation Duration, and visit Duration. b. Visualization: It depends on four tools based on Tobii studio analyses which are: bee swam, gaze plot, heat map, and cluster. c) Pictorial Representation/ research: to shape (VAF) based on the model s requirements as follows: a. Attention features: which are spotlight (focus, fringe, and margin), motion, Intensity, color, location, cluster, respond time (RT) and orientation. b. Pictorial Properties: which are pictorial references, information based identification, constraints on thought, pictorial aspects, pictorial systems, pictorial recognition, pictorial meaning, and pictorial experience. B. Respondents We plan to follow the procedures of shuffling sampling technique. We will collect Fixation (eye movement s data) from eight students, three female and five male aged from 23 to 32 years old in University of Malaya. Respondents will be asked to focus their gaze when watching movie which has totaling approx. (3.13 min). Clip namely Tajwal will present in three scenes: war, resistance and the gender. Fig.1 Paulikevitch s performance artwork, Tajwal II. METHOD A. Design & Methods The purpose of this research is to shape (VAF) toward Tajwal project based on eye movements analyses, pictorial theory through traditional drawing. Create a test design will be the design of this study based on one experimental group. Three scenes of Tajwal project will separate to create three tests: test 1: Gender, test 2: War, and test 3: Resistance. So, the project will be undertaken in three areas: C. Experiment Setup The visual attention through human eye movement will be recorded using an infrared video- based remote eye tracker (Tobii T60) system and Tobii studio software suite to capture and analyze the spectators experience when watching Tajwal movie, fig 2. Respondents will be asked to watch Tajwal clip and focus their gaze in three test trails totaling approx. (3.13 min). The viewing distance between the respondents and the center of the track box will set to (65 cm) from the eye tracker. Stimuli (Tajwal clip: gender, war, and resistance) will collect from YouTube and store at (60 Hz) on the native TFT screen (1280 1024 pixels). First, the researcher will take the gender, age, background information for each one, and then they will be asked to move their gaze on the calibration points. Second, they will be asked to focus their gaze on the three scenes when watching movie. ISBN: 978-1-61804-218-7 98
D. Visualization Process It depends on the research design through the following techniques, fig. 3 : a. Orientation vs. (VAF) Shape: Swam tool is used to display the coded dot of a fixation for all the participants at the same time in different colors. It will use to identify the shape element of (VAF) based on the visual view of dotes distribution, and the total number of participants at N > 0.5 in order to decide of using geometric or organic shapes. color theory wheel by Robert Plutchik. Also, it will use to identify the value tones of (VAF) based on the time stamp, duration, and the principles of spotlight model of attention: Focus, fringe, and margin. e. ter vs. (VAF) form and space: ter tool is used to display the graphic representation with a high attention of fixation clusters on the background. This tool will use to identify the motion, form, and space of (VAF). f. Intensity vs. (VAF) Texture: area of interest (AOIs) tool provides the tables and graphs of attention data after selecting the regions of interest. The researcher will use AOIs to estimate the custom data of attention based on selecting the body area and will define the texture through Intensity based on visit duration results. III. STATEMENT OF SIGNIFICANCE This study could be as an alternative source of an innovation and creation that provide a new technique of painting. It represents objects such motion, time, and attention from invisible meaning into a self-visible experience through using hybrid methods by scientific and artistic approaches. Therefore, this study will carry benefits to the following: a. Visual artists and Designers, students and scholars b. Neuroscience and vision scholar c. critics and philosophers d. psychologists Fig. 2 Tobii hardware, Software and experiment setup b. Selected location vs. (VAF) line: Plot tool is used to display with dot the single view, order of fixation by numbers, and the length of the fixation for each image by each participant. It will use to identify the line element of (VAF) through the feature of location based on plot s direction, and the feature of weight based on plot s order numbers. c. Time& motion vs. (VAF) motion form: also, gaze plot will use through the (RT) numbers sequence and the segment technique to select 6 frames (one each ten second) of each scene in order to identify motion. d. Spotlight vs. (VAF) Color and Value: map tool is used to visualize the behavior of gaze. It consists of transparent background with the highlighted areas of attention. map will use to identify color type of (VAF) for the three senesces based on the area of interest method (AOI) by fixation amount, fixation length through combining the result of gaze with the IV. CONCEPTUAL FRAMEWORK The conceptual framework of this study is the hybrid (VAF) model which based on the combination between two theories, spotlight attention theory by William James, and aspectrecognition theory of depiction by Lopes.Spotlight attention theory will use to identify the attention features, which are spotlight (focus, fringe, and margin), motion, Intensity, color, location, cluster, responds time (RT) and orientation. And aspect- recognition theory of depiction will use to identify and describe the properties of depiction in pictorial representation. The pictorial properties of that will be used based on the theory are pictorial references, information based identification, constraints on thought, pictorial aspects, pictorial systems, pictorial recognition, pictorial meaning, and pictorial experience. The hybrid (VAF) model components illustrated in Fig 4. V. CONCLUSION The expected outcome of this proposed study is 2D and 3D installation artwork which planned to hold it in Art Expo Malaysia fair. It will transfer a new concept of self- visual attention for public experience toward performance art. The procedures of this study will orient visual artists, painters, critics to borrow unfamiliar tools to explore the invisible objects in this world in deeply examination. ISBN: 978-1-61804-218-7 99
Literature Review/ Research: To Identify hybrid (VAF) model Spotlight Attention Theory + Aspect- Recognition Theory Visualization Software/ Research: To record visual data Tajwal Project Participa Statistics Test 1 Test 2 Test 3 AOI Record 1 Record 2 Record 3 Visualization Pictorial Representation/ Research: To investigate Pictorial Properties Hybrid (VAF) Model Fig.3 Research Design and Methods Attention Features Spotlight (focus, fringe, and margin), motion, Intensity, color, location, cluster, respond time ACKNOWLEDGMENT First and foremost I would like to thank Dr. Hamid Abdulla Jalab, Department of Computer System and Technology. I could not have imagined having a better advisor and mentor for this proposal. The combination of friendly comments and constructive criticism was crucial for being able to submit this paper. Lastly, a very special thanks to my family, my son Amgad and my husband Khaled, to whom I dedicate this thesis. Fig.4. Conceptual Framework REFERENCES [1] B. Katerina. "The Opticality of Pictorial Representation." Journal of Aesthetics & Art Criticism 66, no. 2 (2008): 183-192, published. [2] D. J. Berg, S. E. Boehnke, R. A. Marino, D. P. Munoz, and L. P. Munoz. "Free Viewing of Daynamic Stimuli by Humans and Monkeys." Vision Research 9, no. 5:19 (2009): 1-15, published. [3] C. C. Ondine, "Orphans of Modernism: Chicano Art, Public Representation, and Spatial Practice in Southern California." Ph.D., The University of Rochester, 2002. [4] C. Hilary, Creative Research: The Theory and Practice of Research for the Creative Industries. Switzerland: AVA Publishing SA, 2010. [5] D. John, "A Representational Theory of Artefacts and Artworks." British Journal of Aesthetics 41, no. 4 (2001): 353, published. [6] D. S. Howard, "The Image Incarnate: On the Documentary Representation of Painting in Film." Ph.D., The University of Iowa, 2000. [7] S. L. Feagin, "Presentation and Representation." Journal of Aesthetics & Art Criticism, Summer98 1998, 234, published. [8] I. Sherri, "Artworks and Representational Properties " Dialogue 43, no. 2004 (2004): 627-644. [9] L. Itti, and P. Baldi. "Bayesian Surprise Attracts Human Attention." Vision Research 49, no. 10 (2009): 1295-1306, published. [10] Chi Kei Solly. LEUNG, "Explaining Depiction: Recent Debates in the Philosophy of Pictorial Representation " Lingnan University 2009. [11] L. A. Linnell, "Exploring the Experience of a Place through the Visual Representation of Aural Presence." RMIT University, Melbourne 2010, published. [12] J. Malins and C. Gray Appropriate Research Methodologies for Artists, Designers & Craftspersons: Research as a Learning Process. Available: http://carolegray.net/papers%20pdfs/cc.pdf [13] M. Michelle, "The Pedagogy of Space: Visual Representation of Place in Contemporary Art." M.F.A., University of Southern California, 2003. [14] R. Mark, "Pictorial Representation: When Cognitive Science Meets Aesthetics." Philosophical Psychology 12, no. 4 (1999): 387-413, published. [15] T. Massimo, and G. Galfano, "Color, Form and Luminance Capture Attention in Visual Search." Vision Research 40, no. 13 (2000): 1639-1643, published. [16] V. Johan, "Reconsidering Pictorial Representation by Reconsidering Visual Experience." Leonardo 41, no. 5 (2008): 493-497. ISBN: 978-1-61804-218-7 100
[17] W. Richard, "On Pictorial Representation." Journal of Aesthetics & Art Criticism, Summer98 1998, 217, published. [18] N. H. M. Zain, F. H. Abdul Razak, A. Jaafar, and M. F. Zulkipli, "Eye Tracking in Educational Games Environment: Evaluating User Interface Design through Eye Tracking Patterns." In Proceedings of the Second international conference on Visual informatics: sustaining research and innovations - Volume Part II, 64-73. Selangor, Malaysia: Springer- Verlag, 2011, published. ISBN: 978-1-61804-218-7 101