The role of non-visual aesthetics in consumer product evaluation

Transcription

1 Vol. 84, 2015, The role of non-visual aesthetics in consumer product evaluation Andreas Sonderegger a* and Juergen Sauer a a Department of Psychology, University of Fribourg, 1700 Fribourg, Switzerland The present article addresses the impact of non-visual aesthetics of consumer products on the outcomes of usability tests such as perceived usability, user performance and user affect. Building on the research surrounding the impact of visual aesthetics in usability testing, the present work aimed to determine whether the same pattern of effects repeatedly found for visual aesthetics will also be found for non-visual aesthetics. A series of three experiments was carried out, all examining sound and touch as two prominent dimensions of nonvisual aesthetics. To increase the robustness of the results, the experiments were conducted with three different consumer products, which were a smartphone (N = 60), vacuum cleaner (N = 60), and video racing game (N = 60). Although manipulation checks confirmed that users generally experienced both sound and tactile product properties as intended by the experimental manipulation, in none of the studies the results showed an effect on perceived usability. Other outcome variables such as affect and performance showed a few selected effects. Overall, this suggests that findings from the field of visual aesthetics cannot be easily transferred to the domain of non-visual aesthetics. The findings of the present studies are discussed within the framework of Schifferstein s work on sensory dominance. Keywords: haptic aesthetics; auditory aesthetics; consumer product; usability test. Highlights The effects typically observed for visual aesthetics were not found for non-visual aesthetics. Results indicated that haptic and auditory aesthetics did not affect perceived usability. There was no strong effect of non-visual aesthetics on performance and affect. 1 Introduction The work of Kurosu and Kashimura (1995) and of Tractinsky et al. (2000) has established an important new research strand addressing the relationship between visual aesthetics and usability. Their work showed for the first time that the aesthetic properties of a product had an influence on how its usability was perceived by users. This finding has had important implications for the use of product evaluation methods like usability testing since it demonstrated that the outcomes of product evaluation (e.g. perceived product usability) can be influenced by aesthetic properties although they are by definition unrelated to aesthetic product properties. While the link between visual aesthetics and perceived usability has received increasing attention in the research community in the past decade, the link between visual aesthetics and measures relevant in human-computer interaction (e.g. perceived and objective performance, emotions) has received less interest. Including measures of performance and emotion when examining the influence of aesthetics in human-computer * Corresponding author. Tel.: Rue de Faucigny 2, CH-1700 Fribourg. E- mail address: andreas.sonderegger@unifr.ch.

2 interaction is essential for a better understanding of the user experience, especially against the background of recent changes in human-computer interaction. A mere functional view of the user-system interaction has been extended to a more holistic approach, addressing the whole user experience with a stronger focus on user emotions such as fun, pleasure and frustration (Norman, 2004). Virtually all empirical work in that research strand has focused on visual aesthetics while other senses (e.g., taste, smell, hearing and touch) have been largely neglected. Against this background, theoretical and empirical work has addressed the relative importance of the different sensory systems when product features are evaluated (Schifferstein, 2006). Across a range of consumer products, Schifferstein s work found the following overall rank order of senses with regard to their importance: vision, touch, smell, hearing, and taste. However, there were considerable differences between products in the rank order, showing that for about half of the products one of the non-visual senses was considered to be more important for evaluation than the visual sense (e.g. for a vacuum cleaner it was audition and for a computer mouse it was touch). The relative importance of different sensory modalities in product evaluation might hint at a potential effect of non-visual aesthetics on outcome measures of usability tests, similar to the effect of visual aesthetics. Since previous research on visual aesthetics has shown that validity and reliability of usability evaluation methods are influenced by visual aesthetics (see Sonderegger et al. 2012), a similar effect of non-visual aesthetics would imply for usability practitioners that they need to control for non-visual aesthetics in usability tests. Furthermore, very little is known so far about the influence of non-visual aesthetics on the outcomes of usability tests, such as perceived usability, user performance, emotion, and perceived workload. Therefore, in the present article three experiments are reported in which the influence of non-visual aesthetics in usability testing is evaluated. They all adopted a broad methodological approach by measuring a large range of variables relevant in usability testing. The results may then be compared to the findings in the research literature on the effects of visual aesthetics in usability tests. 1.1 Visual aesthetics in product evaluation The influence of the aesthetic appeal of a product on perceived usability has been well researched, producing a rather consistent pattern of findings, with more attractive products also being considered to be more usable (Tractinsky, 2013). The positive effect of aesthetic appeal has been found across a range of national cultures, including Japan (e.g., Kurosu & Kashimura, 1995), Israel (e.g., Tractinsky et al. 2000), Switzerland (e.g., Tuch et al. 2010), Germany (e.g., Thüring & Mahlke, 2007), and United Kingdom (e.g., De Angeli et al. 2006). Furthermore, the positive effect of aesthetic appeal has been found across a range of products, including mobile phones (e.g., Quinn & Tran, 2010), cash machines (e.g., Tractinsky et al. 2000), web pages (e.g., De Angeli et al. 2006; van Schaik & Ling, 2008, 2011) and digital audio players (e.g. Thüring & Mahlke, 2007). As a theoretical explanation for the positive relationship between aesthetics and perceived usability, the halo effect has often been cited (e.g. Tractinsky et al. 2000; Hartmann et al. 2008). It refers to a rater's tendency to overestimate the association between different traits or behaviours of a person that are actually independent (e.g. the judgement of a researcher s quality of teaching could be influenced by the same person s research record). One example for a halo effect is the what 2

3 is beautiful is good -stereotype (Dion et al. 1972), describing the phenomenon that certain salient features of a person (e.g., physical attractiveness) have an influence on the perception of less salient features (e.g. intelligence, personality). The empirical findings mentioned above suggest that the halo effect is not limited to characteristics of humans but also seems to apply to product properties. Recent empirical work also suggests a moderating influence of the general evaluation of the product ( goodness ) on the link between visual aesthetics and perceived usability (Hassenzahl & Monk, 2010; van Schaik et al. 2012). While goodness is an interesting concept, it appears not to be well defined, relying only on a single-item measure. In addition, the work addressing the influence of goodness is based on correlation studies only so that the cause-effect relationships between the different concepts could not yet be established. Other work put forward moderating variables such as affect and emotions (Thüring & Mahlke, 2007; Porat & Tractinsky, 2012), arguing that affective states need to be considered when addressing the influence of aesthetics in usability tests. In their components of user experience model (CUE), Thüring and Mahlke (2007) suggest that the interplay of instrumental (e.g. effectiveness) and non-instrumental experiences (e.g. visual and haptic quality) and the emotional reactions to those experiences influence the product evaluation. In contrast to the robust relationship between aesthetics and perceived usability, the effects of aesthetics on performance have been characterised by some inconsistency. Most often, it was assumed that aesthetics had a positive influence on performance as predicted by the affect mediation model (e.g., Norman, 2004; Moshagen et al. 2009). It suggests that positive aesthetic experience is associated with positive affect. This might influence motivational and behavioural processes (Ashby et al, 1999; Isen, 2001) and is expected to increase user performance. Similarly, a positive effect on performance for aesthetically pleasing products was predicted by the increased motivation -hypothesis (Sonderegger & Sauer, 2010). While such a positive link between aesthetics and performance was demonstrated by several studies (e.g., Moshagen et al. 2009; Sonderegger & Sauer, 2010; Quinn & Tran, 2010), other studies reported no effect of aesthetics on performance (e.g., Chawda et al. 2005, Hartmann et al. 2007, Thüring & Mahlke, 2007). A third cluster of studies showed exactly the opposite pattern, that is, when using an aesthetically pleasing product, user performance decreased (e.g., Ben-Bassat et al. 2006; Sauer & Sonderegger, 2009; Sauer & Sonderegger, 2011). An attempt to explain the negative association between aesthetics and performance in some studies is the prolongation of joyful experience -hypothesis (Sonderegger & Sauer, 2010). It suggests that when users have been distracted by the beauty of the consumer product, they have concentrated less on the task, resulting in decreased user performance. The inconsistent pattern of effects suggests the existence of moderating factors that may have contributed to the conflicting findings. For example, the performance focus of a usage situation may play a moderating role (Rheinberg et al. 2007; Sonderegger et al. 2014). In a domestic work context (e.g. vacuum cleaner usage) with a high need for efficient task performance, an appealing product may increase the user s motivation for efficient task performance. Conversely, when using an appealing interactive product in a leisure context (e.g. video game usage), it may lead users to wish to prolong this pleasant experience (Sonderegger et al. 2014). In addition to perceived usability and performance, research has examined further outcome variables that might be influenced by aesthetic appeal, such as the affective state of the user. While the goal of usability testing initially was to evaluate the functional qualities of a product, experiential aspects such as fun and pleasure have become increasingly important in product evaluation (Desmet & Hekkert, 2007). The aesthetic experience is expected to have an impact on the affective states of the user of an artefact (Leder et al. 2004). In HCI research, aesthetic appeal was generally positively related to users' affective states (Thüring & Mahlke, 3

4 2007; Sonderegger et al. 2012; Cai & Xu, 2011; Porat & Tractinsky, 2012). Affect may also play an important role in the modulation of motivational and behavioural processes and thus influences user behaviour in usability tests (Moshagen et al. 2009; Sonderegger & Sauer, 2010). A further measure to be considered in usability evaluation is the subjective evaluation of mental workload (Jordan, 1998). Mental workload has also been considered as an objective indicator for usability (e.g. van Schaik & Ling, 2009). However, previous research has shown that measures of mental workload are often considerably correlated with subjective measures of usability (e.g. meta-analysis of Hornbæk & Law, 2007). Little is known about its association with visual aesthetics, though one study indicated that visual aesthetics did not show an influence on mental workload (van Schaik & Ling, 2009). For the purpose of the present work, the most relevant finding from the research literature is the highly consistent relationship between visual aesthetics and perceived usability, which was even unaffected by the cultural background of the user and the particularities of specific consumer products. This established relationship is of particular relevance because it raises the important question of whether such a finding could also be observed for the non-visual sensory systems. Among the non-visual sensory systems, we will focus on the auditory and haptic sense. This is because for many interactive products, hearing and touching are of particular importance for user-product interaction (Mahlke et al. 2007). 1.2 Auditory aesthetics The perception of sound is a complex process (Baldwin, 2012). Perception is largely influenced by the two main physical characteristics of sound, which are amplitude (perceived as loudness) and frequency (perceived as pitch). A further factor of influence is the timbre of the sound. The attractiveness of different sounds may vary as a function of these factors. For example, high-pitched voices are perceived as less pleasant to listen to than voices of low or medium pitch (Collins & Missing, 2003). In addition, pitch and loudness may also interact, which results in effects of a complex nature (e.g. higher tones are perceived as higher with increasing amplitude; Baldwin, 2012). Sound plays an important role in several research fields of ergonomics. For example, the importance of sound aesthetics for the comfort and emotions of product users is indicated by research in the car industry. This industry has been involved in sound engineering by testing the auditory pleasantness of different design options for car parts, such as the sound of exhaust pipes, doors, and switches (e.g., Knöferle, 2011). Although this kind of industrial research is usually not published and might not always meet high methodological standards, it is clearly indicative of the importance of sound design for the evaluation of consumer products. The important role of sound was not limited to large consumer products like cars but was also found for smaller artefacts like mobile phones. For example, in a study examining the importance of auditory aesthetics for mobile phones, it also emerged that sound plays an important role for user experience and eventually for the overall evaluation of the product (Mahlke et al. 2007). In their correlational study, acoustic qualities of the mobile phone predicted more of the total variance of the overall product evaluation than visual qualities. In addition to the domains work and designing interactive technology, research on the attractiveness of a speaker s voice showed that auditory stimuli can have a considerable 4

5 influence on human behaviour and perception. For example, people with attractive voices were rated more favourably with regard to their personality traits compared to speakers with unattractive voices (Zuckerman & Driver, 1989). Similarly, male participants preferred pictures of women with high-pitched voices to pictures of women speaking with lowerpitched voices (Feinberg et al. 2005). Both studies showed considerable similarity to the work that originally established a link between physical attractiveness of humans and how their non-physical attributes are rated (i.e. what is beautiful is good -stereotype; Dion et al. 1972). This suggests that these effects are to some extent generalisable across sensory systems. Overall, the review of the research literature showed that sound stimuli have multiple effects on user behaviour (e.g. performance), emotions, and attitudes towards the artefact. While the literature review demonstrated the important role auditory aesthetics can play for human perception and behaviour, only little is known about the influence of auditory aesthetics in usability testing and the domain of consumer ergonomics. 1.3 Haptic aesthetics The haptic experience represents a very complex process, with different active movements being carried out to explore and evaluate objects (Lederman & Klatzky, 1987; Sonneveld & Schifferstein, 2008; Jakesch & Carbon, 2012). Such movements include exerting pressure (to estimate hard- and softness of material), maintaining static contact (to assess temperature), completing lateral movements (to identify texture), maintaining a holding position (to assess weight), and grasping (to gauge shape and size). While the haptic experience is largely an active process (i.e. involving active touching), it also includes the so-called passive touch, that is, a person s haptic experience is brought about by an external object touching the person (Gibson, 1966). Several terms have been used to refer to issues surrounding aesthetics and the haptic experience. We will adopt the term haptic aesthetics suggested by Carbon and Jakesch (2013) to describe the objective qualities of the product (e.g. smoothness, round shapes) while we use the term haptic attractiveness when referring to the user s subjective reaction to these product features (e.g. the degree to which a person believes that the product is aesthetically pleasing when touched). The literature suggests that there is a considerable number of relevant object characteristics that constitutes the haptic user experience, which is reflected in a range of conceptions and models. The most elaborate model distinguishes between six haptic-related object characteristics: hardness (elasticity and plasticity) of material, temperature, surface characteristics, shape and size of object, and weight and balance (Sonneveld & Schifferstein, 2008). A small number of studies looked at the aesthetic evaluation of haptic product dimensions. In an evaluation of the surface of objects (cardboard, sandpaper, etc.), it emerged that the preference ratings of participants showed a positive relationship with the smoothness of the object (Ekman et al. 1965). Similar effects of smoothness were observed in another study, where participants rated the touch of a feather as highly pleasant whereas touching sandpaper was rated as highly unpleasant, with ratings of tree bark lying in between (Peck & Wiggins, 2006). In addition, smoother surfaces and simpler shapes were found to encourage users to touch those (Klatzky & Peck, 2012). The influence of such haptic qualities also extends to specific characteristics of a product in that attitudes towards the product were more positive for a soft sweater that was pleasant to touch compared to a rough sweater which was unpleasant to touch (Peck, 1999). Furthermore, haptic aesthetics proved to have a 5

6 considerable influence on the overall judgment of mobile phones (Mahlke et al. 2007). In a recent experiment, different haptic product qualities such as weight, texture, and hardness were manipulated and found to have a considerable influence on ratings that were completely unrelated to the original stimulus (Ackerman et al. 2010). For example, participants completing a jigsaw puzzle whose pieces were covered with rough sandpaper rated a story they were listening to at the same time to be more difficult to understand than participants who completed a smooth puzzle. In addition to the influence of haptic qualities on user ratings of preference and attractiveness, haptic qualities were also found to have an influence on perceived usability and emotion. Carbon and Jackesch (2013) argued that haptic aesthetics may have an impact on outcome measures of user-product interaction, such as quality ratings or pleasure in usage. This was also shown in a study where haptic characteristics of two different products (pillow case and flashlight) were experimentally manipulated (appealing vs. unappealing characteristics), with the subsequent ratings of overall product quality being significantly higher for products with more appealing haptic characteristics (Grohmann et al. 2007). Work also suggested that sensory systems differed in the degree to which they are ruled by affect. For example, tactile perceptions emerged to be more strongly influenced by affect than visual perceptions (Sonneveld & Schifferstein, 2008). The review of the literature on haptic aesthetics shows some similarity to the one on auditory aesthetics. The literature in both fields revealed that a small number of studies had addressed the question of what features of a product were associated with a pleasant or unpleasant experience. However, only little is known about how haptic and auditory experiences influence perceived usability and affect, and no research has addressed effects on user performance. 1.4 The present studies The main research question in the present studies was concerned with the extent to which the halo effect of visual aesthetics on outcome measures of usability tests (e.g. perceived usability, user affect, perceived workload) reported in previous studies can also be observed for non-visual aesthetics. Being able to determine the degree of influence of non-visual aesthetics in usability testing and product evaluation is important for product designers and manufacturers in order to predict customer responses, in particular, by avoiding possible overestimations of product usability due to highly attractive non-visual product features. The present article presents a series of three experiments, which all examined the influence of auditory and haptic product characteristics as two prominent dimensions of non-visual aesthetics on a broad range of outcome measures. These multiple outcome measures cover different facets of user behaviour and user experience during the operation of consumer products. Due to the nature of the results of the first experiment, we conducted a second one and eventually a third one to challenge the findings of the first. We employed a different interactive product in each study to increase the robustness of the findings. The selection of the three products also allowed us to cover a range of usage domains to examine influencing factors that may influence the impact of aesthetics on performance (e.g. mobile phone: dual domain; vacuum cleaner: domestic work domain; video racing game: leisure domain). Naturally, the use of different products required the adaptation of the experimental manipulations to meet the particularities of each specific product. Similarly, the measures employed also had to be adapted to meet the demands of each product. 6

7 In order to achieve a high level of comparability of results across the three experiments, all studies used a similar methodological approach. The three studies employed a 2 x 2 betweensubjects design, with the two independent variables, auditory aesthetics (high vs. low appeal) and haptic aesthetics (high vs. low appeal). The common methodological standard also included a broad measurement of effects, with perceived usability, performance, affect, and perceived attractiveness being measured as dependent variables. All studies were carried out in a laboratory at the University of Fribourg. The products used in the empirical work were smartphones, vacuum cleaners, and video games. They are suitable products for the purpose our studies for two reasons. First, they share the feature that both the sensory systems of audition and touch enjoy a similar degree of importance when using each of the three products. According to Schifferstein s (2006) data, the vacuum cleaner and the mobile phone had slightly higher ratings for audition than for touch whereas the video racing game showed the reverse pattern. However, audition and touch always emerged in the first three positions when the importance of the five sensory systems was rated for each of the three products. Second, they represent a broad range of products from the domestic domain since they are quite different from each other in a number of features. For example, the three products differ in the degree to which they contribute to relaxation or domestic chores: the video game is strongly associated with relaxation, the vacuum cleaner with domestic chores whereas the smartphone is positioned between the two. Furthermore, the smartphone is different from the two others because it entails a higher prestige value. Using products with such marked differences has the advantage that converging research findings can be generalised with more confidence. It was generally assumed that the same effects found for visual aesthetics would also emerge for the two factors of non-visual aesthetics. More precisely, we hypothesised that products with high aesthetic appeal would be given higher usability ratings and would induce more positive affective states than products with low aesthetic appeal (halo effect of non-visual aesthetics on outcomes of usability tests). Given the considerable (negative) correlation of perceived mental workload with measures of perceived usability, a similar (inverse) effect of non-visual aesthetics on mental workload was expected for perceived usability. This would apply to auditory and haptic aesthetics, with either expected to show a main effect for these dependent variables but no interaction. For performance, we expected a positive influence of aesthetic appeal on performance for products operated in a domestic work context (e.g. vacuum cleaner) and a negative influence products operated in a leisure context (e.g. video game) while for the dual-domain product (mobile phone), no effect of aesthetic appeal on performance was expected. These hypotheses were put forward in all experiments. 2 Study I 2.1 Goals of study The first study aimed to examine the degree of similarity of the effects found for visual aesthetics to non-visual aesthetics. This was done by examining a mobile phone as a dualdomain product which is very commonly employed in the research domain of visual aesthetics. This allowed us to make direct comparisons with the pattern of effects previously found for visual aesthetics. 7

8 2.2 Method Participants The sample of this experiment consisted of 60 students from the University of Fribourg (70% female), aged between 19 and 36 years (M = 23.1, SD = 3.6). Participants were mostly German native speakers (88%), but for some of them, German was their second language (12%). With regard to their own expertise in operating smartphones with a touchpad, they gave a rating of M = 9.5 (SD = 4.8) on a visual analogue scale ranging from 1 to 20. Participants were not paid but received course credits for their participation Experimental design In a 2 x 2 between-subjects design, auditory and haptic aesthetic appeal of a smartphone was experimentally manipulated (N = 15 per cell). Haptic aesthetic appeal was manipulated by applying two different versions of the casing of a smartphone. This was done by using a standard plastic casing (soft and smooth touch, i.e. pleasant haptic experience; HAP + ) and a casing that has been coated with emery (sharp, grained and uncomfortable touch, i.e. unpleasant haptic experience; HAP - ). For the manipulation of the auditory aesthetic appeal of the smartphone, a sound was issued at each user action (e.g. click). This sound was either displeasing (shrill high-frequency beep; AUD - ) or pleasing (mid-frequency gong sound; AUD + ) and was presented for approximately one second. Throughout the experiment, each sound was presented on average 94 times (SD = 16, min = 57, max = 150) over a period of 700 ms at a level of 65 db Measures and instruments Perceived haptic and auditory attractiveness. Haptic qualities of the smartphone were measured by means of a one-item scale. The item focused on the experiential quality of the haptic design ("The smartphone feels pleasant when holding it") and used a 7-point Likert scale as a response format (strongly agree, agree, partly agree, neither agree nor disagree, partly disagree, disagree, strongly disagree). In addition, a one-item scale was used to measure auditory attractiveness ("I perceived the sound of the audio feedback as being pleasant"). Again, a 7-point Likert scale was used as a response format. Perceived usability. To measure perceived usability, the Post Study System Usability Questionnaire (PSSUQ; Lewis, 1995) was used. The PSSUQ is a well-established instrument to measure perceived usability, using a 7-point Likert scale as response format, ranging from strongly agree to strongly disagree. The scale comprises three subscales, 'system usefulness', 'information quality' and 'interface quality'. The psychometric properties of the PSSUQ are very good, with a Cronbach s alpha of.97 for the scale. In the present study, the scale had a Cronbach s alpha of.91. User affect. In order to be able to detect even subtle changes in affect, two scales were used in this study, which were rather different in their underlying theoretical assumptions. The first instrument was the positive and negative affect schedule (PANAS), a verbal scale measuring positive and negative affect as two distinct dimensions (Watson et al. 1988). It consists of 20 adjectives describing positive and negative emotional states (e.g. active, interested, sad, angry) that are to be rated by test participants. As response format, a 5-point Likert scale was used, ranging from "not at all" to "extremely". The scale enjoys good psychometric properties (PA Cronbach s α =.85; NA Cronbach s α =.86; Krohne et al. 1996). In the present study, 8

9 the scale had a Cronbach s alpha of.81 for PA and.84 for NA. The second instrument was the self-assessment manikin (SAM; Bradley and Lang, 1994). Based on the circumplex model of affect (Russell, 1980), this non-verbal instrument assesses three distinct dimensions of affect (valence, arousal and dominance) by means of graphic affect representations in the form of manikins. For this study, only the dimensions valence and arousal were measured. For valence, the five depictions of the SAM scale range from a smiling manikin to a frowning one. For arousal, the five depictions range from a calm manikin with closed eyes to a wideawake and highly aroused manikin. Previous work (Bradley & Lang, 1994) showed that the scores obtained with the non-verbal SAM scales were highly correlated with verbal scales of pleasure (r =.94) and arousal (r =.94). User performance. User behaviour was logged during the whole experiment. Based on the logged data, different measures of user performance were computed. Since the way performance could be measured differed between tasks, performance on each task was analysed separately. For tasks I and II, task completion time (s) and number of user actions were recorded. For task III, the number of user clicks was recorded, distinguishing between on-target clicks and off- target ones. Subjective workload. The concept was measured after completion of each task by means of the well-established NASA Taskload Index (TLX), originally developed by Hart and Staveland (1988). This instrument measures mental, physical and temporal demands as well as performance, effort and frustration related to the task. In the present study, the scale had a Cronbach s alpha of Materials In the present study, the iphone TM 3GS was chosen as a smartphone for testing. In order to avoid that the test participant was influenced by the visual aesthetic appeal of the smartphone, a special experimental set-up was required (see also procedure). The smartphone was connected to a laptop computer (Dell TM XPS M1710) via a WLAN router (CISCO TM 857W- G-E-K9). On the laptop computer, two applications were running: one application handled user inputs while the other presented the tasks and recorded user behaviour. The video output of the computer was projected on a large screen (about 2.0 m x 3.0 m) by using a LCD projector (Eiki TM LC-XB26). This elaborate experimental set-up was required because it had not been feasible to identify tasks on the smartphone that could be completed without having access to a visual interface Pilot study In a pilot study, different options of manipulating auditory and haptic aesthetic appeal of the smartphone were pretested. Ten blindfolded participants evaluated four different smartphone covers (plastic, rubber, sand paper, emery) and six different sounds (XP Ding, XP Notification, XP add hardware, gunshot, nail on steel, high frequency beep) with regard to their haptic and auditory quality on a 10-point Likert scale (ranging from 'very displeasing' to 'very pleasing'). The order of presentation of the different covers and sounds was varied to control for order effects. For the experiment, the most and least pleasing designs were chosen for each aesthetic dimension. For haptic aesthetic appeal, this was the plastic cover (M = 6.8, SD = 1.99) and the emery cover (M = 2.2, SD = 2.49). For auditory aesthetic appeal, this was the XP Ding (M = 6.0, SD = 2.45) and the high frequency beep (M = 1.1, SD = 0.33). 9

10 2.2.6 Tasks Test participants were asked to complete a warm-up task and then carry out three experimental tasks. The warm-up task was to solve a simple jigsaw puzzle of nine pieces. The aim of this task was to allow test participants to familiarize themselves with the smartphone. The first task was a verbal reasoning task which consisted of building a meaningful and grammatically correct sentence with a set of 11 words (e.g., we were walking through the fields and were swimming in the lake). The second task, a spatial reasoning task, consisted of the completion of a jigsaw puzzle (12 pieces). The third task was a probe detection task in which participants were asked to click on a black square which appeared somewhere on a screen (random positioning by the system). When the participant clicked on the square, it disappeared and appeared somewhere else on the screen again. The aim was to click on as many squares as possible during the duration of the task (120 s) Procedure The study was conducted in a usability laboratory at the University of Fribourg. Participants were randomly assigned to one of the four experimental conditions. The experimenter welcomed the participants and explained to them that the purpose of the study was to determine the usability of a newly developed system that was to be operated via a smartphone. The system consisted of software that was running from a personal computer and was projected on an overhead screen. Participants were told that they should use the touch screen of the smartphone as input device to interact with the system. Moving the finger on the touchscreen made the pointer move on the big screen. Tapping the touchscreen allowed for a selection of an object on the screen. The usability test took about 90 min to complete. Participants were asked to carry out several tasks using the smartphone with a view to providing feedback about the usability of the system afterwards. It was emphasised that the idea was to evaluate the system and not the participant. Furthermore, participants were told that the design of the prototype was not yet complete but was still undergoing some further developments. Throughout the whole experiment, participants were unable to see the smartphone because it was hidden behind a cover. Participants were instructed to hold the smartphone in one hand during task completion but not to remove it from behind the screen. This was to prevent an influence of visual aesthetic appeal on user judgements. To begin with, participants filled in a demographic questionnaire (e.g., age, gender, mother tongue) and rated their experience of using smartphones. Then they completed the baseline measures of their current affective state (SAM and PANAS). After that, they completed the warm-up task followed by the three experimental tasks. After completing each task, participants rated their subjective workload using the NASA-TLX. After task completion, participants rated their current affective state (SAM and PANAS) for a second time and responded to the questionnaires on haptic and auditory attractiveness and on perceived usability (PSSUQ) Manipulation check, power analysis and data analysis In order to facilitate the comparison of results between the three studies, all variables were transformed to z-scores before conducting the analysis. The pilot study had already indicated considerable differences in the perception of the manipulations of auditory and haptic aesthetic appeal of the device. This was confirmed in the main experiment. For the measure of haptic attractiveness, the aesthetically pleasing haptic manipulation received higher user ratings than the aesthetically displeasing one (M HAP+ = 0.31, SD = 0.99; M HAP- = -0.31, SD = 0.93; F = 6.5; df = 1, 53; p <.05; η 2 partial =.11; power 1 β =.71). The same pattern was found for auditory attractiveness ratings and the manipulation of auditory aesthetic appeal (M AUD+ = 0.24, SD = 1.08; M AUD- = -0.24; SD = 0.86; F = 5.9; df 10

11 = 1, 53; p <.05; η 2 partial =.10; power 1 β =.67). We also tested for an interaction of the two independent variables on haptic and auditory attractiveness. No such interaction was observed (both Fs < 1). A two-factorial analysis of covariance (ANCOVA) was carried out to analyse the data. Age, touchpad-experience, and mother tongue were entered as covariates. For affect, a baseline measure (taken prior to task completion) was used as an additional covariate. Since these covariates were not related to the research questions, their influence is not discussed in the present article and the outcomes of the analysis are not presented in the results section. Nevertheless it is important that these covariates are entered into the analysis to statistically control for the effects of such variables even if they are not of primary interest. 2.3 Results Perceived usability (PSSUQ). As the descriptive data in table 1 show, there is little difference between experimental conditions. This was confirmed by ANCOVA, which revealed no main effects for haptic and auditory aesthetic appeal on perceived usability (both F < 1). There was no significant interaction between the two factors (F = 2.04; df = 1, 53; p >.05; η 2 partial =.037). A separate analysis of the three subscales of the questionnaire did not reveal any significant effects either. Performance. The analysis of participant performance on the three experimental task revealed no difference between experimental conditions, with one exception out of the six measures (see table 1). In task 1 participants needed significantly more clicks to complete the task for the haptically pleasing device (M = 16.7; SD = 9.7) than for the displeasing one (M = 12.8; SD = 4.26; F = 4.14; df = 1, 53; p =.047; d = 0.53). This main effect was paralleled by a significant interaction between auditory and haptic aesthetic appeal (F = 4.48; df = 1, 53; p =.039; η 2 partial =.078). Bonferroni-corrected pairwise comparisons however revealed no significant differences between the four experimental conditions. The analysis of the other five performance indicators shown in table 1 did not reveal any significant effects (most F <1). Affect: Table 1 presents the affective states that were recorded as a result of participants operating the device. Baseline measures were entered as covariates in the data analysis. All statistical tests carried out on the PANAS data (i.e. positive and negative affect) revealed no significant effects but for the influence of the baseline measure for PA (F = 32.7; df = 1,51; p <.001; η 2 partial =.39) and NA (F = 62.88; df = 1,51; p <.001; η 2 partial =.55). The analysis of the non-verbal SAM-scales showed the same pattern of effects, with none of the tested effects showing a significant result (all F < 1). Baseline measures showed also a significant influence on measures of the SAM scale (F = 32.7; df = 1,51; p <.001; η 2 partial =.39). Subjective workload. With regard to measures of subjective workload, no major differences were found for the different conditions of haptic and auditory aesthetic appeal (see table 1). This was confirmed by the non-significance of the two main effects (both F < 1) and the interaction being not significant either (F = 2.01; df = 1,53; p >.05; η 2 partial =.036). 11

12 Table 1: Measures of perceived usability, performance, affect and subjective workload as a function of auditory and haptic aesthetic appeal; (M = mean; SD = standard deviation; HAP = haptic aesthetic appeal; AUD = auditory aesthetic appeal). Measures AUD + HAP + M (SD) AUD + HAP - M (SD) AUD - HAP + M (SD) AUD - HAP - M (SD) Perceived usability 4.94 (1.01) 4.88 (1.06) 4.60 (1.06) 5.03 (0.80) Performance Task 1: completion time (s) 205 (100) 183 (47) 180 (54) 170 (71) Task 1: number of clicks* 19.7 (13.0) 11.7 (4.0) 13.8 (2.8) 13.9 (4.4) Task 2: completion time (s) 268 (144) 280 (137) 303 (161) 207 (79) Task 2: number of clicks 29.8 (9.9) 25.8 (11.0) 34.1 (15.5) 25.9 (10.9) Task 3: correct clicks 36.9 (7.7) 39.4 (4.6) 38.5 (7.6) 39.7 (7.3) Task 3: incorrect clicks 7.4 (4.4) 15.7 (11.0) 12.9 (11.1) 12.3 (8.5) Affect Arousal (SAM) 2.87 (0.91) 2.80 (0.93) 2.67 (1.11) 2.83 (1.19) Valence (SAM) 3.53 (0.83) 3.53 (0.92) 3.60 (0.74) 3.93 (0.70) Positive affect 3.17 (0.47) 3.05 (0.63) 2.89 (0.58) 3.04 (0.55) Negative affect 1.57 (0.61) 1.31 (0.38) 1.30 (0.42) 1.33 (0.52) Subjective workload 8.94 (3.3) 9.92 (2.5) 9.34 (5.5) 9.01 (2.4) * Significant main effect of haptic aesthetic appeal and interaction between auditory aesthetic appeal and haptic aesthetic appeal 2.4 Discussion The results of this study indicated that auditory and haptic aesthetic appeal hardly had any influence on the outcomes of usability tests. Although our analysis confirmed that the experimental manipulation of haptic and auditory aesthetic appeal was successful, it was surprising that neither factor had influenced the perception of product usability or user affect. Performance was only affected in a specific instance. These results are in contrast to findings of numerous studies addressing the influence of visual aesthetics on the outcomes of usability tests, which generally showed a considerable impact of aesthetics on many variables including perceived usability, performance and affect (e.g., Tractinsky et al. 2000; Thüring & Mahlke, 2007). Some of this work also employed mobile phones (Quinn & Tran, 2010; Sauer & Sonderegger, 2009; Sonderegger & Sauer, 2010, Sonderegger et al. 2012; Thüring & Mahlke, 2007), which suggests that any differences in effects between visual and non-visual aesthetics cannot be attributed to particularities of a specific technical device. Nonetheless, there is still a need to demonstrate that the particular results of the present study were not due to product-specific influences. Especially smartphones are a product with high importance for self-representation (Bouwman et al. 2008). For such products, design features seem to be of particular importance to create a positive user experience. Therefore, there is a need to conduct further studies that make use of different interactive consumer products in order to exclude the possibility that the findings of the present study were caused by product-specific influences associated with mobile phones. 12

13 3 Study II 3.1 Goals of study As a consequence of the non-significant results of the first study, great importance was attached to increasing the effect of the experimental manipulations in the second study. In addition, a product was chosen whose operation requires a stronger physical activity (i.e. the need to hold the appliance tight). This was expected to ensure an intense haptic experience during task completion. Furthermore, the sound was presented this time via headphones in order to increase the effect of the auditory manipulation. While the first study used a smartphone which may be considered a multiple-domain product (i.e. it can be used for work purposes, in a more pleasurable leisure context but also to complete domestic chores), the vacuum cleaner that was chosen as the test product in the present study has a distinct purpose of being used for household chores. This will broaden the range of consumer products being tested in the present series of studies. Furthermore, this will allow us evaluating the moderating influence of product characteristics (e.g. domestic work appliance) on the influence of non-visual appeal on user performance. 3.2 Method Participants Sixty students (82% female), aged between 18 and 51 years (M = 22.4, SD = 5.0) took part in this experiment. 95% of the participants had a vacuum cleaner at home and used it 5.04 times a month on an average (SD = 2.8). The subjective frequency of participants' vacuum cleaning activity was 11.5 (SD = 4.7) on a scale ranging from 0 (very seldom) to 20 (very often). The participants' subjective rating of the importance of cleanness was 16.3 (SD = 2.4) on a scale ranging from 0 (not very important) to 20 (very important). Participants were not paid for their participation but received course credits Experimental design In a 2 x 2 between-subjects design, auditory and haptic aesthetic appeal of a vacuum cleaner was manipulated experimentally (N = 15 per cell). Haptic aesthetic appeal was manipulated by applying two different fabrics to the extension tube (suction pipe) of the vacuum cleaner (c.f. fig. 1), a soft rubber fabric covered with nodules (soft and smooth touch, i.e. pleasant haptic experience, HAP + ) and coarse sandpaper (rough, sharp and uncomfortable touch, i.e. unpleasant haptic experience, HAP - ). The auditory attributes of the vacuum cleaner was manipulated by issuing two sounds that differed in sound quality and volume. They were rated in a pilot study as being particularly pleasing or displeasing. The displeasing sound (AUD - ) was presented throughout the complete task at 79 db (A) whereas the pleasing sound (AUD + ) was presented at 58 db (A). 13

14 Figure 1: Two versions of the vacuum cleaner extension tube, a) coarse sandpaper (unpleasant haptic experience), and b) soft rubber fabric (pleasant haptic experience) Measures and instruments Perceived haptic and auditory attractiveness. Auditory attractiveness was measured by two items ("I experienced the sound of the device as being pleasant" and "I experienced the sound of the device as being harmonic"). The reliability of the scale was satisfactory (Spearman- Brown estimate:.79). Haptic attractiveness was measured by two items ("The extension tube was pleasant to touch", "Touching the extension tube felt good"). The scale obtained good reliability scores (Spearman-Brown estimate:.96). Both attractiveness scales used a 20-point Likert scale as a response format ranging from "I strongly agree" to "I strongly disagree". Perceived usability. To measure perceived usability in the present study, the same instrument (PSSUQ, Lewis, 1995) was used as in the previous study (see for a description of the instrument). However, the instrument needed to be adapted since a number of items were not applicable for use with vacuum cleaners (the questionnaire was originally developed for the evaluation of interactive computer systems which included, for example, items about error messages). Those items were not applicable to vacuum cleaners and were therefore removed in this study. For the present study, only items 1, 2, 3, 4, 5, 6, 8, and 19 of the PSSUQ were used. In addition, the wording of the items was adapted by replacing the term 'system' by 'appliance'. An analysis of the psychometric properties of the modified instrument provided good result, revealing a Cronbach s alpha of.90 for the internal consistency of the adapted version. User performance. Two measures of user performance were recorded. Task completion time (s) was recorded, indicating the time needed to clean the predefined surface. Additionally, cleanliness was calculated by the weight of sand (g) that was picked up by the vacuum cleaner divided by the total amount of sand that was spread out on the carpet. User affect and subjective workload. For both concepts, the same questionnaires were used as in study 1 (see for a description of the instruments). Analysis of reliability of the scales revealed Cronbach s alphas of.86 for PA,.87 for NA and.72 for the NASA TLX. 14

15 Visual attractiveness: As a control variable, visual attractiveness was measured with two items ("the design of the appliance was visually attractive" and "the appliance is visually appealing"). Both items used a 20-point Likert scale as a response format ranging from "I strongly agree" to "I strongly disagree". The scale showed good reliability scores (Spearman- Brown estimate:.92) Materials In this experiment, the usability of a vacuum cleaner was evaluated. For this purpose, a vacuum cleaner (AEG TM CE 4120) was used. The vacuum cleaner was hidden away in a sound-proof box. This was to ensure that visual aesthetic appeal and the original sound of the appliance did not influence the test user. Although the vacuum cleaner could not be moved, it was possible to clean the designated carpet area of about 1.5 m 2 due to the sufficient length of the extension tube. The sounds of the vacuum cleaner (pleasing, displeasing) were stored on an MP3-player (Sony TM NWZ-A818) and were presented to the test participants via headphones (Sennheiser TM HD 202). Two extension tubes were used in the experiment; one was wrapped with nubby soft rubber fabric, the other with coarse sandpaper. Participants were asked to use both hands to complete the tasks. The extension tube was the only part of the vacuum cleaner that was visible to test participants. Both fabrics were in black colour and it was ensured that both extension tubes were of comparable visual attractiveness. A set of precision scales (Kern TM WKr KB 650) was used to weigh the vacuum cleaner bag (before and after task completion) as well as the amount of sand to be spread out on the floor for the vacuum-cleaning task Pilot studies In a first pilot study, various fabrics of different haptic qualities as well as different sounds were evaluated with regard to their aesthetic qualities. Twelve blindfolded participants rated the haptic quality of five different fabrics: nubby soft rubber fabric, nubby hard rubber fabric, smooth rubber fabric, hard textured rubber fabric, and coarse sandpaper. Furthermore, they evaluated auditory quality of the recordings of the sound of five different vacuum cleaners: Philips TM SmallStar 1600 FC8230, Electrolux TM UltraSilencer green ZUS3365, Rowenta TM PowerSpace 2000W RO2125, Dyson TM ball DC37 Origin Extra, and Satrap TM Aspira Basic For both ratings, a 6-point Likert scale (ranging from 'very displeasing' to 'very pleasing') was used. The order of presentation of the different designs was randomized. For the present experiment, the most and least pleasing manipulations were chosen; nubby soft rubber fabric (M = 5.0, SD = 1.86) and coarse sandpaper (M = 1.6, SD = 0.67) for haptic aesthetic appeal, and Philips SmallStar 1600 (M = 5.50, SD = 1.09) and Rowenta PowerSpace 2000 (M = 1.83, SD = 1.47) for auditory aesthetic appeal. A second pilot study was conducted in order to ensure a successful manipulation of auditory aesthetic appeal. In this study, the two sounds were presented to 11 participants at a sound level of 58 db (A) for the quiet and at 79 db (A) for the loud condition. Participants rated the quiet sounds more positively than the loud sounds (M SmallStar(58db) = 5.7, M SmallStar(79db) = 3.8, M PowerSpace(58db) = 2.2, M PowerSpace(79b) = 1.8). Based on this result, the most pleasing stimulus (SmallStar at a sound pressure level of 58 db (A)) and the least pleasing stimulus (PowerSpace at a sound pressure level of 79 db (A)) were chosen for the main study Tasks Participants were asked to clean a designated carpet area (1.0 x 1.5 m) with the vacuum cleaner. Before the participants entered the lab, 200 g of bird sand was spread out evenly on 15