Visual and verbal metaphors in advertisements

Transcription

1 Visual and verbal metaphors in advertisements The effect of multimodal metaphors on the advertisement s comprehension, processing fluency, and aesthetic pleasure Laura Nijs ANR: Communication and Information sciences Business Communication and Digital Media Tilburg School of Humanities Tilburg University, Tilburg Supervisor and first reader: Neil Cohn Second reader: Renske van Enschot August 2017

2 Abstract In order to stand out and still communicate a strong, memorable message, advertisers more and more use tropes. A popular example of such a trope is a metaphor, which uses crossdomain mapping in order to communicate the message and can be applied both verbally and visually. Visually, the replacement metaphor, which replaces the target by the source, is perceived as the most complex metaphor whereas no metaphor is perceived as most convenient. However, both visualizations are considered as the least appreciated. Therefore, the addition of explanatory text could reduce the complexity of a replacement metaphor, and the addition of a verbal metaphor could apply a challenging element to a literal image. Previous studies showed contradictory results concerning the addition of explanatory information via text (i.e. text anchoring), with ad appreciation increased after providing extra information (Rompay, & Veltkamp, 2014), or having no result or a negative effect on likeability towards the ad after provided with full explanation (van Enschot, & Hoeken, 2015; Phillips, 2000). The current study sheds new light on the use of visual and verbal metaphors within advertisements on comprehension, aesthetic pleasure, and processing fluency. The stimuli had 16 scenarios which each contained 4 multimodal versions: literal image with literal text, metaphoric image with metaphoric text, literal image with metaphoric text, and metaphoric image with literal text. In total, 84 participants were exposed to one version per scenario where they rated each ad on processing fluency and aesthetic pleasure. Comprehension was tested via viewing time and whether the participant comprehended the ad or not. Results showed that verbal anchoring increased comprehension compared to a monomodal ad. Verbal challenging, however, hardly increased comprehension. The addition of a verbal metaphor to a visual metaphor resulted into the highest processing fluency, whereas the addition of a verbal metaphor to a literal image resulted into the least processing fluency. Moreover, visual metaphors were considered as more aesthetically pleasurable than literal images. Finally, the comparison between advertisements with a literal text when the ad was comprehended pointed at a possible mediation effect of processing fluency on aesthetic pleasure. Keywords: Advertisements, Aesthetic pleasure, Comprehension, Processing fluency, Verbal anchoring, Verbal challenging, Verbal metaphor, Viewing time, Visual metaphor. 2

3 Table of contents Introduction... 4 Theoretical framework 8 Ground principles of a metaphor... 8 Metaphors within advertisements. 8 Verbal versus visual metaphors Pictorial metaphorical structures The processing fluency theory Multimodal metaphors.. 15 Hypotheses 16 Method Stimuli 19 Procedure 22 Participants. 23 Data analysis.. 23 Results Comprehension.. 25 Aesthetic pleasure.. 30 Processing fluency. 31 Mediation analysis Conclusion and discussion Comprehension Aesthetic pleasure Processing fluency.. 41 Limitations and directions for future research 42 Practical implications.. 44 References.. 46 Appendix 1: Brand name congruency scores. 50 Appendix 2: Stimuli experiment.. 51 Appendix 3: Literal and metaphoric mean plus mean difference per scenario.. 60 Appendix 4: Overview of the 4 lists used in the experiment. 61 3

4 Introduction Due to fast-changing behaviors of customers, advertisers continually need to evoke new desires and tastes towards their products and services. In order to reach their goal of persuading customers to buy their products and services, advertisers are therefore constantly challenged to convince consumers the products they offer are meeting their needs. Over the past decades within print ads, more and more emphasis has been put on pictures instead of words (Anaya, 2008; Phillips, & McQuarrie, 2002; Pollay, 1985). Where text once served as an information tool, it now serves as an explanation of the core concept of the ad (Baker, 1988). Nevertheless, even with reduced focus on text, articulating the message of the product or service is still of major importance, since competitors still need to advertise a unique product. The consequence of not standing out is that comprehension will be negatively influenced when two similarly designed ads of equal products are presented next to each other (Janiszewski, 1990). Advertisers are therefore asked to develop creative, outstanding advertisements which eventually may not only increase comprehension, but also have a positive effect on the customer s attitude towards the company s product or service. In order to increase creativity in advertisements, many advertisers express the content of their ad through artful deviations from audience expectation (Phillips et al., 2002). These techniques include schemes which are superficial observations that are easily recognizable. In contrast to schemes, tropes are meaningful deviations that require more cognitive processing, because of their incongruence between what is pictured and what is meant (van Enschot, Broekhuizen, & Kolthoff, 2015). As a result, tropes require more effort, and are therefore, harder to comprehend compared to schemes. An example of a classical trope is a metaphor. A metaphor conceptualizes one mental, often abstract, domain (Target) in terms of another, concrete, domain (Source) which is named cross-domain mapping. In ordinary life, metaphors are often used in everyday language, and are in fact part of our ordinary way of thinking. An example of a conceptual metaphor that is integrated in our everyday language is the LOVE AS A JOURNEY metaphor in which the term love refers to the target domain, and the term journey refers to the source domain. Explanations like we re stuck and we need to move on are examples related to a journey, and are commonly used sentences within relationships. In this case, we apply our knowledge about travelling to clarify our thoughts concerning a relationship in order to make it more comprehensible (Lakoff, 1993). 4

5 Besides expressing a particular element of an object in terms of another object verbally, such an expression can also be visualized. When visualizing a metaphor, the reader needs to identify the target domain and source domain himself, which does not provide the guarantee that the target and source the reader identifies are equal to the target and source the advertiser intended to communicate (Forceville, 2009). This is in contrast to verbal metaphors, where the chance of a possible misunderstanding is reduced. Thus, metaphors in both modalities serve as an explanation of the intended message, but verbal metaphors are used to clarify a certain message, whereas pictorial metaphors reveal a more challenging effect in order to understand the message. According to Phillips and McQuarrie (2004), pictorial metaphors can be divided into three metaphorical structures: juxtaposition (separate presentations of target and source), fusion (target and source are merged), and replacement (the target is replaced by the source). Juxtaposition and fusion portray both the target and source which help to identify the rationale of the image. Replacement however, is not immediately identified as a metaphor, because of the lack of a perceptual second domain. In this case, the surrounding context plays a crucial role in identifying the target domain and conceptualizing it in terms of the source domain (Forceville, 1994). Figure 1 provides an example of the three metaphorical distributions plus no metaphor applied to the metaphor Glasses are like carrots. In all three appearances, the advertiser tends to communicate that these glasses improve your sight, just like the nutrients of carrots provide when you consume them. Figure 1. Four pictorial appearances of the inverted U-curve (van Mulken et al., 2014) Each of the metaphorical distributions contribute to a certain level of complexity. Van Mulken, van Hooft, and Nederstigt (2014) interpreted the relationship between the conceptual complexity and appreciation of the four pictorial appearances of metaphors as an inverted U- curve. Their study showed that the more complex the metaphor, the higher the appreciation until a tipping point where conceptual complexity of the visualization within the ad leads to 5

6 misunderstanding of the message which results in low appreciation. Accordingly, no metaphor requires low cognitive effort, which leads to fast comprehension, and replacement requires too much cognitive effort compared to the other appearances, which results in longer processing without full comprehension, and eventually, results in low appreciation towards an ad in both cases. Consequently, metaphors are harder to process than literal representations due to the cross-domain mapping metaphors require, but they are applied to increase creativity which positively influences appreciation at least till a certain level of comprehensibility. According to the Processing Fluency Theory of Reber, Schwartz, and Winkielman (2004), a stimulus is considered as more aesthetically pleasurable when the reader is able to easily process the stimulus. Core aspects that maximize processing fluency are easily understandable patterns, symmetry, and clarity (Reber et al., 2004; Gaarman, 2017). Because a metaphor is characterized by its incongruence, which contrasts these aspects, processing fluency may decrease which, according to the Processing Fluency Theory, negatively impacts aesthetic pleasure. On the other hand, the creative aspect of a metaphor evokes several interpretations that allow for elaboration, which sparks a reader s interest, curiosity, and attention, and leads to greater motivation to put cognitive effort into understanding the visualization. This can increase the aesthetic pleasure of the message (McQuarrie, & Mick, 2003; McQuarrie, & Mick, 1999; Goodstein, 1993; Ang, & Lim, 2006; Morgan, & Reichert, 1999). Even though metaphors contribute to an unexpected and moderately complex advertisement that may be harder to process than literal messages, the effort that is required eventually leads to a positive effect in both an informational and aesthetic way (Tanaka, 1994; van Mulken et al., 2014; Phillips et al., 2004). These results are contradictory to Reber et al. s (2004) Processing Fluency Theory. However, Phillips et al. (2004) also emphasize that this pleasurable effect only appears to a certain extent. Readers still need to be able to solve the riddle (i.e. successfully decode the hidden meaning of the metaphor) when exposed to a metaphorical ad, and thus advertisements that are too complex decrease processing fluency to a level that makes message comprehension too difficult. Therefore, an advertisement should aim for a level of complexity that allows for processing, but is also comprehensible enough to be solved (i.e. on the right place in the inverted U-curve) (van Mulken et al., 2014). One aspect that should be mentioned, is that the study of van Mulken et al. (2014) only tested visual advertisements, whereas in current advertisements text often serves as an explanation of the portrayed image (Anaya, 2008). Several studies have analyzed the addition of explanatory text, and revealed that verbal anchoring increases the comprehension and 6

7 decreases the cognitive elaboration of a pictorial metaphor (Jeong, 2008; Phillips, 2000; van Enschot et al., 2015). These anchoring headlines help readers to frame their interpretation of the advertisement s message (Ang et al., 2006). Consequently, text anchoring could reduce the complexity level of the replacement metaphor, which prevents misunderstanding and helps to solve the riddle. According to Phillips et al. (2004) and van Mulken et al. (2014), reducing complexity still allows for putting effort into processing, but also provides a comprehensible message, and thus may increase the pleasurable effect towards an ad. However, research on this effect shows contradictory results. On the one hand, the study by Rompay et al. (2014) indeed shows that providing extra information towards a visual metaphor increases ad appreciation, but on the other hand, van Enschot et al. (2015) and Phillips (2000) found no or even a negative effect of likeability towards the ad when full explanation was given. Despite verbal anchoring leading to no significant effect on ad appreciation, van Enschot et al. (2015) did find a positive effect on both comprehension and likeability when visual anchoring was used. Similar results were found in the study of Ang et al. (2006). In their study, they tested for the use of multimodal metaphors in advertisements on the attitude towards the ad. No differences were found for the combination of a metaphoric image with a metaphoric or literal headline, but with a literal image, the attitude towards the ad was more positive for a metaphoric headline than a literal headline. In both cases, the literal image serves as visual anchoring, so the addition of a metaphoric headline to a literal image increases the complexity and creativity, and could therefore lead to less processing fluency, but also more aesthetic pleasure compared to the monomodal literal advertisement. Because studies show contradictory results towards the addition of anchoring headlines, this study seeks to shed new light on the use of a combination of metaphoric headlines and images in advertisements. The results of a study on multimodal metaphors within advertisements may discover new insights concerning comprehension, processing fluency, and aesthetic pleasure for ads with a literal image and ads with a replacement metaphor. In order to test this effect, the following research questions are conducted: RQ1: What are the differences in cognitive processing between literal and metaphoric advertisements? RQ2: What are the differences in cognitive processing between metaphoric images and text within advertisements? 7

8 RQ3: What is the effect of multimodal metaphors within advertisements on the reader s comprehension, processing fluency, and aesthetic pleasure?. In the following chapter a theoretical framework has been developed based on previous studies that tested for pictorial, verbal, and multimodal metaphors related to comprehension, processing fluency, and aesthetic pleasure. Theoretical framework Basic principles of a metaphor The basic terminology of metaphors is the A IS B principle. A refers to the target domain which is an abstract term (i.e. the term that requires explanation). B refers to the source domain, which is the concrete term to which the target domain can be related (i.e. the term from which we draw metaphorical expressions). In order to infer metaphorical expressions, both domains contain a common feature that maps the source domain onto the target domain (i.e. a mapping feature). This common characteristic explains the target domain in concrete terms in order to provide better understanding of this abstract term (Forceville, 2014; Trezova, 2015). In fact, we unconsciously use this explanatory feature of metaphors repeatedly in our everyday language which is known as the Conceptual Metaphor Theory (CMT) (Lakoff, & Johnson, 1980). The CMT implicates that we tend to find phenomena that we can hear, taste, see, smell, and/or feel (i.e. concrete phenomena) easier to understand than phenomena we cannot. Therefore, we systematically infer abstract terms in concrete terms in order to make them more comprehensible (Forceville, 2009). An example of such a conventional metaphor is the ARGUMENT IS WAR metaphor (e.g. your claims are indefensible) (Lakoff, & Johnson, 1980). When you connect features of both terms, it will become clear that many things we do during arguing are similar to the concept of war. However, arguing reflects a verbal battle whereas war reflects a physical battle. Though, war structures the actions we perform during arguing which positively contributes to the comprehensibility of this abstract term. Metaphors within advertisements The CMT primarily refers to conventional metaphors, but metaphors within advertisements reach outside this conventional conceptual system, because they focus on creativity and imagination in order to interest the customer. Consequently, metaphors within advertisements 8

9 provide the opportunity of giving a new understanding to our current experience (Lakoff et al., 1980). However, advertisements do not always use an abstract metaphor. Since advertisers use their product or service as the target domain to explain its unique selling point via a common characteristic with a source domain, this comparison could also contain a comparison between two concrete terms (Morgan et al., 1999). These common features do not immediately reflect our everyday life. Therefore, concrete metaphors within ads are a challenge that require deeper cognitive processing compared to literal messages and conventional metaphors (Phillips et al., 2004). Readers inferences between the target domain and the source domain start with similarities that are easy to draw, but when they are not able to draw simple inferences, they continue to search for alternatives (McQuarrie et al., 2003). Therefore, when the reader is exposed to a metaphoric message, he will first have a perception of error, but once the message is comprehended, this negative tension disappears (Jeong, 2008). In other words, metaphoric messages within advertisements challenge, but also help, readers to solve a riddle, which require deeper processing and, consequently, take longer to comprehend compared to literal messages (Toncar, & Munch, 2001; Lagerwerf, van Hooijdonk, & Korenberg, 2012). An angle from which comprehension of metaphors could be analyzed more precisely, is via neurocognitive processing. Ortiz, Murcia, & Fernandez (2017) tested this process in an electrophysiological study. They measured the participants electrical brain activity by recording Event-Related Potentials (ERP s). A similar processing pattern appeared for pictorial and verbal metaphors, namely the N400 effect. The N400 is a waveform with a negative direction that has a starting point at approximately 300 ms and peaks at around 400 ms after presentation of a specific content. The N400 appears after inconsistencies with the reader s comprehension of real-world knowledge of what is usual or factually true (Sitnikova, Holcomb, Kiyonaga, & Kuperberg, 2008). This effect indicates that metaphors, in general, require more effortful semantic processing than literal messages. Besides, Sitnikova et al. (2008) and Mudrik, Lamy, & Deouell (2010) investigated the neurocognitive processing on visual real-world events only and found additional results. The real-world events either consisted of a congruent scene (e.g. someone cuts bread with a sharp knife) or an incongruent scene (e.g. someone slides an iron across a piece of bread). The recorded ERP s for incongruent events showed an anterior N400. Besides that, they found a late positivity (P600), which relates to the revision of a mental model. The P600 appeared because, after the N400, the incongruent event is considered as a context violation, and therefore the viewer s brain starts to evaluate and update the current event, which evokes the P600. As a result, 9

10 metaphoric images could, just like incongruent images and video clips, require a longer comprehension time than literal images, because the viewer will notice that the advertisement shows an unusual real-world event which requires evaluation of the image, so perceptions of the pictorial metaphor s context can be conducted in order to comprehend the intended message. In other words, due to the P600, visual metaphoric advertisements evoke less processing fluency than literal ads, which requires more time to comprehend the message. The positive side, however, is that metaphoric messages add more interest, curiosity and attention which serves as a motivation for processing a metaphor within an advertisement (McQuarrie et al., 1999; Goodstein, 1993; Ang et al., 2006; Morgan et al., 1999). This motivation in combination with an ad s creativity results into multiple interpretations, new perspectives, and new ideas towards the communicated product or service (Lubart, & Getz, 1997; McQuarrie, & Phillips, 2005; Forceville, 2009). Because these metaphors allow for multiple interpretations, metaphoric messages require elaboration on the meaning of the ad, which is experienced as aesthetically pleasurable (McQuarrie et al., 2003). Consequently, metaphoric messages are less fluently to process than literal messages within advertisements, but because these messages evoke the motivation to solve the riddle, and allow for multiple interpretations and new perspectives, the aesthetics of the advertisement will be experienced as more pleasurable. The latter is similar to Aristotle s statement to learn gives the liveliest pleasure in which he states that individuals experience pleasure during the cognitive processing, because, during this process, they use more cognitive effort in order to reach comprehension (Stern, 1990; Ang et al., 2006). This desire for comprehension, eventually, will also contribute to the increasing of the aesthetic pleasure, and therefore, moderates the effect of metaphoric messages on aesthetic pleasure. Verbal versus visual metaphors In order to define a multimodal metaphor as meant in the current study, it is important to first define the concept mode. A mode can occur in different varieties, and, according to Forceville (2009), consists of one of the following aspects: written signs, pictorial signs, spoken signs, sounds, gestures, smells, music, touch, and tastes. In advertising, verbal metaphors can occur via spoken language (in TV-commercials) and written language (in print ads). Similarly, visual metaphors occur via an audiovisual form in TV-commercials, and occur via an image in print advertisements. Due to the aim of the current study, the following information will focus on the metaphoric occurrences within print ads. 10

11 When analyzing an advertisement, the reader will be more attracted to the ad s image due to its attention-grabbing ability (Ang et al., 2006). This superiority combined with the ability of a metaphor to provide creativity may result in higher aesthetic pleasure when pictorial metaphors are used compared to verbal metaphors. Indeed, several researchers imply pictorial metaphors as being more open to interpretations than when the same message is communicated via a verbal metaphor, because pictorial metaphors are considered as completely implicit (Eco, 1976; Marchland, 1985). However, even though a visual metaphor allows for more openness towards drawing interpretations, Smith (1991) found that the number of inferences drawn from both the visualized and verbalized version of a message was equal. The number of inferences may therefore not deviate, but because the openness could be perceived as more inviting, pictorial metaphors could still have a more positive effect on the aesthetic pleasure than verbal metaphors. Regarding comprehensibility, Morgan et al. (1999) imply that verbal metaphors require the reader to create his own image. When advertisers use visual metaphors, they choose a specific image that aligns with the message they intend to communicate via the metaphor. Besides that, associating incongruent objects via visualization causes less questioning from readers compared to verbalizing these associations, because readers can take this association for granted when the metaphor is visualized rather than when the same association is verbalized (Messaris, 1997). Therefore, the processing of pictorial metaphors will also be considered as more fluent compared to verbal metaphors. Even though studies show that pictorial metaphors provoke greater aesthetic pleasure than verbal metaphors, for both modalities cognitive processing serves as a precondition for causing a pleasurable attitude towards the advertisement (Ortiz et al., 2017). This implicates that the reader first needs to comprehend the metaphor in order to enjoy the overall advertisement (Lagerwerf et al., 2012). Therefore, comprehension serves as a moderator for the effect of metaphors on aesthetic pleasure. Pictorial metaphorical structures The level of comprehension, processing fluency, and aesthetic pleasure on visual metaphors depends on the way the metaphor is represented. According to Phillips et al. (2004), target and source can be physically depicted in 3 different structures: Juxtaposition: The target and source are depicted alongside each other; Fusion: target and source are merged; Replacement: the target is replaced by the source. 11

12 The three structures differ in level of complexity. Complexity plays a central part in whether the reader processes, and eventually, comprehends the message of the ad correctly. Because juxtaposition visualizes the target and source domain separately, identification of the elements is relatively convenient. Identification of the two elements becomes harder in the fusion structure. The reader needs to correctly separate the target and source domain himself in order to understand the message. The most complex structure is the replacement metaphor. Due to the lack of a target domain, the reader is challenged to identify the missing element and to connect its relation with the present element (Phillips et al., 2004). Van Mulken et al. (2014) set the complexity of visual metaphorical structures and no metaphor against the appreciation of the ad which resulted in the inverted U-curve (Figure 2). The inverted U-curve reveals that the more complex the visualization, the higher the appreciation until a tipping point where the visual metaphor becomes too complex which decreases ad appreciation. Figure 2. The inverted U-curve of van Mulken et al. (2014) The decreased appreciation of no metaphor and replacement metaphor compared to juxtaposition and fusion could refer to the appearance of only the target or source domain. When only the target domain is portrayed, the image literally explains the message of the advertisement, so the reader is not challenged to understand the message the advertiser is trying to communicate. In the replacement metaphor, as provided in Figure 1, the target metaphor was fully replaced by the source domain. This form of replacement is highly complex, and therefore, requires much cognitive effort. Moreover, the reader may draw many interpretations without completely being aware of which interpretation was meant by the advertiser. Thus, exposure to a replacement metaphor will not lead to the feeling of 12

13 completely having solved the riddle whereby the perception of error remains (Jeong, 2008). The level of comprehension (either too easy or too complicated) is therefore in line with the level of complexity of the visual structure (Lagerwerf et al., 2012; Phillips et al., 2004). In case of no metaphor, all relevant pictorial elements are present in order to understand the message, and therefore, offers no challenging cognitive processing for the reader. For replacement metaphors, however, the visual structure is considered as the most complex compared to fusion and juxtaposition, because the number of relevant pictorial elements is too low to comprehend the message. As a result, the lack of relevancy evokes misunderstanding which, consequently, has a negative effect on the appreciation towards the advertisement compared to the other pictorial appearances. The unchallengeable effect of no metaphor and the misleading effect of the replacement metaphor negatively influence the comprehension of the advertisement s message, which not only negatively impacts ad appreciation, but also processing fluency. Besides that, a literal visual representation only shows the target domain which decreases the level of creativity, and therefore decreases aesthetic pleasure. In his study, Schilperoord (2016) provides another form of replacement in which he does not replace the target domain by the source domain, but replaces the context of the target domain by the source s context. He calls this form of replacement substitution (van den Reek, 2016). An example of substitution within an advertisement is given in Figure 3. By adjusting the context instead of the product, a subtle hint has been provided in order to solve the riddle. This tactic is one way to decrease the misleading effect of a replacement metaphor, and helps to comprehend the message the advertiser tends to communicate. As a result, providing such a clue will increase processing fluency, which makes the message more comprehensible. Yet, another way of providing a clue is not by adjusting the image, but by adding another modality: text. 13

14 Figure 3. The use of substitution within an advertisement The Processing Fluency Theory One theory that contradicts the aforementioned theories is the Processing Fluency Theory of Reber et al. (2004) which states that the more fluently the stimulus is processed, the higher the level of aesthetic felt pleasure will be. According to this theory, no metaphor should be the most appreciated representation. Several studies indeed found that an object is considered more aesthetically pleasing when it contains symmetry, clarity, and simple patterns which characterize fluent processing (Reber, Wurz, & Zimmermann, 2004; Reber, 2002; Reber, Winkielman, & Schwarz, 1998). These aspects clearly contradict the characteristics of a visual metaphor. A metaphor, in all its possible constructions, visualizes incongruence and therefore, according to those findings, should yield less aesthetic pleasure in general. However, besides the study of van Mulken et al. (2014), other studies found results that proved the opposite state. Several researchers found evidence for a positive relation between complexity and novelty on aesthetic pleasure (e.g. Hekkert, Snelders, & Van Wieringen, 2003; Landwehr, Labroo, & Herrmann, 2011) which actually should decrease aesthetic pleasure according to the Processing Fluency Theory. Therefore, it remains unclear whether this theory is also applicable to the use of metaphoric messages within advertisements. 14

15 Multimodal metaphors The two aspects that are most relevant for creating a multimodal metaphor in print ads are written and pictorial signs. On basis of the two modalities of text and image, a multimodal metaphor within an advertisement can occur in three varieties. First, the advertisement could contain a visual metaphor and a literal text. Another representation could be a verbal metaphor and a literal image, and lastly, both a verbal and pictorial metaphor. Besides that, an advertisement can consist of a text and an image that communicate the message literally. Since the replacement metaphor and no metaphor yielded the least appreciation, we will further investigate these visual structures in a multimodal representation. Therefore, from now on in this study a pictorial/visual metaphor refers to a replacement metaphor, and a literal image refers to no metaphor. The addition of a literal text to a pictorial metaphor could serve as an explanation. In this way, the explanatory text increases relevant elements verbally, which offers a clue in order to reduce cognitive effort and to increase processing fluency and comprehension (Phillips, 2000). Once the message is comprehended, the error will be converted into understanding of the message, and because the visual metaphor still allows for elaboration, the aesthetic pleasure also remains positive (McQuarrie et al., 2003). In turn, the addition of a verbal metaphor to a literal image could apply a challenging element which increases the advertisement s complexity and, hence, requires more cognitive effort in order to comprehend the message (Toncar et al., 2001; Langerwerf et al., 2012). The increased cognitive effort evokes motivation to solve the riddle and allows for different perspectives (McQuarrie et al., 1999; Goodstein, 1993; Lubart et al., 1997; McQuarrie et al., 2005). Moreover, adding a metaphoric element increases the creativity of the advertisement which leads to higher aesthetic pleasure as a result of the allowance for elaboration (McQuarrie et al., 2003). However, because of the attention-grabbing ability of an image, the reader will first focus on an advertisement s literal element. The exposure to the verbal metaphor after being exposed to the image may therefore have less impact on the aesthetic pleasure compared to the advertisement with a visual metaphor. Eventually, the advertisement with both a verbal and pictorial metaphor will yield the lowest processing fluency of the three multimodal metaphoric representations. Even though the context of the visualization slightly reveals the target domain, adding this relevant verbal element provides information to better understand this context (Lagerwerf et al., 2012). However, the addition of the verbal metaphor still allows for multiple interpretations without fully comprehending which interpretation fits the message of the ad (van Mulken et al., 2014). 15

16 As a result, the verbal and pictorial metaphoric advertisement will be considered as the most complex, which negatively influences processing fluency. On the contrary, metaphors increase creativity, so adding a verbal metaphoric element to a pictorial metaphor increases the level of creativity even more, which positively affects the aesthetic pleasure (Lubart et al., 1997; McQuarrie et al., 2005; McQuarrie et al., 2003). Hypotheses Based on the given literature concerning metaphoric messages, the following hypotheses can be drawn. In line with Toncar et al. (2001) and van Hooijdonk et al. (2012), it is hypothesized that metaphoric messages are harder to comprehend, because metaphors are less fluent to process. Processing fluency therefore mediates the effect of metaphors on comprehension. Because metaphoric messages evoke more interest, curiosity and attention, the reader will be motivated to process the ad (McQuarrie et al., 1999; Goodstein, 1993; Ang et al., 2006; Morgan et al., 1999). Besides that, metaphoric messages are considered as more creative, which positively influences the aesthetic pleasure (Lubart et al., 1997; McQuarrie et al., 2005, McQuarrie et al., 2003). However, according to the Processing Fluency Theory of Reber et al. (2004), processing fluency positively mediates aesthetic pleasure, but other studies (e.g. Landwehr et al., 2011) found contradictory results. Consequently, processing fluency does mediate aesthetic pleasure, but the direction remains unclear. Besides that, a precondition for causing a pleasurable attitude towards the advertisement is message comprehension (Ortiz et al., 2017; Lagerwerf et al., 2012). Therefore, comprehension serves as a moderator for the effect of metaphors on aesthetic pleasure which is also in line with the statement of Aristotle to learn gives the liveliest pleasure (Stern, 1990; Ang et al., 2006). In Figure 4, the conceptual model of the expected effects of metaphoric messages on the three dependent variables is shown. H1: Metaphoric messages are harder to comprehend than literal messages. H2: Metaphoric messages positively influence the aesthetic pleasure of the advertisement compared to literal messages. H3: Metaphoric messages negatively influence the processing fluency of the advertisement compared to literal messages. H4: Processing fluency mediates the effect of metaphoric messages on aesthetic pleasure. H5: Comprehension positively moderates the effect of metaphoric messages on aesthetic pleasure. 16

17 - Processing fluency +/- + Comprehension Metaphoric messages + Aesthetic pleasure - Comprehension Figure 4. The conceptual model of the effect of metaphoric messages within advertisements on aesthetic pleasure, mediated by processing fluency and moderated by comprehension, and on comprehension (H1, H2, H3, H4, and H5) The second aspect that will be investigated, is the effect of different metaphoric modalities on comprehension, aesthetic pleasure, and processing fluency. In general, pictorial metaphors will be considered as more aesthetically pleasurable than verbal metaphors, because metaphoric images have the ability to demand attention and are considered as completely implicit compared to metaphoric text which allows for more elaboration (Ang et al., 2006; Eco, 1976; Marchland, 1985). Besides that, associating metaphors via images causes less questioning than the verbalization of these associations (Messaris, 1997). Thus, the processing of pictorial metaphors could be considered as more fluent compared to verbal metaphors. On the contrary, the visual metaphor does not depict both target and source which is the case for verbal metaphors. Therefore, visual metaphors lack relevancy more than verbal metaphors which, in turn, could decrease processing fluency. Also, for both modalities, the precondition of first comprehending the message before the advertisement yields a pleasurable effect which is in line with H4. According to these theories, the following hypothesis and research question can be formulated: RQ4: Do pictorial metaphors differ in level of processing fluency compared to verbal metaphors? H6: Pictorial metaphors within advertisements evoke higher aesthetic pleasure than verbal metaphors. 17

18 Eventually, the addition of multimodal metaphors towards advertisements will be analyzed. First, the addition of a literal text to a pictorial metaphor provides a clue in order to provide more fluent processing, which increases comprehension (Phillips, 2000). Adding a metaphoric text to a pictorial metaphor, in case of a replacement metaphor, only reveals the target domain, but because the intended message remains unclear, the addition of a metaphoric text is less comprehensible than the addition of a literal text. A literal text provides the understanding of the message which may positively influence the processing fluency, and the combination with the pictorial metaphor makes it possible to elaborate on the message, so the aesthetic pleasure remains positive (McQuarrie et al., 2003). However, metaphors are considered as more creative compared to literal text and images. Adding a verbal metaphor to a pictorial metaphor increases creativity, which in turn, evokes higher aesthetic pleasure compared to the addition of a literal text to a pictorial metaphor. H7a: An advertisement with a pictorial metaphor and a literal text increases comprehension compared to an advertisement with a pictorial and verbal metaphor. H7b: An advertisement with a pictorial metaphor and a literal text evokes higher processing fluency compared to an advertisement with a pictorial and verbal metaphor. H7c: An advertisement with a pictorial and verbal metaphor evokes higher aesthetic pleasure compared to an advertisement with a literal image and a literal text. Adding a verbal metaphor to a literal image induces more cognitive effort, which decreases processing fluency in order to comprehend the message compared to the addition of a literal text, because the verbal metaphor serves as a challenging element (Toncar et al., 2001; Langerwerf et al., 2012). This challenging effect allows for different perspectives (McQuarrie et al., 1999; Goodstein, 1993; Lubart et al., 1997) which positively influences aesthetic pleasure (McQuarrie et al., 2005). H8a: An advertisement with a literal image and a verbal metaphor makes it harder to comprehend the message compared to an advertisement with a literal text and image. H8b: An advertisement with a literal image and a verbal metaphor negatively influences processing fluency compared to an advertisement with a literal text and image. H8c: An advertisement with a literal image and a verbal metaphor positively influences aesthetic pleasure compared to an advertisement with a literal text and image. 18

19 Method Stimuli The experiment used 20 advertisements of fictitious brands containing six versions: literal image only, pictorial metaphor only, pictorial and verbal metaphor, pictorial and verbal literal, pictorial metaphor and verbal literal, and pictorial literal and verbal metaphor. In Figure 5 an Figure 5. Example of an advertisement in all six versions used within the experiment example of advertisements in all six versions is provided. On basis of the metaphorical distributions of Phillips et al. (2004), in the current research the replacement metaphor and no metaphor were used. The verbal metaphor is processed in the advertisement s headline, and is constructed as This target is like a source (e.g. this car is like a rhino). In the literal headline a literal explanation of what the product represents is given (e.g. this car is strong and solid). The metaphors were collected from the stimuli of several studies (van der Horst, 2016; van den Reek, 2016; Gaarman, 2016; Ang et al., 2006; McQuarrie et al., 2005), and were also based on our own implementations. To make the stimuli applicable for the current experiment, pictures were collected from Google images and then reproduced and developed 19

20 in Adobe Photoshop cc 2017 to create novel advertisements. To minimalize the chance that the results are the cause of other elements than the verbal and pictorial metaphors, the advertisements were provided a similar design. All advertisements consisted of a black coloured headline on top with font size 17, font type Perpetua Titling MT, and a 2 pt. white line and drop shadow to make sure the headline was clearly visible in all backgrounds. The headline was followed by an illustration of the product, and the fictitious brand name on the bottom right of the ad (Ang et al., 2006). Furthermore, all images were similarly sized. Five filler ads contained irrelevant verbal and pictorial incongruences in order to distract the participants from the aim of the study. These fillers were all used in each list, and were randomly divided across the items per list. Pre-test experiments To guarantee the metaphors used in the experiment would be recognized as more figurative compared to the literal images and headlines, two pre-tests were conducted with Qualtrics that were both divided into two versions. The two pre-tests contained similar questions, but each referred to one modality; one was related to the images and the other was related to text. As a result, two pre-tests were conducted where participants were randomly exposed to the figurative or literal version per scenario. To measure the figurativeness, five statements were conducted. These statements were based on the dimensions of a metaphor described in the study of Marschark, Katz, and Pavio (1983). These statements included the metaphor goodness (pleasing/unpleasing), imageability (easy to imagine/difficult to imagine), interpretation (easy to interpret, difficult to interpret), degree of metaphoricity (literal/figurative), and familiarity (familiar, unfamiliar), and were rated on a 7-point Likert scale. After that, the participants in the pictorial pre-test were asked to rate the fictitious brand name on congruency with the portrayed product on a 7-point Likert scale. Eventually, all participants were asked to fill out questions concerning their demographics. Pre-test participants A total of 56 participants (31 in the text pre-test and 25 in the image pre-test) completed the survey and were collected via convenience sampling. Of the 25 participants who participated in the image pre-test 10 were male and 15 were female. The average age was (SD = 2.03) and 22 were highly educated. A total of 19 females participated in the text pre-test, and the average age of all participants was (SD = 1.84). Of the participants, 90.3% was 20

21 highly educated. Consequently, the participants of both pre-tests contained similar demographics which allows for generalization of the interpretations. Pre-test results Before analyses were performed, the metaphoric dimension pleasing was reversed, and then a mean score per metaphoric dimension for the literal images and text and metaphoric images and text was calculated. After that, an overall mean score of all five dimensions was computed per condition. In all four cases, reliability was low. However, after excluding the dimension pleasing, Cronbach s Alpha ranged from somewhat good to high, which allowed for calculation of the mean variables image literal, image metaphoric, text literal, and text metaphoric. Table 1 provides an overview of the reliability before and after exclusion of the dimension pleasing. Table 1. Overview of the reliability scores before and after exclusion of "pleasing" Variable Reliability of all five dimensions Reliability after excluding pleasing Image literal Image metaphoric Text literal Text metaphoric Image pre-test: In order to analyse if the participants indicated metaphoric images at a higher metaphoric level than literal images, a dependent t-test was performed. Z-scores indicated that the data was somewhat skewed (z-score skewness = 2.65, z-score kurtosis =.98). However, the KS-test was nonsignificant (D(24) =.17, p >.05). Therefore, we should carefully interpret the outcome of the t-test. The t-test outcome showed a significant difference between the two variables (t(24) = -8.64, p <.001). Overall, participants rated the intended metaphoric images (M = 4.58, SD =.87) as being more metaphoric than the intended literal images (M = 2.83, SD =.10). The effect size for this result was large (r =.87). Text pre-test: To analyse if metaphoric headlines were considered as more metaphoric compared to the literal headlines, again a dependent t-test was performed. The data showed some kurtosis (z-score skewness = 1.67, z-score kurtosis = 3.04). However, the KS-test was 21

22 nonsignificant (D(31) =.097, p >.05), so also in this case we should be careful with interpreting the outcome of the t-test. The t-test outcome showed a significant difference between the two variables (t(30) = -8.38, p <.001). Overall, participants rated the intended metaphoric headlines (M = 3.82, SD =.81) as being more metaphoric than the intended literal headlines (M = 2.28, SD =.87). The effect size of this result was large (r =.84). Finally, the brand name congruency score was analysed per scenario. A brand name was considered as congruent when the congruency score was lower than 3.5. Brand names of seven scenarios were considered as incongruent. The complete table with all brand name congruency scores is added to Appendix 1. Final stimuli creation Based on the rating study, the four images that contained the lowest metaphor scores were excluded. This left 16 scenarios and 5 fillers in the final experiment. In Appendix 2, the final stimulus set is provided, and Appendix 3 shows a table with the literal and metaphoric means per scenario plus the difference between both means. The five remaining incongruent brand names (two incongruent brand names were part of the removed scenarios) were adjusted. Per brand name two or three potential brand names were presented to 8 persons (50% female) who selected the most congruent brand name for each scenario. Eventually, the brand name that contained the highest number of votes was selected per scenario. Finally, four lists were conducted that counterbalanced the four conditions such that each list contained the same number of conditions and only one condition per scenario. Consequently, across all lists all conditions would be viewed across participants. The four lists are presented in Appendix 4. Participants For the main experiment a total of 84 participants were collected via convenience sampling. Before performing the experiment, all participants gave their informed consent form to participate. Of all participants 56 were female. The participants age ranged from 18 to 75 years old (M = 26, SD = 9.24). Most participants were Dutch (96.3%). The remaining nationalities were American (2.5%) and Chinese (1.3%). Because the advertisements contained English headlines, the participants were asked to rate their knowledge of the English language on a five-point scale, which was rated as somewhat above average (M = 22

23 3.95, SD =.86). Only two participants rated their English knowledge somewhat below average. Procedure Participants were invited via convenience sampling to take part in the online experiment. After clicking on the link, the participant entered the online experiment in Qualtrics. They were able to read a short introduction concerning the experiment and the consent form. By clicking on the next button, they gave permission to use their outcomes for experimental results. Before entering the actual experiment, the participants were asked to fill out questions concerning their demographics and English knowledge. After that, each participant was randomly assigned to one of the four lists. During the experiment, the participants were exposed to 21 advertisements: 16 manipulated advertisements and 5 fillers. For each trial, the participants first saw either the metaphorical image or the literal image without headline. After pressing a button, they rated their understanding of the image, by pressing the 1 button (yes) or the 0 button (no). Then, they were exposed to the next screen where they saw the same image with either the literal or metaphorical headline. After pressing a button, they were asked the same question concerning the comprehension of the ad s message. Finally, they rated the advertisement on processing fluency and aesthetic pleasure on a 5-point Likert scale. These actions were repeated for all 21 advertisements. To measure the processing fluency and the aesthetic pleasure, three scaled questions were used, and were measured with a 5-point semantic differential which were derived from the study of Van der Horst (2016). The processing fluency (Was the advertisement easy to comprehend?) was measured with the semantic differential easy to comprehend-hard to comprehend. The other two semantic differentials which were used to measure the aesthetic pleasure (How attractive was this advertisement?; How enjoyable was this advertisement?) were attractive-unattractive and enjoyable-unenjoyable. The participants were informed to rate these questions on basis of the multimodal advertisement. Data Analysis In order to analyze the effect of verbal and pictorial metaphors within advertisements on the participants comprehension, processing fluency, and aesthetic pleasure, a 2 (Text type: literal vs. metaphoric) x 2 (Image type: literal vs. metaphoric) within-subject design was conducted for each measurement. Because the participants were exposed to different conditions per 23

24 advertisement, averages across items were calculated. In the end, an overall viewing time, comprehension rating, processing fluency rating, and aesthetic pleasure rating was computed per condition. Because some participants not always used the 1 or 0 button to rate their comprehension, the mean scores for these participants were excluded from the analysis. To measure the aesthetic pleasure, two measurements were combined into a mean score per condition. For all 4 conditions, Cronbach s alpha of the combined measurements was calculated and reported in order to indicate if the measurements were reliable. Viewing time and comprehension ratings were calculated by averaging conditions across participants, both for monomodal ads (first image) and multimodal ads (second image). Finally, to analyse the change in viewing after the addition of text, viewing times of the monomodal ads were subtracted from the viewing times of the multimodal ads. The same action was performed for comprehension ratings. Eventually, viewing time and comprehension ratings were tested for correlation to indicate a possible relationship. Correlations were also measured between viewing time and comprehension ratings on aesthetic pleasure in order to indicate a possible moderation effect. Another correlation tested whether processing fluency mediates the effect of multimodal metaphoric advertisements on comprehension. Mediation was tested via the statistical program MEMORE which allowed for mediation analysis on within-subject designs. However, this program is also associated with some practical limitations. MEMORE currently does not allow for moderation analysis, so comprehension was used as a split file condition within the mediation analysis in order to obtain separate results for not comprehended and comprehended advertisements. Besides, the program is only capable of comparing two levels per analysis. Therefore, 8 separate data files were conducted and tested for mediation. Since there might be a chance of alpha-inflation which cannot be corrected, the results should be interpreted carefully. First, a mean comprehension score per comparison was computed. After that, the data file was split on comprehension in which <.5 was considered as not comprehended, and >.5 was considered as comprehended. Eventually, 8 mediation analyses were performed: 4 comparisons on no comprehension, and 4 comparisons on comprehension. Per comparison, three effects were reported: the first effect was the indirect effect (i.e. the mediation effect) that showed whether the difference of the two compared conditions (the dummy variable X) differed on aesthetic pleasure via processing fluency. The second effect was the direct effect which showed if the difference in aesthetic pleasure was the direct cause of X, and the third effect was the total 24

25 effect that showed whether X differed on aesthetic pleasure via both the direct and indirect pathways (Hayes, & Preacher, 2013). Results Comprehension In order to test the comprehensiveness of metaphors and text anchoring, multiple analyses were performed with multimodal and monomodal advertisements on viewing time and comprehension ratings. Viewing time Multimodal advertisements In order to test the comprehensiveness of metaphors and text anchoring, a factorial repeated measure 2 (image type) x 2 (text type) ANOVA was performed on viewing time. No condition was normally distributed. Therefore, the results should be interpreted carefully. Cook s distance was not larger than 1, and for both conditions less than 5% of the residuals ranged outside (-)2. A marginally significant main effect of image type shows that a metaphoric image within a multimodal advertisement was viewed longer (M = , SE = 368.0) compared to a literal image within a multimodal advertisement (M = , SE = ), F(1,83) = 3.77, p =.06, ω 2 =.04. Concerning text type, no significant effect appeared which implicates no difference in viewing time between a metaphoric text (M = , SE = ) and a literal text (M = , SE = ), F(1,83) =.32, p =.58, ω 2 =.004. No interaction effect was found between text type and image type (F(1,83) = 1.41, p =.24, ω 2 =.02). The average viewing time per condition is shown in Figure 6. Monomodal advertisements The difference scores of viewing time show non-normally distributed data. Therefore, the 95% bootstrapped confidence interval is reported. A dependent t-test showed a marginally significant effect of monomodal advertisements on viewing time, t(83) = 1.90, p =.061, which implicates that literal images (M = , SD = ) were viewed longer than metaphoric images (M = , SD = ). The 95% bootstrapped CI [60.10, ] does not cross zero which indicates that the results are generalizable to the population. 25

26 Difference scores A repeated measure factorial ANOVA was performed on the calculated differences between monomodal and multimodal images in order to test if the addition of text increased or decreased viewing time. Only the difference in viewing time of the completely metaphoric advertisement was normally distributed (z-score skewness =.11, z-score kurtosis = 1.58). Therefore, the p-values should be interpreted carefully. According to the residuals and Cook s distance, the data can be considered as healthy. A significant main effect of image type occurred on viewing time, F(1,83) = 6.58, p =.012, ω 2 =.07, because the difference in viewing time on the literal image after adding text (M = , SE = ) is larger than the difference in viewing time on a metaphoric image (M = , SE = ). Also, the addition of text decreased viewing time on a literal image whereas the viewing time on a metaphoric image slightly increased. Regarding to text type, a nonsignificant effect appeared (F(1,83) =.56, p =.46, ω 2 =.007). Therefore, the addition of a literal text did not evoke a larger difference in viewing time between the multimodal and monomodal ad (M = , SE = ) than a metaphoric text (M = , SE = ). Figure 6. Average viewing time of the ad per condition. Figure 7. Difference in viewing time of multimodal compared to monomodal ads Overall, no significant interaction effect appeared (F(1,83) =.91, p =.34, ω 2 =.01). The difference score of viewing time after the addition of text per condition is visualized in Figure 7. 26

27 Comprehension ratings Multimodal advertisements Next, we analysed the participant s comprehension of the multimodal advertisement. The data was highly skewed, so the p-value should be interpreted carefully. However, more than 95% of the residuals lies between -2 and 2, and in all cases Cook s distance was < 1. A significant main effect of text type indicated that a literal text (M =.80, SE=.02) was easier to comprehend than a metaphoric text (M =.74, SE =.03), F(1,59) = 5.26, p =.03, ω 2 =.08. No significant main effect of image type appeared on comprehension ratings (F(1,59) =.33, p =.57, ω 2 =.005), suggesting that there was no difference between literal images (M =.762, SE =.03) and metaphoric images (M =.779, SD =.03) on the comprehension of the ad after the addition of text. Figure 8 shows the pattern of the results. Figure 8. Outcome of the comprehension ratings per condition Figure 9. Multiple line graph of the interaction effect between text and image on comprehension An interaction suggested that an advertisement with a literal text and a literal image (M =.84, SD =.21) is better comprehensible than an advertisement with a literal text and a metaphoric image (M =.77, SD =.24), but the other way around appeared for the conditions with the metaphoric text (literal image, metaphoric text: M =.69, SD =.31; metaphoric image, metaphoric text: M =.79, SD =.26), F(1,59) = 10.89, p =.002, ω 2 =.16 (Figure 9). One-sample t-tests were performed in order to test whether the outcomes significantly differed from the midpoint (a rating of 4), which also tested for incongruence. All multimodal 27

28 advertisements were strongly significantly rated above average except for the fillers which were strongly significantly below average. The outcomes are shown in Table 2. Table 2. Outcome one sample t-tests of the rating scores on comprehension compared to the average score.5. Condition t p R Visual literal, Text literal < Visual literal, Text metaphor 4.50 < Visual metaphor, Text literal 8.51 < Visual metaphor, Text metaphor 8.41 < Filler < Monomodal advertisements For comprehension, the data was normally distributed. A literal visual advertisement was considered as better comprehensible (M =.67, SD =.25) than a metaphorical visual advertisement (M =.48, SD =.29), t(60)= 5.35, p <.001, r =.57. The comprehensibility of literal images was rated above average, t(59) = 5.02, p <.001, r =.25, but fillers were rated below average, t(59) = , p <.001, r =.82. Metaphoric images were not significantly different from the average score (M =.48, SD =.24, t(59) = -.70, p =.49, r =.008). Difference scores We tested if the addition of text increased or decreased comprehension ratings compared to the monomodal ad. Except for one slightly skewed variable, the data was normally distributed. Furthermore, Cook s distance was < 1 and more than 95% of the residuals were between -2 and 2. The ANOVA showed a significant main effect of image type, indicating that comprehension on multimodal ads increased more on visual metaphors (M =.31, SE =.031) than on literal images (M =.098, SE =.035), F(1,59) = 38.51, p <.001, ω 2 =.41. Furthermore, a significant main effect of text type implicated that the addition of a literal text increased comprehension ratings more (M =.24, SE =.030) compared to a metaphoric text (M =.16, SE =.036), F(1,59) = 5.55, p =.022, ω 2 =

29 Finally, an interaction indicated that the addition of a metaphoric text to a literal image increased comprehension ratings the least compared to all other conditions whereas the addition of a metaphoric text to a metaphoric image resulted in the largest increasing of comprehension scores compared to monomodal ads (F(1,59) = 10.96, p =.002, ω 2 =.16). Compared to a completely literal advertisement (M =.19, SD =.29), the difference in comprehension ratings of a completely metaphoric ad was very high (M =.32, SD =.31), but the metaphoric advertisement only showed a slightly higher increased comprehension rating compared to the addition of a literal text to a metaphoric image (M =.29, SD =.28) which is also much higher compared to the addition of either a literal or metaphoric text to a literal image (Figure 10). The addition of a metaphoric text to a literal image clearly shows the least difference in comprehension score compared to the monomodal ad (M =.006, SD =.37). Figure 10. Multiple line graph of the comprehension ratings for monomodal and multimodal advertisements Correlation Viewing times were correlated with comprehension ratings. Only viewing times of literal ads significantly correlated with comprehension ratings of monomodal ads, r = -.30, p =.022, 95% BCa [.011,.69], and multimodal ads, r = -.27, p =.036, 95% BCa [-.51, -.024]. This suggested that shorter viewing times aligned with comprehension scores on literal advertisements. Viewing times did not correlate with comprehension ratings for any other monomodal (rs <.1, ps >.44) or multimodal ads (rs <.18, ps >.18). 29

30 Aesthetic pleasure Cronbach s alpha showed high reliability, which allowed us to combine the 2 differentials that tested for aesthetic pleasure into a mean score. Table 3 shows the reliability per condition. Table 3. Reliability score for aesthetic pleasure per condition Condition Cronbach s Alpha (α) Literal image, Literal text.91 Literal image, Metaphoric text.90 Metaphoric image, Literal text.92 Metaphoric image, Metaphoric text.86 To measure the effect of text type and image type on aesthetic pleasure, a factorial ANOVA with repeated measures was performed. Except for one slightly skewed variable, all data was normally distributed. No more than 7.5% of the residuals ranged outside -3 and 3, and Cook s distance was < 1. The results show a marginally significant main effect of image type on aesthetic pleasure, F (1, 83) = 3.94, p =.05, ω 2 =.05, implying that the ads with a literal image (M = 3.68, SE =.08) were considered as less aesthetically pleasurable than advertisements with a metaphoric image (M = 3.84, SE =.08). No significant difference of text type was found (F(1,83) =.05, p =.83, ω 2 =.001). Therefore, a metaphoric text (M = 3.77, SE =.07) is not considered as more aesthetically pleasurable than a literal text (M = 3.75, SD =.07). No significant interaction appeared between text type and image type (F(1,83) = 2.10, p =.15, ω 2 =.03). Table 4 shows the mean and standard deviation of each condition. 30

31 Table 4. Mean and Standard Deviation of the aesthetic pleasure ratings on the multimodal advertisements Condition M SD Literal image, Literal text Literal image, Metaphoric text Metaphoric image, Literal text Metaphoric image, Metaphoric text Processing fluency To test for a possible effect of text and image type on processing fluency, again a factorial ANOVA with repeated measures was performed. The assumption of normality was met, no more than 5% of the standardized residuals crossed -2 or 2, and Cook s distance was <1. A main effect of text type indicated that a literal text within a multimodal ad evokes higher processing fluency (M = 4.74, SD =.09) compared to metaphoric text (M = 4.46, SD =.09), F(1,83) = 11.45, p =.001, ω 2 =.12. Furthermore, a marginally significant main effect of image type, F(1,83) = 3.09, p =.08, ω 2 =.04, indicated that literal images in a multimodal advertisement are slightly easier to process (M = 4.69, SD =.09) than metaphoric images (M = 4.51, SD =.09). Figure 11. Multiple line graph of the interaction effect between text and image on processing fluency 31

32 An interaction effect between text type and image type (F(1,83) = 7.61, p =.007, ω 2 =.08) suggested that advertisements with a literal text and literal image (M = 4.95, SD = 1.09) are processed more fluently whereas advertisements with a literal text and a metaphoric image (M = 4.43, SD =.97) caused less processing fluency. The opposite, though smaller in difference, effect appeared for metaphoric text combined with either a literal or metaphoric image. Advertisements with a metaphoric text and literal image (M = 4.52, SD = 1.01) are slightly less easy to process compared to advertisements with a metaphoric text and metaphoric image (M = 4.50, SD = 1.00). Figure 11 visualizes the direction of the interaction effect. Mediation effect Correlation To investigate a possible mediation effect of processing fluency on aesthetic pleasure, and a possible moderation effect of comprehension on aesthetic pleasure, these variables were correlated. Figure 12 provides an overview of the correlations between the variables. Except for a negative correlation of advertisements with a literal image and a metaphoric text, viewing time did not correlate with aesthetic pleasure. Comprehension, however, positively correlated in most cases. Only completely literal advertisements did not correlate with aesthetic pleasure. Therefore, there could be a chance that comprehension moderates aesthetic pleasure. Finally, processing fluency correlated with all four conditions. Besides completely literal advertisements, which showed a medium positive relationship, all other conditions positively correlated with a large effect size. This outcome points at a possible mediation effect between processing fluency and aesthetic pleasure. 32

33 Figure 12. Correlation coefficients for viewing time, comprehension, and processing fluency on aesthetic pleasure. Dashed line = negative correlation, thick solid line = positive correlation, thin solid line = not significant; *p <.05, **p <.005, ***p <.001 Mediation analysis The outcome of the three effects of the 8 mediation analyses is shown in Table 5. Because only 2 participants had a mean comprehension score of <.5 on advertisements with a literal text, this comparison did not allow for mediation analysis. Even though correlations pointed at a possible mediation effect, no mediation effect appeared when the ad was not comprehended. Only the comparison between advertisements with a visual and verbal metaphor and advertisements with a literal image and verbal metaphor indicated a significant total prediction on aesthetic pleasure (b = -.46). This total effect implicates that aesthetic pleasure may be negatively influenced via both pathways (i.e. via processing fluency as well as directly) by the difference in image type when a metaphoric text is added. 33

34 For comprehended advertisements, a significant direct effect (b = -.30) of the comparison between advertisements with a literal text appeared on aesthetic pleasure. The outcome of the direct effect suggests that the difference in image type negatively influences aesthetic pleasure when a literal text is added. The indirect effect (b =.21), however, hints at a positive mediation effect which suppresses the direct effect and clarifies the non-significant total effect (CI [-.58,.017]). A dependent t-test on processing fluency significantly showed that the completely literal ad (M = 5.10, SD = 1.09) was easier to process than the ad with a literal text and a visual metaphor (M = 4.56, SD = 1.01), t(58) = 3.05, p =.003, when the ad was comprehended. Therefore, when the advertisement was comprehended, metaphoric images were not particularly considered as more aesthetically pleasurable than literal images, because the aesthetic pleasure was suppressed by processing fluency, which was rated higher for literal images compared to visual metaphors. 34

35 Table 5. Outcome of the mediation analyses on the 2(image type) x 2(text type) comparisons with the data split by comprehension. Definition per condition: VlTl (Visual literal/text literal); VlTm (Visual literal/text metaphor); VmTl (Visual metaphor/text literal); VmTm (Visual metaphor/tekst metaphor). Red items indicate a significant effect. Total effect Direct effect Indirect effect No Comprehension Comprehension VlTl-VlTm VlTl-VlTm b =.11, t(7) =,29, p =.78, CI [-.78, 1.01] b = -.25, t(5) = -1.11, p =.32, CI [-.82,.33] b =.36, CI [-.35,.91] VlTl-VmTl ERROR (sample < 3) ERROR (sample < 3) ERROR (sample < 3) VlTm-VmTm b = -.46, t(10) = -2.48, p =.03, CI [-.87, -.046] VmTl-VmTm b = -.18, t(3) = -1.51, p =.23, CI [-.55,.20] b =.082, t(52) =.77, p =.44, CI [-.13,.29] VlTl-VmTl b = -.095, t(58) = -.70, p =.49, CI [-.58,.017] VlTm-VmTm b = -.22, t(45) = -1.88, p =.066, CI [-.45,.015] VmTl-VmTm b = -.035, t(58) = -.39, p =.70, CI [-.21,.14] b = -.40, t(8) = -2.19, p =.06, CI [-.82,.021] b = -.18, t(1) = , p =.03, CI [-.24, -.13] b =.016, t(50) =.14, p =.87, CI [-.21,.24] b = -.30, t(56) = -2.47, p =.017, CI [-.55, -.057] b = -.22, t(43) = , p =.04, CI [-.42, -.011] b = -.034, t(56) = -.43, p =.67, CI [ ] b = -.055, CI [-.32,.22] b =.006, CI [-.01,.03] b =.066, CI [-.019,.19] b =.21, CI [.060,.40] b = -.003, CI [-.12,.11] b = , CI [-.082,.093] 35

36 Conclusion and discussion This experiment investigated the comprehension, aesthetic pleasure, and processing fluency of multimodal metaphoric advertisements. The main results show that metaphors are harder to process and comprehend than literal messages. However, adding text increases comprehension on metaphoric images more than on literal images. Moreover, adding a literal or metaphoric text to visual metaphors does not evoke large differences in comprehension. On the contrary, for literal images, adding a metaphoric text makes the ad less comprehensible compared to a literal text. Regarding aesthetic pleasure, visual metaphors scored higher than literal images. Finally, the comparison between advertisements with a literal text when the ad was comprehended pointed at a possible mediation effect of processing fluency on aesthetic pleasure. This section further explains the experimental findings. Per dependent variable, the hypotheses will be addressed and discussed based on the current results and previous research. In the end, limitations of the current study, directions for future research, and practical implications will be provided. Comprehension Monomodal advertisements Participants indicated the literal visual advertisement as more comprehensible than the visual metaphoric advertisement as predicted by H1. However, literal visual ads were viewed longer than the visual metaphoric ads. This implicates that metaphoric images were considered as less comprehensible, which made the participants spend less attention to visual metaphors in order to comprehend the message compared to the literal visual ad. A reason for this finding could be that the replacement metaphor, which is visualized in the ad, offers no relevant pictorial elements and therefore provokes misunderstanding (Lagerwerf et al., 2012). As a result, the lack of relevancy demotivates the participant to process the image thoroughly compared to a literal image. The lack of relevant elements could also explain the low comprehension scores on the fillers. The difference between the fillers and the metaphoric ads, though, is that the fillers showed irrelevant incongruences whereas the incongruences depicted in metaphors should eventually contribute to the message of the ad in order to formulate message-congruent impressions. These impressions are relevant for processing and comprehending the message. In other words, the reader needs to find a thematic relationship (Heckler, & Childers, 2009). Two dimensions that are of main importance to find a thematic relationship are relevancy and 36

37 expectancy. A text or image is considered as relevant when the underlying information that is communicated directly contributes to the message. Expectancy derives from pre-existing knowledge structures which help to cause a logical pattern of the information that is evoked by the message (Heckler et al., 1992). Because the fillers were the only advertisements that scored below average on comprehension, this indicates that fillers were considered as irrelevant incongruences, and therefore served well in order to distract the participants from the study s main goal. Multimodal advertisements Image type How text affects the comprehensibility of metaphoric images will be further discussed in this subsection. When text was added, participants took more time to comprehend the metaphoric image compared to the ad with a literal image. This result is consistent with the greater amount of time people spent on watching the visual metaphor in the multimodal ad compared to the monomodal ad. The viewing time on literal images largely decreased compared to the monomodal ad. The addition of literal text explains the metaphoric image, and metaphoric text provides a clue by mentioning the target and source domain. This addition stimulates participants to elaborate more on the metaphoric image (Lagerwerf, 2012; Philips, 2000) which clarifies the increased viewing time. For literal images the addition of text does not immediately provide more comprehensibility, because the image itself is already easy to comprehend. Therefore, people might spend less time on, or not even pay attention to, the added text which decreases the time required to view the multimodal ad compared to the monomodal ad. However, no significant differences in comprehension ratings were found between literal and metaphoric images. This outcome is the opposite of the results of the monomodal ads regarding comprehension. Yet, the difference in comprehension ratings of metaphoric images was larger compared to literal images. This indicates that the comprehension of metaphoric images largely increased after the addition of text whereas comprehension of literal images only slightly increased. This effect is also known as anchoring, where adding a full explanation or providing a clue increases comprehension of the message, because of the addition of verbal relevant elements (Phillips, 2000). Here, text anchoring yields more motivation to elaborate on the message of a metaphoric image, which increases comprehension time, and results in greater comprehension to such a degree that message 37

38 comprehension of literal and metaphoric images do not significantly differ anymore. In addition, due to a deficiency of pictorial relevant elements, the metaphoric monomodal ad could have evoked inconsistencies with the reader s comprehension of real-world knowledge (Sitnikova et al., 2008). The addition of text allowed the reader to evaluate and update the current message (i.e. text anchoring requires a revision process), which took more time to process than the monomodal ad (Sitnikova et al., 2008; Mudrik et al., 2009). These findings contradict other studies which show that text anchoring increases comprehension, but decreases cognitive elaboration. Van Enschot et al. (2015) and Philips (2000) both found that the more explanatory information was given, the higher the comprehension of the ad would be. However, full explanation framed the intended message which resolved the challenge of solving the riddle, and eventually resulted in less elaboration (Ang et al., 2006). Jeong (2008) found similar results and related them to the elaboration likelihood model (Petty, & Cacioppo, 1986) and the heuristic-systematic model (Eagly, & Chaiken, 1993). These models indicate that audience factors, such as need for cognition and personal relevance, lead to greater involvement in cognitive elaboration. The challenging aspect of a metaphor motivates people to comprehend the message which increases need for cognition. The metaphor s incongruence evokes a lack of personal relevancy which increases elaboration in order to make it more relevant (Jeong, 2008). Because the current study found that viewing time increased after text anchoring, this result could indicate that text anchoring does evoke more elaboration in general due to more motivation. However, the current result only reveals a main effect of image, but does not distinguish the two types of verbal anchoring. Therefore, text anchoring could yield different levels of elaboration. This possibility will be discussed later on in this chapter. Text type Based on the comprehension ratings, a literal text is easier to comprehend than a metaphoric text. Therefore, using a literal text within an advertisement increases the comprehension of the intended message. This result confirms H1, which states that literal messages are easier to comprehend than metaphoric messages, also on a verbal level. This conclusion can be explained by the difference in characteristics between metaphoric and literal images. As already mentioned, a literal text fully explains a metaphoric image whereas a metaphoric text adds a clue to a metaphoric message. Adding a metaphoric text to a literal image, however, requires more cognitive effort, because now the pictorial relevant elements of the image are added with a verbal irrelevant element (Langerwerf et al., 2012). This verbal challenging is 38

39 in fact the opposite of verbal anchoring. Whereas verbal anchoring increases comprehension, verbal challenging decreases comprehension. Thus, adding a verbal metaphor to a visual metaphor serves as verbal anchoring, and adding a verbal metaphor to a literal image serves as verbal challenging. However, literal text increases comprehension more than metaphoric text in both cases, so a verbal metaphor, regardless of the image, will require more cognitive effort compared to a literal text. The nonsignificant difference in viewing time between metaphoric and literal text could be derived from the fact that images are dominant to text. The attention-grabbing ability of images are superior to text, which make people focus on the image first, and will take time to elaborate on it while text is used to clarify the message (Mackenzie, 1986; Ang et al., 2006; Eco, 1976; Marchland, 1985). Therefore, viewing time more or less depends on the image instead of on the verbalization which causes the nonsignificant difference in text. Interaction of text and image type According to H8a, a completely literal ad is more comprehensible than an ad with a literal text and metaphoric image which is in line with the experimental outcome. However, a metaphoric text with a literal image is not more comprehensible than a completely metaphoric advertisement. In fact, a completely metaphoric advertisement is even considered as slightly more comprehensible. These results suggest that the addition of a metaphoric text to a literal image does indeed make the advertisement less comprehensible compared to a literal image only. The verbal metaphor suddenly adds a comparison that is not visualized which requires more cognitive effort in order to figure out the intended meaning. Regarding verbal anchoring, H7a states that adding a literal text to a pictorial metaphor increases comprehension more than when a verbal metaphor is added. However, the visual metaphor added with the same textual metaphor does not extremely increase the complexity. It is even considered as slightly more comprehensible than adding a full literal explanation to the metaphor, which rejects the hypothesis. Two possible explanations can be derived from this result. First, people are more motivated to comprehend the message when provided with a clue compared to a full explanation (McQuarrie et al., 1999; Goodstein, 1993; Ang et al., 2006; Morgan et al., 1999). The verbal metaphor reveals the target domain, which serves as a clue to find the resemblance between the conceptual domains, but does not provide full explanation which is the case when a literal text is added. Adding a clue allows for multiple perspectives and interpretations that trigger motivation to solve the riddle in contrast to a literal text (Lubart et al., 1997; McQuarrie et al., 2005). As mentioned before, the 39

40 addition of different kinds of text could lead to different levels of elaboration, which is confirmed by this explanation. Revealing the target domain provides room for own interpretations, but revealing the whole context completely frames the message which limits the amount of elaboration (Ang et al., 2008; Lubart et al., 1997; McQuarrie et al., 2005). A remarkable, though not significant, result that contradicts this explanation is that the viewing time of the ad with the visual metaphor and literal text was slightly larger than the viewing time of the completely metaphoric ad. Therefore, another possible reason for the increased comprehension and decreased viewing time of the metaphoric ad compared to the advertisement with a visual metaphor and a literal text could be that the participants did not comprehend the mapped feature of both domains in some scenarios which makes literal explanation incongruent to the visual metaphor (Heckler et al., 1992). When provided with a verbal metaphor, participants are allowed to seek for a mapped feature based on their own knowledge and experience, so the advertisement fulfils expectancy and relevancy which increases comprehension (Heckler et al., 1992). Aesthetic pleasure H2 predicted that metaphoric messages yield more aesthetic pleasure than literal messages. The results show that metaphoric images were considered as more aesthetically pleasurable than literal images, and therefore confirm H2, but only on a visual level. The reason for this single main effect could depend on the attention-grabbing ability of images that puts focus on the image s aesthetics rather than the verbalization (Mackenzie, 1986; Ang et al., 2006). In fact, text mainly serves as an explanation of the communicated image, so aesthetics are less of interest (Anaya, 2008; Baker, 1985). H6, which states that pictorial metaphors evoke higher aesthetic pleasure compared to verbal metaphors, is therefore confirmed. Compared to literal images, metaphoric images increase creativity (Phillips et al., 2002). Due to this creative aspect, visual metaphors yield multiple interpretations which allow for elaboration. Metaphors evoke interest, curiosity and attention, which all positively contribute to the advertisement s aesthetic pleasure (McQuarrie et al., 2003; McQuarrie et al., 1999; Goodstein, 1993; Ang et al., 2006; Morgan et al., 1999). Literal images lack these aspects, and are therefore considered as less aesthetically pleasurable than metaphoric images. The completely metaphoric ad was only slightly more aesthetically pleasurable than the visual metaphoric advertisement with a literal text. However, this difference was not significant and in such a small degree that we cannot confirm that a completely metaphoric message is more aesthetically pleasurable than an advertisement with a verbal metaphor and 40

41 literal text (H7c). H8c states that completely literal ads evoke less aesthetic pleasure than ads with a literal image and verbal metaphor. However, the opposite effect appeared: on ads with a literal image and metaphoric text, the aesthetic pleasure was slightly, yet not significantly, lower than a completely literal ad. People could consider the addition of an incongruent text as irrelevant rather than creative (Heckler et al., 1992). Due to the irrelevancy, providing people with a challenging element after already being exposed to a straightforward, explanatory element could therefore decrease aesthetic pleasure which rejects H8c. Metaphoric images take longer to comprehend, and are considered as more aesthetically pleasurable than literal images. Including a metaphoric image with text provides an opportunity to solve the riddle (Lagerwerf, 2012; Philips, 2000). Therefore, people are motivated to elaborate on different perspectives, which increases comprehension time, and eventually gains insight into the creativity of the ad which yields aesthetic pleasure (Philips et al., 2002; Lubart et al., 1997; McQuarrie et al., 2005; McQuarrie et al., 2003). The nonsignificant differences in comprehension ratings show that the level of comprehension does not influence the aesthetic pleasure, which clarifies that no clear differences in mediation effects were found between whether the ad was comprehended or not. Therefore, H5, which predicted a significant moderation effect of comprehension on multimodal metaphoric ads and aesthetic pleasure, is rejected. Comprehension namely does not moderate aesthetic pleasure at least in this study. Processing fluency In H3 we predicted that metaphoric messages are harder to process than literal messages. The results of processing fluency align with the results of comprehension. Literal text and images evoke higher processing fluency compared to metaphoric text and images due to the full, straightforward explanation the literal message provides. Thus, H3 is confirmed. H8b more specifically predicts that completely literal images are easier to process than advertisements with a literal image and a verbal metaphor. The inclusion of a challenging aspect, which in this case refers to a metaphor, indeed makes the advertisement more effortful to process. Adding a verbal metaphor increases cognitive effort, and therefore evokes less fluent processing of the advertisement compared to the addition of a literal text to a literal advertisement (McQuarrie et al., 2005), which confirms H8b. H7b states that a completely metaphoric advertisement requires more effort to process than an advertisement with a metaphoric image and a literal text. Adding a metaphoric text to a metaphoric image, however, is not immediately more challenging, but rather serves as a clue 41

42 which makes processing more convenient. The latter can be explained by taking the inverted U-curve of van Mulken et al. (2014) into account. The visual metaphoric advertisement contained too little relevant pictorial elements in order to easily process the advertisement. When adding a clue, one reaches a tipping point where the advertisement is still challenging, but also solvable enough which increases processing fluency (Philips et al., 2004; van Mulken et al., 2014). The results of completely metaphoric ads and an advertisement with a pictorial metaphor and a literal text on processing fluency were similar to the results on comprehension. The same possible explanations are addable to this result; people are more motived to solve the riddle (Lubart et al., 1997; McQuarrie et al., 2005), plus they are allowed to seek a mapped feature that is based on their own expectations (Heckler et al., 1992). As a result, H7b is rejected. Whether verbal and pictorial metaphors differ in processing fluency (Q4), therefore strongly depends on the combination between literal and metaphoric text and images within multimodal advertisements. Finally, H4 states that processing fluency mediates the relationship between multimodal metaphoric ads and aesthetic pleasure. Results showed a negative direct result of image type after the addition of a literal text which is suppressed by a positive indirect (i.e. mediation) effect of processing fluency on aesthetic pleasure. This outcome shows that even though metaphoric images may yield higher aesthetic pleasure, the positive impact of processing fluency on aesthetic pleasure, which is rated higher for literal images, suppresses this direct effect which causes a nonsignificant total effect of the comparison between completely literal advertisements and advertisements with a metaphoric image and a literal text on aesthetic pleasure. People therefore appreciate the creativity of a metaphor, but also want to be able to easily process an advertisement. This conclusion is in line with the outcome of van Enschot et al. (2015) who found that even though incomplete anchoring evoked higher perceived likeability, this effect was suppressed by the lower perceived comprehensibility scores which positively mediated perceived likeability, and which were higher in case of complete anchoring. Both outcomes therefore suggest that an ad needs to be creative, yet understandable enough to be solved. Limitations and directions for future research The current study mainly focussed on comprehension. In order to limit the duration of the experiment, only one question was asked concerning processing fluency, and two questions were provided to indicate the aesthetic pleasure. Future research could investigate these two variables in combination with comprehension more thoroughly. For processing pleasure, one 42

43 can use the 4 items that Mohanty & Ratneshwar (2016) conducted in order to test processing fluency, and to derive more solid conclusions in combination with viewing time. To measure aesthetic pleasure, the five 7-point semantic differentials that were used in the study by Van Enschot and Van Hooijdonk (2016) could be implemented. Conducting a study that covers all three variables equally provides stronger evidence for possible correlations, and mediation and moderation effects. Moreover, the within-subject design complicated the analysis on a possible moderation effect since the statistical program MEMORE does not allow for moderation analysis yet. The other limitation refers to the fact that MEMORE only allows for comparisons between two conditions at a time. Therefore, future research could implement a between-subject design in order to derive more valid conclusions concerning moderation and mediation via the statistical program PROCESS. This program is capable of performing a stronger in-depth mediation and moderation analysis, so more solid conclusions can be derived from the use of multimodal metaphors on comprehension, processing fluency, and aesthetic pleasure. Another remark that refers to the experimental design, is that participants were only asked to answer whether they comprehended the message with yes or no, but it remains unclear if they indeed fully understood the intended message one was trying to communicate. Therefore, future research should ask more in-depth questions in the form of a multiple choice question or an open question in order to analyse if the participant comprehended the message correctly which positively contributes to the content validity. This study not only showed the consequences of verbal anchoring, but also introduced a new phenomenon called verbal challenging. In order to further investigate these two phenomena, it would be interesting to investigate whether differences occur when people are first exposed to text which is anchored or challenged visually. Van Enschot et al. (2015) investigated the effect of verbal and visual anchoring on TV-commercials and found different outcomes between visual and verbal anchoring. Complete verbal anchoring yielded less product appreciation whereas product appreciation did not decrease when complete visual anchoring was used. This difference can be explained by the fact that visual anchoring still allows for elaboration. Verbal anchoring immediately reveals the intended interpretation which decreases ad appreciation (van Enschot et al., 2015). Regarding comprehension, when the reader is first exposed to a text, they first need to create an image. When the text adds an image that is specifically chosen by the advertiser in order to communicate the intended message, readers are able to align the text with the image which should result into comprehension that is more congruent with the intended message compared to the use of text 43

44 only (Morgan et al., 1999). Future research could more deeply investigate visual anchoring on print ads by performing a similar experiment, but in a reversed order to analyze if verbal metaphors are also more comprehensible when added with a literal image, and if literal text is less comprehensible after the addition of a visual metaphor. Finally, the intended communicative effect where metaphors are used for in the current study mainly refers to persuasion and motivation (Lentz, & Pander Maat, 2004). Due to its creativity, metaphors provide motivation to process and comprehend the message of the ad which should result in for example the purchase of a particular product or service (Ang et al., 2006). Perhaps, in order to evoke more attention to comprehend the message, metaphors can also be applicable to, for example, instructional messages such as medical information. However, the question is whether metaphors yield similar attitudes towards this kind of information as what is shown in the current experiment. Therefore, future research could dive deeper into whether multimodal metaphors evoke similar attitudes when applied to different messages with other communicative effects. Practical implications Overall, the current study proposes three practical implications advertisers should take into account when conducting novel advertisements. Metaphors within advertisements, both verbal and visual, are less comprehensible and less fluently to process than literal messages. Furthermore, a verbal metaphor is not particularly more aesthetically pleasing than a literal text. Therefore, verbal metaphors should only be applied when they serve as an explanatory element. This implies that verbal metaphors are most sufficiently when they are applied as a clue rather than as a challenging element. Visual metaphors, however, increase aesthetic pleasure. Implementing a visual metaphor in an advertisement could therefore demand attention, which is of main importance when persuading someone to buy a particular product or service. A visual metaphor, which increases creativity, makes the advertisement aesthetically pleasurable, and grabs the attention of the reader. Nevertheless, the advertisement still needs to be challenging yet solvable enough for the reader. Therefore, advertisers should keep in mind that, in case of the replacement metaphor, the image should be clarified with text in order to solve the riddle. If the explanatory text should be literal or metaphoric, it depends on whether the advertiser wants to implement multiple metaphors or rather sticks with only one metaphoric element. The current 44

45 study namely showed that a completely metaphoric ad evoked only slightly higher comprehension compared to an ad with a metaphoric image and a literal text. Thus, the main conclusion is that verbal and visual metaphors should not be applied separately, but will have the most positive effect on comprehension, processing fluency, and aesthetic pleasure when they complement each other. Combining a verbal and visual metaphor provides challenge and explanation, which eventually is the most successful key to an attractive, comprehensible ad. 45

46 References Anaya, M. A. (2008). Changing appearances: magazine advertising layout and design, (Unpublished master s thesis). San Jose State University, US. Ang, S. H., & Lim, E. A. C. (2006). The influence of metaphors and product type on brand personality perceptions and attitudes. Journal of Advertising, 35(2), Baker, S. (1988). The advertiser's manual. New York: Wiley. Eagly, A. H., & Chaiken, S. (1993). The psychology of attitudes. San Diego: Harcourt Brace Jovanovich College Publishers. Eco, U. (1976). A Theory of Semiotics. Bloomington: Indiana University Press. Forceville, C Non-verbal and multimodal metaphor in a cognitivist framework: agendas for research. De Gruyter Mouton, Forceville, C. (1994). Pictorial metaphor in advertisements. Metaphor and Symbol, 9(1), Forceville, C. (2014). Relevance Theory as model for analysing visual and multimodal communication. Visual communication, 4, 51. Gaarman, E. (2017). Visual metaphors in advertising. How fluency and comprehension. affect aesthetic pleasure and interest at different complexity levels (Unpublished master s thesis). Tilburg University, the Netherlands. Goodstein, R. C. (1993). Category-based applications and extensions in advertising: Motivating more extensive ad processing. Journal of Consumer Research, 20(1), Hayes, A. F., & Preacher, K. J. (2014). Statistical mediation analysis with a multicategorical independent variable. British Journal of Mathematical and Statistical Psychology, 67(3), Heckler, S. E., & Childers, T. L. (1992). The role of expectancy and relevancy in memory for verbal and visual information: what is incongruency?. Journal of Consumer Research, 18(4), Hekkert, P., Snelders, D., & van Wieringen, P.C. (2003). Most advanced, yet acceptable : Typicality and novelty as joint predictors of aesthetic preference in industrial design. British journal of psychology, 94(1), Janiszewski, C. (1990). The influence of print advertisement organization on affect toward a brand name. Journal of Consumer Research, 17, Jeong, S. H. (2008). Visual metaphor in advertising: Is the persuasive effect attributable to visual argumentation or metaphorical rhetoric?. Journal of Marketing Communications, 14(1),

47 Lakoff, G. (1993). The contemporary theory of metaphor. In Metaphor and thought (pp ). New York, NY: Cambridge University Press. Lakoff, G., & Johnson, M. (1980). Conceptual metaphor in everyday language. The journal of Philosophy, 77(8), Lakoff, G., & Johnson, M. (1980). Metaphors we live by. Chicago: University of Chicago Press. Lagerwerf, L., van Hooijdonk, C. M., & Korenberg, A. (2012). Processing visual rhetoric in advertisements: Interpretations determined by verbal anchoring and visual structure. Journal of Pragmatics, 44(13), Landwehr, J.R., Labroo, A.A., & Herrmann, A. (2011). Gut liking for the ordinary: Incorporating design fluency improves automobile sales forecasts. Marketing Science, 30(3), Lentz, L., & Pander Maat, H. (2004). Functional analysis for document design. Technical communication, 51(3), Lubart, T. I., & Getz, I. (1997). Emotion, metaphor, and the creative process. Creativity research journal, 10(4), Marchand, R. (1985). Advertising and the American Dream: Making Way for Modernity. Berkeley: University of California Press. Marschark, M., Katz, A. N., & Paivio, A. (1983). Dimensions of metaphor. Journal of Psycholinguistic Research, 12(1), McQuarrie, E. F., & Mick, D. G. (1999). Visual rhetoric in advertising: Text-interpretive, experimental, and reader-response analyses. Journal of consumer research, 26(1), McQuarrie, E. F., & Mick, D. G. (2003). Visual and verbal rhetorical figures under directed processing versus incidental exposure to advertising. Journal of consumer research, 29(4), McQuarrie, E. F., & Phillips, B. J. (2005). Indirect persuasion in advertising: How consumers process metaphors presented in pictures and words. Journal of advertising, 34(2), Messaris, Paul (1997), Visual Persuasion: The Role of Images in Advertising. Communications, 23(3), Mohanty, P., & Ratneshwar, S. (2016). Visual Metaphors in Ads: The Inverted-U Effects of Incongruity on Processing Pleasure and Ad Effectiveness. Journal of Promotion Management, 22(3),

48 Morgan, S. E., & Reichert, T. (1999). The message is in the metaphor: Assessing the comprehension of metaphors in advertisements. Journal of Advertising, 28(4), Mudrik, L., Lamy, D., & Deouell, L. Y. (2010). ERP evidence for context congruity effects during simultaneous object scene processing. Neuropsychologia, 48(2), Ortiz, M. J., Murcia, M. G., & Fernandez, E. (2017). Brain processing of visual metaphors: An electrophysiological study. Brain and Cognition, 113, Petty, R. E. & Cacioppo, J. T. (1986) Communication and Persuasion: Central and Peripheral Routes to Attitude Change. In Communication and persuasion (pp ). New York, NY: Springer Verlag Ney York Inc. Phillips, B. J. (2000). The impact of verbal anchoring on consumer response to image ads. Journal of advertising, 29(1), Phillips, B. J., & McQuarrie, E. F. (2002). The development, change, and transformation of rhetorical style in magazine advertisements Journal of Advertising, 31(4), Phillips, B. J., & McQuarrie E. F. (2004) "Beyond visual metaphor: A new typology of visual rhetoric in advertising " Marketing theory, 4(1-2), Pollay, R. (1985) The Subsidizing Sizzle: A Descriptive History of Print Advertising, , Journal of Marketing, 48(Summer), Reber, R. (2002). Reasons for the preference for symmetry. Behavioral and Brain Sciences, 25, doi: /s x Reber, R., Schwarz, N., & Winkielman, P. (2004). Processing fluency and aesthetic pleasure: Is beauty in the perceiver's processing experience?. Personality and social psychology review, 8(4), Reber, R., Winkielman, P., & Schwarz, N. (1998). Effects of perceptual fluency on affective judgments. Psychological Science, 9(1), Reber, R., Wurtz, P., & Zimmermann, T.D. (2004). Exploring fringe consciousness: The subjective experience of perceptual fluency and its objective bases. Consciousness and cognition, 13(1), Rompay, T. J., & Veltkamp, M. (2014). Product packaging metaphors: Effects of ambiguity and explanatory information on consumer appreciation and brand perception. Psychology & marketing, 31(6), Schilperoord, J. (2016). Ways with Pictures: Visual incongruities and metaphor (Manuscript to appear). Tilburg University, the Netherlands. Sitnikova, T., Holcomb, P. J., Kiyonaga, K. A., & Kuperberg, G. R. (2008). Two 48

49 neurocognitive mechanisms of semantic integration during the comprehension of visual real-world events. Journal of cognitive neuroscience, 20(11), Smith, R. A. (1991). "The Effects of Visual and Verbal Advertising Information on Consumers' Inferences." Journal of Advertising, 20(4), Stern, B. B. (1990). Beauty and joy in metaphorical advertising: The poetic dimension. Association for Consumer Research, 17, Tanaka, K. (1999). Advertising language: A pragmatic approach to advertisements in Britain and Japan. London: Routledge. Toncar, M., & Munch, J. (2001). Consumer responses to tropes in print advertising. Journal of Advertising, 30(1), Trezova, L. (2015). Metaphors in psychotherapy and coaching [PowerPoint slides]. Retrieved from Van der Horst, S. (2017). Adding context to metaphors. Research into the influence of exposure time on felt fluency, processing pleasure, and aesthetic pleasure of different metaphor structures with and without context (Unpublished master s thesis). Tilburg University, the Netherlands. Van den Reek, A. (2017). The effects of visual metaphor in advertising at different processing routes. Testing the Pleasure-Interest Model of Aesthetic Liking for different metaphor structures at a long and short exposure time (Unpublished master s thesis). Tilburg University, the Netherlands. van Enschot, R., Broekhuizen, E., & Kolthoff, M. (2015). De reclamemaker centraal. Tijdschrift voor Taalbeheersing, 37(3), van Enschot, R., & Hoeken, H. (2015). The occurrence and effects of verbal and visual anchoring of tropes on the perceived comprehensibility and liking of TV commercials. Journal of Advertising, 44(1), Van Enschot, R., & Van Hooijdonk, C. (2016). The pleasure of processing visual metaphor in advertising. International Journal of Marketing & Business Communication, 5(4), van Mulken, M., van Hooft, A., & Nederstigt, U. (2014). Finding the tipping point: Visual metaphor and conceptual complexity in advertising. Journal of Advertising, 43(4),

50 Appendix 1: Brand name congruency scores Table 6. Mean and standard deviation of the brand name congruency scores per scenario. Red items indicate incongruent scores. Product Mean Standard Deviation Car Bathroom cleaner Car refresher Condom Drumkit Energy drink Suitcase Pillow Sanitary napkin Belt Sauna Chair Duster Toilet paper Tablet Detergent Alarm Bike Wineglass Sunblock

51 Appendix 2: Stimuli experiment Scenario 1: NextTime Scenario 2: AllClean 51

52 Scenario 3: Gizelle Scenario 4: Road Driver 52

53 Scenario 5: Purely Scenario 6: Armano 53

54 Scenario 7: Durix Scenario 8: Golden Sound 54

55 Scenario 9: GoodNight Scenario 10: Hotwood 55

56 Scenario 11: Travelin Scenario 12: Megasun 56

57 Scenario 13: Dusty Scenario 14: Calgen 57

58 Scenario 15: Pagi Scenario 16: Kick-it 58

59 Fillers 59