Book review. visual communication

Transcription

1 668684VCJ / Visual Communication research-article2016 visual communication Arianna Maiorani and Christine Christie (eds), Multimodal Epistemologies: Towards an Integrated Framework. New York: Routledge, 2014; 294 pp.; ISBN (hbk); ISBN (ebk) Multimodal Epistemologies: Towards an Integrated Framework, edited by Arianna Maiorani and Christine Christie, is a 2014 addition to the Routledge Studies in Multimodality series, edited by Kay O Halloran. This volume features 16 chapters selected by the editors based on papers originally given at a conference they organized titled Analysing Multimodal Discourse: SFL [systemic functional linguistics] Meets Pragmatics at the English and Drama Department of Loughborough University (UK) in September The aim of this conference, and in turn the papers assembled in this collection, is to develop a new, comprehensive, more flexible and adaptable epistemology (p. 1) for multimodal research, capable of capturing rapid technological and cultural developments in such areas as new media communication and interactive gaming and learning. Such developments have, in the editors words, challenged the relevance and scope of the epistemologies that have until now defined and informed the study of communication and discourse (p. 1), specifically SFL (Halliday and Matthiessen, 2004), which has strongly influenced work in multimodality to date, including the pioneering work by Gunther Kress and Theo van Leeuwen (Kress, 2010; Kress and Van Leeuwen, 2006; Van Leeuwen, 2005) and influences, to a greater and lesser extent, the chapters in this collection. By synthesizing a Hallidayan approach with aspects of other interdisciplinary approaches to multimodal discourse, each chapter demonstrates how bottom-up approaches offered by SFL can be fruitfully combined with more top-down perspectives to enrich the analysis of multimodal texts, in so doing answering appeals made by the likes of Forceville (2009), that strictly Hallidayan approaches risk leading to overly-descriptive accounts of multimodal texts. The 16 analytical chapters in this volume are resolved into three parts, based on three ways of interpreting multimodality as an approach to knowledge (pp. 1 2); these are: (1) multimodality as a semiotic perspective, (2) multimodality as a tool for cultural research, and (3) multimodality as a way to analyse contemporary narrative processes. Visual Communication 2016 Vol. 0(0) The Author(s) 2016 Reprints and permissions: sagepub.co.uk/journalspermissions.nav DOI /

2 The first section, titled Multimodality as a Semiotic Perspective contains six chapters which adopt complex and multi-faceted analytical frameworks in order to decipher the various semiotic mechanisms at work in contemporary multimodal communication. The section begins with a chapter based on an experimental study by Bergh and Beelders, who combine approaches to cognition, metaphor and multimodal discourse in order to demonstrate the significance of human bodies and faces as conceptual archetypes in advertising texts. In Chapter 2, Ozga integrates multimodal, semantic and pragmatic approaches in order to explore how demotivators that is, a conjunction of an eloquent picture or photo with a caption which comments on its content usually produc[ing] an ironical effect (p. 28) function as multimodal communicative acts in a corpus of user-generated demotivators taken from Facebook. In Chapter 3, Pagani contributes to the under-researched topic of multimodal legitimation. The author provides a case study in which he integrates linguistic and visual analysis frameworks with a material analysis framework to show how Norwich Bus Station is linguistically and visually legitimized in local council documents and in the architectural features of the building itself. In the fourth chapter, Mazzali-Lurati and Pollaroli fruitfully apply Congruitive Theory (Rigotti, 2005) to the analysis of a Nike Air Print advertisement in order to explicate, among other things, the argumentative and rhetorical functions of the text as accomplished multimodally. In Chapter 5, Riley undertakes a systemic-functional semiotic analysis of art school drawings, arguing that the shift from representationalism towards abstraction in European art history can be explained in Hallidayan terms, with recognition of the potential of the material qualities of paint to carry negotiable meanings. In the sixth and final chapter of this section, Starc develops a model of discourse analysis informed by, amongst others, Hoey s (2001) text pattern theory, Halliday s (2002) theory of text and discourse, Van Leeuwen s social semiotic theory of communication (2005) and Kress and Van Leeuwen s visual grammar (2006) to examine historical newspaper advertisements. The second section, titled Multimodality as a Tool for Cultural Research contains five chapters featuring studies that adopt multimodal approaches to examine how cultural phenomena are created, elaborated and consumed in sociocultural contexts (p. 6). In the first chapter in this section, Christie (one of the volume editors) fuses a pragmatic approach to relevance (Sperber and Wilson, 1995[1986]) with a multimodal approach to metonymy in order to investigate how power is indexed in the prologue to Kazuo Ishiguro s (1989) novel The Remains of the Day and its film adaptation. In Chapter 8, Krasni examines the construal of collective memory in internet representations of Second World War bombings of the cities of Coventry, Belgrade and Dresden. Integrating the frameworks of Key Visuals (Kramer and Ludes, 2010) and Kress and Van Leeuwen s (2006) visual grammar, the analysis demonstrates the importance of collective public remembrance for the construction of, amongst other things, identity within a particular cultural setting. The ninth chapter 140 Visual Communication 16(1)

3 in this collection features a study by Rinn, who applies a multimodal rhetorical analysis to Holocaust denial websites, revealing how semiotic resources are carefully arranged to attempt to convince the audience that the atrocities committed by the Nazis during the Second World War did not happen. In Chapter 10, Trklja combines semiotic theory with corpus linguistics and narratology to examine semantic change across 500 versions of the Uncle Sam Wants You US Army recruitment poster, finding, for instance, that modified versions of the poster adopt a tone that is less persuasive but more obligatory than the original. The final chapter of section two introduces an innovative approach to the study of literary translation that integrates methods from translation studies and visual analysis to account not only for linguistic translation, but also for identification of similarities and differences between the illustrations accompanying the original text and its translation. Comparing Rudyard Kipling s (1899) The Jungle Book and its (1928) Italian translation, Turci identifies how semiotic resources are organized in such a way as to reduce the effect of foreignisation (p. 182) by rendering the jungle in an artistic style more familiar to Italian readers. The final analytical section, titled Multimodality as a Way to Analyse Contemporary Narrative Processes contains five chapters which adopt a variety of mixed-methods approaches to examine contemporary narratives as multimodal texts. The initial chapter in this section features a study by Freddi and Malagori of discourse markers in the Italian dubbing of English films. Combining approaches from corpus linguistics, discourse analysis and translation studies, the authors investigate how discourse markers are translated from English to Italian, demonstrating the difficulty of conveying the sentiment of pragmatic functions in multimodal texts. The 13th chapter is authored by the other editor, Maiorani, who integrates multimodal discourse analysis, systemic functional linguistics and corpus linguistics to examine the semiotics of space in English films and their Italian-dubbed versions. Like its predecessor, this chapter demonstrates the multi-layered impact of dubbing (particularly of richly multimodal texts) and also suggests that online environments may have had an impact on the ways that contemporary films are offered to audiences as interactive multimodal texts. In Chapter 14, Taylor develops a multi-method approach for analysing children s face-to-face interactions in classroom contexts that fuses aspects of linguistic ethnography and anthropology with social semiotics. Focusing in particular on meaning-making in this context, Taylor s analysis reveals the multimodal nature of metaphor as a cohesive device, involving, in particular, such modes as speech, posture and gesture. In the next chapter, Chapter 15, Wahl demonstrates the utility of a computational transcription system for the analysis of multimodal texts. Examining a Mercedes-Benz advertisement as a case study, Wahl s method reveals the narrative functions of various levels of semiosis, excluding language, which does not contribute to the telling of the on-screen story as such, but plays a mediating role, offering the audience a particular interpretation of the narrative. In the 141

4 16th and final analytical chapter in this volume, Wildfeuer combines aspects of pragmatics and discourse semantics to examine the logic of film discourse interpretation (p. 260), showing that the inference of meanings conveyed through combinations of semiotic resources is contingent upon the organization of those resources adhering to a particular cultural logic. This is a strong book that offers an accessible, flexible range of tools that can be used to analyse a broad range of multimodal texts across an even broader range of cultural contexts. The book s most telling contribution is to research that sits at the interface of traditional multimodality and other approaches to text analysis, linguistic and otherwise. Indeed, the argument that multimodal approaches particularly those influenced by SFL can be greatly enriched when combined with other, usually top-down (but also bottom-up) approaches is made quite convincingly throughout the volume. In this respect, this book makes an innovative contribution to interdisciplinary multimodal research, incorporating an impressive range of perspectives into multimodal analysis, including from aspects of pragmatics, semantics, corpus linguistics, translation studies and cognitive linguistics, all underpinned by a varying level of commitment to SFL. There has, to my knowledge, been little attempt to integrate multimodal discourse analytic approaches with many of these perspectives, for instance pragmatics, as Christie does in Chapter 7, or translation studies, as Turci and then Freddi and Malagori do in Chapters 11 and 12, respectively. Moreover, the chapters in this book also consider a wide range of multimodal text types, from the Nike Air Print advertisement analysed by Mazzali-Lurati and Pollaroli in Chapter 4, to the architectural features of Norwich Bus Station examined by Pagani in Chapter 3. The strong emphases placed on interdisciplinarity, mixed-methods and broad notions of multimodality across a wide range of texts mean that, collectively, the chapters in this volume provide abundant evidence of how not just SFL but a range of approaches to discourse can be fruitfully combined to enrich analysis, yielding more sophisticated and, I would suggest, context-sensitive accounts of a wide range of multimodal texts. Although I have argued that the range of text types analysed in this book is wide, I do feel that the flexibility of the tools introduced could have been demonstrated more emphatically with the inclusion of an even broader range of multimodal data, in particular complex data that present challenges to scholars interested in multimodality at the cutting edge of technological developments, such as fast-paced advancements in social media and mobile communication. The inclusion of such data might have fostered a more explicit commentary on the evolution and adaptation of methods for multimodal analysis to meet the demands of increasingly complex multimodal texts, potentially with some reflection on and speculation as to how such methods should develop to capture the increasingly multimodal landscape of everyday communication in the future. 142 Visual Communication 16(1)

5 Notwithstanding this suggestion, this book constitutes a strong addition to this series and should encourage much-needed interdisciplinary projects in multimodal discourse analysis. The extent to which the editors appeal for more meaningful methodological dialogue in multimodal research well made throughout this book will be answered by researchers will only be truly evaluable in the future. Even should this appeal not be answered to the extent that the editors would wish, this book has at least succeeded in generating debate about the merits of interdisciplinary, mixed-methods approaches for studies in multimodality. References Forceville, C. (2009) Metonymy in visual and audio visual discourse. In: Ventola, E. and Moya Guijarro, A.J. (eds) The World Told and the World Shown. Basingstoke: Palgrave, Halliday, M. A. K. (2002) Text as semantic choice in social contexts. In: Webster, J (ed.) M.A.K. Halliday: Linguistic Studies of Text and Discourse. London: Continuum, Halliday, M. A. K. and Matthiessen, C. M. I. M. (2004) An Introduction to Functional Grammar. London: Arnold. Hoey, M. (2001) Textual Interaction: An Introduction to Written Discourse Analysis. London: Routledge. Ishiguro, K. (1989) Remains of the Day. London: Faber & Faber. Kramer, S. and Ludes, P. (2010) Networks of Culture: The World Language of Key Visuals, 2nd edn. Berlin: Li. Kress, G. (2010) Multimodality: A Social Semiotic Approach to Contemporary Communication. London: Routledge. Kress, G. and Van Leeuwen, T.J. (2006) Reading Images: The Grammar of Visual Design, 2nd edn. London: Routledge. Rigotti, E. (2005) Congruity theory and argumentation: Argumentation in dialogic interaction. Special Issue of Studies in Communication Sciences: Sperber, D. and Wilson, D. (1986/1995) Relevance: Communication and Cognition. Oxford: Blackwell. Van Leeuwen, T.J. (2005) Introducing Social Semiotics. London: Routledge. GAVIN BROOKES University of Nottingham, UK [ g.brookes@lancaster.ac.uk] 143