Usability of Computer Music Interfaces for Simulation of Alternate Musical Systems Dionysios Politis, Ioannis Stamelos {Multimedia Lab, Programming Languages and Software Engineering Lab}, Department of Informatics, Aristotle University of Thessaloniki, Thessaloniki 540 06 GREECE {dpolitis,stamelos}@csd.auth.gr http://www.csd.auth.gr/~dpolitis SUMMARY This paper introduces the framework of Computer Music interfaces within the context of Human Computer Interaction (HCI). It examines the unique problems posed by the use of computers by composers and performers of music. The paper presents the various interfacing protocols and software products that apply HCI design methodologies on the world of music, producing special modules like: composition and synthesis languages, GUIs for score editing, performance instruments, digital audio processing tools. A presentation of the basic concepts involved in the Computer Music world prevails the discussion on the evaluation criteria of Computer Music interfaces. New criteria are introduced, like knowledge representation of musical data and alternate musical interfaces materialization. A survey of various systems, based on the previously mentioned criteria is committed. KEYWORDS: Computer Music, Interfaces Design, Usability, Music Languages, Alternate Musical Systems. INTRODUCTION One of the most intriguing fields of Human Computer Interaction involves the communication aspects of Computer Music interfaces. Music is a rich communication medium, and Computer Music is the amalgam of interface science and musical praxis forming a dynamic subset of Human Computer Interaction. There are structural similarities between the job of a music composer and that of a User Interface designer (although their objectives may be different). While sound has been used in general purpose interfaces as an object, its use has been deteriorated at a primitive level, that of signal-processing approach. However, music composition and performance are highly abstract human activities involving a semantic and a symbolic mechanism of human intellectual activity. It is surprising that there have been so few attempts exploring the possibilities of computer composed and synthesized music in the context of object oriented graphical user interfaces. This paper analyzes the unique problems posed by the use of computers by composers and performers of music. It presents the HCI predicates involved in the chain of musical interaction with computer devices, commencing from the abstract part of symbolic composition, then coping with usability issues of the graphical user interfaces implemented for musical scripting and concluding to a synthesis stage which produces digitized sounds that enhance or replace protypal analog audio signals. The evaluation of HCI elements for Computer Music under the prism of usability aims at the development of new graphical tools, new symbolic languages and finally better user interfaces. Technological advances in these fields challenge for increasing demands on the quality of the user interface and offer the potential to further progress the functionality of Computer Music devices. Under this prism, human-computer interaction becomes more central to the design and development of Computer Music systems investigating functionality that does not exist for the user or functionality that is not virtually usable. Computer Music Interfaces In the early stages of microcomputer evolution, various protocols had been developed in order to achieve interconnection between computers and instruments. The milestone of Computer Music proved however to be the MIDI specification. MIDI, the Musical Instrument Digital Interface, is a protocol which specifies both a hardware interface and a language for passing musically meaningful messages. It was developed in 1983 in response to the increasing sophistication, and corresponding complexity, of commercial electronic instruments, especially synthesizers. Therefore, MIDI is a protocol specifying how electronic musical instruments may be controlled remotely. In brief, MIDI is a very successful and inexpensive protocol that has reshaped the Computer Music landscape. However, it can not overcome easily its representation limitations, especially on alternative music notations [1].
MIDI, albeit the dominant, is not the most expandable and modular interface. Recently, other interfaces like the SKINI [2] or the SMS [3] physical modeling interfaces have appeared. These interfaces are purely computer software inventions and lack the hardware orientation of MIDI. However, they are more adaptive in expressing alternate musical forms and interfaces. Score Writing and Notation Creation This category of interfaces consists of state of the art, easy to use graphical user interfaces that provide ways to create, enter, edit, hear, view, lay out, and ultimately print music in staff notation. Usually these programs have complete control over every aspect of music printing and publishing. Generally, they are perceived as mature products, satisfying the musician in the same sense Figure 2: A Byzantine Music client for visual composition. apart from having a significant diachrony, serves also as an intuitively alternate interface, since it uses the notification methodology of ancient Delta systems. In Figure 2 is presented a prototypal visual client, the first of its kind, that has served for Synthesis with Byzantine Music semantics [4]. Figure 1: Entry level Score Writer, Cakewalk s notation handling module. that a good word processing system satisfies the author enough to shift from handwriting to electronic processing. However, their expression format is staff based, and therefore they can satisfy users needs as long as the Common Music Notation (CMN) can satisfy the expression of the melody accurately. For Western music users, or for systems that have adapted to CMN, there seems to be little or no problem. A typical interface for CMN composition is shown in Figure 1. However, the world of music is not unified. Especially in the East, we do have alternate musical interfaces which use different semantics. A classical example is that of Byzantine Music. This kind of music, Recording Systems and Production Systems This category of products, undoubtedly the flagship of Computer Music industry, produces complete professional music recording systems. Usually they combine high resolution MIDI recording channels with audio recording in either 16 or 24 bit formats. This way they offer state of the art multi-track digital input capabilities and thus they simulate and gradually replace classical analog recording studios having in the meanwhile the advantage of inherent communication with digital instruments. Although the hardware and the low level capabilities of these systems rely heavily on the MIDI specification and therefore do not offer substantial improvements over the performance limitations of MIDI, their high level perception, i.e. their graphical user interfaces (GUIs) challenge the Computer Music community for the invention of new conceptually and functionally composing and synthesizing schemas. To great extend this has been solved with the various plug-ins that accompany and complement the basic systems. A typical interface of this kind can be seen in Figure 3.
Figure 3: Typical audio track based musical interfaces. A noticeable consequence is that the plug-ins hanging around visualize spatially and semantically polymorphic and diversified views of Computer Music predicates. Another key element of these systems is that they are real time systems, adding another perspective to (software) musical compositions and equalling thus the performance gain of these systems to hardware synthesizers. In musical composition, the strength of these systems lies on the symbolic processing of CMN. However, these systems are not merely notation interfaces that perform via the MIDI specification. Good performance criteria should also apply. In this field, the advances in HCI challenge for innovations beyond the limited, file-based, single-data-type applications. The momentum is towards models supporting richer data types, visualization paradigms and distributed storage such as the model behind the rapidly evolving World Wide Web. The multi-dimensionality of musical data begs for higher dimensions of control and representation impossible with the current "paper on a desktop" metaphor [5] (Figure 4). Figure 4: Typical recording systems interfaces implementing the recording console metaphor. In an effort for more natural interaction with the Computer Music systems, we need support for input devices with higher control bandwidth and dimensionality than the mouse [6]. There is also need to integrate new kinds of keyboards and a broader range of physical gestures and non-human control sources. However, the critical point in this kind of systems is to produce in semantic form a broad range of musical signals. They should be more judged according to the fidelity, the range of musical produce than according to the polymorphism of the input devices. Afterall, they should help musicians make music. In this sense there is need for code libraries to help programmers write programs that help musicians make music. Waveform processing systems This category of products, is related with the signal processing aspect of Computer Music. Although these systems allow the production of virtually any sound, they are not short term composition systems and they will not be considered. PROBLEM FORMULATION The use of Computer Music interfaces aims at producing melodic pieces. The instrument used in this case is a computer program, perhaps in conjunction with a keyboard hardware interface communicating via the MIDI- IN and OUT ports. Producing melodic lines is a matter of inspiration and not an arbitrary or disciplined procedure. In terms of Human Computer Interaction it means that the computer program used must have functionality and usability features that enable the user to record in symbolic form the music he has conceived. Usually, five criteria are used in order to evaluate the usability of an interface according to the ISO/DIS 9241-11 directive: a) Learnability for the use of the new system. b) Effectiveness, i.e. the extent to which the intended goals of musical synthesis and composition are achieved. c) Efficiency, when used by experienced and trained users, i.e. the amount of resources that have to be expended to achieve the intended goals. This criterion is more procedural than quantitative in Computer Music. d) Satisfaction, in the sense of the extent to which the user finds the use of the product acceptable and desirable. e) Capability to use the system from users not familiar with its musical categories and predicates after a long time. In order to evaluate the performance of Computer Music systems on alternate musical interfaces a heuristic evaluation will be performed. According to Nielsen heuristic evaluation [7] is a usability engineering method for finding the usability problems in a user interface design so that they can be attended to as part of an itera-
tive design process. Heuristic evaluation involves having a small set of evaluators, experts in their field, examining the interface and judging its compliance with recognized usability principles (the "heuristics"). For each category of musical interface products, evaluation takes place according to the previously mentioned criteria. RESULTS The evaluation of specific Computer Music interfaces is based on the previously mentioned usability criteria. These criteria however are adjusted to the specific communication content of each interface. The evaluation is calibrated with the following ratings of confidence whether a task can be performed: - : weak confidence, +/- : plausible, + : strong confidence. Interfaces The usability criteria for the category of Computer Music protocols and specifications has to do mainly with the ability to simulate a broad range of musical data, to perform them acceptably and to expand to alternate musical forms. An evaluation of some schemas is shown in Table 1. By the term simulation is described the ability to render musical sounds close to the real time performance data. By the term interconnection is implied the ability to communicate with other digital musical devices. By the term expandability is described the ability to engulf alternate musical systems and events. By the term acceptability is measured the propagation of the protocol to alternate musical traditions users. By the term learnability is implied how easily the users of a specific product learn to produce alternate musical sounds and predicates. Protocol Simul ation Interconnection Expandability Acceptability Learnability +/- + - + + MIDI (Industry standard) PM + + + - + Extended MIDI + + +/- +/- + Table 1: Usability evaluation of Computer Music protocols on their ability to simulate alternate musical sounds. By PM Physical Modeling techniques are implied. Score Writing and Production In this combined category, the evaluation criteria are adjusted to the pool of Computer Music users attempting to compose not abstract CMN melodies but melodies which will be performed and propagated to listeners of alternate musical systems. Modern Greek Pop music has been taken into account. Also, hardware incarnations of such systems were considered; the basic criterion for their acceptance is the existence of a corresponding software module which at least can create notation or symbolic scripting of the performed music. For instance, if we have a keyboard performing Arab or Byzantine tunes, it is prerequisite to have software module that can write melodic lines according to this system. It is desirable but not obligatory for these systems to communicate. The comparison of such systems is performed in Table 2. The well-known ISO/DIS 9241-11usability criteria are applied. Some variations and extension of these criteria have to do with: (a) Whether the system has room for symbolic representation of the alternate musical form. (b) Whether the system is learnable for users expressed mainly in alternate forms and not in CMN. (c) Whether the produced sound or the symbolic scripting of a melody are close to the alternate music predicates. (d) Whether the system is modular and can co-operate with other Computer Music instruments and gadgets. (e) Whether expert users of computer software and alternate music theory and practice can produce alternative music. (f) Whether the listeners of alternate music forms accept the audio result of the simulation. Product Group Alternate Symbolic Represent ation Learnability Simulation Effectiveness Expand ability Efficiency Satis faction - +/- - +/- - - CMN Based System Based Alternate - +/- +/- + +/- +/- + + + - + + Table 2: Usability evaluation of Computer Music software modules on their ability to track down alternate musical predicates and to produce adequate sounds. CONCLUSIONS Already several prototypal and research projects have been focusing on alternate music representation, authoring and scripting. It is expected soon that the advances in GUI software engineering will enable the production of commercial products that can compete the more than a decade old Western Music counterparts. Since the
Western Music interfaces have a very interesting evolution from the HCI point-of-view, a first performance evaluation on their outsourcing capabilities has been achieved. ACKNOWLEDGEMENTS We would like to thank our students K. Zaharaki and M. Almasidou for the bibliographic research they committed on the Computer Music instrumentation field. BIBLIOGRAPHY 1. Roads, C. The computer music tutorial. MIT Press, Massachusssets, 1996. 2. Cook, P. The SKINI Interface, Princeton, 1996. 3. Serra, X, Bonada, J., Herrera, P., Loureiro, R., Integrating complementary spectral models in the design of a musical synthesizer, International Computer Music Confernece ICMC 97, Thessaloniki, September 25-30, 1997. Proceedings, pp. 152-159. 4. Politis, D., Tsoukalas, A., Linardis, P., Bakalakos, A., VIDI-A Voice Instrument Digital Interface for Byzantine Music, International Computer Music Conference ICMC 97, Thessaloniki, September 25-30, 1997. Proceedings, pp. 403-407. 5. Freed, A., Center for New Music and Audio Technologies, Berkeley. http://www.cnmat.berkeley.edu 6. Kabbash, P., MacKenzie, I.S. & Buxton, W. Human performance using computer input devices in the preferre and non-preferred hands. Proceedings of InterCHI '93, pp. 474-481. 7. Nielsen, J. Heuristic evaluation. In Nielsen, J., and Mack, R.L. (Eds.), Usability Inspection Methods. John Wiley & Sons, New York, 1994.