Distributed Virtual Music Orchestra DMITRY VAZHENIN, ALEXANDER VAZHENIN Computer Software Department University of Aizu Tsuruga, Ikki-mach, AizuWakamatsu, Fukushima, 965-8580, JAPAN Abstract: - We present the main features of the Distributed Virtual Music Orchestra (DVMO), which simulates music band/orchestra by means of a distributed computing system. In this orchestra, each computer is associated with the one musician or a group of musicians playing its own melody. The DVMO software includes a conductor assigning melodies to corresponding computers, a set of servers (musicians) as well as special multimedia tools for composing/decomposing music melodies to be efficiently played on the DVMO. This paper discusses approaches to the orchestra model design, synchronization strategies as well as the user interface features and demonstrates some examples of system implementation. Key-Words: - Internet and Distributed Computing, Computer Music, Virtual Music Orchestra, Client/Server Technology, MIDI, Synchronization Strategies 1 Introduction Multimedia has developed explosively in recent years, and integrated several media of graphics, images, music and so on in the literal sense of the word. At the same time, it is possible to say, that the Internet has become to be a common place for us. Social life is now inseparable from the network. Art has a power of new expression due to the development of the Internet and multimedia. Computer music is also affected significantly. Since its beginnings, music has developed by the latest technical advancement for increasing the power of artistic expression. The power and cost of modern computers allow to model a real orchestra [1]. Some models of a Virtual Music Orchestra are shown in [2]. Gil Weinberg and Seum-Lim Gan wrote this article on performance interfaces, which investigates how multiple players, who may be musically untrained, can control a single system in interdependent manner. The controller in this case consists of a set of gal balls that the performers squeeze and pull, and the control data is converted to music by way of a Max patch that sends MIDI data to external synthesizers. The implementations use of a Max patch should allow composers to use the author s controller while easily substituting their own mappings. In the FlexiMusic Orchestra [3] multiple keyboards can be connected to one PC, and multiple persons can play music at the same time.. They can create unlimited number of play boards. Each play board can have all input devices (like keyboard etc.) as sub boards. Each play board will have its own set of sound assignment. During the playing, the user can change over to next play board to play completely a new set of music. It is possible to choose any standard instrument/drum from midi bank, to use sample of wave file obtained from Internet. The distributed, real-time object system, Aura [4,5], offers a carefully designed architecture for distributed real-time processing. In contrast to streaming audio or MIDI-over-LAN systems, Aura offers a general real-time message system capable of transporting audio, MIDI, or any other data between objects, regardless of whether objects are located in the same process or on different machines. Measurements of audio synthesis and transmission to another computer demonstrate about 20 ms of latency. Local-area networks offer a means to interconnect personal computers to achieve more processing, input, and output for music and multimedia performances. The purpose of this research is in developing Distributed Virtual Music Orchestra (DVMO), which simulates music bands/orchestra by means of a distributed computing system [6]. We consider a DVMO as a distributed application combining both multimedia tools and parallel synchronous computations. In this orchestra, each computer is
associated with the one musician or a group of musicians playing its own melody. The Conductor of a DVMO is a system assigning the melodies to the corresponding computers, as well as controlling and synchronizing the execution of music programs. This paper discusses approaches to the orchestra model design, synchronization strategies as well as the user interface features and presents some practical examples of system implementation. Some further directions of the DVMO development will also be discussed. 2 DVMO concept 2.1 Basic model overview As shown in Figure 1, the Distributed Virtual Music Orchestra (DVMO) can be considered as a set of computers reflecting behavior of musicians. In this orchestra, each computer corresponds to a musician who uses a concrete musical instrument like piano, violin, trumpet, etc. To implement music, each musician should operate with expressive playing information or an instructions (notes) sequence defining how to perform music. All players synthesize their own instrumental melodies according to the conductor's direction and their own rules in a real orchestra. In each computer, melodies are generated using local sound devices. The DVMO plays MIDI-files [7]. Usually, a MIDI-file contains data about all instruments using to play melodies on a single computer. To play music on the DVMO, it is necessary to decompose an initial MIDI-file into several items each of which has data for concrete instrument. Therefore, the DVMO Conductor should implement assignment of music files to computers as well as control for playing in order to achieve synchronization between musicians. The DVMO model is a client-server architecture consisting of multiple identical servers (musicians) and one client (conductor). It is important to note, that it is not enough to use only traditional fat-client or thin-client models for providing good music performance because of specifics in client functions. For example, it is necessary for the client to implement data management operations for the distribution melodies. Each server executes the application processing as well as the thin-client models while performing music. As shown in Fig. 2, each server (musician) provides communication with the client, accepts music files, starts/stops playing music on its own hardware. The client (conductor) stores the melodies (music files), creates a server list, attaches music files to the corresponding musicians, broadcasts network to check availability of each server, and controls play/stop music. Figure 2: DVMO: Conductor Musician Transaction Figure 1: Distributed Virtual Music Orchestra Servers have to play music simultaneously. This means that it is necessary to synchronize servers with acceptable delays between instruments like in
a real orchestra. The concrete value of such a possible delay depends on quality estimations provided by ordinary audience as well as experts. Figure 2 also illustrates a communication mechanism between conductor and musicians, which includes the following four major steps: 1. Checking servers connections, 2. Transfer corresponding files from client to servers, 3. Broadcasting synchronization message to play music, 4. Sending messages to stop/wait playing, and reinitialize each server. The important feature of such a strategy is that we do not use the streaming audio, and try to avoid transferring the musical data during playing the music. This allows achieving a harmony of playing because of communications via very short control messages. 2.2 Software structure As shown in Figure 3, the DVMO architecture consists of four subsystems: Conductor (client), Servers (musicians), Decomposer and Room Editor. Remote Method of Invocation (for OS UNIX). When MS Window systems are used, these programs should be preloaded before launching the Conductor. The Room Editor and Decomposer are tools allowing to the users to provide comfortable debugging of melodies taking into account not only network characteristics but also features of auditorium. Actually, they are GUI-applications supporting object-oriented manipulations. The DVMO Decomposer is to prepare music files and plan of instrument distributions on the computer network. The input data for the Decomposer are a MIDI-file containing data about all instruments and the so-called Room Description Data reflecting features of the auditorium used for the DVMO placement. According to this, each computer (conductor and players) has several properties such as hostname, IP address, and coordinates in the room. The Conductor uses the output Decomposer files. To simulate a real music orchestra, room conditions are a very important factor. Positions of computers (music devices) and/or objects such as tables and obstacles influence significantly to feelings of the audience. The Room Editor is a GUI application devoted to create/edit description of rooms/halls in which the DVMO used. It operates with room images/objects as well as with whole rooms of different shapes. The user can specify the shape and size of a room, the position and number of computers, obstacles, etc. Figure 3: The DVMO Software Architecture The main DVMO module is the Conductor assigning melodies to corresponding computers as well as controlling and synchronizing execution of music programs. The input data are the set of MIDI-files and a plan of their distribution on the server network. The server part of the DVMO is a set of identical programs loading directly via the Conductor. The DVMO initialization can be implemented manually (for MS Windows OS) or automatically using the 3 DVMO User Interface In this section, we show the user's interface features, and some practical examples of system implementation. The programming language is Java. The system can play MIDI files using standard Java MIDI player tools with Java sound APIs. The application was tested on different architectures and OS like UNIX and MS Windows. 3.1 DVMO Conductor Figure 4 depicts the interface of the DVMO Conductor assigning melodies to the selected computers as well as performing the music on them. The assigning procedure implements specifying IP addresses/names of servers and corresponding musical channels/ instruments.
Figure 4: DVMO Conductor Window As shown in Figure 4, the Conductor Window has three interface zones: Input/output and Control zone, Network Settings and Activity Diagram. The Control Zone is to read/save MIDI-files, network data and instruments assignment information. It is possible to provide simple on-line editing operations like remove/insert servers and instrument distribution. For example, the instrument channels can be reallocated by pushing the check at the left of the Activity Diagram. It is also possible to distribute two or more channels to one server. After completing all settings, the user should press the "broadcast" button. Each server will be tested, and notifications about broken links will be displayed. If there are no errors, corresponding music data will be sent to servers automatically. After melody loading, servers wait for "play" or "stop" signals. To play music on the DVMO, the user should press the play button. At any time, the user can press the "stop" button to finish the music performance and the reinitialize servers. The Network Setting zone displays information about the servers state. After broadcasting, it shows the list of broken and good servers. The Activity Diagram shows a structure of music files according to the channel/instrument activities. When finished reading the MIDI-file, the activity diagram draws in an analytical result of using the channel/instrument data. During playing, the special indicator shows the occurrence part of MIDI events. 3.2 Decomposer Figure 5 depicts the DVMO Decomposer Window and describes an example of an allocation of instrument channels. In this example, we used the computer exercise room of the University of Aizu. The main Decomposer window has six areas: Room Data, Channel Assignment, Color Chooser, Host Names Table and Instrument Names Table. The Room Data Area shows the room shape as well as computers placement inside this room. This data should be loaded from the Room Description File. The Channel Assignment Area is to show an instrument distribution in the loaded MIDI-file. The Color Chooser and Host Names Areas are to assign instrument by coloring corresponding computers. Finally, the Instrument Names Area contains instrument description for better understanding the melody structure. It uses standard instrument coding system, but can be modified.
Figure 5: DVMO Decomposer Window The decomposing process can be divided into the following stages: User should to input a music file and old distribution list if necessary. Decomposer retrieves data and displays the musical instruments for each channel. Decomposer reads and displays the description of an assumed room made from the Room Editor. Each white circle shows each computer, which becomes a server. User can assign instrument to the computers. All musical instrument channels are also coded by color data, and relate the server to the channel according to this color. By clicking the circle of left graphics, the circle is painted out with the color of the channel specified by the radio-button. The list of allocated servers is displayed in the host area. Musical instrument color can freely be chosen from the combo box. If the multiple channels have the same color, the allocated server plays these musical instruments at the same time. After completing all settings, the output distribution list can be saved for using by the DVMO Conductor. 3.3 Room Editor The DVMO Room Editor is to provide efficient room image editing and format of the room data. Example of a room is shown in Figure 6. Figure 6: The Room Example The editing operations include Draw/Delete/Move Objects functions, Open/Save Room Data, etc. As shown in Figure 6, the user can create a room description based on real placement of computers in room. There are several basic room shapes like square, rectangle, oval, circle, etc. Each computer
(conductor and players) also has several properties such as hostname, IP address, size (width and height), and coordinates in the room. 3.4 Testing The system can play MIDI-files using standard Java MIDI-player tools. It was tested on different architectures and operating systems like UNIX and MS Windows. Experiments provided show that the best results can be achieved, when servers are organized on the same computers, and have the same operating systems. There is no problem with different OS and computers used for the Conductor and servers. The maximal number of computers used to play symphonic- and pop- music was about 40. To achieve the good sound volume and space effects, we divided all workstations into groups to simulate the limited number of musical instruments. Importantly, each melody has its specific type of such a grouping. The experiments were provided for MIDI-melodies requiring the playing time up to 6-7 minutes, and showed relatively good performance of this system. 4 Conclusion We have presented main architectural elements and features of human/computer interactions with the Conductor of a Virtual Music Orchestra. The practical experiments with the system confirm its good adaptability to different architectures and operating systems. Our future work is also to develop the modern visualization system based on presentation of application methods in the film format [9, 10]. Within the framework of this technology, self-explained series of stills presenting computational schemes are used. Each scheme reflects some knowledge about data processing. It defines sets of points and/or moving objects in multi-dimensional space-time, and possibly a partial order of scanning of these points and objects. Each film can be considered as a 4D space-time object or component. The user embeds his/her algorithmic ideas into a computational scheme (by making this scheme more precise) or into a composition of such schemes. We expect that the combination of the DVMO with self-explanatory films and multimedia format for data representation will be a very attractive basis for knowledge-on-demand applications. References: [1] E. R. Miranda, Composing Music with Computers. Oxford (UK): Focal Press, 2001. [2] Gil Weinberg, Seum-Lim Gan. The Squeezables: Toward an Expressive and Interdependent Multi-player Musical Instrument. Computer Music Journal, MIT Press, Vol. 25, N 2, 2001. [3] "FlexiMusic Orchestra - Overview", http://www.fleximusic.com/orchestra/overview.htm [4] Brandt, E. and R. B. Dannenberg. Brandt, E. and R. B. Dannenberg. 1998. Low-Latency Music Software Using Off-the-Shelf Operating Systems. Proc. of the Int. Comp. Music Conf. San Francisco: Int. Comp. Music Association, 1998, pp. 137-141. [5] B. Roger, R.B. Dannenberg, P. van de Lageweg. A System Supporting Flexible Distributed Real-Time Music Processing. Proc. of the 2001 Int. Comp. Music Conf. San Francisco: Int. Comp. Music Association, 2001, pp. 267-270. [6] A. Vazhenin, D. Vazhenin, Conductor of Virtual Music Orchestra, The Journal of Three Dimensional Images, Vol. 16, No. 4, 2002, pp. 226-230. [7] K. Giesing, ``A Brief History of MIDI'', \\http://www-ccrma.stanford.edu/\~~kgiesing/midi/ index.html. [8] Midisoft Studio Recording Session 2002 XP, http://www.midisoft.com. [9] N. Mirenkov, A. Vazhenin, R. Yoshioka, T. Ebihara, T. Hirotomi, T. Mirenkova, Self-explanatory Components: a New Programming Paradigm, International Journal of Software Engineering and Knowledge Engineering, World Sci., Vol.11, N 1, 2001, pp. 5-36. [10] A. Vazhenin, N. Mirenkov, Visual Programming System VIM, Programming and Computer Software, Kluwer Pub., Vol. 27, N 4, 2001, pp. 217-231.