melodic c2 melodic c3 melodic

Transcription

1 An Interactive Constraint-Based Expert Assistant for Music Composition Russell Ovans y and Rod Davison z Expert Systems Lab Centre for Systems Science Simon Fraser University Burnaby, B.C., Canada V5A 1S6 Abstract A novel use of constraint propagation within an expert system for music composition is described. The task of composing contrapuntal music is modelled as a constraint satisfaction problem, and consistency techniques are utilized to present the user { as each note is chosen { with a graphical projection of the relaxed constraint graph. The expert system's role is to prevent the user from violating any rule of counterpoint composition. This system illustrates the potential of separating generative (search strategy) from restrictive (constraints) knowledge in interactive expert systems. 1 Introduction This paper describes an interactive tool for generating rst species counterpoint of note against note, a highly structured historical style of music. The tool is essentially a graphical interface to a counterpoint expert system, and would be useful to a beginning student of counterpoint. Our approach is to formulate the task as a constraint satisfaction problem (CSP), and the composition process as the navigation of a constrained search-space. A CSP is the problem of assigning discrete values to a nite set of variables such that a set of constraints is satised. The project described here is a natural extension to our earlier work on viewing music composition as a CSP [Ovans, 1990]. We view the composition process as the selection of note attributes (pitch, duration, etc.), from nite and discrete domains, such that a set of constraints is satised. In this context, the constraints impose acceptable structure, coherence, and aesthetic on a composition. The user's task is to compose a counterpoint for a given melody. The user, as composer, is responsible for Appears in Proceedings: Ninth Canadian Conference on Articial Intelligence, pp ythe nancial support of PRECARN Associates, Ottawa, is gratefully acknowledged. ovans@cs.sfu.ca. zthe nancial support of the Science Council of B.C. is gratefully acknowledged. Author's present address: Vertigo Technology Inc.; Homer St.; Vancouver, B.C.; V6B 2X6. generating a musical solution whereas the expert system is responsible for policing the constraints. We begin in Section 2 of this paper with a description of the tool from the user's perspective. The underlying theory of CSPs and a description of how the composition task is formulated as one is found in Section 3. In Section 4 we provide some detail about the actual implementation, and conclude in Section 5 with a short discussion. 2 Expert-Assisted Counterpoint Composition Counterpoint is a style of music composition that predates our modern notion of harmony. Whereas harmony is concerned mainly with vertical relationships among notes, chords in succession, and supporting a dominant melodic theme, counterpoint is polyphonic composition: the combination of several independent and equally interesting melodies into a coherent whole. The rules of counterpoint, which constrain the allowable note combinations occurring in a composition, were codied in 1725 by Johann Fux, a translation of which appears in [Mann, 1965]. The normal manner of composition in counterpoint is to craft additional melodic lines (called counterpoints) to be sung, or played, along with a previously composed given melody, usually taken from a book of chorales. The new voices are said to be in counterpoint to the given melody. The program interface is presented in Figure 1. The window is titled \Fux Composition Tutor" in deference to Johann Fux. The user has a choice of composing a counterpoint of four, seven, or twelve bars in length. In the example, 12 Bar has been chosen and a given melody automatically generated. The rules of counterpoint are such that a given melody completely denes a searchspace of permissible counterpoints. The corresponding counterpoint choices are presented as in the gure: the whole notes in each bar of the counterpoint represent allowable choices, not chords. The user's task is to make choices within the counterpoint search-space until a single note remains in each bar. There are 5,930 \correct" counterpoints for the given melody presented in Figure 1 [Ovans, 1992]. The expert system's role is to insure the user nds one; that is, that he or she does not violate any of the rules of counter-

2 Figure 1: A 12-bar given melody and its resulting counterpoint search-space. point. The task is made interesting by the simple fact that not every option is consistent with every other: user choices result in a narrowing of the search-space. The search-space is navigated by selecting { with a mouse { a series of notes, which need not be done in the conventional and sequential left-to-right manner. With each selection the expert system propagates constraints and lters the remaining consistent choices in the other bars. Figure 2 reveals the resulting search-space after the user has selected a G in the fth bar. Because consistency is maintained as each variable is instantiated, the user can make a choice for a pitch value and see the ramications in the form of the resulting constraint propagation (e.g., \...if I choose a fth for the interval in this bar, I see that it rules out the A in the previous bar."). User choices can be undone by selecting a note a second time. Figure 3 is the state of the composition after two subsequent choices are made and the original choice of Figure 2 is repealed. As well, the user need not nish the task: at any time the expert system can be asked to solve the problem. It does so without overriding the user: previous choices must appear in the solution (see Figure 4). 3 The Underlying Theory Our approach is unique in its recognition that composing counterpoint can be represented as a constraint satisfaction problem: a user of this tutoring system is interacting with a constraint graph. In this section we present a short summary of the topic of CSPs and show how the task of composing rst species counterpoint can be formulated as a CSP. 3.1 Constraint Satisfaction Problems A ubiquitous problem in articial intelligence is the constraint satisfaction problem, which can be succinctly stated as follows: given a nite set of variables X = fx 1 ; x 2 ;... ; x n g whose elements range respectively over the nite (and not necessarily numeric) domains D 1 ; D 2 ;... ; D n, nd a value for each variable such that a nite set of constraints is satised. The role of the constraints is to reduce the cartesian solution space D = D 1 D 2 D n. Each constraint is a relation on a subset of X that states which values are consistent with each other. CSPs are solved by nding one (all) point(s) in the nite discrete space D that simultaneously satises the constraints. A good survey of CSPs is found in [Mackworth, 1992]. CSPs are conveniently represented by constraint graphs [Mackworth, 1977]. In a constraint graph, each node is a variable to be assigned a value. Adjacent nodes must satisfy the constraint denoted by the arc connecting them. Unary constraints are expressed with loops. The constraint graph for a corresponding graph colouring problem is in fact the graph to be coloured. To represent k-ary constraints, k > 2, hyperarcs (arcs that connect more than two nodes) are required. Depth-rst chronological backtracking search is a general purpose algorithm for solving CSPs. Despite the elimination of subspaces from the solution space D with each failed instantiation, in the worst case backtracking is exponential in the number of variables. Attempts at improving the performance of backtracking algorithms led to the incorporation of consistency techniques. Consistency techniques prune the search-space before failure occurs thus improving the eciency of tree search by reducing the number of backtrack points. The searchspace is reduced by decreasing the tree's branching factor; values from a variable's domain that cannot possibly participate in any solution to the problem are eliminated, thus avoiding a possibly costly failure later on in the search tree. Consistency techniques are weak inference methods: they always work, but will not always solve a CSP. Arc consistency [Mackworth, 1977] is a technique applicable to connected nodes in a constraint graph. Assuming a binary constraint P is acting on variables x i and x j, the arc is consistent i (8a 2 D i )(9b 2 D j )P (a; b) (8b 2 D j )(9a 2 D i )P (a; b)

3 Figure 2: A choice is made resulting in a reduced search-space. Figure 3: Two subsequent choices plus the original choice revoked. Figure 4: The composition as completed by the expert system.

4 Note that arc consistency is generalizable to hyperarcs as well, in which case we call it k-ary arc consistency. A constraint graph is said to be arc consistent i all adjacent nodes are consistent with each other. Arc consistency may occur fortuitously, but is assured when a graph is subjected to a full arc consistency algorithm, which in any of its forms essentially controls the propagation of constraints until quiescence. Many polynomialtime arc consistency algorithms have been reported, for example Waltz's procedure [Waltz, 1975] and AC3 [Mackworth, 1977]. A unied survey of many of the CSP-solving algorithms in terms of tree search and arc consistency is given in [Nadel, 1989]. 3.2 Counterpoint as a CSP The problem of making a good piece of music is a problem of nding a structure that satises a lot of dierent constraints. Marvin Minsky [Roads, 1980] This section describes how the problem of generating contrapuntal music can be formulated as a CSP. Specically, compositions of rst species counterpoint that adhere to a fairly complete set of rules taken from [Mann, 1965] are modelled. The rst species is counterpoint of the simplest form: two or more voices comprised of notes of equal length. This restriction to rst species eliminates the myriad of representation problems concerned with rhythm and allows us to focus solely on the constraint relationships. The only attribute of a note that requires representation is its pitch. Any representation scheme that facilitates the denition of constraints on pitch values would be acceptable. We have chosen the Musical Instrument Digital Interface (MIDI) standard: pitch is an integer from 1 to 127 corresponding to an ordering of the semi-tones with middle-c equal to 60. A rst species counterpoint composition in two voices n bars in length is thus comprised of n counterpoint variables fc 1 ;... ; c n g and n melody variables fm 1 ;... ; m n g. It is convenient to think of the notes of the given melody as instantiated variables rather than constants. Using this notation, the ith bar of a composition is comprised of note m i in the melody and c i in the counterpoint. The task is thus to assign values to fc 1 ;... ; c n g that satisfy the rules of rst species counterpoint given an instantiation for each of fm 1 ;... ; m n g. First species counterpoint, as codied in [Mann, 1965], is dened by nine rules restricting allowable note combinations. Each of these rules can be represented as a set of local constraints enforced by the same predicate [Ovans, 1992]; in other words, each rule is comprised of many constraints. A sample rule is the following: because they are dicult to sing, melodic intervals greater than a minor sixth or equal to a tritone are forbidden. A constraint dening allowable melodic intervals is thus: melodic(c i ; c i+1 ), j c i? c i+1 j 8 ^ j c i? c i+1 j 6= 6 This constraint is applied to successive neighbouring pairs of counterpoint variables. Note that this constraint would normally also apply to the melody variables, but given that our system works only with supplied (and correct) melodies, constraints acting solely on melody variables are superuous. A constraint graph that enforces the nine rules of note against note rst species counterpoint is presented in Figure 5. This graph is for compositions of length four { the shortest compositions to include all the rules of rst species counterpoint { and can easily be extended for compositions of greater length. It is in actuality a hypergraph: the skip-step constraints are ternary, while the quaternary constraints embody both the forbidden parallel motion to a perfect consonance and forbidden progressions to an octave by a skip rules. For clarity, unary constraints are not included in the graph of Figure 5. The unary constraints restrict the pitch values for the notes of the counterpoint to those of the Aeolian mode since given melodies are assumed to be in the Aeolian mode. This provides a convenient mechanism for prohibiting a change of key. Since counterpoint is generally written for voice, we impose an arbitrary two-octave range: mode(c i ), c i 2 f45; 47; 48; 50; 52; 53; 55; 57; 59; 60; 62; 64; 65; 67; 69g The mode constraint is applied to each counterpoint variable, save the second to last. To facilitate the formation of the proper cadence, the unary predicate cadence is applied to the second to last counterpoint variable: cadence(c n?1 ), c n?1 2 f56; 68g With an instantiated set of melody variables and the domains of the counterpoint variables initialized to re- ect these unary constraints, a full arc consistency algorithm applied to the constraint graph results in a drastically reduced branching factor for the search tree of permissible counterpoints [Ovans, 1992]. It is a projection of this resulting relaxed constraint graph that is presented to the user in Figure 1. 4 The Implementation The interface is a C++ program that utilizes the Inter- Views user interface toolkit. The expert system is an Echidna knowledge-base. Echidna is a new constraint logic programming language { strongly inuenced by CHIP [Van Hentenryck, 1989] { suitable for model-based expert systems applications. Echidna improves on existing expert system shells by combining facets of objectoriented programming, dataow dependency backtracking, and constraint logic programming [Havens et al., 1990]. The Echidna knowledge-base consists of a set of schema (object class) declarations, including those for composition, counterpointnote, and melodynote classes. The constraints of counterpoint are realized as methods (Horn clauses) within these schema declarations. Bidirectional constraints between schema instances (objects) are created via message passing. The Echidna reasoning engine maintains k-ary arc consistency on all discrete constraints. Echidna's dependency-directed backtracking provides ecient means for undoing user selections.

5 m1 m2 m3 m4 first quaternary harmonic quaternary quaternary last c1 melodic c2 melodic c3 melodic c4 skip-step skip-step Figure 5: A constraint graph for rst species counterpoint, n = 4. The graphical interface and Echidna interpreter are separate processes. The two processes communicate through a Unix pipe, and need not reside on the same CPU. 1 The protocol governing their communication is dened by Echidna's external object protocol. For a process external to Echidna, this protocol provides the ability to issue goals to the knowledge-base and \share" logic variables. In our composition system, the pitch value for each counterpointnote instance is shared with the interface process. Any change to the domain of a shared variable is communicated from the interface to Echidna (in the case of a user choice), and vice-versa (in the case of constraint propagation). 5 Discussion Rule-based automated music composition systems (like that of [Ebcioglu, 1984; Thomas, 1985]) typically contain a set of constraints that eliminate unmusical compositions. The problem with these systems is two-fold. First, passive constraints are computationally expensive. The incorporation of consistency techniques can help to ameliorate inecient generation of compositions due to excessive backtracking [Ovans, 1992], but they are by no means a panacea. Second, composition is not paint-by-numbers. In other words, the satisfaction of constraints alone is not sucient for musical results [Ebcioglu, 1984]. The composer's skill is in how he or she prunes and navigates the constrained, but still very large, search-space of possibilities. How to represent this skill in a declarative knowledge base is not well understood and indeed might be impossible for deep humanistic domains like music [Loy, 1991]. The problem, as it also appears in other expert systems, is as follows. Knowledge of how to generate a \good" solution is dicult to represent. Conversely, constraints that restrict allowable solutions are often 1 We normally run the Echidna process on a NeXTstation and the interface under OpenWindows on a SPARCstation. easy to represent. This dichotomy is reinforced by the experiences of others who, in an attempt to devise rules for nding good solutions, have ended up with systems that contain hundreds of rules yet generate mediocre compositions (e.g., [Schottstaedt, 1984; Ebcioglu, 1984]). The design of our composition tutor is inuenced by the belief that, in this case, the user best provides the generative knowledge. Let the expert system represent and propagate constraints; this it can do well and eciently. Let the user search for the good solutions for this he or she (currently) does better than the expert system. The constraint-based approach is useful in this domain and others like it (e.g., interactive scheduling) where it is more important for the computer to prevent mistakes than to generate solutions. References [Ebcioglu, 1984] Kemal Ebcioglu. An expert system for schenkerian synthesis of chorales in the style of J. S. Bach. In Proceedings of the International Computer Music Conference, pages 135{142, [Havens et al., 1990] William S. Havens, Susan Sidebottom, Greg Sidebottom, John Jones, Miron Cuperman, and Rod Davison. Echidna constraint reasoning system: Next-generation expert system technology. Technical Report CSS-IS TR 90-09, Centre for Systems Science, Simon Fraser University, [Loy, 1991] D. Gareth Loy. Connectionism and musiconomy. In Proceedings of the International Computer Music Conference, pages 364{374, [Mackworth, 1977] Alan K. Mackworth. Consistency in networks of relations. Articial Intelligence, 8:99{118, [Mackworth, 1992] Alan K. Mackworth. Constraint satisfaction. In Stuart C. Shapiro, editor, Encyclopedia of Articial Intelligence, Second Edition, pages 285{293. John Wiley & Sons, New York, 1992.

6 [Mann, 1965] Alfred Mann, editor. The Study of Counterpoint from Johann Joseph Fux's Gradus Ad Parnassum. W. W. Norton, New York, [Nadel, 1989] Bernard A. Nadel. Constraint satisfaction algorithms. Computational Intelligence, 5(4):188{224, [Ovans, 1990] Russell Ovans. Music composition as a constraint satisfaction problem. In Proceedings of the International Computer Music Conference, pages 317{319, [Ovans, 1992] Russell Ovans. Ecient music composition via consistency techniques. Technical Report CSS-IS TR 92-02, Centre for Systems Science, Simon Fraser University, [Roads, 1980] Curtis Roads. Interview with Marvin Minsky. Computer Music Journal, 4(3):25{39, [Schottstaedt, 1984] B. Schottstaedt. Automatic species counterpoint. Technical Report STAN-M-19, Centre for Computer Research in Music and Acoustics, Stanford University, [Thomas, 1985] Marilyn Taft Thomas. Vivace: A rule based AI system for composition. In Proceedings of the International Computer Music Conference, pages 267{274, [Van Hentenryck, 1989] Pascal Van Hentenryck. Constraint Satisfaction in Logic Programming. MIT Press, Cambridge, Massachusetts, [Waltz, 1975] David Waltz. Understanding line drawings of scenes with shadows. In Patrick Henry Winston, editor, The Psychology of Computer Vision. McGraw-Hill, New York, 1975.