A Cmpsitinal Tl fr Cmputer-Aided Musical Orchestratin Marcel Caetan Sund and Music Cmputing grup
Musical Orchestratin Refers t cmpsing music fr an rchestra Initially rchestratin was simply the assignment f instruments t pre-cmpsed parts f the scre Gradually rchestratin has becme part f the cmpsitinal prcess Nwadays musical rchestratin invlves timbral cmbinatins Timbral cmbinatins invlve playing multiple instruments simultaneusly t achieve desired effects 2
Timbral Cmbinatins I ask yu Can yu play the instruments f an rchestra t resemble a persn screaming? Why wuld anyne want t d that? Yu ask me back Well 3
Timbral Cmbinatins 4
Cmputer-Aided Musical Orchestratin (CAMO) Large databases f musical instrument sunds Find cmbinatin f ntes frm musical instruments that best apprximates a given target sund perceptually Target Orchestratins 5
Applicatins f CAMO Music Cmpsitin Autnmus explratin f instrument cmbinatins Music educatin and training Orchestratin classes Sund design and synthesis Explratin f timbral cmbinatins f synthetic sunds Cmputer music Cntrl f sund cmbinatins via perceptual features Cinema and vide Generatin f rchestral pieces fr mvie scres Pst prductin Re-rchestrate existing pieces Scientific research and develpment Musical instrument timbre perceptin 6
Frmalizatin f CAMO Autnmus explratin f instrument cmbinatins Cmplexity f timbre perceptin [1] Multidimensinal perceptual phenmenn Cmpsers use nn frmalized knwledge Cdify perceptual similarity between sunds Cmbinatrial ptimizatin prblem Cmbinatrial explsin Knapsack prblem NP-cmplete Cnstraints What instruments are available Hw many f each instrument [1] S. McAdams, B. Girdan (2009). The perceptin f musical timbre. In S. Hallam, I. Crss, & M. Thaut (Eds.), The Oxfrd handbk f music psychlgy, 72 80. New Yrk, NY: Oxfrd University Press. 7
Musical Instrument Timbre Timbre Perceptin The ways in which sunds are perceived t differ Perceptual difference nt accunted fr by pitch, ludness, spatial psitin, duratin, and envirnmental characteristics Categrical view: sund surce recgnitin, identificatin, tracking in time Sensry view: multidimensinal set f attributes assciated with timbre spaces Timbre Space [2] [2] J Grey. Multidimensinal Perceptual Scaling f Musical Timbres. Jurn. Acust. Sc. Am., 61(5):1270 1277, 1977. 8
Knapsack Prblem Resurce allcatin Knapsack has limited capacity W N items with weight (z) and benefit ($) Pack n N items t maximize ttal benefit B withut exciding ttal capacity f the knapsack W 9
CAMO as a Knapsack Prblem Allcatin f musical instruments t an rchestratin Orchestratin accmmdates a limited number f instrumental sunds The spectral energy is the cunterpart t weight The similarity with the target is the benefit f adding a sund Add sunds t maximize similarity withut exciding the capacity f the rchestratin 10
Knapsack Prblem is NP Cmplete Brute frce slutin requires evaluatin f 2 n candidate slutins Expnential time cmplexity: O(nW) NP cmplete (nn deterministic plynmial time) N prf that NP cmplete prblems can be slved in plynmial time N knwn plynmial cmplexity algrithm t determine whether slutin is ptimal Slving NP cmplete prblems Apprximatin Parameterizatin Restrictin Heuristics: Genetic algrithms 11
Genetic Algrithms Meta-heuristics inspired by the prcess f natural selectin and the principle f survival f the fittest Ppulatin f candidate slutins evlves tward a lcal ptimum by recursive applicatin f bi-inspired peratrs mutatin, crssver, and selectin Fitness functin evaluates the quality f candidate slutins 12
Genetic Algrithms 13
CAMO with Genetic Algrithms Orchidée [3] is the state f the art fr CAMO Orchidée perfrms cnstrained ptimizatin Orchidée uses Genetic Algrithms and lcal search Lcal search explres the neighbrhd f a slutin Target Orchestratins [3] G. Carpentier, G. Assayag, and E. Saint-James. Slving the Musical Orchestratin Prblem using Multibjective Cnstrained Optimizatin with a Genetic Lcal Search Apprach. Jurn. Heuristics, 16(5), pp. 681 714 (2010). 14
Prpsed Apprach Genetic Algrithms Artificial Immune System [4] [4] L. N. de Castr and J. Timmis. An Artificial Immune Netwrk fr Multimdal Functin Optimizatin. In Prc. Cngress n Evlutinary Cmputatin, vl. 1, pp. 699 704 (2002). 15
CAMO with Artificial Immune Systems Genetic Algrithms Lss f diversity Cnverges t a single slutin Slutin usually crrespnds t a lcal ptimum Cnstraints n database t btain different slutins Artificial Immune System [4] Maintenance f diversity Cnverges t multiple slutins in parallel Capable f returning all ptima upn cnvergence Allws greater autnmy n database [4] J. Abreu, M. Caetan, R. Penha. Cmputer-Aided Musical Orchestratin Using an Artificial Immune System. In: C. Jhnsn, V. Ciecielski, J. Crreia, P. Machad (Eds.) Evlutinary and Bilgically Inspired Music, Sund, Art and Design. Springer Internatinal Publishing, vl. 9596, pp. 1-16 (2016) 16
CAMO with Artificial Immune Systems Immune Orchestra Target: Car Hrn Orchestratins 17
CAMO with Artificial Immune Systems Immune Orchestra Target: Ahh Orchestratins 18
CAMO with Artificial Immune Systems Immune Orchestra Target: Scream Orchestratins 19
Acknwledgments This wrk was partially supprted by FurEyes, a Research Line within prject TEC4Grwth Pervasive Intelligence, Enhancers and Prfs f Cncept with Industrial Impact/NORTE-01-0145-FEDER-000020 financed by the Nrth Prtugal Reginal Operatinal Prgramme (NORTE 2020), under the PORTUGAL 2020 Partnership Agreement, and thrugh the Eurpean Reginal Develpment Fund (ERDF).