1 On the mathematics of beauty: beautiful music A. M. Khalili Abstract The question of beauty has inspired philosophers and scientists for centuries, the study of aesthetics today is an active research area in fields as diverse as psychology, neuroscience, and computer science. In this paper, we will study the simplest kind of beauty that can be found in a simple piece of music and can be appreciated universally. The proposed model suggest that there is a link between beautiful music and the Dyson distribution which seems to be the result of a deeper optimization process between randomness and regularity. Then we show that beautiful music need to satisfy a more fundamental law that seeks to deliver the highest amount of information using the least amount of energy. The proposed model is tested on a set of beautiful music pieces. Index Terms Evolutionary Art, Music Aesthetic Assessment, Information Theory. T I. INTRODUCTION he study of aesthetics started with the works of Greek philosophers such as Plato, and recently became an active research area in fields as diverse as neuroscience [1], psychology [2], and computer science. Baumgarten [3] argued that aesthetic appreciation is the result of objective reasoning. David Hume [4] took the opposing view that aesthetic appreciation is due to induced feelings. While Immanuel Kant argued that there is a universal aspect to aesthetic [5]. Shelley et al. [6] discussed the relative influence of subjective versus objective factors in aesthetic appreciation. Research on empirical aesthetics shows that despite the subjectivity of aesthetic appeal, there is still a general agreement on what is considered beautiful and what isn t [7]. Predicting the aesthetic appeal of music is beneficial for a number of applications, such as retrieval and recommendation in multimedia systems. The development of social media and the rapid growth in media content, has increased the requirements of high quality aesthetic assessment systems. Automating the aesthetic judgement is still an open problem, and developing a model of aesthetic judgement is one of the major challenges. Recently, convolutional neural networks (CNN), which can automatically learn the aesthetic features, have been applied to aesthetic quality assessment [8], [9], [10], and [11], promising results were reported. Neural networks have also been used in music composition, some early approaches include [12], [13], [14], [15], [16], [17], [18], and [19]. The development of a model of aesthetic judgement is also one of the major challenges in evolutionary art [20], [21], and [22]. where only music of high aesthetic quality should be generated. Birkhoff [23] proposed an information theory approach to aesthetics, he proposed a mathematical measure, where the measure of aesthetic quality M = O/C is in direct relation to the degree of order O, and in reverse relation to the complexity C. Eysenck [24], [25], and [26] conducted a series of experiments on Birkhoff s model, he suggested that the aesthetic measure function should be in a direct relation to complexity rather than an inverse relation M = O C. Javid et al. [27] conducted a survey on the application of entropy to quantify order and complexity, they proposed a computational measure of complexity, the model is based on the information gain from specifying the spacial distribution of pixels and their uniformity and non-uniformity. Herbert Franke [28] proposed a theory based on psychological experiments that suggested that working memory can t take more than 16 bits per second of visual information. Then he argued that artists should produce a flow of information of about 16 bits per second for their works of art to be perceived as beautiful and harmonious. Manaris et al. [29], Investigated Arnheim s view [30], [31], and [32] that artists tend to produce art which create a balance between chaos and monotony. They presented results of applying Zipf s Law to music. They created a large set of metrics based on Zipf s Law which measure the distribution of various parameters in music, such as pitch, duration, harmonic consonance, and melodic intervals. They applied these metrics to a large collection of music pieces, their results suggest that metrics based on Zipf s Law, capture essential aspect of proportion in music as it relates to music aesthetic. Simple Zipf metrics have a key limitation, they examine the music piece as a whole, and ignore potentially significant details. For example, sorting a piece s notes in different order of pitch would produce an unpleasant musical artefact, this artefact exhibits the same distribution as the original piece. Therefore Fractal metrics were used in [29] to handle the limitation of simple metric, the fractal metric will capture how many subdivisions of the piece exhibit the same distribution at many levels of granularity. For example, they recursively applied the simple pitch metric to the piece s half subdivisions, quarter subdivisions, etc. However, as stated by the authors, this condition is a necessary but not sufficient condition; furthermore, switching two notes won t affect the distribution but will produce an unpleasant musical artefact. In this paper we will propose a novel model to classify and generate beautiful music. The first contribution of this paper is showing that there is a link between aesthetically appealing music and the Dyson distribution. The second contribution of A. M. Khalili is with the Faculty of Computing, Engineering and Science, Staffordshire University, United Kingdom, (e-mail: abd.iptmide@gmail.com).
2 this paper is showing that there is a link between aesthetically appealing music and a deeper optimization process that seeks to deliver the highest amount of information using the least amount of energy. The paper is organised as follows: Section II describes the proposed model. The results are listed in Section III, and the paper is concluded in Section IV. II. PROPOSED MODEL In this paper, we will study the simplest kind of beauty that can be found in a short piece of music and can be appreciated universally. Our brain is able to distinguish very short piece of music and determine whether it is beautiful or not. This may suggest that there is a more fundamental law that is taking place. Therefore, we won t study the whole pieces of music, we will only study the most aesthetically pleasing part of the piece. The most important aspect in identifying beautiful music is by studying the spacing or the transition pattern between different notes, i.e. the difference between the frequencies of successive notes. For simplicity, the frequencies of the studied pieces will be scaled down to a scale close to the human voice. By plotting the transition patterns of different beautiful pieces, a distribution similar to Dyson distribution shows up, which match up with broad range empirical spacing distribution. Fig. 1 shows the distribution of the transitions of parts of three beautiful pieces by different composers (And the waltz goes on, gulumcan, and khagali alshadeed najat). However the distribution alone can t capture the spatial arrangement of the piece; therefore, we will use the multilevel approach to represent the spatial arrangement of the piece. Fig. 1. The distribution of the transition pattern of three beautiful pieces. If we take the following piece [120, 160, 170, 145], the values here represent the frequencies of the notes, the corresponding transition pattern will be [40, 10, -25]. We notice that there is a change in the direction (sign change). The transition pattern will be divided into two directions, the positive direction [40, 10] and the negative direction [-25]. We will use the following representation that will produce two more levels, the second level represents the spatial arrangement in the same direction by subtracting the transitions in each direction, and the third level represents the spatial arrangement of the two directions by subtracting the sum of the two directions. For example the second level representation for the previous piece will be [40-10,-25] = [30, -25] and the third level will be (40+10)-(-25) = 75. The second level will give different values if we switch two transitions in the same direction, and the third level will give different values if we switch the two whole directions. If we combine the three levels and plot their distribution we will get a distribution similar to Dyson distribution Fig.2. Fig. 2. The three levels distribution of a beautiful piece. However, again this condition is not enough because it shows up for many pieces, many of which are not beautiful. The distribution seems to be the result of a deeper optimization process between randomness & regularity, but a more fundamental law is needed to capture beautiful pieces among all pieces that satisfy the distribution. Dyson distribution has shown up in many phenomena where there is a balance between randomness and regularity [33]. We will use entropy as a measure of randomness, and energy as a measure of regularity. Statistical metric such as entropy can t distinguish between different arrangements of the piece, because it is based on the distribution of the values, and not on their spatial arrangement. Furthermore, the Dyson distribution has shown up for all the levels, which suggests that the same law should be satisfied at each level. Therefore, we will use the multilevel approach again to represent the spatial arrangement of the piece, where the first level will represent the frequencies of the notes, the second level represents the transition pattern of the notes, and the third level will represent the difference of the transition pattern of the notes in the same direction. We will use the same hypothesis used in [34] that beautiful patterns are the results of an optimization process between order (the lower the energy the higher the order) and disorder (the higher the entropy the higher the disorder), and beautiful patterns are those which are closer to this optimal point. This optimization process can also be seen as an optimization process that seeks to deliver the highest amount of information using the least amount of energy. The entropy energy ratio over multiple level will be used as a measure of aesthetic quality, the measure of aesthetic quality M is given by (1)
3 M = Entropy(Li)/Energy (Li) (1) i L 1, L 2, and L 3 represent the levels described earlier, for example, if we take the following piece [120, 160, 170, 145], L 1 will be [120, 160, 170, 145], L 2 will be [40, 10, -25], and L 3 will be [30, -25]. Then, we shift the values of L 1 such that the minimum value will be 1, for example [120, 160, 170, 145] will be [1, 41, 51, 26]. Entropy is Shannon entropy; however, because of the small number of notes of the tested set, we can t use the Shannon entropy; therefore, we will use different entropy representation [35] given by (2). Although this entropy representation was able to fit the pieces in the set as will be shown, it doesn t give an accurate representation of the original model where the Shannon entropy is used. Entropy(Li) = Li(n) 2 Log (Li(n) 2 ) (2) n Energy is the sum of the square of the values of L i and it is given by (3) Energy(Li) = Li(n) 2 n Equation (1) will be maximised when we have the lowest energy and the highest entropy, the lowest energy case (the most ordered case) will give a pulse at the minimum transition value, and the highest entropy case (the most random case) will give a uniform distribution. The Dyson distribution seems to be the result of this optimization process between order and randomness, which means that we should have lower number of high transitions and higher number of low transitions according to the Dyson distribution. If we take all possible music transitions (5,10,15,..., 40), and built multiple distributions from these transitions such that the sum of the transitions is the same, we notice that the Dyson distribution has the highest entropy energy ratio R. Fig. 3 shows the ratio R for different distributions. (3) delivering the highest amount of information using the least amount of energy. Similar to [34], pieces will only be compared with other pieces in the same energy level, i.e. the sum of the transitions of each level should be the same. We will use the same hypothesis used in [34] that aesthetically appealing patterns will have higher M value when they are compared with patterns in the same energy level. III. RESULTS To test the proposed model, five pieces with four notes length will be considered. The pieces with their M value are shown in Table 1. Fig. 4 shows a visual representation of the pieces, where the x axis represents the duration of the notes, and the y access represents the frequency of the notes. Because we are using different entropy representation, we will require that the pieces satisfy both maximisation of the M value, and showing the same distribution of Fig. 2. All previous pieces showed the same distribution. Fig. 5 shows the three level distribution of the five pieces, where three main clusters can be identified, the middle cluster with three values, the left cluster with two values, and the right cluster with one value. Fig. 3. Entropy energy ratio for different distributions. Fig. 4. Visual representation of the five pieces. Maximising the entropy energy ratio can then be seen as
4 [15, -5, -5], [-15, 5, 5], etc. We will ignore pieces that don t satisfy the distribution condition. At the 25 energy level (25 is the sum of the transitions) we got P 4 and P 5 with the highest M value. At the 45 energy level, we got P 3 with the highest M value. At the 60 energy level, we got P 2 with the highest M value. And at the 75 energy level, we got P 1 with the highest M value. The reported work is a preliminary results. The original model use the Shannon entropy; however, because of the small number of notes of the tested set, the entropy equation used in this paper is a different representation of the entropy, and although it was able to fit the pieces in the set, it doesn t give an accurate representation of the original model where the Shannon entropy is used. Therefore, future work will test larger set of pieces with larger number of notes, where the Shannon entropy can be used. IV. CONCLUSION This paper has presented a novel approach in identifying beautiful music. The proposed model showed that there is a link between beautiful music and a deeper optimisation process that seeks to deliver the highest amount of information using the least amount of energy. We also showed that there is a link with the Dyson distribution which seems to be the result of this optimisation process. The results showed that the proposed model was able to identify beautiful music. Fig. 5. The distributions of the five pieces. Then at each energy level, we will generate all possible combinations and keep only those that satisfy both the distribution and the maximisation of the M value, and see if we will get the same pieces. Table. 1. The tested pieces with their M value Piece M value P1 = [120, 160, 170, 145] 2.118 P2 = [120, 155, 150, 130] 2.055 P3 = [120, 125, 130, 95] 2.098 P4 = [120, 135, 140, 135] 1.513 P5 = [120, 125, 120, 105] 1.513 Firstly, we will compare each piece with its all possible transition arrangements. For example, if we take P 1, the possible transition arrangements will be: [40, 10, -25], [10, 40, -25], [40, -25, 10], [-25, 10, 40], [-25, 40, 10], [10, -25, 40]. For all of the previous pieces, the entropy energy ratio of the first level of the original transition arrangement give the highest values among all arrangements that satisfy the same distribution. Then, we will compare each piece with all possible transition arrangements that have the same energy level. For example, if we take P 4, it will be compared with all arrangements of the following transition patterns: [15, 5, -5], REFERENCES [1] A. Chatterjee, Neuroaesthetics: a coming of age story, Journal of Cognitive Neuroscience, vol. 23, no. 1, pp. 53 62, 2011. [2] H. Leder, B. Belke, A. Oeberst, and D. Augustin, A model of aesthetic appreciation and aesthetic judgments, British Journal of Psychology, vol. 95, no. 4, pp. 489-508, 2004. [3] K. Hammermeister, The German aesthetic tradition, Cambridge University Press, 2002. [4] T. Gracyk, Hume s aesthetics, Stanford encyclopedia of Philosophy, winter 2011. [5] D. Burnham, Kant s aesthetics Internet encyclopedia of philosophy, 2001. [6] J. Shelley, The concept of the aesthetic, Stanford encyclopedia of Philosophy, spring 2012. [7] E. A.Vessel, and N. Rubin, Beauty and the beholder: highly individual taste for abstract but not real-world images, Journal of Vision, vol. 10, no. 2, 2010. [8] X. Lu, Z. Lin, H. Jin, J. Yang, and J. Z. Wang, Rapid: Rating pictorial aesthetics using deep learning, in Proc. ACM Int. Conf. Multimedia, pp. 457 466, 2014. [9] Y. Kao, C. Wang, and K. Huang, Visual aesthetic quality assessment with a regression model, in Proc. IEEE Int. Conf. Image Process., pp. 1583 1587, 2015. [10] X. Lu, Z. Lin, X. Shen, R. Mech, and J. Z. Wang, Deep multi-patch aggregation network for image style, aesthetics, and quality estimation, in Proc. IEEE Int. Conf. Comput. Vis., pp. 990 998, 2015. [11] L. Mai, H. Jin, and F. Liu, Composition-preserving deep photo aesthetics assessment, in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., pp. 497 506, 2016. [12] J.J. Bharucha, and P.M. Todd, Modeling the perception of tonal structure with neural nets, Computer Music Journal, vol. 13, no. 4, pp. 44 53, 1989. [13] M.C. Mozer, Neural network music composition by prediction: Exploring the benefits of psychoacoustic constraints and multi-scale processing, Connection Science, vol. 6, pp. 2-3, 1996. [14] C.J. Chen, and R. Miikkulainen, Creating melodies with evolving recurrent neural networks, In International Joint Conference on Neural Networks, 2001.
[15] D. Eck, and J. Schmidhuber, A first look at music composition using lstm recurrent neural networks, 2002. [16] N. Boulanger-lewandowski, Y. Bengio, and P. Vincent, Modeling temporal dependencies in high-dimensional sequences: Application to polyphonic music generation and transcription, In ICML, 2012. [17] C. Thornton, Hierarchical markov modeling for generative music, 2009. [18] T. M. Browne, and C. Fox, Global Expectation-Violation as fitness function in evolutionary composition, In Applications of Evolutionary Computing, p. 538 546, 2009. [19] A. E. Coca, R. A. Romero, and L. Zhao, Generation of composed musical structures through recurrent neural networks based on chaotic inspiration, In Proceedings of the International Joint Conference on Neural Networks, p. 3220 3226, 2011 [20] M. Johnson, D. R. Tauritz, and R. Wilkerson, Evolutionary computation applied to melody generation, In Proc. of the Artificial Neural Networks in Engineering (ANNIE), 2004. [21] M. Alfonseca, M. Cebri an, and A. Ortega, Evolving computer-generated music by means of the normalized compression distance, In WSEAS Transactions on Information Science and Applications, vol. 2, pp. 1367 1372, 2005. [22] J. H. Jensen, Evolutionary music composition: A quantitative approach, Ph.D. thesis, 2011. [23] G. Birkhoff, Aesthetic Measure, Harvard University Press, 1933. [24] H.J. Eysenck, An experimental study of aesthetic preference for polygonal figures, The Journal of General Psychology, vol. 79, no. 1, pp. 3 17, 1968. [25] H.J. Eysenck, The empirical determination of an aesthetic formula, Psychological Review, vol. 48, no. 1, 1941. [26] H.J. Eysenck, The experimental study of the good gestalt a new approach, Psychological Review, vol. 49, no. 4, 1942. [27] MA. Javid, T. Blackwell, R. Zimmer, and MM. Al-Rifaie, Correlation between Human Aesthetic Judgement and Spatial Complexity Measure, International Conference on Evolutionary and Biologically Inspired Music and Art, pp. 79-91, Mar, 2016. [28] H.W. Franke, A cybernetic approach to aesthetics, Leonardo, vol. 10, no. 3, pp. 203 206, 1977. [29] B. Manaris, J. Romero, P. Machado, D. Krehbiel, T. Hirzel, W. Pharr, and RB. Davis, Zipf's law, music classification, and aesthetics, Computer Music Journal, vol. 29, no. 1, pp. 55-69, 2005. [30] R. Arnheim, Art and visual perception: A psychology of the creative eye, Univ of California Press, 1954. [31] R. Arnheim, Towards a psychology of art/entropy and art an essay on disorder and order, The Regents of the University of California, 1966. [32] R. Arnheim, Visual thinking, Univ of California Press, 1969. [33] P. Deift, Universality for mathematical and physical systems, arxiv preprint math-ph/0603038, Mar, 2006. [34] A. Khalili, On the mathematics of beauty: beautiful images, arxiv preprint arxiv:1705.08244, May, 2017. [35] R.R. Coifman, M.V. Wickerhauser, Entropy-based Algorithms for best basis selection, IEEE Trans. on Inf. Theory, vol. 38, no. 2, pp. 713 718, 1992. 5