Analysis, Synthesis, and Perception of Musical Sounds

Transcription

1 Analysis, Synthesis, and Perception of Musical Sounds The Sound of Music James W. Beauchamp Editor University of Illinois at Urbana, USA 4y Springer

2 Contents Preface Acknowledgments vii xv 1. Analysis and Synthesis of Musical Instrument Sounds 1 James W. Beauchamp 1 Analysis/Synthesis Methods Harmonie Filter Bank (Phase Vocoder) Analysis/Synthesis Frequency Deviation and Inharmonicity Heterodyne-Filter Analysis Method Window Functions Harmonie Analysis Limits Synthesis from Harmonie Amplitudes and Frequency Deviations Signal Reconstruction (Resynthesis) and the Band-Pass Filter Bank Equivalent Sampled Signal Implementation Analysis Step Synthesis Step Piecewise Constant Amplitudes and Frequencies Piecewise Linear Amplitude and Frequency Interpolation Piecewise Quadratic Interpolation of Phases Piecewise Cubic Interpolation of Phases Spectral Frequency-Tracking Method Frequency-Tracking Analysis Frequency-Tracking Algorithm Fundamental Frequency (Pitch) Detection 33

3 xviii Contents Reduction of Frequency-Tracking Analysis to Harmonie Analysis Frequency-Tracking Synthesis Frequency-Tracking Additive Synthesis Residual Noise Analysis/Synthesis Frequency-Tracking Overlap-Add Synthesis 40 2 Analysis Results Using SNDAN Analysis File Data Formats Phase-Vocoder Analysis Examples for Fixed-Pitch Harmonie Musical Sounds Spectral Centroid Spectral Envelopes Spectral Irregularity Phase-Vocoder Analysis of Sounds with Inharmonic Partiais Inharmonicity of Slightly Inharmonic Sounds: The Piano Measurement of Tones with Widely Spaced Partiais: TheChime Measurement of a Sound with Dense Partiais: The Cymbal Spectrotemporal Incoherence Inverse Spectral Density: Cymbal, Chime, and Timpani Frequency-Tracking Analysis of Harmonie Sounds Frequency-Tracking Analysis of Steady Harmonie Sounds Frequency-Tracking Analysis of Vibrato Sounds: The Singing Voice Frequency-Tracking Analysis of Variable-Pitch Sounds 81 3 Summary 82 References Fundamental Frequency Tracking and Applications to Musical Signal Analysis 90 Judith C. Brown 1 Introduction to Musical Signal Analysis in the Frequency Domain 90 2 Calculation of a Constant-Q Transform for Musical Analysis Background Calculations Results 96

4 Contents xix 3 Musical Fundamental-Frequency Tracking Using a Pattern-Recognition Method Background Calculations Results High-Resolution Frequency Calculation Based on Phase Differences Introduction Results Using the High-Resolution Frequency Tracker Applications of the High-Resolution Pitch Tracker Frequency Ratios of Spectral Components of Musical Sounds Background Calculation Results Cello Alto Flute Discussion Perceived Pitch Center of Bowed String Instrument Vibrato Tones Background Experimental Method Sound Production and Manipulation Listening Experiments Results Experiment 1: NonProfessional-Performer Listeners Experiment 2: Graduate-Level and Professional Violinist Listeners Experiment 3: Determination of JND for Pitch Summary and Conclusions 116 Appendix A: An Efficient Algorithm for the Calculation of a Constant-Q Transform 116 Appendix B: Single-Frame Approximation Calculation of Phase Change for a Hop Size of One Sample 117 References Beyond Traditional Sampling Synthesis: Real-Time Timbre Morphing Using Additive Synthesis 122 Lippold Haken, Kelly Fitz, and Paul Christensen 1 Introduction Additive Synthesis Model Real-Time Synthesis 124

5 xx Contents 2.2 Envelope Parameter Streams Noise Envelopes Additive Sound Analysis Sinusoidal Analysis Noise-Enhanced Sinusoidal Analysis Spectral Reassignment Time Reassignment Frequency Reassignment Spectral-Reassignment Summary Navigating Source Timbres: Timbre Control Space Creating a New Timbre Control Space Timbre Control Space with More Control Dimensions Producing Intermediate Timbres: Timbre Morphing Weighting Functions for Real-Time Morphing Time Dilation Using Time Envelopes Morphed Envelopes Low-Amplitude Partiais New Possibilities for the Performer: The Continuum Fingerboard Previous Work Mechanical Design of the Playing Surface Final Summary 142 References A Compact and Malleable Sines+Transients+Noise Model for Sound 145 Scott N. Levine and Julius O. Smith III 1 Introduction History of Sinusoidal Modeling Audio Signal Models for Data Compression and Transformation Chapter Overview System Overview Related Current Systems Time-Frequency Segmentation Reasons for the Different Models Multiresolution Sinusoidal Modeling Analysis Filter Bank Sinusoidal Parameters Sinusoidal Tracking Masking Sinusoidal Trajectory Elimination Sinusoidal Trajectory Quantization Switched Phase Reconstruction Cubic-Polynomial Phase Reconstruction 160

6 Contents xxi Phaseless Reconstruction Phase S witching Transform-Coded Transients Transient Detection A Simplified Transform Coder Time-Frequency Pruning Noise Modeling Bark-Band Quantization Line-Segment Approximation Applications Sinusoidal Time-Scale Modification Transient Time-Scale Modification Noise Time-Scale Modification Conclusions Acknowledgment 171 References Spectral Envelopes and Additive + Residual Analysis/Synthesis 175 Xavier Rodet and Diemo Schwarz 1 Introduction Spectral Envelopes and Source-Filter Models Source-Filter Models Source-Filter Models Represented by Spectral Envelopes Spectral Envelopes and Perception Source and Spectrum Tilt Properties of Spectral Envelopes Spectral Envelope Estimation Methods Requirements Autoregression Spectral Envelope Disadvantage of AR Spectral Envelope Estimation Cepstrum Spectral Envelope Disadvantages of the Cepstrum Method Discrete Cepstrum Spectral Envelope Improvements on the Discrete Cepstrum Method Regularization Stochastic Smoothing (the Cloud Method) Nonlinear Frequency Scaling Estimation of the Spectral Envelope of the Residual Signal Representation of Spectral Envelopes Requirements Filter Parameters 206

7 xxii Contents 4.3 Frequency Domain Sampled Representation Geometrie Representation Formants Formant Wave Functions Basic Formants Fuzzy Formants Discussion of Formant Representation Comparison of Representations Transcoding and Manipulation of Spectral Envelopes Transcodings Converting Formants to AR-Filter Coefficients Formant Estimation Manipulations Morphing Shifting Formants Shifting Fuzzy Formants Morphing Between Well-Defined Formants Summary of Formant Morphing Synthesis with Spectral Envelopes Filter Synthesis Additive Synthesis Additive Synthesis with the FFT" 1 Method Applications Controlling Additive Synthesis Synthesis and Transformation of the Singing Voice Conclusions Summary 220 Appendix: List of Symbols 221 References A Comparison of Wavetable and FM Data Reduction Methods for Resynthesis of Musical Sounds 228 Andrew Homer 1 Introduction Evaluation of Wavetable and FM Methods Comparison of Wavetable and FM Methods Generalized Wavetable Matching Wavetable-Index Matching Wavetable-Interpolation Matching Formant-FM Matching Double-FM Matching Nested-FM Matching Results TheTrumpet 241

8 Contents xxiii 4.2 The Tenor Voice ThePipa Conclusions 245 Acknowledgments 247 References The Effect of Dynamic Acoustical Features on Musical Timbre 250 John M. Hajda 1 Introduction Global Time-Envelope and Spectral Parameters Salience of Partitioned Time Segments Relational Timbre Studies Temporal Envelope Spectral Energy Distribution Spectral Time Variance The Experimental Control of Acoustical Variables Conclusions and Directions for Future Research 267 References Mental Representation of the Timbre of Complex Sounds 272 Sophie Donnadieu 1 Timbre: A Problematic Definition The Notion of Timbre Space Continuous Perceptual Dimensions Spectral Attributes of Timbre Temporal Attributes of Timbre Spectrotemporal Attributes of Timbre The Notion of Specificities Individual and Group Listener Differences Evaluating the Predictive Power of Timbre Spaces Perceptual Effects of Sound Modifications Perception of Timbral Intervals The Role of Timbre in Auditory Streaming Context Effects Verbal Attributes of Timbre Semantic Differential Analyses Relations Between Verbal and Perceptual Attributes or Analyses of Verbal Protocols Categories of Timbre Studies of the Perception of Causality of Sound Events Categorical Perception: A Speech-Specific Phenomenon 301

9 xxiv Contents Definition of the Categorical Perception Phenomenon Musical Categories: Plucking and Striking vs Bowing Are the Same Feature Detectors Used for Speech and Nonspeech Sounds? Categorical Perception in Young Infants The McGurk Effect for Timbre Is There a Perceptual Categorization of Timbre? Conclusions 312 References 313 Index 320