Rechnergestützte Methoden für die Musikethnologie: Tool time!

Rechnergestützte Methoden für die Musikethnologie: Tool time! André Holzapfel MIAM, ITÜ, and Boğaziçi University, Istanbul, Turkey andre@rhythmos.org 02/2015 - Göttingen André Holzapfel (BU/ITU) Tool time! 02/2014 1 / 29

Outline 1 Introduction 2 Pitch Analysis: Praat 3 Sonic Visualizer 4 Tarsos 5 Python André Holzapfel (BU/ITU) Tool time! 02/2014 2 / 29

Outline 1 Introduction 2 Pitch Analysis: Praat 3 Sonic Visualizer 4 Tarsos 5 Python André Holzapfel (BU/ITU) Tool time! 02/2014 3 / 29

Praat: What is it? Software for speech analysis developed by Paul Boersma and David Weenink of the University of Amsterdam. First version 1995, most recent version Dec. 2014. Freely available at http://www.fon.hum.uva.nl/praat/. Versions for all operation systems available. Suitable for: Pitch analysis of monophonic music signals. Spectral and timbre analysis of instruments or vocal signals. André Holzapfel (BU/ITU) Tool time! 02/2014 4 / 29

Praat: Let s look at it! Visualizations for: spectrum, spectrogram, pitch analysis, intensity, formant analysis etc... Many possibilities to manipulate signal analysis. Highly accurate analysis for monophonic signals. Very convenient for the production of figures for documentation/publication. André Holzapfel (BU/ITU) Tool time! 02/2014 5 / 29

Praat: Let s look at it! To open wav: Praat objects window: Open:Read from file. Look at wavform: Praat objects window: View and edit button. You can switch on and off spectrogram, pitch contour, and intensity. Clicking on the buttons below the display causes audio replay. André Holzapfel (BU/ITU) Tool time! 02/2014 6 / 29

Outline 1 Introduction 2 Pitch Analysis: Praat 3 Sonic Visualizer 4 Tarsos 5 Python André Holzapfel (BU/ITU) Tool time! 02/2014 7 / 29

Sonic Visualizer: Basics Download from http://www.sonicvisualiser.org/ On order to do advanced things, you need plugins: http://www.vamp-plugins.org/ Nice new gimmick: you can use the vamp plugins in audacity as well (http://audacity.sourceforge.net/) Valuable tutorial: http://www.charm.rhul.ac.uk/analysing/p9_0_1.html Sonic Visualizer is not an audio editor. Basic idea: put different sound visualizations on top of each other. Several of these layers can be grouped into different panes, to enable for different views in parallel. André Holzapfel (BU/ITU) Tool time! 02/2014 8 / 29

Sonic Visualizer: Spectrogram Use the wheels in the bottom right to zoom horizontally and vertically....if the wheels are not there: View:Show Zoom wheels Typical settings that enable for a nice overview of melodic and spectral properties: 1 Scale: dbv 2, Normalize Visible Area. 2 Window: 4096 or 8192, Large overlap. 3 Bins: All Bins, log arithmic axis. In the defaults colormap you go from high to low energy using the colors red-yellow-green-black André Holzapfel (BU/ITU) Tool time! 02/2014 9 / 29

Sonic Visualizer: Example Annotate sama in Carnatic music How to do live annotations. How to edit their positions. How to edit their labels. How to store them: 1 Store the whole sonic visualizer session if you want to continue later (File->Save Session). 2 Store the annotation only (File->Export Annotation Layer). Usually, csv or txt files are the most convenient. André Holzapfel (BU/ITU) Tool time! 02/2014 10 / 29

Plug-in: BBC speech/music segmenter Potentially useful to segment e.g. music seminar recordings. Problems: cryptic parameters, apparently not very robust to noisy enviroments. Recipe: use spectrogram in combination with segmentation result plotted as CURVE. Example of a seminar recording André Holzapfel (BU/ITU) Tool time! 02/2014 11 / 29

Plug-in: BBC speech/music segmenter Label interesting segments in Sonic Visualizer: 1 Add new time instants layer 2 Change between Navigate and Draw tools 3 Then export labels ( File:Export Annotation Layer ) Read audio into audacity, import labels ( File:Import:Labels ) Export segments separately ( File:Export Multiple: ) Unfortunately, you have to delete every second of the exported files manually. André Holzapfel (BU/ITU) Tool time! 02/2014 12 / 29

Plug-in: pyin (monophonic pitch) good way to estimate pitch for monophonic sounds in Sonic Visualizer Use "pyin:smoothed Pitch Track" Estimation can be manually corrected: 1 Activate layer with the pitch estimation. 2 Choose "DRAW"tool, in order to spot mistakes set Scale to Auto-Align". 3 Spot problems of pitch estimation on top of spectrogram, set Scale to "Linear". 4 Edit the line, switch back to Auto-Align". Solo voice recording and pitch estimated by pyin André Holzapfel (BU/ITU) Tool time! 02/2014 13 / 29

Plug-in: melodia (pitch from polyphonic) Good method to estimate a lead melody out of a mixture of instruments 1 (Optional:) Select the regions in the file you want to examine. 2 Do not use it for monophonic pitch, use pyin or Praat. 3 Set frequency borders according to the source you want to estimate. 4 Pull up the voicing tolerance to maximum. If necessary delete artifacts afterwards. Parameters for Melodia André Holzapfel (BU/ITU) Tool time! 02/2014 14 / 29

Plug-in: Silvet note transcription Plugin for estimation of multiple pitches. It has an option for micro-tonality, and can serve for visualization as well. Export to a text file gives you a list of notes with microtonal information. Parameters for Silvet André Holzapfel (BU/ITU) Tool time! 02/2014 15 / 29

Plug-ins: Beat, Tempo Queen Mary: Bar and Beat tracker, Tempo and Beat tracker INESC Porto beat tracker. and others... Currently, all beat tracking tools available as plug-ins in SV were tailored for Western music. Therefore, using these plugins for other musics is trial-and-error. Carl Bussey s tempogram is an interesting visualization where beat tracking fails. André Holzapfel (BU/ITU) Tool time! 02/2014 16 / 29

Outline 1 Introduction 2 Pitch Analysis: Praat 3 Sonic Visualizer 4 Tarsos 5 Python André Holzapfel (BU/ITU) Tool time! 02/2014 17 / 29

Tarsos: a java library for sound processing Get it: http://0110.be/releases/tarsos/tarsos-latest/tarsos-latest.jar Get started: http://tarsos.0110.be/releases/tarsos/tarsoslatest/tarsos-latest-manual.pdf You need a recent Java Runtime Environment (JRE) to run it. André Holzapfel (BU/ITU) Tool time! 02/2014 18 / 29

Tarsos: detecting pitch classes Load an audio example. During playback a pitch-class-profile will build up. Determine peaks in this profile, you can listen to the pitches. Beyati makam André Holzapfel (BU/ITU) Tool time! 02/2014 19 / 29

Outline 1 Introduction 2 Pitch Analysis: Praat 3 Sonic Visualizer 4 Tarsos 5 Python André Holzapfel (BU/ITU) Tool time! 02/2014 20 / 29

Why Python? With all graphical user interfaces, you will always reach point where you want to do a more flexible analysis. At this point some code needs to be written. For this, you have two choices: André Holzapfel (BU/ITU) Tool time! 02/2014 21 / 29

Why Python? With all graphical user interfaces, you will always reach point where you want to do a more flexible analysis. At this point some code needs to be written. For this, you have two choices: 1 Find an engineer for a collaboration. André Holzapfel (BU/ITU) Tool time! 02/2014 21 / 29

Why Python? With all graphical user interfaces, you will always reach point where you want to do a more flexible analysis. At this point some code needs to be written. For this, you have two choices: 1 Find an engineer for a collaboration. 2 Learn some programming. For Python many free tools are available, and it is relatively easy to use. Documentation is rich, but quite technical. André Holzapfel (BU/ITU) Tool time! 02/2014 21 / 29

ipython: an ethnomusicologist-friendly compromise Do annotations, pitch estimations, or whatever else in Sonic Visualizer, Praat,... Export the data you want to look at to a csv file. Import them to Python, and plot and examine the data. ipython notebook Python, as user-friendly as it can get. Integrated into the webbrowser. Valuable tool for demonstrations, and for publications. All is freeware, and all works on Windows, Mac, Linux Drawbacks: 1 You need to install some things (python environment, maths and plotting libraries) 2 Some work with a command line needs to be done. André Holzapfel (BU/ITU) Tool time! 02/2014 22 / 29

ipython: an ethnomusicologist-friendly compromise An example: Visualizing pitch on a cent scale Compute a pitch contour with YIN in Sonic Visualizer and export it to a csv file. Import the data to ipython. Convert Hz to cent (relative to tonic). Plot histograms, and pitch contours. Advantage: You might be able to get these plots in other tools, but using ipython you can bring it into shape for a publication. André Holzapfel (BU/ITU) Tool time! 02/2014 23 / 29

ipython: an ethnomusicologist-friendly compromise An example: Visualizing pitch on a cent scale Pitch contour of an excerpts. Scale intervals shown as dotted lines. Pitch histogram of the whole recording. André Holzapfel (BU/ITU) Tool time! 02/2014 24 / 29

Emmanouil Benetos and Andre Holzapfel, INCORPORATING PITCH CLASS PROFILES FOR IMPROVING AUTOMATIC TRANSCRIPTION OF TURKISH MAKAM MUSIC. Proceedings of the 4th Workshop on Folk Music Analysis, 2014. Emmanouil Benetos and Andre Holzapfel, Automatic Transcription of Turkish Makam Music. Proceedings of ISMIR - International Conference on Music Information Retrieval, 2013. André Holzapfel (BU/ITU) Tool time! 02/2014 25 / 29