Developing multitrack audio e ect plugins for music production research

Similar documents
TEN YEARS OF AUTOMATIC MIXING

A Semantic Approach To Autonomous Mixing

Crossroads: Interactive Music Systems Transforming Performance, Production and Listening

Convention Paper Presented at the 145 th Convention 2018 October 17 20, New York, NY, USA

Convention Paper Presented at the 139th Convention 2015 October 29 November 1 New York, USA

Jacob A. Maddams, Saoirse Finn, Joshua D. Reiss Centre for Digital Music, Queen Mary University of London London, UK

Convention Paper 9700 Presented at the 142 nd Convention 2017 May 20 23, Berlin, Germany

Towards an Automated Multitrack Mixing Tool using Answer Set Programming

Autonomous Multitrack Equalization Based on Masking Reduction

Towards a better understanding of mix engineering

TOWARD AN INTELLIGENT EDITOR FOR JAZZ MUSIC

Abbey Road TG Mastering Chain User Guide

Semantic description of timbral transformations in music production

soothe audio processor Manual and FAQ

Award Winning Stereo-to-5.1 Surround Up-mix Plugin

Enhancing Music Maps

Advance Certificate Course In Audio Mixing & Mastering.

Voxengo Soniformer User Guide

Perceptual Mixing for Musical Production

1 Introduction. 2 Features. Welcome to CS-3301, a channel strip plugin with gate, EQs, compressor, saturation and oversampling.

Reference Guide Version 1.0

LA xlimit. Manual. by tb-software (C) tb-software 2015 Page 1 of 6

M-16DX 16-Channel Digital Mixer

Liquid Mix Plug-in. User Guide FA

Perceptual Evaluation and Analysis of Reverberation in Multitrack Music Production

+10db Compressor User s Manual. +10db Compressor. User s Manual

Navigating the mix space : theoretical and practical level balancing technique in multitrack music mixtures

DW Drum Enhancer. User Manual Version 1.

AB-Loudness Match. Manual. by TBProAudio (C) TBProAudio 2018 Page 1 of 9

Sound, Music Lighting and Live Production. Short Courses

Eventide Inc. One Alsan Way Little Ferry, NJ

Witold MICKIEWICZ, Jakub JELEŃ

LIO-8 Quick Start Guide

Piano Transcription MUMT611 Presentation III 1 March, Hankinson, 1/15

Dynamic Spectrum Mapper V2 (DSM V2) Plugin Manual

An ecological approach to multimodal subjective music similarity perception

CLA MixHub. User Guide

White Paper Measuring and Optimizing Sound Systems: An introduction to JBL Smaart

Voxengo PHA-979 User Guide

A prototype system for rule-based expressive modifications of audio recordings

Introducing the New Daking Console

This project will work with two different areas in digital signal processing: Image Processing Sound Processing

ZYLIA Studio PRO reference manual v1.0.0

Eventide Inc. One Alsan Way Little Ferry, NJ

Audio and Video II. Video signal +Color systems Motion estimation Video compression standards +H.261 +MPEG-1, MPEG-2, MPEG-4, MPEG- 7, and MPEG-21

Analysis of Peer Reviews in Music Production

Eventide Inc. One Alsan Way Little Ferry, NJ

The Digital Audio Workstation

WAVES Cobalt Saphira. User Guide

ULN-8 Quick Start Guide

Advanced I Multi-Track Recording. Class requirements. 1) Cover tune. 2) "Anything goes" with permission of instructor.

Analog Code MicroPlug Manual. Attacker Plus

Toward a Computationally-Enhanced Acoustic Grand Piano

^ÄtÇz{xÄÅ IVGI 2. Table of Contents. Introduction 2 Installation 3 Controls 4 Credits 5

Eventide Inc. One Alsan Way Little Ferry, NJ

Interframe Bus Encoding Technique for Low Power Video Compression

Eventide Inc. One Alsan Way Little Ferry, NJ

Dr. Speaker Blower and Presents

SPL Analog Code Plug-ins Manual Classic & Dual-Band De-Essers

Technical Datasheet. Overview. Qu-SB v1.9 Technical Datasheet 1 Allen & Heath Ltd

Chapter 4 Signal Paths

User Manual Tonelux Tilt and Tilt Live

1 Prepare to PUNISH! 1.1 System Requirements. Plug-in formats: Qualified DAW & Format Combinations: System requirements: Other requirements:

Using Cubase SE with DSP Factory

Bionic Supa Delay Disciples Edition

Intelligent Monitoring Software IMZ-RS300. Series IMZ-RS301 IMZ-RS304 IMZ-RS309 IMZ-RS316 IMZ-RS332 IMZ-RS300C

OPTIMIZATION OF MIDI SYNTHESIZER ON THE ILLUSTRATION OF MOVIE MUSIC

StudioLive RM32AI RM16AI Applications Guide

OUTER SPACE USER GUIDE

Eventide Inc. One Alsan Way Little Ferry, NJ

Automatic Music Clustering using Audio Attributes

SPL Analog Code Plug-in Manual

Speech Recognition and Signal Processing for Broadcast News Transcription

Set-Top Box Video Quality Test Solution

TF5 / TF3 / TF1 DIGITAL MIXING CONSOLE. TF StageMix User's Guide

CTP 431 Music and Audio Computing. Course Introduction. Graduate School of Culture Technology (GSCT) Juhan Nam

Contents. Welcome to LCAST. System Requirements. Compatibility. Installation and Authorization. Loudness Metering. True-Peak Metering

THE PROJECT MOMS/ PRIRODA MAIN RESULTS IN H.-D. Bettac*, K.-D. Missling*, V. Panchenko**, T. Tyan**, V. Voroncov**

TABLE OF CONTENTS TABLE OF CONTENTS TABLE OF CONTENTS. 1 INTRODUCTION 1.1 Foreword 1.2 Credits 1.3 What Is Perfect Drums Player?

Multiroom Multimedia Distribution

1608-II and 2448 Recording Console Operator s Manual

Audio Source Separation: "De-mixing" for Production

BOUNCE. COMPRESSOR with Analog Sound & Digital Transparency USER MANUAL

Interframe Bus Encoding Technique and Architecture for MPEG-4 AVC/H.264 Video Compression

Fast thumbnail generation for MPEG video by using a multiple-symbol lookup table

«Limiter 6» Modules and parameters description

Introduction! User Interface! Bitspeek Versus Vocoders! Using Bitspeek in your Host! Change History! Requirements!...

How to overcome/avoid High Frequency Effects on Debug Interfaces Trace Port Design Guidelines

Long-term Preservation of Acousmatic Works: Toward a Generic Model of Description

StudioLive RM32AI RM16AI Applications Guide

NOTICE. (Formulated under the cognizance of the CTA R4 Video Systems Committee.)

UNIVERSAL SPATIAL UP-SCALER WITH NONLINEAR EDGE ENHANCEMENT

Design and Implementation of Partial Reconfigurable Fir Filter Using Distributed Arithmetic Architecture

Music Similarity and Cover Song Identification: The Case of Jazz

Real Time Face Detection System for Safe Television Viewing

A REAL-TIME SIGNAL PROCESSING FRAMEWORK OF MUSICAL EXPRESSIVE FEATURE EXTRACTION USING MATLAB

A Study of Synchronization of Audio Data with Symbolic Data. Music254 Project Report Spring 2007 SongHui Chon

New forms of video compression

Smart Traffic Control System Using Image Processing

ONE SENSOR MICROPHONE ARRAY APPLICATION IN SOURCE LOCALIZATION. Hsin-Chu, Taiwan

Transcription:

Developing multitrack audio e ect plugins for music production research Brecht De Man Correspondence: <b.deman@qmul.ac.uk> Centre for Digital Music School of Electronic Engineering and Computer Science Queen Mary, University of London 9 November 205 @BrechtDeMan Multitrack plugins for music production research 9 November 205 / 2

$whoami Research at the Centre for Digital Music (C4DM) 60+ academics, post-docs, PhD students Area: audio engineering, intelligent music production @BrechtDeMan Multitrack plugins for music production research 9 November 205 2 / 2

Why a (JUCE) plugin? I Rapid prototyping Idea Algorithm Stand-alone software Hardware I Compatibility with existing audio production tools I Focus on processing Platform independence Audio input and output taken care of (real time!) Easy GUI I JUCE encourages good coding practice, made my life a lot easier is very well thought through, well integrated, and just Great! @BrechtDeMan Multitrack plugins for music production research 9 November 205 3 / 2

Why multitrack? I Cross-adaptive functionality I Avoid sidechaining clutter I Modelling of larger system I Simple setup I Spatial applications 2 @BrechtDeMan Multitrack plugins for music production research 9 November 205 4 / 2

Why multitrack? I Cross-adaptive functionality I Avoid sidechaining clutter I Modelling of larger system I Simple setup I Spatial applications @BrechtDeMan Multitrack plugins for music production research 9 November 205 4 / 2

Why multitrack? I Cross-adaptive functionality I Avoid sidechaining clutter I Modelling of larger system I Simple setup I Spatial applications @BrechtDeMan Multitrack plugins for music production research 9 November 205 4 / 2

Why multitrack? I Cross-adaptive functionality I Avoid sidechaining clutter I Modelling of larger system I Simple setup I Spatial applications @BrechtDeMan Multitrack plugins for music production research 9 November 205 4 / 2

Why multitrack? I Cross-adaptive functionality I Avoid sidechaining clutter I Modelling of larger system I Simple setup I Spatial applications @BrechtDeMan Multitrack plugins for music production research 9 November 205 4 / 2

Why multitrack? I Cross-adaptive functionality I Avoid sidechaining clutter I Modelling of larger system I Simple setup I Spatial applications Researchers, developers, artists,... @BrechtDeMan Multitrack plugins for music production research 9 November 205 4 / 2

Example: Automatic faders @BrechtDeMan Multitrack plugins for music production research 9 November 205 5 / 2

Example: Automatic EQ @BrechtDeMan Multitrack plugins for music production research 9 November 205 6 / 2

Example: Audio alignment @BrechtDeMan Multitrack plugins for music production research 9 November 205 7 / 2

Example: Listening test @BrechtDeMan Multitrack plugins for music production research 9 November 205 8 / 2

Example: Semantic compressor @BrechtDeMan Multitrack plugins for music production research 9 November 205 9 / 2

Others Full applications @BrechtDeMan Multitrack plugins for music production research 9 November 205 0 / 2

Others TouchKeys @BrechtDeMan Multitrack plugins for music production research 9 November 205 / 2

Others Non-multitrack (<= 2)plugins SAFE plugins (semanticaudio.co.uk) 4:30pm today: The SAFE JUCE module: A System for Managing Music Production Metadata by Sean Enderby intelligent Audio Switch Box Audio E ects (Reiss & McPherson) QMUL Digital Audio E ects course... @BrechtDeMan Multitrack plugins for music production research 9 November 205 2 / 2

DAW multitrack plugin support Few DAWs support multitrack VSTs/AUs @BrechtDeMan Multitrack plugins for music production research 9 November 205 3 / 2

DAW multitrack plugin support Cockos Reaper @BrechtDeMan Multitrack plugins for music production research 9 November 205 3 / 2

DAW multitrack plugin support AudioMulch @BrechtDeMan Multitrack plugins for music production research 9 November 205 3 / 2

Avid Pro Tools AAX not accessible to us lowly researchers You must be willing to make your product available for commercial sale to end-users. We don t o er our developer materials or services for academic use, experimentation, or just to check them out. (https://www.avid.com/us/partners/audio-plugin-dev-program) @BrechtDeMan Multitrack plugins for music production research 9 November 205 4 / 2

Complexity I HeapBlock I OwnedArray I for (int trk = 0; trk < numtracks; ++trk) @BrechtDeMan Multitrack plugins for music production research 9 November 205 5 / 2

CPU I Scaling factor I Inter-dependencies, cross-adaptiveness,... I O(?) I E ciency more critical @BrechtDeMan Multitrack plugins for music production research 9 November 205 6 / 2

CPU @BrechtDeMan Multitrack plugins for music production research 9 November 205 6 / 2

Interfaces Large, complex @BrechtDeMan Multitrack plugins for music production research 9 November 205 7 / 2

Interfaces Pagination @BrechtDeMan Multitrack plugins for music production research 9 November 205 7 / 2

Concluding remarks I Multitrack audio plugins provide unique opportunities and challenges Cross-adaptive audio streams, complex architectures Compatibility, complexity I Academic focus 6= industryfocus Prototyping, data collection Computational e ciency, slick interfaces I JUCE supports multitrack plugins and enables quick development Use cases for multitrack plugins are limited, but support could stimulate development @BrechtDeMan Multitrack plugins for music production research 9 November 205 8 / 2

References I S. Mansbridge, S. Finn, and J. D. Reiss, Implementation and evaluation of autonomous multi-track fader control, in 32nd Convention of the Audio Engineering Society, April202. E. Perez-Gonzalez and J. D. Reiss, Automatic gain and fader control for live mixing, IEEE Workshop on applications of signal processing to audio and acoustics, October2009. E. Perez-Gonzalez and J. D. Reiss, Automatic equalization of multi-channel audio using cross-adaptive methods, 27th Convention of the Audio Engineering Society, October2009. S. Hafezi and J. D. Reiss, Autonomous multitrack equalisation based on masking reduction, to appear in Journal of the Audio Engineering Society, 205. A. Cli ord and J. D. Reiss, Calculating time delays of multiple active sources in live sound, in 29th Convention of the Audio Engineering Society, 200. A. Cli ord and J. D. Reiss, Reducing comb filtering on di erent musical instruments using time delay estimation, Journal of the Art of Record Production, vol.issue5,july20. N. Jillings, A. Cli ord, and J. D. Reiss, Performance optimization of gcc-phat for delay and polarity correction under real world conditions, in 34th Convention of the Audio Engineering Society, 203. Z. Ma, B. De Man, P. D. Pestana, D. A. A. Black, and J. D. Reiss, Intelligent multitrack dynamic range compression, Journal of the Audio Engineering Society, vol.63,pp.42 426,June205. T. Wilmering, G. Fazekas, and M. B. Sandler, High-level semantic metadata for the control of multitrack adaptive digital audio e ects, in Audio Engineering Society Convention 33, 0202. A. McPherson, TouchKeys: Capacitive multi-touch sensing on a physical keyboard, in Proc. NIME, 202. @BrechtDeMan Multitrack plugins for music production research 9 November 205 9 / 2

References II R. Stables, S. Enderby, B. De Man, G. Fazekas, and J. D. Reiss, SAFE: A system for the extraction and retrieval of semantic audio descriptors, in 5th International Society for Music Information Retrieval Conference (ISMIR 204), October204. J. D. Reiss and A. McPherson, Audio E ects: Theory, Implementation and Application. CRC Press, 205. S. Mansbridge, S. Finn, and J. D. Reiss, An autonomous system for multi-track stereo pan positioning, in 33rd Convention of the Audio Engineering Society, October202. E. Perez-Gonzalez and J. D. Reiss, Automatic mixing: Live downmixing stereo panner, in 0th International Conference on Digital Audio E ects (DAFx-0), 2007. E. Perez Gonzalez and J. D. Reiss, A real-time semiautonomous audio panning system for music mixing, EURASIP Journal on Advances in Signal Processing, 200. @BrechtDeMan Multitrack plugins for music production research 9 November 205 20 / 2

Q&A b.deman@qmul.ac.uk brechtdeman.com @BrechtDeMan github.com/brechtdeman linkedin.com/in/brechtdeman See also: 4:30pm today: The SAFE JUCE module: A System for Managing Music Production Metadata by Sean Enderby @BrechtDeMan Multitrack plugins for music production research 9 November 205 2 / 2