MULTIMEDIA SIGNALS AND SYSTEMS

Transcription

1 MULTIMEDIA SIGNALS AND SYSTEMS

2 THE KLUWER INTERNATIONAL SERIES IN ENGINEERING AND COMPUTER SCIENCE

3 MULTIMEDIA SIGNALS ANDSYSTEMS Mrinal Kr. Mandal University of Alberta, Canada SPRINGER SCIENCE+BUSINESS MEDIA, LLC

4 Additional material to this book can be downloaded from Library of Congress Cataloging-in-Publication Data Mandal, Mrinal Kr. Multimedia Signals and Systems / Mrinal Kr. Mandal. p.cm.-(the Kluwer international series in engineering and computer science; SECS 716) lncludes bibliographical references and index. ISBN ISBN (ebook) DOI / Multimedia systems. 2. Signal processing-digitial techniques. I. Title. II. Series. QA M dc Copyright 2003 by Springer Science+Business Media New York Originally published by Kluwer Academic Publishers in 2003 Softcover reprint of the hardcover lst edition 2003 AH rights reserved. No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, mechanical, photocopying, record ing, or otherwise, without the prior written permission of the publisher. MATLAB is a registered trademark ofthe MathWorks, Inc. Printed an acid-free paper.

5 Table of Contents 1. INTRODUCTION 1.1. Development of Multimedia Systems 1.2. Classification of Media 1.3. Properties of Multimedia Systems 1.4. Multimedia Computing 1.5. Different Aspects of Multimedia References Questions Part I: MULTIMEDIA SIGNALS 2. AUDIO FUNDAMENTALS Characteristics of Sound The Human Auditory System Audio Recording Audio Signal Representation Wavefonn method Musical Instrument Digital Interface 24 References 30 Questions THE HUMAN VISUAL SYSTEM AND PERCEPTION Introduction The Human Visual System Relative Luminous Efficiency Weber's Law Modulation Transfer Function HVS Model Color Representation Three-Receptor Model Color Matching Tristimulus Value Chromaticity Diagram Color Models and Transformation of Primaries 48

6 VI Multimedia Signals and Systems NTSC Receiver Primary NTSC Transmission System CIE-UCS Color coordinates CMYModel Temporal Properties of Vision 54 References 55 Questions MULTIMEDIA DATA ACQUISITION Sampling of Audio Signals Sampling of Two-Dimensional Images Anti-Aliasing Filters Digitization of Audio Signals Analog to Digital Conversion Audio Fidelity Criteria MIDI versus Digital Audio Digitization of Images Visual Fidelity Measures 79 References 81 Questions 81 Part ll: SIGNAL PROCESSING AND COMPRESSION 5. TRANSFORMS AND SUBBAND DECOMPOSITION I-D Unitary Transfonn I-D Discrete Fourier Transfonn I-D Discrete Cosine Transfonn Digital Filtering and Subband Analysis Digital Filters Subband Analysis Transfonns and Digital Filtering I-D Discrete Wavelet Transfonn D Unitary Transfonn D Discrete Fourier Transfonn D Discrete Cosine Transfonn D Discrete Wavelet Transfonn 116 References 118 Questions 119

7 Table of Contents vii 6. TEXT REPRESENTATION AND COMPRESSION Text Representation Principles of Text Compression Statistical Redundancy Probability Density Function and Entropy Shannon's Noiseless Source Coding Theorem Huffman Coding Arithmetic Coding Dictionary-based Compression LZ77 Technique LZ78 Technique Summary 143 References 143 Questions DIGITAL AUDIO COMPRESSION Audio Compression Principles Rate Distortion Function Statistical Redundancy Companding and Expanding Temporal Redundancy Perceptual Audio Coding Audio Compression Standards MPEG-l Audio Compression Standard MPEG-2 Audio Compression Standard AC Audio Compression Standards Comparison of Compression Algorithms Audio Formats 166 References 166 Questions DIGITAL IMAGE COMPRESSION TECHNIQUES Principles oflmage Compression Low Complexity Compression Techniques Entropy Coding Run-length Coding Predictive Coding Transform Coding 175

8 VIII Multimedia Signals and Systems 8.3 Transfonn Coding Unitary Transfonn Block Transfonn Wavelet Coding Comparison ofdct and Wavelets Other Coding Techniques Vector Quantization Fractal Image Compression Image Compression Standards The JPEG Image Compression Standard Baseline Sequential Mode Other JPEG Modes The JPEG 2000 Standard Image Fonnats 199 References 200 Questions DIGITAL VIDEO COMPRESSION TECHNIQUES Principles of Video Compression Digital Video and Color Redundancy Temporal Redundancy Reduction Block-based Motion Estimation Fast Motion Estimation Algorithms Video Compression Standards Motion JPEG The MPEG-I Video Compression Standard The MPEG-2 Video Compression Standard The MPEG-4 Video Compression Standard Video Coding Scheme The H.261 Video Compression Standard H.263, H.263+ and H.26L Standards Comparison of Standard Codecs 232 References 235 Questions DIGITAL AUDIO PROCESSING Audio Filtering Techniques 10.2 Audio Equalization 10.3 Audio Enhancement Noise Suppression by Digital Filtering

9 Table of Contents IX Spectral Subtraction Method Editing MIDI Files Digital Audio and MIDI Editing Tools 254 References 255 Questions DIGITAL IMAGE AND VIDEO PROCESSING Basic Image Processing Tools Image Resizing Cropping 11.2 Image Enhancement Techniques Brightness and Contrast Improvement Contrast Stretching Histogram Equalization Image Sharpening 11.3 Digital Video Special Effects and Gradual Transition Wipe Dissolve FadeIn/Out Video Segmentation Camera Operations 11.4 Image and Video Editing Softwares 11.5 Summary References Questions Part m: MULTIMEDIA SYSTEMS 12. ANALOG AND DIGITAL TELEVISION 12.1 Analog Television Standards 12.2 Raster Scanning 12.3 Color Space for TV Transmission NTSC System PAL System 12.4 NTSC Television System Channel Assignment

10 x Multimedia Signals and Systems NTSC Encoder and Decoder Component and S-Video Digital Television Grand Alliance HDTV Standard 299 References 303 Questions CONTENT CREATION AND MANAGEMENT Multimedia Authoring Authoring Steps Multimedia Authoring Tools CardlPage-Based Tools Icon-Based Tools Time-Based Tools Object Oriented Tools Multimedia Documents Hepertext and Hypermedia Nonlinear Information Chain Hypertext and Hypermedia Systems Mark-up Languages HTML XML Web Authoring Tools Multimedia Standards The MPEG-7 Standard The MPEG-21 Standard The MHEG Standard Summary 329 References 330 Questions OPTICAL STORAGE MEDIA Physical Medium Cross-section ofa CD Digital Versatile Disc (DVD) Physical Formats and Speeds Playback of CD and DVD CD-ROM, CD-R, and CD-RW Advantages of Optical Technology CD and DVD Standards 342

11 Table of Contents xi Video CD and DVD-Video Standards References Questions 15. ELECTRONIC DISPLAYS 15.1 Important Display Parameters 15.2 Cathode-Ray Tube 15.3 Field Emission Display 15.4 Plasma Display 15.5 Liquid Crystal Display 15.6 Digital Micromirror Display References Questions APPENDIX: About the CD-ROM INDEX

12 PREFACE Multimedia computing and communications have emerged as a major research and development area. Multimedia computers in particular open a wide range of possibilities by combining different types of digital media such as text, graphics, audio and video. The emergence of the World Wide Web, unthinkable even two decades ago, also has fuelled the growth of multimedia computing. There are several books on multimedia systems that can be divided into two major categories. In the first category, the books are purely technical, providing detailed theories of multimedia engineering, with an emphasis on signal processing. These books are more suitable for graduate students and researchers in the multimedia area. In the second category, there are several books on multimedia, which are primarily about content creation and management. Because the number of multimedia users is increasing daily, there is a strong need for books somewhere between these two extremes. People with engineering or even non-engineering background are now familiar with buzzwords such as JPEG, GIF, W A V, MP3, and MPEG files. These files can be edited or manipulated with a wide variety of software tools. However, the curious-minded may wonder how these files work that ultimately provide us with impressive images or audio. This book intends to fill this gap by explaining the multimedia signal processing at a less technical level. However, in order to understand the digital signal processing techniques, readers must still be familiar with discrete time signals and systems, especially sampling theory, analog-todigital conversion, digital filter theory, and Fourier transform. The book has 15 Chapters, with Chapter 1 being the introductory chapter. The remaining 14 chapters can be divided into three parts. The first part consists of Chapters 2-4. These chapters focus on the multimedia signals, namely audio and image, their acquisition techniques, and properties of human auditory and visual systems. The second part consists of Chapters These chapters focus on the signal processing aspects, and are strongly linked in order to introduce the signal processing techniques step-by-step. The third part consists of Chapters 12-15, which presents a few select multimedia systems. These chapters can be read independently. The objective of including this section is to introduce readers to the intricacies of a few select frequently used multimedia systems.

13 XIV Preface including this section is to introduce readers to the intricacies of a few select frequently used multimedia systems. The chapters in the first and second parts of the book have been organized to enable a hierarchical study. In addition to the Introductory Chapter, the following reading sequence may be considered. i) Text Representation: Chapter 6 ii) Audio Compression: Chapters 2, 4,5,6, 7 iii) Audio Processing: Chapters 2, 4, 5, 10 iv) Image Compression: Chapters 3, 4, 5, 6, 7, 8 v) Video Compression: Chapters 3, 4, 5, 6, 7, 8, 9 vi) Image & Video Processing: Chapters 3, 4, 5, 11 vii) Television Fundamentals: Chapters 3, 4, 5, 6, 7, 8, 9, 12 Chapters can be read in any order. A major focus of this book is to illustrate with examples the basic signal processing concepts. We have used MATLAB to illustrate the examples since MATLAB codes are very compact and easy to follow. The MATLAB codes of most examples, wherever appropriate, in the book are provided in the accompanying CD so that readers can experiment on their own. Any suggestion and concern regarding the book can be ed to the author at the address:mandai@ee.ualberta.ca. There would be a follow-up website ( where future updates will be posted. I would like to extend my deepest gratitude to all my coworkers and students who have helped in the preparation of this book. Special thanks are due to Sunil Bandaru, Alesya Bajoria, Mahesh Nagarajan, Shahid Khan, Hongyu Liao, Qinghong Guo, and Sasan Haghani for their help in the overall preparation. I would also like to thank Drs. Philip Mingay, Bruce Cockburn, Behrouz Nowrouzian, and Sethuraman Panchanathan (from Arizona State University) for their helpful suggestions to improve the course content. Jennifer Evans and Anne Murray from Kluwer Academic Publishers have always lent a helping hand. Last but not least, I would like to thank Rupa and Geeta, without whose encouragement and support this book would not be completed. August 2002 Mrinal Kr. Mandai