VIDEO ANALYSIS IN MPEG COMPRESSED DOMAIN

Size: px
Start display at page:

Download "VIDEO ANALYSIS IN MPEG COMPRESSED DOMAIN"

Transcription

1 VIDEO ANALYSIS IN MPEG COMPRESSED DOMAIN THE PAPERS COLLECTED HERE FORM THE BASIS OF A SUPPLICATION FOR THE DEGREE OF DOCTOR OF PHILOSOPHY AT THE DEPARTMENT OF COMPUTER SCIENCE AND SOFTWARE ENGINEERING OF THE UNIVERSITY OF WESTERN AUSTRALIA By Lifang Gu September 2002

2 c Copyright 2002 by Lifang Gu

3 Abstract The amount of digital video has been increasing dramatically due to the technology advances in video capturing, storage, and compression. The usefulness of vast repositories of digital information is limited by the effectiveness of the access methods, as shown by the Web explosion. The key issues in addressing the access methods are those of content description and of information space navigation. While textual documents in digital form are somewhat self-describing (i.e., they provide explicit indices, such as words and sentences that can be directly used to categorise and access them), digital video does not provide such an explicit content description. In order to access video material in an effective way, without looking at the material in its entirety, it is therefore necessary to analyse and annotate video sequences, and provide an explicit content description targeted to the user needs. Digital video is a very rich medium, and the characteristics in which users may be interested are quite diverse, ranging from the structure of the video to the identity of the people who appear in it, their movements and dialogues and the accompanying music and audio effects. Indexing digital video, based on its content, can be carried out at several levels of abstraction, beginning with indices like the video program name and name of subject, to much lower level aspects of video like the location of edits and motion properties of video. Manual video indexing requires the sequential examination of the entire video clip. This is a time-consuming, subjective, and expensive process. As a result, there is an urgent need for tools to automate the indexing process. In response to such needs, various video analysis techniques from the research fields of image processing and computer vision have been proposed to parse, index and annotate the massive amount of digital video data. Howi

4 ever, most of these video analysis techniques have been developed for uncompressed video. Since most video data are stored in compressed formats for efficiency of storage and transmission, it is necessary to perform decompression on compressed video before such analysis techniques can be applied. Two consequences of having to first decompress before processing are incurring computation time for decompression and requiring extra auxiliary storage. To save on the computational cost of decompression and lower the overall size of the data which must be processed, this study attempts to make use of features available in compressed video data and proposes several video processing techniques operating directly on compressed video data. Specifically, techniques of processing MPEG-1 and MPEG-2 compressed data have been developed to help automate the video indexing process. This includes the tasks of video segmentation (shot boundary detection), camera motion characterisation, and highlights extraction (detection of skin-colour regions, text regions, moving objects and replays) in MPEG compressed video sequences. The approach of performing analysis on the compressed data has the advantages of dealing with a much reduced data size and is therefore suitable for computationally-intensive low-level operations. Experimental results show that most analysis tasks for video indexing can be carried out efficiently in the compressed domain. Once intermediate results, which are dramatically reduced in size, are obtained from the compressed domain analysis, partial decompression can be applied to enable high resolution processing to extract high level semantic information. ii

5 Acknowledgements First of all, I would like to thank my thesis supervisor, Professor Robyn Owens, for providing me with the opportunity to write this thesis. I thank her for her support, inspiration, and encouragement during the course of this study. I have benefited tremendously from her insight and vision. I am very grateful to David Keightley, who provided me the opportunity to enter into the exciting research area of MPEG video processing and to Dr. Graham Reynolds for allowing me the freedom and resources to pursue this area of research within the group of Digital Media Information Systems (DMIS) in the CSIRO Division of Mathematical and Information Sciences (CMIS). My thanks also go to Dr. Ken Tsui and Dr. Don Bone who have been especially helpful with technical discussions and suggestions. I would also like to thank the members of DMIS group for the rewarding and friendly research environment. Finally, I thank my family for their love and support. Especially, I would like to express my eternal gratitude to my mother. Without her support and help, this thesis might never have been written. iii

6 Contents Abstract Acknowledgements List of Papers Included Contribution of Candidate to Submitted Work i iii vi viii 1 Introduction Digital Video and Research Challenges Scope and Themes of the Study Organisation of the Thesis MPEG Introduction MPEG Stream Structure MPEG Compression Algorithm Motion Compensation DCT Transform Information Available From an MPEG Stream Preprocessing and Minimal Decoding Reconstruction of DC Images Preprocessing of Motion Vectors in B-Pictures Conclusions iv

7 3 Overview of Included Papers Shot Boundary Detection Cut Detection Dissolve Detection Discussion Motion Analysis Panning and Zooming Detection Global Motion Estimation and Moving Object Detection Discussion Skin Colour Region Detection Skin Colour Model Skin Colour Region Detection Based on Region Growing Discussion Text Region Detection and Extraction Text Region Detection Text Extraction Discussion Replay Detection in Sports Video Exact Replay Detection Slow Motion Replay Detection Discussion Conclusions Included Papers 51 4 Summary and Conclusions 92 Bibliography 98 v

8 Papers Included in this Thesis Paper 1 (Refereed) Gu, L., Tsui, K., Keightley, D., 1997a. Dissolve detection in MPEG compressed video, In Proc. of IEEE International Conference on Intelligent Processing Systems (ICIPS), Beijing, China, October 1997, pp Paper 2 (Refereed) Srinivasan, U., Gu, L., Tsui, K., and Simpson-Young, W.G., 1997b. A data model to support content-based search in digital video libraries. The Australian Computer Journal, November 1997, Vol. 29, No. 4, pp Paper 3 (Refereed) Gu, L., Scene analysis of video sequences in the MPEG domain, In Proc. of International Conference on Signal and Image Processing (SIP), Las Vegas, USA, October 1998, pp Paper 4 (Refereed) Gu, L. and Bone, D., 1999a. Skin colour region detection in MPEG video sequences, In Proc. of International Conference on Image Analysis and Processing (ICIAP), Venice, Italy, September vi

9 Paper 5 (Refereed) Gu, L., Bone, D. and Reynolds, G., 1999b. Replay detection in sports video sequences, In Proc. of the Eurographics Workshop on Multimedia, Multimedia 99, Eds. Correia N., Chambel T. and Davenport D., Springer Verlag, Milan, Italy, September 1999, pp Paper 6 (Refereed) Gu, L., Text detection and extraction in MPEG video sequences, In Proc. of the IEEE Workshop on Content-Based Multimedia Indexing (CBMI), Brescia, Italy, September 2001, pp vii

10 Contribution of Candidate to Submitted Work Paper 1 90% contribution. Developed and implemented algorithms, and wrote the paper. Ken Tsui reviewed the paper and David Keightley was the project manager. Paper 2 30% contribution. Wrote the section on video analysis. Paper 3 Sole author. Paper 4 90% contribution. Developed and implemented algorithms, and wrote the paper. Don Bone offered some technical support through discussion and reviewing the paper. He also presented the paper at the conference while the candidate was on maternity leave. Paper 5 90% contribution. Developed and implemented algorithms, and wrote the paper. Don Bone reviewed the paper. Graham Reynolds was the project leader and presented the paper at the workshop while the candidate was on maternity leave. Paper 6 Sole author. viii

11 ix

12 Chapter 1 Introduction The six papers comprising this thesis for the degree of Doctor of Philosophy are the principal outcomes of a still-active research program commenced in This work is relevant to the fields of Digital Video Processing, Digital Video Compression, and Content-Based Search and Indexing. The nature of conference papers and journal articles causes inevitable differences between a collection of such works and a conventional doctoral dissertation. This introduction and the following two chapters attempt to describe how the papers relate to each other to form a cohesive study within the above-mentioned fields. Section 1.1 gives some background on research in the field of digital video processing and points out some of the research challenges. The scope and themes of this study are outlined in Section 1.2. The organisation of the rest of the thesis is then outlined in Section Digital Video and Research Challenges The availability of digital images and video, the integration of information from heterogeneous and distributed sources, the accessibility of fast communication networks, and very powerful computers have led to a flood of emerging technologies for multimedia systems, digital libraries, interactive television, telemedicine, virtual classrooms, and the like. With the advances in capturing and scanning technologies, more and more archived analog video 1

13 CHAPTER 1. INTRODUCTION 2 material is being converted into digital formats. In addition, new digital acquisition units capture content into bit streams (compressed or uncompressed) directly at the creation time. Digital video is therefore becoming an increasingly common data type in the new generation of multimedia databases [AB95, BMS95, DM95, Fox91, Jai95]. Many broadcasters are switching to digital formats for broadcasting, and some of them already have a significant amount of video material available in digital format for previewing. Improved compression technologies and increased Internet bandwidth have made webcasting a reality. The production of multimedia material distributed on CD-roms has been increasing dramatically during the last few years and the introduction of the new DVD technology is offering consumers unprecedented new experiences of interacting with video data. The ever growing amount of digital video poses new challenges of storage, transmission and access as vast repositories are being built at an increasing pace. Research has made significant progress in data compression technology to reduce the data size of images and videos for efficient transmission and storage. International standards have been established for lossy compression of still images through JPEG [Wal91] and lossy compression of moving pictures (video) through MPEG [LeG91]. In addition, faster communication networks have been emerging and have become available for transmitting digital video data. As shown by the Web explosion, the usefulness of vast repositories of digital information is limited by the effectiveness of the access methods. The key issues are those of content description and of information space navigation. While textual documents in digital form are somewhat self-describing (i.e., they provide explicit indices, such as words and sentences that can be directly used to categorise and access them), digital video does not provide such an explicit content description. In order to access video material in an effective way, without looking at the material in its entirety, it is therefore necessary to annotate video sequences and provide an explicit content description targeted to the user needs. Digital video is a very rich medium, and the characteristics in which users may be interested are quite diverse, ranging from the structure of the video,

14 CHAPTER 1. INTRODUCTION 3 i.e. its decomposition into shots and scenes, to the most representative frames of sequences, the identity of the people who appear in it, their movements and dialogues, and the accompanying music and audio effects. Indexing digital video, based on its content, can be carried out at several levels of abstraction, beginning with indices like the video program name and name of subject, to much lower level aspects of video like the location of edits and motion properties of video [ZSW + 95, HJW94a]. The cost of manually generating such indices is inversely related to the level of abstraction of the index. For example, the indexing effort for video library applications which model video by title is much less than the indexing effort for a multimedia authoring application which organises video based on the content and style of the shots used to compose the video. Manual video indexing requires the sequential examination of the entire video clip in order to annotate it. This is a time-consuming, subjective, and expensive process; an hour long footage can take up to 10 hours (depending on its content) to be completely described and archived. The automation of the indexing process thus becomes essential as the granularity of the video access becomes finer. A growing body of research studies the problem of how video indexing could benefit from the use of automated procedures [FL95, ABL95, AL96] to make the process faster and cheaper. A variety of algorithms and systems have been developed to help automate some of the indexing tasks [HJW94a, ZSW + 95, FSN + 95, SK97]. It is expected that in future, a significant portion of all digital video will be in various compressed formats. However, most of the current video analysis techniques, such as those for video indexing, have been developed for uncompressed video. It would be necessary to perform decompression on compressed video before such processing or analysis techniques could be applied. Two consequences of having to first decompress before processing are incurring computation time for decompression and requiring extra auxiliary storage. This study thus investigates video analysis techniques which operate directly on compressed video data.

15 CHAPTER 1. INTRODUCTION Scope and Themes of the Study As more and more video data has been and will continue to be stored and distributed in compressed formats, it would be advantageous to develop processing algorithms that operate directly on compressed representations, saving on the computational cost of decompression and lowering the overall size of the data which must be processed. This eliminates the time-consuming decompression and often leads to computationally more efficient algorithms [CM95]. Consequently, this study explores the possibility of making use of features available in compressed video data and proposes several video processing techniques operating directly on compressed video data. Specifically, techniques of processing MPEG-1 and MPEG-2 compressed data are developed to help automate the video indexing process. This is because MPEG-1 and MPEG-2 are international standards for video compression and have been the widely used video formats in many applications. The study links features available from MPEG compressed data to features useful for video indexing. Figure 1.1 shows the structure of a typical computer-assisted video indexing system. There are four main modules in this system: video analysis, audio analysis, image indexing, and video access. In this diagram, it is assumed that all modules operate directly on compressed data. As a result, another decompression module at the very beginning would be required in order to apply conventional uncompressed analysis techniques. Following the approach proposed in this thesis, incoming compressed video can be directly fed into the video and audio analysis modules for processing. Analysis results are then used for video annotation and building high-level (abstract) video structures. These abstracts can be used for video browsing and retrieval. This study focuses on the video analysis module only. The components covered by this study are shaded in the diagram. In particular, the problem of temporal video segmentation (shot boundary detection) is considered first since it is often the first task in video indexing. This segmentation divides a video into some manageable units (shots), which are used as temporal limits for the annotations by professional archivists, and basic units for nonlinear access to the video stream. Once a video stream

16 CHAPTER 1. INTRODUCTION 5 Figure 1.1: The structure of a typical computer-assisted video indexing system in the compressed domain. The shaded blocks are covered in this study. is decomposed into shots, features within each shot can be extracted for indexing. The majority of features used in the past have been low-level image descriptions extracted from the key frames, such as colour and texture [FSN + 95]. These low-level features do not match the semantic representation used by most people to categorise scene content. In this study, attempts are made to extract semi-semantic features, such as moving objects, skin colours and captions. Although general object recognition is still beyond the reach of current techniques in the field of computer vision, object detection and even recogni-

17 CHAPTER 1. INTRODUCTION 6 tion within limited domains are possible. Two important examples are faces and captions, which often have the role of highlights. As a result, techniques for detecting skin colour regions and text regions directly in MPEG video sequences are developed in this study. As another example of highlights extraction, replay detection in sports video sequences is also addressed to demonstrate how this can be effectively achieved by using features in MPEG compressed video data. Camera operation information is very important for the analysis and classification of video shots, since it often explicitly reflects the communication intentions of the film (video) director. Motion analysis techniques based on motion vectors in MPEG streams are developed to detect and classify camera operations (e.g. panning), and to detect and track moving objects. Both the types of camera operations and trajectories of moving objects can be very useful indices for subsequent search and retrieval. In summary, the aim of this thesis study is to develop a number of techniques for extracting the following features directly from MPEG compressed video data: Shot boundaries Camera motion Object motion Skin colour Text captions Action replays All techniques proposed in this thesis provide essential tools for automatic and efficient video indexing and content-based search in very large video databases. The study demonstrates that these tasks can be performed effectively directly in the MPEG domain.

18 CHAPTER 1. INTRODUCTION Organisation of the Thesis The rest of the thesis is organised as follows: Chapter 2 gives some background knowledge on MPEG standards and briefly describes the basic MPEG syntax and structures. It then describes the method for reconstructing DC images, which are images reduced 64 times in size, from an MPEG video sequence, and proposes some pre-processing of motion vectors used for motion analysis. In essence, Chapter 2 provides background on MPEG compression schemes and presents the argument for processing MPEG video data directly and what minimal processing is needed to provide the input data required by the algorithms proposed in this study. Chapter 3 gives an overview of the problems addressed by each of the included papers. Specifically, it discusses problems of shot boundary detection, camera/object motion detection, skin colour region detection, text region detection, and replay detection directly in MPEG compressed video. This chapter also summarises the approach taken in this study and adds comments and discussions, which cannot be included in the original papers due to the limitations of page numbers. Chapter 4 concludes the thesis with a summary of the contributions this research has made and some suggestions for future research work.

19 Chapter 2 MPEG 2.1 Introduction MPEG (Moving Picture Experts Group) is a working group of ISO/IEC (the International Standards Organizaton/the International Electro-technical Commission) in charge of the development of standards for coded representation of digital audio and video. Established in 1988, the group has produced MPEG-1, the standard on which such products as Video CD and MP3 are based, MPEG-2, the standard on which such products as Digital Television set top boxes and DVD are based, and MPEG-4, the standard for interactive multimedia for the fixed and mobile web. More recently, MPEG-7, the standard for description and search of audio and visual content, was finalised in July Work on the new standard MPEG-21 Multimedia Framework has started in June 2000 and its different parts are at different stages of development. According to the MPEG-21 time schedule, the standard will be finalised as Final Draft International Standard by July MPEG-1 and MPEG-2 have been widely used both for storage and transmission purposes up to now [LeG91, ISO93, ISO96]. MPEG-1 is targeted to applications with digital storage media, such as Video CD, at up to about 1.5 Mbit/s while MPEG-2 is designed for applications requiring higher resolution. However, they both use a suite of similar techniques to reduce both spatial and temporal redundancy in a video sequence to achieve high com- 8

20 CHAPTER 2. MPEG 9 pression ratios. Most existing video processing operations require full frame decompression in order to operate on the compressed sequences. The use of information directly in compressed format without decompression or with minimal decoding for processing saves both the time to perform full frame decompression and the additional storage for holding the decompressed data. It thus offers the possibility of computationally more efficient algorithms. While MPEG-1 and MPEG-2 can significantly reduce the number of bits needed to represent a video sequence without appreciable degradation of image quality, the compressed format does not lend itself to easy video processing. In this chapter, the structure of an MPEG-1 stream is first described in Section 2.2. This is followed by the description of MPEG-1 video compression algorithms in Section 2.3. These descriptions are mostly valid for MPEG-2 as well. Information directly available in an MPEG compressed stream is then listed in Section 2.4. Pre-processing and minimal decoding are discussed in Section 2.5. In particular, DC image reconstruction is given in Section Pre-processing of motion vectors, which are used for motion analysis, is finally described in Section MPEG Stream Structure The difficult challenge in the design of the MPEG compression algorithm is the following: on one hand the quality requirements demand a very high compression ratio not achievable with intra-frame coding alone; on the other hand, the random access requirement is best satisfied with pure intra-frame coding. Inter-frame coding can achieve high compression while it does not promise random access. This requires a delicate balance between intra- and inter-frame coding, and between recursive and non recursive temporal redundancy reduction. This is achieved by forming a group of pictures, which usually include one intra-coded frame and several inter-coded frames. The intracoded frame serves as a random access point while the inter-coded frames facilitate a high compression ratio. Figure 2.1 shows the hierarchical structure of an MPEG video stream,

21 CHAPTER 2. MPEG 10 which is represented by the following six layers: Figure 2.1: The hierarchical structure of an MPEG video stream. Video Sequence: A sequence is the top level of the MPEG video coding. It is composed of several groups of pictures (GoPs) and used as a random context access unit. For video analysis, information, such as frame rate, picture width and height, aspect ratio, and video bit rate, can be obtained from the video sequence layer. Group of Picture (GoP): A GoP consists of a series of pictures and is used as a random access unit for video coding. Information, such as the number of pictures in the GoP, is available from the GoP layer. Picture: A picture is the primary coding unit and consists of several slices. Information, such as picture type (I, P and B) and motion vector resolution, is available for video analysis from this layer. Slice: A slice is used as a re-synchronisation unit and consists of several macroblocks.

22 CHAPTER 2. MPEG 11 Macroblock: A macroblock contains a pixel region of luminance component and the spatially corresponding 8 8 pixel region of each chrominance component since chrominance components are sampled at half the luminance resolution. It thus has four 8 8 luminance blocks and two 8 8 chrominance blocks. For video analysis, the macroblock layer provides the type of coding (intra versus non-intra) and the motion vector. Block: A block is 8 8 pixels in size and is the unit of subsequent Discrete Cosine Transform (DCT). It provides 64 DCT coefficients (either of original pixel values or of the residues after the motion compensation). MPEG uses a component colour representation for each colour pixel, namely one luminance (Y ) and two chrominance components (C b and C r ). The conversion from Y C b C r to conventional RGB space can be carried out by a linear mapping (a 3 3 matrix). Since the human visual system (HVS) is most sensitive to the resolution of an image s luminance component, the Y values are encoded at the full resolution. The HVS is less sensitive to the chrominance information. As a result, the two chrominance components are encoded at half the resolution of their luminance counterpart. This considerably reduces the amount of information to be compressed. 2.3 MPEG Compression Algorithm The MPEG compression algorithm relies on two basic techniques: blockbased motion compensation for the reduction of the temporal redundancy and transform domain-based compression for the reduction of the spatial redundancy. MPEG uses two inter-frame coding techniques: predictive and interpolative. This results in three basic picture types in an MPEG stream: I-, P- and B-pictures. An I-picture is completely intra-coded. It provides access points for random access but only with moderate compression. A P-picture is predictively coded with reference to a past picture, which can be either an I- or a P-picture. A P-picture will in general be used as a

23 CHAPTER 2. MPEG 12 reference for future prediction. A B-picture is bi-directionally coded. It is similar to a P-picture, but requires both a past and a future reference picture for prediction. B-pictures provide the highest amount of compression. The relation between the three picture types is illustrated in Figure 2.2. Figure 2.2: The relationship between the three picture types Motion Compensation Motion-compensated prediction assumes that locally the current picture can be modeled as a translation of the picture at some previous time. Locally means that the magnitude and the direction of the displacement need not be the same everywhere in the picture. The local unit used in MPEG-1 is the pixel macroblock. This is the result of a trade-off between the coding gain provided by the motion information and the cost associated with coding the motion information. Each macroblock in a P-picture is matched to the most similar group of pixels in its past reference picture. This process is called motion estimation. Motion estimation obtains the motion vector, which is the displacement between a macroblock and its predictor candidate, by minimising a cost function measuring the mismatch between the two macroblocks.

24 CHAPTER 2. MPEG 13 If no match is found within a specified search range, a macroblock will be intra-coded. A macroblock in a P-picture can also be skipped, meaning it is exactly the same as the macroblock at the same location in the reference picture. As a result, a skipped macroblock will not be coded at all. For each macroblock in a B-picture, it can be forward-predicted, backwardpredicted, or bi-directionally predicted. As a result, its motion information consists of one forward motion vector, one backward motion vector, or both of the forward and backward motion vectors. Once the motion vector for each macroblock is estimated, the prediction error or residue, the difference between a macroblock and its matched candidate, is calculated. The residue will then be intra-coded by the DCT transform method described in Section DCT Transform Both still-image and difference image (residue) signals have a very high spatial redundancy [Wal91]. Because of the block-based nature of the motioncompensation process and a relatively straightforward implementation, the two dimensional Discrete-Cosine Transform (DCT) is chosen as the basis of compression of each I-picture and of the residue images from P- and B- pictures. As explained above, each macroblock is divided into four 8 8 luminance blocks and two 8 8 chrominance blocks (because of sub-sampling of the chrominance components). Each such 8 8 block is fed to the Forward DCT. The Forward and Inverse DCT are defined as follows: c(i, j) = k(i)k(j) f(x, y) cos x=0 y=0 f(i, j) = k(x)k(y)c(x, y) cos 4 x=0 y=0 (2x + 1)iπ 16 (2i + 1)xπ 16 cos cos (2y + 1)jπ (2.1) 16 (2j + 1)yπ (2.2) 16

25 CHAPTER 2. MPEG 14 where c(i, j) is the DCT coefficient, f(i, j) is the original pixel value, i, j = 0, 1,...7 and k(i) = 1 2 i = 0; In terms of matrix notation, we can write 1 otherwise. (2.3) C = T F T t (2.4) F = T t CT (2.5) where the 8 8 matrices C = [c(i, j)] and F = [f(i, j)] are the 64 DCT coefficients and the original pixel values respectively, and matrix T t denotes the transpose of matrix T, which is the DCT matrix with entries t(i, j) given by t(i, j) = 1 2 (2i + 1)jπ k(j) cos. (2.6) 16 Among the 64 DCT coefficients, c(0, 0) is the weighted value for the DCT basis function, which has no frequencies in the horizontal or vertical direction, and is therefore referred to as the DC term. The other 63 DCT coefficients, c(i, j), for i, j = 0, 1,..7 and (i, j) (0, 0), are generally referred to as the AC coefficients. The DC term c(0, 0) is related to the pixel values f(i, j) by c(0, 0) = f(x, y), (2.7) 8 x=0 y=0 which is 8 times the average intensity of the block. From the Forward DCT, we have 64 coefficients for each block. These coefficients are quantised, zig-zag ordered, run-length and then Huffman coded to reduce spatial redundancy. Note that the DC terms are usually processed separately to the AC terms and as a result are more accessible in the coded (compressed) stream.

26 CHAPTER 2. MPEG Information Available From an MPEG Stream Since there are three different picture types in an MPEG-1/2 stream, different kinds of information are available from these three picture types. For each I-picture, all macroblocks are intra-coded. As a result, the 64 DCT coefficients for its four luminance blocks and two chrominance blocks are available after entropy decoding and de-quantisation. Operations of these two steps are straightforward and require very little computation. For each P-picture, macroblocks can be either intra-coded or motion compensated (MC). For intra-coded macroblocks, the same information (64 DCT coefficients for each of the six blocks) as that of macroblocks in an I-picture is available. For each MC macroblock, a motion vector and the DCT coefficients of the difference blocks are available. For each B-picture, macroblocks can be intra-coded, MC with a forward motion vector, MC with a backward motion vector and MC with both forward and backward motion vectors. The same information is available for intracoded, MC macroblocks with either a forward or backward motion vector as those in a P-picture. For each MC macroblock with both forward and backward motion vectors, two motion vectors (forward and backward) and the DCT coefficients of the difference blocks are obviously available. 2.5 Preprocessing and Minimal Decoding After the discussion on MPEG compression algorithms in the previous section, we now describe preprocessing of MPEG compressed data and extraction of relevant information. Efficient processing can be achieved by developing analysis algorithms, which make use of these features directly available from MPEG compressed video data, and thereby avoiding full decompression. Section studies the reconstruction of spatially reduced images from the different picture types of MPEG video. Such reduced images use the DC coefficient of each block only and are therefore called DC images. The size of

27 CHAPTER 2. MPEG 16 a DC image is 64 times smaller than its original image. Such images, though greatly reduced in size, still capture important global image features useful for many analysis purposes, such as shot boundary detection and skin colour region detection. Section then discusses preprocessing of motion vector information in B-pictures to obtain more reliable motion vectors for motion analysis, such as global motion estimation and moving object detection Reconstruction of DC Images Since each image in an MPEG stream is divided into macroblocks, which in turn consist of four 8 8 luminance blocks and two 8 8 chrominance blocks, the number of luminance blocks in each dimension is reduced 8 times compared with the number of pixels while the number of chrominance blocks in each dimension is reduced 16 times. In the following, only the reconstruction of the luminance DC images is discussed. Chrominance DC images can be similarly reconstructed. In the DCT domain, each block has one DC coefficient and 63 AC coefficients. The DC coefficient of each block is equal to 8 times the average of its original pixel values. If only the DC coefficient is used, this corresponds to using one pixel to represent every 8 8 block. As a result, such an image is reduced 8 times in each dimension and is called a DC image. For intra-coded I-pictures, reconstruction of such DC images is trivial since the DCT DC value of each block can be directly obtained from an MPEG stream. For the predictively coded P-pictures and bi-directionally coded B-pictures, reconstruction of DC images is not straightforward since macroblocks in a P- or B-picture can be either intra-coded or motion compensated (MC). The DC values of an intra-coded macroblock in a P-picture can be similarly obtained as those in an I-picture. Extraction of exact DC values for MC macroblocks in a P-picture is given by Chang and Messerschmitt [CM95] and is computationally expensive. Here we describe an approximation method proposed by Meng et al. [MJC95]. An MC macroblock in a P-picture has a motion vector and four blocks of DCT coded MC errors. The motion vector

28 CHAPTER 2. MPEG 17 allows us to trace back the macroblock to its matching counterpart in the previous reference picture. Each of the four luminance blocks will be matched in a location in the reference picture as shown in Figure 2.3. The matching block may overlap as many as four blocks in the reference picture. Assume that the DC values of the reference picture are available and the luminance variance within each block is small. Then the DC value of an MC block in a P-picture can be approximated by taking the overlapping area-weighted average of the four blocks in the reference picture pointed by the motion vector plus its DC value of the residues: DC(b) = 1 4 h i w i DC(b i ) + DC(b residue ), (2.8) 64 i=1 where DC(b i ) is the DC value of block i in the reference picture, and w i and h i are the overlapping width and height respectively. Their values are related to the motion vector (u, v) as follows: w 1 = w 3 = u, w 2 = w 4 = 8 u, h 1 = h 2 = v and h 3 = h 4 = 8 v. The term DC(b residue ) is the residue DC value of the current block, b. Figure 2.3: Illustration on the relation between the reference block (b ref ), the current block, and the motion vector.

29 CHAPTER 2. MPEG 18 The DC values of a B-picture can be similarly reconstructed for MC macroblocks with either a forward or a backward motion vector. For those MC macroblocks with both forward and backward motion vectors, their DC values can be calculated as the average of those reconstructed from the previous reference picture and the future reference picture plus the DC values of their residues. Using the above method, we can reconstruct a DC image sequence from an MPEG stream, no matter what picture types (I, P or B) it contains. Subsequent analysis algorithms presented in this thesis can then be applied to these DC image sequences. Figure 2.4 shows some examples of DC images reconstructed from the three picture types using the above method. The size of original frames in this MPEG-1 sequence is pixels. It can be seen that they capture global features of the original images well although they are greatly reduced in size. Accuracy of the reconstructed DC values of MC macroblocks in P- and B-frames can be assessed by comparing them to the true values computed from the fully decompressed images (the approximation error). Obviously, it depends on several factors such as the picture type, the accuracy of motion vector estimation and the scene content. Examples of error DC images are given by Yeo [Yeo96], which showed that over 90% of blocks have errors smaller than 5 in their reconstructed DC values. Processing time for reconstructing a DC image is obviously dependent on the picture type. For a GoP pattern of IBBPBBPBBPBBP of MPEG-1 sequences with a size of pixels, the average speed of DC image extraction is about 100 frames per second on an 167 MHz Sun workstation, roughly 4 times faster than real-time Preprocessing of Motion Vectors in B-Pictures The MPEG syntax specifies how to represent the motion information. It is specified by one or two motion vectors per macroblock of the picture depending on the type of motion compensation: one for forward-

30 CHAPTER 2. MPEG 19 Figure 2.4: Examples of some reconstructed DC images. The images of the three columns are reconstructed from the I-, P- and B-pictures in an MPEG- 1 stream respectively. The size of the reconstructed DC images is pixels while the size of original frames is pixels. The images from top to bottom rows illustrate scenes of baby in the bath, woman walking, two men walking, revolving door, and close-up of a document.

31 CHAPTER 2. MPEG 20 predicted and backward-predicted, and two for bi-directionally interpolated. The MPEG standard does not specify how such vectors are to be computed, however. Because of the block-based motion representation, block-matching techniques [JJ81, GM90, LZ93] are usually used. In a block-matching technique, the motion vector is obtained by minimising a cost function measuring the mismatch between a block and its predicted candidate block. Let M i be a macroblock in the current picture P c, and v the displacement with respect to the reference picture P r. Then the optimal displacement ( motion vector ) is obtained by the formula: v i = min v V x M i D[P c (x) P r (x + v)], (2.9) where the search range V of the possible motion vectors and the selection of the cost function D are left entirely to the implementation. Exhaustive searches, where all the possible motion vectors are considered, are known to give good results, but at the expense of a very large complexity for large search ranges. The decision of tradeoff quality of the motion vector field versus complexity of the motion estimation process is for the implementer to make. No matter how large the search range is and therefore how accurate the motion vectors are, a motion vector in MPEG only represents the approximated amount of motion for a macroblock because of the nature of the block-based representation and the assumed translation motion. In addition, there is the aperture problem, existing for any motion estimation technique [Tek95, pages 78 79]. The aperture problem applies particularly to blocks with uniform grey-level values or one-dimensional features such as edges. Assume that the motion in the scene, whether it is the global camera motion or local object motion, is smooth in the temporal direction. Since each macroblock in a B-picture can have both forward and backward motion vectors, the two motion vectors (normalised to the displacement per frame) should not differ a lot in either direction or magnitude for macroblocks with sufficient texture if they are correctly estimated. On the other hand, forward

32 CHAPTER 2. MPEG 21 and backward motion vectors for macroblocks with uniform grey-level values or 1D features tend to have a random relationship and hence their difference will be large. For macroblocks with only a forward or a backward motion vector, we can use the variance of the four DC values or sum of the absolute AC coefficients of the difference blocks to measure their texture levels and hence the reliability of their motion vectors. Based on the above observation, we develop the following algorithm of extracting reliable motion vectors from B-pictures. A brief illustration of the algorithm is shown in Figure 2.5. each MB intra check type MC (one mv) End MC (two mvs) DC variance > D no no MV difference < T yes End End yes keep the MV calculate the average MV End End Figure 2.5: Process of extracting reliable motion vectors in a B-picture. The DC variance measure can be replaced with other alternatives such as the sum of the AC coefficients magnitudes of the 4 residue blocks. The type of each macroblock in a B-picture is first checked. If it is intracoded, no motion vector is available. For MC macroblocks with both motion vectors, their difference is calculated and compared with a threshold, which is empirically set to 5. If the difference is smaller than this threshold, the average of the forward and backward motion vectors is calculated and used as its motion vector. Otherwise, the motion vector is not reliable and will not be used further for any subsequent processing. For an MC macroblock with

33 CHAPTER 2. MPEG 22 only one motion vector, variance of its DC values or some other measures for texture levels are calculated. If the macroblock has enough texture (i.e., large variance), the motion vector is kept. Otherwise, it is declared as unreliable and excluded from further processing. Once reliable motion vectors in B-pictures are extracted, they can be used for any subsequent motion analysis. 2.6 Conclusions In this chapter, we have briefly discussed the MPEG-1 standard and some important features directly available from the compressed MPEG data. We have also shown how reduced (DC) images can be reconstructed from any picture type (I, P and B) of MPEG compressed video. These DC images contain important global features useful for video analysis. The benefits of processing on such reduced images are three folded: no full decompression is necessary; the storage needed for DC images is small (approximately 1/64) compared to the fully decompressed data and complexity is reduced because of the small data size. Based on some simple observations, we have developed a novel algorithm for extracting reliable motion vectors from B-pictures.

34 Chapter 3 Overview of Included Papers This chapter gives an overview of the problem that each of the included papers is trying to solve, summarises the approach taken in this study, and adds some comments and discussions. Section 3.1 discusses the problem of video segmentation, especially dissolve detection in MPEG compressed video sequences using reconstructed DC image sequences. Section 3.2 describes camera motion detection and moving object detection using motion vectors extracted from MPEG streams. Sections 3.3 and 3.4 describe algorithms for extracting highlights (skin colour regions and text regions) from MPEG video sequences. Finally, Section 3.5 proposes an efficient algorithm for detecting replays in sports video sequences. 3.1 Shot Boundary Detection An important first task in video analysis is to segment a video sequence into temporal shots, each representing an event or continuous sequence of actions. A shot is what is captured between record and stop operations. Further scene analysis and interpretation can then be performed on such shots. The segmented video sequences can also be used for browsing, in which only one or a few key frames of each shot is displayed. The boundaries between video shots are commonly called scene changes 1 and the act of segmenting a video 1 This is actually a misnomer; in film production, a scene is a collection of shots and is a division of an act presenting continuous action in one place. 23

35 CHAPTER 3. OVERVIEW OF INCLUDED PAPERS 24 into shots is commonly referred to as scene change detection. In this study, shot boundary detection is used instead to refer to the process of scene change detection in order to reflect the true meaning of the segmentation process. There are two types of shot transitions: abrupt and gradual. In the first case, the change from one shot to the next occurs from one frame to the next, as illustrated in Figure 3.1. This type of shot transition is also called a camera break or cut. In the second case, the change occurs over a longer period of time. This is often the result of applying special editing techniques, such as fade, dissolve and wipe, to join two shots smoothly. An example of a gradual transition (dissolve) is shown in Figure 3.2, where a shot of a document scene slowly changes to that of an outdoor scene and the transition occurs over a period of about 2 seconds (39 frames). Figure 3.1: Example of an abrupt shot transition. Figure 3.2: Example of a gradual shot transition dissolve.

36 CHAPTER 3. OVERVIEW OF INCLUDED PAPERS Cut Detection The difference in grey level as well as the colour information between two consecutive frames is usually large at an abrupt shot boundary due to the content dissimilarity of the two shots. Many of the early methods for cut detection [ZKS93, HJW94b, ZMM95, AL96] were based on difference metrics, such as pixel intensity value difference and histogram difference. One of the problems with these difference based algorithms is that they are sensitive to busy scenes, in which intensities change substantially from frame to frame due to camera/object motion. Since the availability of MPEG video, several algorithms for detecting cuts directly in the MPEG domain have emerged [AHC93, YL95, SD95, ZLS95, LZ95, FLM96, KDR96, IP97, GHP98, JHEJ98, KKC99, MIP99]. These methods use the information directly available from an MPEG stream, such as DCT coefficients, motion vectors and bit-rates, to calculate the frame dissimilarity. A full review of cut detection algorithms operating directly on MPEG video data is given by the candidate [Gu00] in a commercial-inconfidence report. An efficient cut detection algorithm [GTK96] using the motion vector information directly available from an MPEG stream has been designed and implemented by the candidate. It uses one single measure for different types of frames in MPEG streams and is therefore fast, simple and reliable. This algorithm has been commercialised and become part of a series of MPEG video processing products of Mediaware Pty Ltd [Med]. Due to the intellectual property issue, this algorithm is not included as a part of this thesis study Dissolve Detection Since the difference between two consecutive frames is small at a gradual transition, difference measures introduced for cut detection are not suitable for detecting gradual transitions. Different editing techniques result in different types of gradual transitions, which, in turn, have different characteristics [Oha93]. Detection of dissolves, which are the most common gradual

37 CHAPTER 3. OVERVIEW OF INCLUDED PAPERS 26 transitions present in movies and TV programs, is considered here. A dissolve is a gradual transition from one shot to another, in which the first shot fades out and the second shot fades in. Mathematically, a dissolve operation from shot S 1 to shot S 2 is a sequence of frames represented by the following formula: G(x, y, t) = g 1 (x, y, t)[1 α(t)] + g 2 (x, y, t)α(t) (3.1) where G(x, y, t) represents the intensity function of the editing frames at time t, g 1 (x, y, t) and g 2 (x, y, t) are the intensity functions of shots S 1 and S 2 respectively, and α(t) = (t t s )/(t e t s ) increases from 0 to 1 during the dissolve. Here t s and t e stand for the start and end times of the dissolve. It is assumed in Equation 3.1 that the fade-in and fade-out start at the same time. It can be seen from Equation 3.1 that a fade can be regarded as a special dissolve with the intensity values g 1 (x, y, t) of the first shot being constant for a fade-in and intensity values g 2 (x, y, t) of the second shot being constant for a fade-out. Very little work on dissolve detection has been reported, especially in the compressed domain. Meng at al. [MJC95] proposed an algorithm for detecting dissolves directly in MPEG compressed video. It was observed that frame intensity variances during an ideal dissolve follow a clear parabolic shape. The algorithm uses the DC images reconstructed from MPEG video streams for calculating the intensity variances. However, frame intensity variances are often affected by factors such as motion and thus the parabolic shape is not always present during a real dissolve involving motion. In addition when the variance of one shot is much larger than that of the other shot, one half of the parabolic shape almost disappears and thus such dissolve detection algorithms will fail in such cases. Yeo [Yeo96] also proposed a method of detecting dissolves using the DC image sequence reconstructed from an original MPEG video sequence. It first calculates the difference between a frame and the following kth frame. It is then observed that a sequence of such frame differences show a plateau during a dissolve. As before, other factors also contribute to the frame difference

38 CHAPTER 3. OVERVIEW OF INCLUDED PAPERS 27 and therefore plateaus exhibited during ideal dissolves might be disrupted by intensity changes caused by camera/object motion. Paper 1 [Gu et al. 1997a] addresses the problem of detecting dissolves directly in MPEG compressed video. Two reliable methods are proposed, based on the characteristics that intensity values (both local and global) in the editing frames change linearly during a dissolve. DC images are used to calculate intensity values. Dissolve Detection Using Average Frame Intensity The method of dissolve detection based on global intensity information attempts to reduce the intra-shot intensity changes caused by noise and motion by using the average frame intensity values to calculate inter-frame changes. The average frame intensity values can be calculated from the DC images, which are reconstructed from an MPEG bit stream using the method described in Section of Chapter 2. Assuming that the average intensity values of the two shots are g 1 and g 2, the rate of change of the average intensity during a dissolve can be written as follows (from Equation 3.1): R = g 2 g 1 t e t s. (3.2) This rate is relatively stable during the dissolve and its value depends on the dissolve duration, t e t s, and the average intensity difference of the two shots, g 2 g 1. This implies that the average inter-frame differences also change little during the dissolve. This leads to unique rectangular shapes on the average intensity difference curve. The width of the rectangular shape is the dissolve duration while its height is determined by the change rate R. Consequently, dissolves can be detected by finding rectangular shapes on the average intensity difference curve. Dissolve Detection Using Average Block Intensity When the two shots being connected have similar average intensity values, the above method will have difficulty in detecting the dissolve because the average intensity difference between two consecutive editing frames will be

39 CHAPTER 3. OVERVIEW OF INCLUDED PAPERS 28 too small to be detected. However, if we look at the individual blocks, their corresponding intensity values in the two shots are different since the two shots have different contents. As a result, our second dissolve detection method looks at the average intensity changes of each 8 8 luminance block, the basic unit in an MPEG stream. The average intensity value of an 8 8 luminance block corresponds to one eighth of its block DC value in an MPEG stream, which is directly available for I-frames and can be reconstructed for P- and B-frames by minimal partial decoding as described in Section of Chapter 2. From Equation 3.1 it can be derived that the DC value of each block also changes linearly during a dissolve. The DC value difference of each block between any two editing frames falls within a certain range, which is determined by its corresponding DC difference of the two shots. We calculate the percentage of those blocks with large DC value differences between two consecutive frames as a measure. This measure will be large and constant during a dissolve and small otherwise. As a result, dissolves can be detected by finding periods with such block percentage values consistently larger than a given threshold and with a length in the range of a typical dissolve duration Discussion The above two algorithms for dissolve detection have been implemented on top of a general MPEG parsing/decoding library. The DC image sequences are obtained from an MPEG stream by the reconstruction of DC values for motion-compensated blocks in P- and B-frames, as detailed in Chapter 2. Both algorithms have been tested on several MPEG video sequences. Table 3.1 shows the results of dissolve detection of the proposed algorithms on these sequences. For video sequences with small camera/object motion, both algorithms can reliably detect all dissolves with a false detection rate of 10%. These false positives mostly correspond to scenes involving consistent small motion. For video sequences with large camera panning, both algorithms will detect these camera panning segments as dissolves and thus lead to a higher false detection

40 CHAPTER 3. OVERVIEW OF INCLUDED PAPERS 29 Number of dissolves Video sequence (type) true detected missed false test1.mpg (commercial) test2.mpg (TV news) test3.mpg (movies) test4.mpg (documentary) Table 3.1: Results of dissolve detection in several MPEG-1 video sequences. rate. However, segments with camera motion can be easily identified by camera motion detection algorithms, to be described in the next section. In addition, these camera motion segments usually last longer than the normal dissolves. As a result, such false detection can be easily removed. While the method based on the average frame intensity values is simple and fast, it will have difficulty for video sequences containing shots with similar overall intensity values. On the other hand, the method based on the average block intensity values can detect dissolves in video sequences involving substantial object motion, even during dissolves. Figure 3.3 shows 3 snapshot frames at the beginning, middle and end of such a dissolve involving large motion. It can be seen that the singer moves substantially from right to left during the dissolve. Figure 5 in Paper 1 shows the response of the proposed method for the video sequence containing this dissolve (the first peak). The result clearly shows that the proposed method is able to reliably detect this dissolve. Zabih et al. [ZMM95] compared the results of several pixel domain algorithms applied to this video sequence and concluded that only their feature-based algorithm can detect this kind of dissolves. Nevertheless, their algorithm operates on pixel data only and requires several computationally intensive steps (Gaussian smoothing, edge detection, and edge tracking) on top of the time-consuming decompression. A typical speed of 2 frames per second for detecting dissolves in MPEG sequences was quoted in the paper. On the other hand, the two algorithms proposed in this thesis use the features directly available in MPEG compressed data and are thus fast and efficient. The average speed of the two algorithms (including DC image extraction, difference calculation, and final decision making) on an

41 CHAPTER 3. OVERVIEW OF INCLUDED PAPERS 30 Figure 3.3: An example of dissolves involving large motion.

Video compression principles. Color Space Conversion. Sub-sampling of Chrominance Information. Video: moving pictures and the terms frame and

Video compression principles. Color Space Conversion. Sub-sampling of Chrominance Information. Video: moving pictures and the terms frame and Video compression principles Video: moving pictures and the terms frame and picture. one approach to compressing a video source is to apply the JPEG algorithm to each frame independently. This approach

More information

MPEG has been established as an international standard

MPEG has been established as an international standard 1100 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 9, NO. 7, OCTOBER 1999 Fast Extraction of Spatially Reduced Image Sequences from MPEG-2 Compressed Video Junehwa Song, Member,

More information

Motion Video Compression

Motion Video Compression 7 Motion Video Compression 7.1 Motion video Motion video contains massive amounts of redundant information. This is because each image has redundant information and also because there are very few changes

More information

Chapter 10 Basic Video Compression Techniques

Chapter 10 Basic Video Compression Techniques Chapter 10 Basic Video Compression Techniques 10.1 Introduction to Video compression 10.2 Video Compression with Motion Compensation 10.3 Video compression standard H.261 10.4 Video compression standard

More information

Video coding standards

Video coding standards Video coding standards Video signals represent sequences of images or frames which can be transmitted with a rate from 5 to 60 frames per second (fps), that provides the illusion of motion in the displayed

More information

COMP 249 Advanced Distributed Systems Multimedia Networking. Video Compression Standards

COMP 249 Advanced Distributed Systems Multimedia Networking. Video Compression Standards COMP 9 Advanced Distributed Systems Multimedia Networking Video Compression Standards Kevin Jeffay Department of Computer Science University of North Carolina at Chapel Hill jeffay@cs.unc.edu September,

More information

Chapter 2 Introduction to

Chapter 2 Introduction to Chapter 2 Introduction to H.264/AVC H.264/AVC [1] is the newest video coding standard of the ITU-T Video Coding Experts Group (VCEG) and the ISO/IEC Moving Picture Experts Group (MPEG). The main improvements

More information

Module 8 VIDEO CODING STANDARDS. Version 2 ECE IIT, Kharagpur

Module 8 VIDEO CODING STANDARDS. Version 2 ECE IIT, Kharagpur Module 8 VIDEO CODING STANDARDS Lesson 27 H.264 standard Lesson Objectives At the end of this lesson, the students should be able to: 1. State the broad objectives of the H.264 standard. 2. List the improved

More information

Wipe Scene Change Detection in Video Sequences

Wipe Scene Change Detection in Video Sequences Wipe Scene Change Detection in Video Sequences W.A.C. Fernando, C.N. Canagarajah, D. R. Bull Image Communications Group, Centre for Communications Research, University of Bristol, Merchant Ventures Building,

More information

PAL uncompressed. 768x576 pixels per frame. 31 MB per second 1.85 GB per minute. x 3 bytes per pixel (24 bit colour) x 25 frames per second

PAL uncompressed. 768x576 pixels per frame. 31 MB per second 1.85 GB per minute. x 3 bytes per pixel (24 bit colour) x 25 frames per second 191 192 PAL uncompressed 768x576 pixels per frame x 3 bytes per pixel (24 bit colour) x 25 frames per second 31 MB per second 1.85 GB per minute 191 192 NTSC uncompressed 640x480 pixels per frame x 3 bytes

More information

Research Topic. Error Concealment Techniques in H.264/AVC for Wireless Video Transmission in Mobile Networks

Research Topic. Error Concealment Techniques in H.264/AVC for Wireless Video Transmission in Mobile Networks Research Topic Error Concealment Techniques in H.264/AVC for Wireless Video Transmission in Mobile Networks July 22 nd 2008 Vineeth Shetty Kolkeri EE Graduate,UTA 1 Outline 2. Introduction 3. Error control

More information

Reducing False Positives in Video Shot Detection

Reducing False Positives in Video Shot Detection Reducing False Positives in Video Shot Detection Nithya Manickam Computer Science & Engineering Department Indian Institute of Technology, Bombay Powai, India - 400076 mnitya@cse.iitb.ac.in Sharat Chandran

More information

Understanding Compression Technologies for HD and Megapixel Surveillance

Understanding Compression Technologies for HD and Megapixel Surveillance When the security industry began the transition from using VHS tapes to hard disks for video surveillance storage, the question of how to compress and store video became a top consideration for video surveillance

More information

Contents. xv xxi xxiii xxiv. 1 Introduction 1 References 4

Contents. xv xxi xxiii xxiv. 1 Introduction 1 References 4 Contents List of figures List of tables Preface Acknowledgements xv xxi xxiii xxiv 1 Introduction 1 References 4 2 Digital video 5 2.1 Introduction 5 2.2 Analogue television 5 2.3 Interlace 7 2.4 Picture

More information

Module 8 VIDEO CODING STANDARDS. Version 2 ECE IIT, Kharagpur

Module 8 VIDEO CODING STANDARDS. Version 2 ECE IIT, Kharagpur Module 8 VIDEO CODING STANDARDS Lesson 24 MPEG-2 Standards Lesson Objectives At the end of this lesson, the students should be able to: 1. State the basic objectives of MPEG-2 standard. 2. Enlist the profiles

More information

Multimedia Communications. Image and Video compression

Multimedia Communications. Image and Video compression Multimedia Communications Image and Video compression JPEG2000 JPEG2000: is based on wavelet decomposition two types of wavelet filters one similar to what discussed in Chapter 14 and the other one generates

More information

Multimedia Communications. Video compression

Multimedia Communications. Video compression Multimedia Communications Video compression Video compression Of all the different sources of data, video produces the largest amount of data There are some differences in our perception with regard to

More information

An Overview of Video Coding Algorithms

An Overview of Video Coding Algorithms An Overview of Video Coding Algorithms Prof. Ja-Ling Wu Department of Computer Science and Information Engineering National Taiwan University Video coding can be viewed as image compression with a temporal

More information

Implementation of an MPEG Codec on the Tilera TM 64 Processor

Implementation of an MPEG Codec on the Tilera TM 64 Processor 1 Implementation of an MPEG Codec on the Tilera TM 64 Processor Whitney Flohr Supervisor: Mark Franklin, Ed Richter Department of Electrical and Systems Engineering Washington University in St. Louis Fall

More information

Overview: Video Coding Standards

Overview: Video Coding Standards Overview: Video Coding Standards Video coding standards: applications and common structure ITU-T Rec. H.261 ISO/IEC MPEG-1 ISO/IEC MPEG-2 State-of-the-art: H.264/AVC Video Coding Standards no. 1 Applications

More information

Introduction to Video Compression Techniques. Slides courtesy of Tay Vaughan Making Multimedia Work

Introduction to Video Compression Techniques. Slides courtesy of Tay Vaughan Making Multimedia Work Introduction to Video Compression Techniques Slides courtesy of Tay Vaughan Making Multimedia Work Agenda Video Compression Overview Motivation for creating standards What do the standards specify Brief

More information

ELEC 691X/498X Broadcast Signal Transmission Fall 2015

ELEC 691X/498X Broadcast Signal Transmission Fall 2015 ELEC 691X/498X Broadcast Signal Transmission Fall 2015 Instructor: Dr. Reza Soleymani, Office: EV 5.125, Telephone: 848 2424 ext.: 4103. Office Hours: Wednesday, Thursday, 14:00 15:00 Time: Tuesday, 2:45

More information

Digital Video Telemetry System

Digital Video Telemetry System Digital Video Telemetry System Item Type text; Proceedings Authors Thom, Gary A.; Snyder, Edwin Publisher International Foundation for Telemetering Journal International Telemetering Conference Proceedings

More information

Research Article. ISSN (Print) *Corresponding author Shireen Fathima

Research Article. ISSN (Print) *Corresponding author Shireen Fathima Scholars Journal of Engineering and Technology (SJET) Sch. J. Eng. Tech., 2014; 2(4C):613-620 Scholars Academic and Scientific Publisher (An International Publisher for Academic and Scientific Resources)

More information

COMP 9519: Tutorial 1

COMP 9519: Tutorial 1 COMP 9519: Tutorial 1 1. An RGB image is converted to YUV 4:2:2 format. The YUV 4:2:2 version of the image is of lower quality than the RGB version of the image. Is this statement TRUE or FALSE? Give reasons

More information

A Novel Approach towards Video Compression for Mobile Internet using Transform Domain Technique

A Novel Approach towards Video Compression for Mobile Internet using Transform Domain Technique A Novel Approach towards Video Compression for Mobile Internet using Transform Domain Technique Dhaval R. Bhojani Research Scholar, Shri JJT University, Jhunjunu, Rajasthan, India Ved Vyas Dwivedi, PhD.

More information

Audio and Video II. Video signal +Color systems Motion estimation Video compression standards +H.261 +MPEG-1, MPEG-2, MPEG-4, MPEG- 7, and MPEG-21

Audio and Video II. Video signal +Color systems Motion estimation Video compression standards +H.261 +MPEG-1, MPEG-2, MPEG-4, MPEG- 7, and MPEG-21 Audio and Video II Video signal +Color systems Motion estimation Video compression standards +H.261 +MPEG-1, MPEG-2, MPEG-4, MPEG- 7, and MPEG-21 1 Video signal Video camera scans the image by following

More information

Video 1 Video October 16, 2001

Video 1 Video October 16, 2001 Video Video October 6, Video Event-based programs read() is blocking server only works with single socket audio, network input need I/O multiplexing event-based programming also need to handle time-outs,

More information

Principles of Video Compression

Principles of Video Compression Principles of Video Compression Topics today Introduction Temporal Redundancy Reduction Coding for Video Conferencing (H.261, H.263) (CSIT 410) 2 Introduction Reduce video bit rates while maintaining an

More information

CM3106 Solutions. Do not turn this page over until instructed to do so by the Senior Invigilator.

CM3106 Solutions. Do not turn this page over until instructed to do so by the Senior Invigilator. CARDIFF UNIVERSITY EXAMINATION PAPER Academic Year: 2013/2014 Examination Period: Examination Paper Number: Examination Paper Title: Duration: Autumn CM3106 Solutions Multimedia 2 hours Do not turn this

More information

Analysis of a Two Step MPEG Video System

Analysis of a Two Step MPEG Video System Analysis of a Two Step MPEG Video System Lufs Telxeira (*) (+) (*) INESC- Largo Mompilhet 22, 4000 Porto Portugal (+) Universidade Cat61ica Portnguesa, Rua Dingo Botelho 1327, 4150 Porto, Portugal Abstract:

More information

H.261: A Standard for VideoConferencing Applications. Nimrod Peleg Update: Nov. 2003

H.261: A Standard for VideoConferencing Applications. Nimrod Peleg Update: Nov. 2003 H.261: A Standard for VideoConferencing Applications Nimrod Peleg Update: Nov. 2003 ITU - Rec. H.261 Target (1990)... A Video compression standard developed to facilitate videoconferencing (and videophone)

More information

Visual Communication at Limited Colour Display Capability

Visual Communication at Limited Colour Display Capability Visual Communication at Limited Colour Display Capability Yan Lu, Wen Gao and Feng Wu Abstract: A novel scheme for visual communication by means of mobile devices with limited colour display capability

More information

Video Compression. Representations. Multimedia Systems and Applications. Analog Video Representations. Digitizing. Digital Video Block Structure

Video Compression. Representations. Multimedia Systems and Applications. Analog Video Representations. Digitizing. Digital Video Block Structure Representations Multimedia Systems and Applications Video Compression Composite NTSC - 6MHz (4.2MHz video), 29.97 frames/second PAL - 6-8MHz (4.2-6MHz video), 50 frames/second Component Separation video

More information

Motion Re-estimation for MPEG-2 to MPEG-4 Simple Profile Transcoding. Abstract. I. Introduction

Motion Re-estimation for MPEG-2 to MPEG-4 Simple Profile Transcoding. Abstract. I. Introduction Motion Re-estimation for MPEG-2 to MPEG-4 Simple Profile Transcoding Jun Xin, Ming-Ting Sun*, and Kangwook Chun** *Department of Electrical Engineering, University of Washington **Samsung Electronics Co.

More information

INTRA-FRAME WAVELET VIDEO CODING

INTRA-FRAME WAVELET VIDEO CODING INTRA-FRAME WAVELET VIDEO CODING Dr. T. Morris, Mr. D. Britch Department of Computation, UMIST, P. O. Box 88, Manchester, M60 1QD, United Kingdom E-mail: t.morris@co.umist.ac.uk dbritch@co.umist.ac.uk

More information

AUDIOVISUAL COMMUNICATION

AUDIOVISUAL COMMUNICATION AUDIOVISUAL COMMUNICATION Laboratory Session: Recommendation ITU-T H.261 Fernando Pereira The objective of this lab session about Recommendation ITU-T H.261 is to get the students familiar with many aspects

More information

Analysis of MPEG-2 Video Streams

Analysis of MPEG-2 Video Streams Analysis of MPEG-2 Video Streams Damir Isović and Gerhard Fohler Department of Computer Engineering Mälardalen University, Sweden damir.isovic, gerhard.fohler @mdh.se Abstract MPEG-2 is widely used as

More information

MPEG-2. ISO/IEC (or ITU-T H.262)

MPEG-2. ISO/IEC (or ITU-T H.262) 1 ISO/IEC 13818-2 (or ITU-T H.262) High quality encoding of interlaced video at 4-15 Mbps for digital video broadcast TV and digital storage media Applications Broadcast TV, Satellite TV, CATV, HDTV, video

More information

Shot Transition Detection Scheme: Based on Correlation Tracking Check for MB-Based Video Sequences

Shot Transition Detection Scheme: Based on Correlation Tracking Check for MB-Based Video Sequences , pp.120-124 http://dx.doi.org/10.14257/astl.2017.146.21 Shot Transition Detection Scheme: Based on Correlation Tracking Check for MB-Based Video Sequences Mona A. M. Fouad 1 and Ahmed Mokhtar A. Mansour

More information

ITU-T Video Coding Standards

ITU-T Video Coding Standards An Overview of H.263 and H.263+ Thanks that Some slides come from Sharp Labs of America, Dr. Shawmin Lei January 1999 1 ITU-T Video Coding Standards H.261: for ISDN H.263: for PSTN (very low bit rate video)

More information

An Efficient Low Bit-Rate Video-Coding Algorithm Focusing on Moving Regions

An Efficient Low Bit-Rate Video-Coding Algorithm Focusing on Moving Regions 1128 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 11, NO. 10, OCTOBER 2001 An Efficient Low Bit-Rate Video-Coding Algorithm Focusing on Moving Regions Kwok-Wai Wong, Kin-Man Lam,

More information

Part1 박찬솔. Audio overview Video overview Video encoding 2/47

Part1 박찬솔. Audio overview Video overview Video encoding 2/47 MPEG2 Part1 박찬솔 Contents Audio overview Video overview Video encoding Video bitstream 2/47 Audio overview MPEG 2 supports up to five full-bandwidth channels compatible with MPEG 1 audio coding. extends

More information

Reduced complexity MPEG2 video post-processing for HD display

Reduced complexity MPEG2 video post-processing for HD display Downloaded from orbit.dtu.dk on: Dec 17, 2017 Reduced complexity MPEG2 video post-processing for HD display Virk, Kamran; Li, Huiying; Forchhammer, Søren Published in: IEEE International Conference on

More information

CERIAS Tech Report Preprocessing and Postprocessing Techniques for Encoding Predictive Error Frames in Rate Scalable Video Codecs by E

CERIAS Tech Report Preprocessing and Postprocessing Techniques for Encoding Predictive Error Frames in Rate Scalable Video Codecs by E CERIAS Tech Report 2001-118 Preprocessing and Postprocessing Techniques for Encoding Predictive Error Frames in Rate Scalable Video Codecs by E Asbun, P Salama, E Delp Center for Education and Research

More information

Digital Image Processing

Digital Image Processing Digital Image Processing 25 January 2007 Dr. ir. Aleksandra Pizurica Prof. Dr. Ir. Wilfried Philips Aleksandra.Pizurica @telin.ugent.be Tel: 09/264.3415 UNIVERSITEIT GENT Telecommunicatie en Informatieverwerking

More information

The H.26L Video Coding Project

The H.26L Video Coding Project The H.26L Video Coding Project New ITU-T Q.6/SG16 (VCEG - Video Coding Experts Group) standardization activity for video compression August 1999: 1 st test model (TML-1) December 2001: 10 th test model

More information

Adaptive Key Frame Selection for Efficient Video Coding

Adaptive Key Frame Selection for Efficient Video Coding Adaptive Key Frame Selection for Efficient Video Coding Jaebum Jun, Sunyoung Lee, Zanming He, Myungjung Lee, and Euee S. Jang Digital Media Lab., Hanyang University 17 Haengdang-dong, Seongdong-gu, Seoul,

More information

Video Processing Applications Image and Video Processing Dr. Anil Kokaram

Video Processing Applications Image and Video Processing Dr. Anil Kokaram Video Processing Applications Image and Video Processing Dr. Anil Kokaram anil.kokaram@tcd.ie This section covers applications of video processing as follows Motion Adaptive video processing for noise

More information

FLEXIBLE SWITCHING AND EDITING OF MPEG-2 VIDEO BITSTREAMS

FLEXIBLE SWITCHING AND EDITING OF MPEG-2 VIDEO BITSTREAMS ABSTRACT FLEXIBLE SWITCHING AND EDITING OF MPEG-2 VIDEO BITSTREAMS P J Brightwell, S J Dancer (BBC) and M J Knee (Snell & Wilcox Limited) This paper proposes and compares solutions for switching and editing

More information

UC San Diego UC San Diego Previously Published Works

UC San Diego UC San Diego Previously Published Works UC San Diego UC San Diego Previously Published Works Title Classification of MPEG-2 Transport Stream Packet Loss Visibility Permalink https://escholarship.org/uc/item/9wk791h Authors Shin, J Cosman, P

More information

HEVC: Future Video Encoding Landscape

HEVC: Future Video Encoding Landscape HEVC: Future Video Encoding Landscape By Dr. Paul Haskell, Vice President R&D at Harmonic nc. 1 ABSTRACT This paper looks at the HEVC video coding standard: possible applications, video compression performance

More information

Joint Optimization of Source-Channel Video Coding Using the H.264/AVC encoder and FEC Codes. Digital Signal and Image Processing Lab

Joint Optimization of Source-Channel Video Coding Using the H.264/AVC encoder and FEC Codes. Digital Signal and Image Processing Lab Joint Optimization of Source-Channel Video Coding Using the H.264/AVC encoder and FEC Codes Digital Signal and Image Processing Lab Simone Milani Ph.D. student simone.milani@dei.unipd.it, Summer School

More information

Implementation of MPEG-2 Trick Modes

Implementation of MPEG-2 Trick Modes Implementation of MPEG-2 Trick Modes Matthew Leditschke and Andrew Johnson Multimedia Services Section Telstra Research Laboratories ABSTRACT: If video on demand services delivered over a broadband network

More information

The Development of a Synthetic Colour Test Image for Subjective and Objective Quality Assessment of Digital Codecs

The Development of a Synthetic Colour Test Image for Subjective and Objective Quality Assessment of Digital Codecs 2005 Asia-Pacific Conference on Communications, Perth, Western Australia, 3-5 October 2005. The Development of a Synthetic Colour Test Image for Subjective and Objective Quality Assessment of Digital Codecs

More information

Story Tracking in Video News Broadcasts. Ph.D. Dissertation Jedrzej Miadowicz June 4, 2004

Story Tracking in Video News Broadcasts. Ph.D. Dissertation Jedrzej Miadowicz June 4, 2004 Story Tracking in Video News Broadcasts Ph.D. Dissertation Jedrzej Miadowicz June 4, 2004 Acknowledgements Motivation Modern world is awash in information Coming from multiple sources Around the clock

More information

International Journal for Research in Applied Science & Engineering Technology (IJRASET) Motion Compensation Techniques Adopted In HEVC

International Journal for Research in Applied Science & Engineering Technology (IJRASET) Motion Compensation Techniques Adopted In HEVC Motion Compensation Techniques Adopted In HEVC S.Mahesh 1, K.Balavani 2 M.Tech student in Bapatla Engineering College, Bapatla, Andahra Pradesh Assistant professor in Bapatla Engineering College, Bapatla,

More information

SUMMIT LAW GROUP PLLC 315 FIFTH AVENUE SOUTH, SUITE 1000 SEATTLE, WASHINGTON Telephone: (206) Fax: (206)

SUMMIT LAW GROUP PLLC 315 FIFTH AVENUE SOUTH, SUITE 1000 SEATTLE, WASHINGTON Telephone: (206) Fax: (206) Case 2:10-cv-01823-JLR Document 154 Filed 01/06/12 Page 1 of 153 1 The Honorable James L. Robart 2 3 4 5 6 7 UNITED STATES DISTRICT COURT FOR THE WESTERN DISTRICT OF WASHINGTON AT SEATTLE 8 9 10 11 12

More information

Improvement of MPEG-2 Compression by Position-Dependent Encoding

Improvement of MPEG-2 Compression by Position-Dependent Encoding Improvement of MPEG-2 Compression by Position-Dependent Encoding by Eric Reed B.S., Electrical Engineering Drexel University, 1994 Submitted to the Department of Electrical Engineering and Computer Science

More information

Skip Length and Inter-Starvation Distance as a Combined Metric to Assess the Quality of Transmitted Video

Skip Length and Inter-Starvation Distance as a Combined Metric to Assess the Quality of Transmitted Video Skip Length and Inter-Starvation Distance as a Combined Metric to Assess the Quality of Transmitted Video Mohamed Hassan, Taha Landolsi, Husameldin Mukhtar, and Tamer Shanableh College of Engineering American

More information

Colour Reproduction Performance of JPEG and JPEG2000 Codecs

Colour Reproduction Performance of JPEG and JPEG2000 Codecs Colour Reproduction Performance of JPEG and JPEG000 Codecs A. Punchihewa, D. G. Bailey, and R. M. Hodgson Institute of Information Sciences & Technology, Massey University, Palmerston North, New Zealand

More information

06 Video. Multimedia Systems. Video Standards, Compression, Post Production

06 Video. Multimedia Systems. Video Standards, Compression, Post Production Multimedia Systems 06 Video Video Standards, Compression, Post Production Imran Ihsan Assistant Professor, Department of Computer Science Air University, Islamabad, Pakistan www.imranihsan.com Lectures

More information

New forms of video compression

New forms of video compression New forms of video compression New forms of video compression Why is there a need? The move to increasingly higher definition and bigger displays means that we have increasingly large amounts of picture

More information

MPEGTool: An X Window Based MPEG Encoder and Statistics Tool 1

MPEGTool: An X Window Based MPEG Encoder and Statistics Tool 1 MPEGTool: An X Window Based MPEG Encoder and Statistics Tool 1 Toshiyuki Urabe Hassan Afzal Grace Ho Pramod Pancha Magda El Zarki Department of Electrical Engineering University of Pennsylvania Philadelphia,

More information

Coded Channel +M r9s i APE/SI '- -' Stream ' Regg'zver :l Decoder El : g I l I

Coded Channel +M r9s i APE/SI '- -' Stream ' Regg'zver :l Decoder El : g I l I US005870087A United States Patent [19] [11] Patent Number: 5,870,087 Chau [45] Date of Patent: Feb. 9, 1999 [54] MPEG DECODER SYSTEM AND METHOD [57] ABSTRACT HAVING A UNIFIED MEMORY FOR TRANSPORT DECODE

More information

Lecture 2 Video Formation and Representation

Lecture 2 Video Formation and Representation 2013 Spring Term 1 Lecture 2 Video Formation and Representation Wen-Hsiao Peng ( 彭文孝 ) Multimedia Architecture and Processing Lab (MAPL) Department of Computer Science National Chiao Tung University 1

More information

Improved Error Concealment Using Scene Information

Improved Error Concealment Using Scene Information Improved Error Concealment Using Scene Information Ye-Kui Wang 1, Miska M. Hannuksela 2, Kerem Caglar 1, and Moncef Gabbouj 3 1 Nokia Mobile Software, Tampere, Finland 2 Nokia Research Center, Tampere,

More information

complex than coding of interlaced data. This is a significant component of the reduced complexity of AVS coding.

complex than coding of interlaced data. This is a significant component of the reduced complexity of AVS coding. AVS - The Chinese Next-Generation Video Coding Standard Wen Gao*, Cliff Reader, Feng Wu, Yun He, Lu Yu, Hanqing Lu, Shiqiang Yang, Tiejun Huang*, Xingde Pan *Joint Development Lab., Institute of Computing

More information

Evaluation of Automatic Shot Boundary Detection on a Large Video Test Suite

Evaluation of Automatic Shot Boundary Detection on a Large Video Test Suite Evaluation of Automatic Shot Boundary Detection on a Large Video Test Suite Colin O Toole 1, Alan Smeaton 1, Noel Murphy 2 and Sean Marlow 2 School of Computer Applications 1 & School of Electronic Engineering

More information

The H.263+ Video Coding Standard: Complexity and Performance

The H.263+ Video Coding Standard: Complexity and Performance The H.263+ Video Coding Standard: Complexity and Performance Berna Erol (bernae@ee.ubc.ca), Michael Gallant (mikeg@ee.ubc.ca), Guy C t (guyc@ee.ubc.ca), and Faouzi Kossentini (faouzi@ee.ubc.ca) Department

More information

Color Image Compression Using Colorization Based On Coding Technique

Color Image Compression Using Colorization Based On Coding Technique Color Image Compression Using Colorization Based On Coding Technique D.P.Kawade 1, Prof. S.N.Rawat 2 1,2 Department of Electronics and Telecommunication, Bhivarabai Sawant Institute of Technology and Research

More information

AN IMPROVED ERROR CONCEALMENT STRATEGY DRIVEN BY SCENE MOTION PROPERTIES FOR H.264/AVC DECODERS

AN IMPROVED ERROR CONCEALMENT STRATEGY DRIVEN BY SCENE MOTION PROPERTIES FOR H.264/AVC DECODERS AN IMPROVED ERROR CONCEALMENT STRATEGY DRIVEN BY SCENE MOTION PROPERTIES FOR H.264/AVC DECODERS Susanna Spinsante, Ennio Gambi, Franco Chiaraluce Dipartimento di Elettronica, Intelligenza artificiale e

More information

WYNER-ZIV VIDEO CODING WITH LOW ENCODER COMPLEXITY

WYNER-ZIV VIDEO CODING WITH LOW ENCODER COMPLEXITY WYNER-ZIV VIDEO CODING WITH LOW ENCODER COMPLEXITY (Invited Paper) Anne Aaron and Bernd Girod Information Systems Laboratory Stanford University, Stanford, CA 94305 {amaaron,bgirod}@stanford.edu Abstract

More information

Drift Compensation for Reduced Spatial Resolution Transcoding

Drift Compensation for Reduced Spatial Resolution Transcoding MERL A MITSUBISHI ELECTRIC RESEARCH LABORATORY http://www.merl.com Drift Compensation for Reduced Spatial Resolution Transcoding Peng Yin Anthony Vetro Bede Liu Huifang Sun TR-2002-47 August 2002 Abstract

More information

MULTIMEDIA TECHNOLOGIES

MULTIMEDIA TECHNOLOGIES MULTIMEDIA TECHNOLOGIES LECTURE 08 VIDEO IMRAN IHSAN ASSISTANT PROFESSOR VIDEO Video streams are made up of a series of still images (frames) played one after another at high speed This fools the eye into

More information

OVE EDFORS ELECTRICAL AND INFORMATION TECHNOLOGY

OVE EDFORS ELECTRICAL AND INFORMATION TECHNOLOGY Information Transmission Chapter 3, image and video OVE EDFORS ELECTRICAL AND INFORMATION TECHNOLOGY Learning outcomes Understanding raster image formats and what determines quality, video formats and

More information

Content storage architectures

Content storage architectures Content storage architectures DAS: Directly Attached Store SAN: Storage Area Network allocates storage resources only to the computer it is attached to network storage provides a common pool of storage

More information

SHOT DETECTION METHOD FOR LOW BIT-RATE VIDEO CODING

SHOT DETECTION METHOD FOR LOW BIT-RATE VIDEO CODING SHOT DETECTION METHOD FOR LOW BIT-RATE VIDEO CODING J. Sastre*, G. Castelló, V. Naranjo Communications Department Polytechnic Univ. of Valencia Valencia, Spain email: Jorsasma@dcom.upv.es J.M. López, A.

More information

White Paper : Achieving synthetic slow-motion in UHDTV. InSync Technology Ltd, UK

White Paper : Achieving synthetic slow-motion in UHDTV. InSync Technology Ltd, UK White Paper : Achieving synthetic slow-motion in UHDTV InSync Technology Ltd, UK ABSTRACT High speed cameras used for slow motion playback are ubiquitous in sports productions, but their high cost, and

More information

Advanced Computer Networks

Advanced Computer Networks Advanced Computer Networks Video Basics Jianping Pan Spring 2017 3/10/17 csc466/579 1 Video is a sequence of images Recorded/displayed at a certain rate Types of video signals component video separate

More information

Information Transmission Chapter 3, image and video

Information Transmission Chapter 3, image and video Information Transmission Chapter 3, image and video FREDRIK TUFVESSON ELECTRICAL AND INFORMATION TECHNOLOGY Images An image is a two-dimensional array of light values. Make it 1D by scanning Smallest element

More information

(12) Patent Application Publication (10) Pub. No.: US 2006/ A1

(12) Patent Application Publication (10) Pub. No.: US 2006/ A1 (19) United States US 20060222067A1 (12) Patent Application Publication (10) Pub. No.: US 2006/0222067 A1 Park et al. (43) Pub. Date: (54) METHOD FOR SCALABLY ENCODING AND DECODNG VIDEO SIGNAL (75) Inventors:

More information

INTERNATIONAL JOURNAL OF ELECTRONICS AND COMMUNICATION ENGINEERING & TECHNOLOGY (IJECET)

INTERNATIONAL JOURNAL OF ELECTRONICS AND COMMUNICATION ENGINEERING & TECHNOLOGY (IJECET) INTERNATIONAL JOURNAL OF ELECTRONICS AND COMMUNICATION ENGINEERING & TECHNOLOGY (IJECET) International Journal of Electronics and Communication Engineering & Technology (IJECET), ISSN 0976 ISSN 0976 6464(Print)

More information

Introduction to image compression

Introduction to image compression Introduction to image compression 1997-2015 Josef Pelikán CGG MFF UK Praha pepca@cgg.mff.cuni.cz http://cgg.mff.cuni.cz/~pepca/ Compression 2015 Josef Pelikán, http://cgg.mff.cuni.cz/~pepca 1 / 12 Motivation

More information

Color Quantization of Compressed Video Sequences. Wan-Fung Cheung, and Yuk-Hee Chan, Member, IEEE 1 CSVT

Color Quantization of Compressed Video Sequences. Wan-Fung Cheung, and Yuk-Hee Chan, Member, IEEE 1 CSVT CSVT -02-05-09 1 Color Quantization of Compressed Video Sequences Wan-Fung Cheung, and Yuk-Hee Chan, Member, IEEE 1 Abstract This paper presents a novel color quantization algorithm for compressed video

More information

Understanding IP Video for

Understanding IP Video for Brought to You by Presented by Part 3 of 4 B1 Part 3of 4 Clearing Up Compression Misconception By Bob Wimmer Principal Video Security Consultants cctvbob@aol.com AT A GLANCE Three forms of bandwidth compression

More information

Impact of scan conversion methods on the performance of scalable. video coding. E. Dubois, N. Baaziz and M. Matta. INRS-Telecommunications

Impact of scan conversion methods on the performance of scalable. video coding. E. Dubois, N. Baaziz and M. Matta. INRS-Telecommunications Impact of scan conversion methods on the performance of scalable video coding E. Dubois, N. Baaziz and M. Matta INRS-Telecommunications 16 Place du Commerce, Verdun, Quebec, Canada H3E 1H6 ABSTRACT The

More information

DWT Based-Video Compression Using (4SS) Matching Algorithm

DWT Based-Video Compression Using (4SS) Matching Algorithm DWT Based-Video Compression Using (4SS) Matching Algorithm Marwa Kamel Hussien Dr. Hameed Abdul-Kareem Younis Assist. Lecturer Assist. Professor Lava_85K@yahoo.com Hameedalkinani2004@yahoo.com Department

More information

DELTA MODULATION AND DPCM CODING OF COLOR SIGNALS

DELTA MODULATION AND DPCM CODING OF COLOR SIGNALS DELTA MODULATION AND DPCM CODING OF COLOR SIGNALS Item Type text; Proceedings Authors Habibi, A. Publisher International Foundation for Telemetering Journal International Telemetering Conference Proceedings

More information

A video signal consists of a time sequence of images. Typical frame rates are 24, 25, 30, 50 and 60 images per seconds.

A video signal consists of a time sequence of images. Typical frame rates are 24, 25, 30, 50 and 60 images per seconds. Video coding Concepts and notations. A video signal consists of a time sequence of images. Typical frame rates are 24, 25, 30, 50 and 60 images per seconds. Each image is either sent progressively (the

More information

Performance Evaluation of Error Resilience Techniques in H.264/AVC Standard

Performance Evaluation of Error Resilience Techniques in H.264/AVC Standard Performance Evaluation of Error Resilience Techniques in H.264/AVC Standard Ram Narayan Dubey Masters in Communication Systems Dept of ECE, IIT-R, India Varun Gunnala Masters in Communication Systems Dept

More information

Fast MBAFF/PAFF Motion Estimation and Mode Decision Scheme for H.264

Fast MBAFF/PAFF Motion Estimation and Mode Decision Scheme for H.264 Fast MBAFF/PAFF Motion Estimation and Mode Decision Scheme for H.264 Ju-Heon Seo, Sang-Mi Kim, Jong-Ki Han, Nonmember Abstract-- In the H.264, MBAFF (Macroblock adaptive frame/field) and PAFF (Picture

More information

Workload Prediction and Dynamic Voltage Scaling for MPEG Decoding

Workload Prediction and Dynamic Voltage Scaling for MPEG Decoding Workload Prediction and Dynamic Voltage Scaling for MPEG Decoding Ying Tan, Parth Malani, Qinru Qiu, Qing Wu Dept. of Electrical & Computer Engineering State University of New York at Binghamton Outline

More information

Case Study: Can Video Quality Testing be Scripted?

Case Study: Can Video Quality Testing be Scripted? 1566 La Pradera Dr Campbell, CA 95008 www.videoclarity.com 408-379-6952 Case Study: Can Video Quality Testing be Scripted? Bill Reckwerdt, CTO Video Clarity, Inc. Version 1.0 A Video Clarity Case Study

More information

OBJECT-BASED IMAGE COMPRESSION WITH SIMULTANEOUS SPATIAL AND SNR SCALABILITY SUPPORT FOR MULTICASTING OVER HETEROGENEOUS NETWORKS

OBJECT-BASED IMAGE COMPRESSION WITH SIMULTANEOUS SPATIAL AND SNR SCALABILITY SUPPORT FOR MULTICASTING OVER HETEROGENEOUS NETWORKS OBJECT-BASED IMAGE COMPRESSION WITH SIMULTANEOUS SPATIAL AND SNR SCALABILITY SUPPORT FOR MULTICASTING OVER HETEROGENEOUS NETWORKS Habibollah Danyali and Alfred Mertins School of Electrical, Computer and

More information

Midterm Review. Yao Wang Polytechnic University, Brooklyn, NY11201

Midterm Review. Yao Wang Polytechnic University, Brooklyn, NY11201 Midterm Review Yao Wang Polytechnic University, Brooklyn, NY11201 yao@vision.poly.edu Yao Wang, 2003 EE4414: Midterm Review 2 Analog Video Representation (Raster) What is a video raster? A video is represented

More information

PACKET-SWITCHED networks have become ubiquitous

PACKET-SWITCHED networks have become ubiquitous IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 13, NO. 7, JULY 2004 885 Video Compression for Lossy Packet Networks With Mode Switching and a Dual-Frame Buffer Athanasios Leontaris, Student Member, IEEE,

More information

Fast thumbnail generation for MPEG video by using a multiple-symbol lookup table

Fast thumbnail generation for MPEG video by using a multiple-symbol lookup table 48 3, 376 March 29 Fast thumbnail generation for MPEG video by using a multiple-symbol lookup table Myounghoon Kim Hoonjae Lee Ja-Cheon Yoon Korea University Department of Electronics and Computer Engineering,

More information

SCALABLE video coding (SVC) is currently being developed

SCALABLE video coding (SVC) is currently being developed IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 16, NO. 7, JULY 2006 889 Fast Mode Decision Algorithm for Inter-Frame Coding in Fully Scalable Video Coding He Li, Z. G. Li, Senior

More information

Speeding up Dirac s Entropy Coder

Speeding up Dirac s Entropy Coder Speeding up Dirac s Entropy Coder HENDRIK EECKHAUT BENJAMIN SCHRAUWEN MARK CHRISTIAENS JAN VAN CAMPENHOUT Parallel Information Systems (PARIS) Electronics and Information Systems (ELIS) Ghent University

More information