"Digital Media Primer" Yue- Ling Wong, Copyright (c)2011 by Pearson EducaDon, Inc. All rights reserved.

"Digital Media Primer" Yue- Ling Wong, Copyright (c)2011 by Pearson EducaDon, Inc. All rights reserved. 1

Chapter 6 Fundamentals of Digital Video Part 6 MPEG Compression Streaming Video and Progressive Download 2

In this lecture, you will learn: Basic concepts of MPEG compression Types of MPEG ApplicaDons of MPEG What GOP (group of pictures) in MPEG is How GOP sevngs affect MPEG video file size How GOP sevngs affect MPEG picture quality What streaming video and progressive download are 3

MPEG Moving Pictures Experts Group Commi\ee who derives standards for encoding video Allow high compression MPEG- 1, MPEG- 2, MPEG- 4 4

What Happend to MPEG- 3? NOT MP3 (which is audio format) Intended for HDTV HDTV specificadons was merged into MPEG- 2 5

MPEG- 1 Video quality comparable to VHS Originally intended for Web and CD- ROM playback Frame sizes up to 352 240 pixels Video format for VCD (VideoCD) before DVD became widespread 6

MPEG- 2 Supports DVD- video, HDTV, HDV standards For DVD video producdon: Export video into DVD MPEG- 2 format For HDV video producdon: Export video into HDV's MPEG- 2 format 7

MPEG- 4 Newer standard of MPEG family Different encoding approach from MPEG- 1 and MPEG- 2 (will discuss aeer MPEG- 1 and MPEG- 2 compression in this lecture) 8

MPEG- 1 and MPEG- 2 How the Compression Works 9

ProperDes of Typical Video Neighboring frames are very similar This property is called Temporal redundancy. MPEG compression exploits temporal redundancy to reduce video file size by looking for modon difference from one frame to the next This technique is called mo/on compensa/on. 10

Basic Ideas of MoDon CompensaDon 1. An video frame image is read in as a reference frame 2. Read the next video frame image 3. Compare the image content between this current frame (target frame) with the reference frame one block of pixels at a Dme 11

Comparing Reference and Target Frames Case 1: If a pixel block is idendcal at the same locadon in both frames: No need to encode the target frame's Just save an instrucdon to refer to the block in the reference image requires less space than encoding the whole pixel block 12

Comparing Reference and Target Frames Case 2: If a pixel block at the same locadon in both frames is not idendcal: Search for the reference image for a match (because the content may be moved to another locadon) Case 2a: No match is found Case 2b: A match is found 13

Case 2a: No Match is Found The whole pixel block is encoded Thus, no saving in file size 14

Case 2b: A Match is Found The displacement informadon of the block is saved. Displacement informadon: A 2- dimensional value, indicadng how much the block moves horizontally and verdcally from the reference frame to the target frame Called mo/on vector Much smaller size than encoding the whole pixel blocks (thus the saving in file size) 15

IllustraDon of the Ideas Let's consider the first 4 frames of a video Frame 1 Frame 2 Frame 3 Frame 4 The square grid cell in a frame represents a block of pixels. 16

IllustraDon of the Ideas Frame 1 is read in. Frame 1 Frame 2 Frame 3 Frame 4 17

IllustraDon of the Ideas Frame 1 is used as a reference frame. Frame 1 Frame 2 Frame 3 Frame 4 The red color highlights the pixel blocks that are encoded. The whole frame of a reference frame is encoded. No saving in file size. 18

IllustraDon of the Ideas Frame 2 is read in. Frame 1 Frame 2 Frame 3 Frame 4 19

IllustraDon of the Ideas Each pixel block in Frame 2 is searched in Frame 1 trying to find a match. Frame 1 Frame 2 Frame 3 Frame 4 20

IllustraDon of the Ideas Each pixel block in Frame 2 is searched in Frame 1 trying to find a match. Frame 1 Frame 2 Frame 3 Frame 4 The blue color highlights some pixel blocks that are found in Frame 1. The yellow color highlights some pixel blocks that are found in Frame 1 (but later in the process they are found in the next reference frame.) 21

IllustraDon of the Ideas Frame 3 is read in. Frame 1 Frame 2 Frame 3 Frame 4 22

IllustraDon of the Ideas Each pixel block in Frame 3 is searched in Frame 1 trying to find a match. Frame 1 Frame 2 Frame 3 Frame 4 The process is repeated similar to that for Frame 2. 23

IllustraDon of the Ideas about MoDon Vector Take Frames 1 and 2 as an example Frame 1 Frame 2 24

IllustraDon of the Ideas about MoDon Vector Let's look at the pixel block showing the car's windshield Frame 1 Frame 2 25

IllustraDon of the Ideas about MoDon Vector Let's arrange the frames vertically to see the displacement easier Frame 1 Frame 2 26

IllustraDon of the Ideas about MoDon Vector Let's arrange the frames vertically to see the displacement easier Frame 1 Frame 2 27

IllustraDon of the Ideas about MoDon Vector Let's arrange the frames vertically to see the displacement easier Frame 1 Frame 2 28

IllustraDon of the Ideas about MoDon Vector Let's arrange the frames vertically to see the displacement easier Frame 1 Frame 2 The red line indicates the displacement of the pixel block. 29

So how are reference frames chosen? We will need to understand an important concept of group of pictures (GOP) first. 30

31

MoDvaDon of Understanding GOP To understand: the terminology related to GOP because it is used by MPEG which in turns used by DVD video and HDTV it comes up in video producdon applicadon dialog boxes when you export video into MPEG format the impact of the GOP parameters on video file size and quality why some video edidng programs may not support frame accurate edidng of MPEG 32

33

Group of Pictures (GOP) Specifies the grouping structure of frames Frame types in a structure: I- frames P- frames B- frames An MPEG video contains 1 or more repeadng GOPs 34

Number of Frames in a GOP Each GOP in a video has a fixed number of frames The number is the N parameter Example: DVD- compliant MPEG- 3: N = 15 HDV: N = 15 35

I- frames Stands for Intraframes Encoded only using the informadon within the frame- - intracoding. Use spadal compression Similar to JPEG compression No temporal compression Least compressed of the three types Reference frame in the previous 4- frame example 36

I- frames Each GOP starts with an I- frame Each GOP has only one I- frame 37

P- frames Stands for predicted frames Encoded using the informadon from the previous I- or P- frames as the reference frame if a match of a pixel block is found The 2 nd and 3 rd frames in the previous 4- video example 38

B- frames Stands for bidirec/onal frames Between I- and P- frames Encoded using informadon from the previous and subsequent I- and/or P- frame as the reference frames 39

Example GOP DVD- compliant MPEG- 2: I B B P B B P B B P B B P B B N = 15 i.e., number of frames in a GOP = 15 40

Example GOP DVD- compliant MPEG- 2: I B B P B B P B B P B B P B B M = 3 i.e., number of frames between non-b-frames plus one = 2+1 = 3 41

Revisit the 4- frame video example Suppose N = 15, M = 3 42

Example GOP: N = 15, M = 3 I B B P B B P B B P B B P B B Frame 1 Frame 2 Frame 3 Frame 4 43

Example GOP: N = 15, M = 3 I B B P B B P B B P B B P B B Frame 1 Frame 2 Frame 3 Frame 4 I-frame: Least compressed 44

Example GOP: N = 15, M = 3 I B B P B B P B B P B B P B B Frame 1 Frame 2 Frame 3 Frame 4 P-frame: Based on changes from Frame 1 45

Example GOP: N = 15, M = 3 I B B P B B P B B P B B P B B Frame 1 Frame 2 Frame 3 Frame 4 B-frame: Most compressed. Based on changes from Frames 1 and 4 46

Example GOP: N = 15, M = 3 I B B P B B P B B P B B P B B Frame 1 Frame 2 Frame 3 Frame 4 B-frame: Most compressed. Based on changes from Frames 1 and 4 47

How GOP SeVngs Affect File Size Shorter GOP (i.e., lower N): larger file size RaDonales: A MPEG- 2 consists of a repeadng GOP structure Each GOP contains one I- frame I- frames are the least compressed among the three types of frames, and thus take up more storage space Shorter GOP means more GOPs in a MPEG- 2 More GOPs means more I- frames in a MPEG- 2 48

How GOP SeVngs Affect Picture Quality Shorter GOP (i.e., lower N): be\er picture quality RaDonales: I- frames are compressed based on the frame informadon, rather than predicted from other frames 49

How GOP Structure Affects Frame- Accurate Video EdiDng Some video edidng programs may not support frame- accurate video edidng of MPEG- 2 because it is more complex than other video format RaDonales: The informadon for a P- frame depends on the informadon of its previous I- frame. A B- frame depends on the informadon of its previous and subsequent I- or P- frames. Thus, it is more complex to trim out frames from a MPEG- 2 than other video format. 50

ApplicaDons of MPEG- 4 Cover a wide range of data rate Low end of the data rate: Video playback on mobile devices High end of the data rate: HDTV Handheld and portable game devices (e.g., Sony PSP) 51

MPEG- 4 Coding Approach Uses media objects A scene may contain separate media objects Not the frame- based coding used in MPEG- 1 and MPEG- 2 ConvenDonal frame- based video can be converted to MPEG- 4 because a frame can be treated as a media object- - a degenearted case. 52

Ways of Playing Video 53

Two Ways of Playing Video Play from disk Play over a network 54

Play from Disk An endre clip needs to be on disk before it can be played Played from hard drive, CD, or DVD 55

Play over a Network The video can be played while it is being downloaded Can be played from disk Streaming video Progressive download 56

Streaming Video Play video as soon as enough data has arrived Examples: Streaming QuickTime Real Video Window Media Video (WMV) 57

Streaming Video Require a streaming server to stream video Allow saving several different compression levels of a video in a single file The server chooses the compression level to match the speed of network connecdon Buffering: Wait Dme depends on network speed 58

Progressive Download Play video as soon as enough data has arrived Does not require special servers Example: QuickTime fast- start Created by saving the QuickTime movie as self- contained using QuickTime Pro 59

Review QuesDons Note to instructor: Depending on your preference, you may want to go over the review quesdons at the end of this lecture as an instant review or at the beginning of next lecture to refresh students' memory of this lecture. 60

Review QuesDon True/False: The MP3 audio is a MPEG- 3. 61

Review QuesDon provides a video quality comparable to VHS and is the file format for VCD. A. MPEG- 1 B. MPEG- 2 C. MPEG- 3 D. MPEG- 4 62

Review QuesDon support the DVD- video, HDV, and HDTV standards. A. MPEG- 1 B. MPEG- 2 C. MPEG- 3 D. MPEG- 4 63

Review QuesDon True/False: A typical MPEG- 2 consists of a repeadng GOP structure. 64

Review QuesDon MoDon compensadon is a key technique in. A. asymmetric B. lossless C. lossy D. spadal E. temporal 65

Review QuesDon is encoded using only the informadon within that frame. A. B- frame B. I- frame C. P- frame 66

Review QuesDon is encoded using only the previous I- or P- frame as the reference frame. A. B- frame B. I- frame C. P- frame 67

Review QuesDon is encoded using the previous and subsequent I- and/or P- frame as the reference frames. A. B- frame B. I- frame C. P- frame 68

Review QuesDon is the least compressed among the three frame types. A. B- frame B. I- frame C. P- frame 69

Review QuesDon The N parameter of the GOP refers to. A. the number of B- frames in a GOP B. the number of I- frames in a GOP C. the number of P- frames in a GOP D. the total number of frames in a GOP E. one plus the number of frames between the I- and P- frame, the P- and P- frame, and the P- and next GOP's I- frame 70

Review QuesDon The M parameter of the GOP refers to. A. the number of B- frames in a GOP B. the number of I- frames in a GOP C. the number of P- frames in a GOP D. the total number of frames in a GOP E. one plus the number of frames between the I- and P- frame, the P- and P- frame, and the P- and next GOP's I- frame 71