Lecture 23: Digital Video The Digital World of Multimedia Guest lecture: Jayson Bowen
Plan for Today Digital video Video compression HD, HDTV & Streaming Video
Audio + Images Video Audio: time sampling 8kHz 44.1kHz samples/sec, depending on sound Images: space sampling PPI (pixels per inch), depending on display Video: (Orsak et al. pp. 135-137, 157-158) Time sampling for audio: as above Space sampling for images Time sampling for images: 24-30 frames/sec typical in movies & TV Lower frame rates for time lapse motion (flower blooming) Higher frame rates for scientific studies (glass shattering)
Sampling for Video Analogous to spatial sampling: our eyes do the low pass filtering to recover the signal Space: if pixel size is small enough, our eyes don t perceive individual squares Time: if image frames change rapidly enough, our eyes have the sensation of viewing smooth motion Low sampling rate: Space: blocky image, possibly aliasing Time: Jerky movements (as in early movies, video conferencing), possibly aliasing
Frame Rates in Video Human perception: If a light is blinking at greater than N times per second, people perceive it as constant (few people have N>75) Movie frame rates 60i (interlaced) [used in TVs for decades] Transmit odd & even rows in separate frames 30p (progressive) [good for high speed w/o interlace artifacts] 24p (progressive) [35mm movie cameras] (In Europe: 60 50, 24 25)
Progressive vs. Interlaced Scans Progressive: scan side-to-side, top-tobottom, one frame at a time Interlaced: same except that even and odd lines are scanned in alternate frames Can reduce flicker in some cases (perceived higher frame rate) Can make it worse in others (images with horizontal stripes) From wikipedia
Progressive Interlaced (odd/even fields) Interlaced motion artifact (simulated)
Video Temporal Aliasing Mostly the same idea as other types of aliasing: frequencies look lower when they are aliased One difference: in video, periodic motion can appear to be in the reverse direction We saw this before with the aliasing demo, revisit here...
Storage Requirements 10 second CD-quality audio: 10 sec x 44100 samples/sec x 16 bits/sample x 2 chan 1 MB Typical song is 4 minutes (42 MB) 10 second video (excluding sound) 10 s x 30 frames/s x??megapixels/frame x 3colors x 8 bit/color = IPOD: 320x240 pixels 77K pixels/frame 69 MB SDTV-DVD: 720x480 pixels 300K pixels/frame 276 MB HDTV: 1920x1080 pixels 2000K pixels/frame 1.8 GB Typical movie 120 minutes (200 GB or 1300 GB)
Storage Requirements Typical movie 120 minutes (200 GB or 1300 GB) Common storage devices: DVD: BD: IPOD: 4.7 GB (single layer), 8 GB (dual layer) 25 GB (single layer), 50 GB (dual layer) 30 GB can t fit one movie -- need compression! Other concerns: transmission costs for video clips on the web, video on demand, video teleconferencing, Network bandwidth: Cable, satellite, over-air,
Lossy Compression for Videos Take advantage of perception: Spatial: as in image compression Temporal: insensitivity to fine detail in areas of rapid motion Take advantage of redundancy: Spatial redundancy, as image compression Temporal redundancy: successive frames don t change much, so just code the difference MPEG-2 reduces movie to less than 2.5% of its original bit size! (DVD video up to 1.2 MB/s)
Color Space R G B Y Cb Cr
Video Compression Basics of MPEG Like MP3 (part of MPEG) and JPEG, MPEG can be used at a range of qualities Leveraging temporal redundancy Estimate motion direction Code difference in successive frames after motion compensation + motion vector (can be at block level) Use 8x8 DCT etc. similar to JPEG in image coding Reference Frame Current Frame
Video Compression Basics of MPEG Code frames in 3 ways I-frames (intra-picture): code as is (no prediction), to avoid error propagation and handle scene changes P-frames (predictive): use previous I or P frame to predict the next frame, subtract that from actual frame and code difference B-frames (bi-directional predictive): use previous and next I or P frames for prediction
MPEG Block Diagram From http://cnx.org/content/m11144/latest/
DTV vs. HDTV Digital TV (DTV) need not be high definition High-definition TV (HDTV) has More pixels per frame (better for big screen) (720x480 vs. up to 1920x1080) Better audio, support for surround sound Hence, requires higher data rate
HD, HDTV & Streaming Video Factors in digital video quality: The original signal: Number of pixels per inch in a frame (e.g. 1024x768, 1920x1080) The device you are displaying on Communication (transmission) errors Types of communication Dedicated channel (e.g. specified frequency band, satellite transmission) -- could be analog or digital Noise issues (e.g. weather impacts satellite transmission) Packet based (internet) [streaming] purely digital Delay issues (e.g. internet traffic)
Digital vs. Analog: Transmission Errors When everything works, digital is much higher quality, but when it doesn t. Problems in analog TV (annoying) snow (noisy signal) Ghosting (multipath: multiple signals received with delay, as in an echo) Vertical rolling of image Problems in digital TV (can be unwatchable) Pixellation A series of still pictures (interframe errors) No picture at all