Digital Media Lecture 10: Video & Compression Georgia Gwinnett College School of Science and Technology Modified from those of Dr. Jim Rowan
Coping with Video Size Consider human vision limitations 1) Chrominance sub-sampling Compression - two versions 2) Spatial 3) Temporal differencing vectoring
Coping with Video Size Consider human vision limitations 1) Chrominance sub-sampling Compression - two versions 3) spatial 4) Temporal differencing vectoring
Chrominance sub-sampling Humans can t distinguish changes in color as well as they can distinguish luminance changes http://en.wikipedia.org/wiki/chroma_subsam pling In our cameras Of every 4 frames store the luminance for each frame only store a proportion of the color info 4:2:0
Chrominance sub-sampling http://dougkerr.net/pumpkin/articles/subsampling.pdf
Chrominance sub-sampling Luminance, Cr, Cb http://en.wikipedia.org/wiki/chroma_subsampling#sampling_systems_and_ratios
Coping with Video Size Aside from screen size and frame rate... Consider human vision limitations 1) Chrominance sub-sampling Compression - two versions 2) spatial 3) Temporal differencing vectoring
Coping with Video Size Spatial compression Individual images can be compressed using the techniques discussed in the bitmapped section Doesn t result in very much compression for video Doesn t take into consideration the other frames that come before or after it Our video cameras do this, compressing each frame to jpeg
Coping with Video Size Aside from screen size and frame rate... Consider human vision limitations 1) Chrominance sub-sampling Compression - two versions 2) spatial 3) Temporal differencing vectoring
Temporal Compression differencing Use the Difference in two frames A naive approach can result in good compression Works well for a small amount of movement Security cameras spend most of their time seeing the same thing all night long A Tarantino film? not so much Most pixels change with nearly every frame
Image Differencing To subtract one image from the next Do it one pixel at a time red minus red green minus green blue minus blue Store the difference To play it back Play frame one 1 Add frame 2 to frame 1 Next, an example in black & white
Temporal Compression differencing
Example 1, the difference of two identical images
Example 1, the difference of two identical images The result ===>
Example 2, the difference of two similar images
Example 2, the difference of tw similar images The result ===>
Temporal Compression vectoring When an OBJECT moves compute its trajectory fill in the resulting exposed background Captured Frame 1 Captured Frame 2 Stored Background Stored Object BUT there s a problem... why isn t this an easy thing to do? Movement Vector
More on differencing The differencing can happen in a forward manner and a backward manner It might be more economical (in data size) to create a frame from a frame that follows it...
MPEG terminology MPEG-2 iframe (interframe prediction) pframe (forward predicted) bframe (backward predicted) GOP (group of pictures) http://en.wikipedia.org/wiki/inter_frame http://en.wikipedia.org/wiki/i-frames#i ntra_coded_frames_.28or_slices_or_i-fra mes_or_key_frames.29
MPEG terminology iframe: -a keyframe -spatially compressed (a fully specified image) pframe -predicted frame -contains only the difference between the current frame and the previous iframe (smaller in size than iframe) bframe -bi-predicted frame -contains difference between current and both the preceding and following iframes -even smaller in size than iframe
MPEG terminology Group(s) Of Pictures (GOPs) All spatially (intra-frame) compressed The largest Smaller but more computation Spatially compressed and predictive (difference) Spatially compressed forward and backward predictive (difference) Play sequence: 1 2 3 4 5 6 7 8 9 10 Transmit sequence: 1 4 2 3 7 5 6 10 8 9 Smallest but more computation and it is transported out of order!