Digital it Video Processing 김태용 Contents Rounding Considerations SDTV-HDTV YCbCr Transforms 4:4:4 to 4:2:2 YCbCr Conversion Display Enhancement Video Mixing and Graphics Overlay Luma and Chroma Keying Video Scaling Scan Rate Conversion Noninterlaced to Interlaced Conversion DCT-Based Compression 2/45
Rounding Considerations Truncation drops any fractional data Conventional Rounding If the fractional data is 0.5 or greater, rounding up should be performed Error Feedback Rounding never throw anything away residue of a truncation and adding it to the next video sample Dynamic Rounding The fractional data is compared to the output of a random number generator 3/45 Rounding Considerations Error Feedback Rounding Dynamic Rounding 4/45
SDTV - HDTV YCbCr Transforms SDTV and HDTV applications have different colorimetric characteristics YCbCr data should be processed to compensate for the different colorimetric characteristics. SDTV to HDTV HDTV to SDTV before processing, the 8-bit DC offset (16 for Y and 128 for CbCr) must be removed, then added back in after processing 5/45 444t 4:4:4 to 4:2:2 422YCbCr Conversion Y Filtering There may be many cascaded conversions (up to 10 were envisioned) filters were designed to adhere to very tight tolerances to avoid a buildup of visual artifacts keep the passband as flat as possible. 6/45
444t 4:4:4 to 4:2:2 422YCbCr Conversion Y Filtering 7/45 444t 4:4:4 to 4:2:2 422YCbCr Conversion CbCr Filtering The Cb and Cr lowpass filtering requires a sharp cut-off to prevent repeated conversions from producing a cumulative resolution loss The sharp cut-off produces ringing that is more noticeable than for Y Aliasing is less noticeable in color difference signals The attenuation at half the sampling frequency is only 6 db. 8/45
444t 4:4:4 to 4:2:2 422YCbCr Conversion CbCr Filtering 9/45 Display Enhancement Hue, Contrast, Brightness, and Saturation Color Transient Improvement Sharpness 10/45
Hue, Contrast, Brightness, and Saturation 11/45 Color Transient Improvement 12/45
Sharpness 13/45 Sharpness 14/45
Video Mixing i and Graphics Overlay alpha mixing is used alpha mixing implementation out = (alpha_0)(in_0) + (alpha_1)(in_1) + When only two video sources are mixed out = (alpha)(in_0) + (1 alpha)(in_1) using a single multiplier out = (alpha)(in_0 in_1) + in_1 15/45 Video Mixing i and Graphics Overlay 16/45
Video Mixing i and Graphics Overlay 17/45 Luma and Chroma Keying Keying involves specifying a desired foreground color Areas containing this color are replaced with a background image Luminance Keying involves specifying a desired foreground luminance level; soft keying by specifying two luminance values of the foreground image: Y H and Y L if Y FG > Y H K = 1 = background only if Y FG < Y L K = 0 = foreground only if Y H ε Y FG ε Y L Chroma Keying K = (Y FG Y L )/(Y H Y L ) =mix 18/45
Luminance Keying 19/45 Luminance Keying 20/45
Chroma Keying 21/45 Chroma Keying 22/45
Key Generation 23/45 Key Suppression 24/45
Background Key Generation 25/45 Video Scaling The spacing between output samples can be defined by a Target Increment (tarinc) value: tarinc = I / O The first and last output samples may be aligned with the first and last input samples by adjusting tarinc = (I 1) / (O 1) 26/45
Video Scaling Pixel Dropping and Duplication The simplest form of scaling down is pixel dropping Simple upscaling can be accomplished by pixel duplication Scaling using pixel dropping or duplication is not recommended due to the visual artifacts and the introduction of aliasing components. Linear Interpolation An improvement in video quality of scaled images is possible using linear interpolation. 27/45 Video Scaling Anti-Aliased Resampling One approach is a variable-bandwidth antialiasing filter followed by a combined interpolator/resampler. t l A second approach is a combined filter/interpolator followed by a resampler. A third approach is an interpolator followed by a combined filter/resampler. 28/45
Scan Rate Conversion Frame or Field Dropping and Duplicating Simple scan-rate conversion Disadvantage: jerky motion, motion judder 29/45 Scan Rate Conversion Temporal Interpolation 30/45
Scan Rate Conversion Motion compensation Used by MPEG incorporate motion compensation to minimize motion artifacts. For scan rate conversion, it is important to determine true motion information Motion Compensation in MPEG In MPEG, the goal is to minimize the displaced frame difference The resulting motion vectors do not necessarily correspond to true motion vectors. motion vector errors are self-correcting since the residual difference between the predicted macroblocks is encoded. 31/45 Scan Rate Conversion 32/45
Scan Rate Conversion 3-2 Pulldown When converting to PAL or SECAM, each film frame is usually mapped into 2 video fields (2-2 pulldown), resulting in the video program being 4% too fast When converting film to NTSC (59.94-Hz field rate), 3-2 pulldown is used The film speed is slowed down by 0.1% to 23.976 (24/1.001) Varispeed is commonly used to cover up problems such as defects, splicing, censorship cuts, or to change the running time of a program. The 3-2 relationship between the film and video is disrupted long enough to ensure a smooth temporal rate. 33/45 Noninterlaced-to-Interlaced Conversion Noninterlaced to interlaced conversion must be performed on component video signals such as R G B or YCbCr Scan Line Decimation sharp vertical transition of color or intensity will flicker 34/45
Noninterlaced-to-Interlaced Conversion Vertical Filtering A better solution is to use two or more lines of noninterlaced data. Fast vertical transitions are smoothed out over several interlaced lines Using more than 3 lines usually results in excessive blurring Implementation uses IIR rather than FIR filtering 35/45 Interlaced-to-Noninterlaced Conversion Intrafield Processing The computer industry has coined this as bob. The resulting vertical resolution is always limited by the content of the original field. Scan Line Duplication no increase in the vertical resolution. 36/45
Interlaced-to-Noninterlaced Conversion Scan Line Interpolation Although the number of active scan lines is doubled, the vertical resolution is not. Fractional Ratio Interpolation similar to the polyphase filtering used for scaling only performed in the vertical direction. Variable Interpolation 37/45 Interlaced-to-Noninterlaced Conversion Interfield Processing The computer industry has coined this as weave. This method can provide higher vertical resolution uses content from more than a single field Field Merging merges two consecutive fields together to produce a frame of video pair of fields combine to generate a frame 38/45
Interlaced-to-Noninterlaced Conversion Motion Adaptive Deinterlacing 각각의픽셀을처리하여과하질이미지구현이가능함 Motion Compensated Deinterlacing Several orders of magnitude more complex than motion adaptive deinterlacing. Requires calculating motion vectors between fields for each sample Interpolating along each sample s motion trajectory. Motion vectors must be found that pass through each of the missing samples. The motion vectors must have sub-pixel accuracy. Motion vector errors will produce artifacts. 39/45 DCT-Based Compression 40/45
DCT DCT-Based Compression processes an 8 8 block 41/45 DCT-Based Compression 42/45
DCT-Based Compression Quantization Zig-Zag g Scanning 43/45 DCT-Based Compression 44/45
DCT-Based Compression 45/45