Rochester Institute of Technology RIT Scholar Works Presentations and other scholarship 2007 A Color Scientist Looks at Video Mark D. Fairchild Rochester Institute of Technology Follow this and additional works at: http://scholarworks.rit.edu/other Recommended Citation Fairchild, M. (2007). A color scientist looks at video [PowerPoint slides]. Presented at the 3rd International Workshop on Video Processing and Quality Metrics for Consumer Electronics, Scottsdale, AZ. This Presentation is brought to you for free and open access by RIT Scholar Works. It has been accepted for inclusion in Presentations and other scholarship by an authorized administrator of RIT Scholar Works. For more information, please contact ritscholarworks@rit.edu.
A Color Scientist Looks at Video... Mark D. Fairchild RIT Munsell Color Science Laboratory 3rd International Workshop on Video Processing and Quality Metrics VPQM Scottsdale 2007 An Original Image
Which is Sharper? A B Which is Sharper? A B
Colorimetry of Imaging Systems Image Colorimetry Device Dependent (e.g. RGB, CMYK) Device Independent (e.g. XYZ, L*a*b*) Viewing-Conditions Independent (e.g. JCh)
Image Appearance Spatial, Temporal & Image Quality Questions Remain... Which degraded image is better? And by how much? Where is Video? In Reality... Device Dependent Appears Device Independent But Display Centric
Objectives in Color Reproduction Hunt s Objectives 1. Spectral 2. Colorimetric 3. Exact 4. Equivalent 5. Corresponding 6. Preferred
Fairchild s Levels 1. Color 2. Pleasing 3. Colorimetric 4. Color Appearance 5. Color Preference Where is Video? v' 560 570 580 Hunt s Preferred Fairchild s Pleasing But Potential to Move Toward Preference 0.55 0.45 Grass Skin Skin on Television Sky 0.40 0.15 0.20 0.25 0.30 u'
Display-Centric Video From the moment of image capture... from C. Poynton... the colorimetry of the scene is often lost. And the video signal is encoded for some standard display.
A Problem? Not if Recognized or Artistic Intent Display Centric (Output Referred) Scene Centric (Scene Referred) Careful Camera Calibration AND Characterization could Bridge the Two What is YCBCR? Linear or Gamma Corrected? Which Transfer Function? What Primary Set? What Luminance Coefficients? Is It Even Luminance? (or Luma?) What Scaling & Quantization? Not to mention other spaces such as YUV?
Why Does it Work? Few, if any, displays match the standard? But displays are adjusted to look acceptable. Unknown and variable viewing conditions might negate many issues. Video Processing
Linear vs. Nonlinear Linear RGB to Y Transformation (Rec. 709) Y = Luminance Y = 0.2126R + 0.7152G + 0.0722B Nonlinear R G B to Y Transformation (Rec. 709) Y! Y! Luminance (It s called luma.) Y = 0.2126R + 0.7152G + 0.0722B R = R 0.45 (or similar) Linear vs. Nonlinear Linear Nonlinear 1.00 1.00 1.00 1.00 0.75 0.75 0.25 0.21 0.21 0.21 0.21 0.21 0.25 0.21 0 R G B Y 0 R G B Y Gray Red
Orthogonal vs. Nonorthogonal Does the Luminance Dimension Include Color- Difference Variation? Do the Color-Difference Dimensions include Luminance Variation? Orthogonal Color Space = No to Both Orthogonal vs. Nonorthogonal Linear Nonlinear 1.00 1.00 0.75 0.75 0.25 0.21 0.21 0.21 0.21 0.25 0.37 0.37 0.37 0.37 0 0.11 0.11 0.11 0.11 R G B Y CB CR 0 R G B Y CB CR Gray Darker Gray No Crosstalk for Gray
Orthogonal vs. Nonorthogonal Linear Nonlinear 1.00 1.00 1.00 1.00 0.75 0.79 0.75 0.73 0.79 0.25 0 0.39 0.21 0.21 0.11 0.11 R G B Y -CB CR 0.25 0 0.58 0.21 0.21 0.16 0.16 R G B Y -CB CR Red Darker Red 100% Crosstalk for Red... Y = -CB Orthogonal vs. Nonorthogonal CB = B - Y CR = R - Y Not a Problem for Encoding/Decoding Problematic with Differential Processing Shouldn t Assume Y Contains All the Luminance Information and CBCR None
Nonorthogonal Filtering Original: Luminance Only Blur Chroma More than Lightness: Orthogonal Space Blur Chroma More than Lightness: Nonorthogonal Space 4:1:0 Chroma Subsampling A: No Luminance Crosstalk B: 25% Luminance Crosstalk
4:1:0 Chroma Subsampling A: No Luminance Crosstalk B: 25% Luminance Crosstalk Song et al.! C C!!! $%&'()!"*+,!-./!01+2),!$(34!5)67!73!(%&879!78)!78%(:9!6%(079!+;:!0)23;:!:%4);0%3;,!! A!!! $%&'()!"*<,!.=>!?@A!01+2),!$(34!5)67!73!(%&879!?9!@9!+;:!A,!! A C B C B PCA3 PCA3!!! $%&'()!"*2,!=-B!01+2),!$(34!5)67!73!(%&879!=9!-9!+;:!B! PCA2 PCA2! PCA1 A PCA1!!! $%&'()!"*:,!@.<.(!01+2),!$3(4!5)67!73!(%&879!@9!.<9!+;:!.(,! A B B! For Y average =0.29! # # # " V 1 V 2 V 3 $!'0.0048 0.0196 0.9998 $! X $ & # & # & & = #'0.9983 0.0578 ' 0.0059& * # Y & & # % "'0.0579 ' 0.9981 0.0193& # % " Z & %
Appearance vs. Tristimulus Values Even with Perfect Colorimetry Tristimulus Matching Requires Identical Viewing Conditions Color Appearance Matching Accounts for Changes in Viewing Conditions e.g. Philips Ambilight Display Adjustment & Characterization
Colorimetric vs. Preferred There is no color management in video. RGB always maps into an RGB device. New displays will make this more difficult. Accuracy and utility of user adjustments could be improved with a device-independent approach. e.g., Samsung My Color Control Gamuts: Chromaticity vs. Appearance Almost no information on appearance!
Gamuts: Chromaticity vs. Appearance Heckaman et al. Perceived gamut volume matches volume in color appearance space.
Heckaman et al. Perceived gamut volume can be manipulated drastically without changing the primary chromacities! Careful use of dynamic range is more important. Perceived Gamuts
Image Examples Image Examples
Summary Reality/Practice Display-Centric Capture (Scene Color Lost) Becomes Irreversible After Differential Processing of Luma/Color Difference Largely Unknown Displays & Environment
Ideal Theory HDR Colorimetric Capture Processed/Edited/Encoded in Perceptually Meaningful Spaces No Crosstalk or Nonlinearity Confusions Characterized Displays and Environment Possible Practical Improvements More Accurate Cameras & Metadata Process in Meaningful Spaces, not the Encoding Spaces More Consistent Displays and Ambient Adjustment through Characterization
Thank you for this opportunity... mcsl.rit.edu