A Color Scientist Looks at Video

Transcription

1 Rochester Institute of Technology RIT Scholar Works Presentations and other scholarship 2007 A Color Scientist Looks at Video Mark D. Fairchild Rochester Institute of Technology Follow this and additional works at: Recommended Citation Fairchild, M. (2007). A color scientist looks at video [PowerPoint slides]. Presented at the 3rd International Workshop on Video Processing and Quality Metrics for Consumer Electronics, Scottsdale, AZ. This Presentation is brought to you for free and open access by RIT Scholar Works. It has been accepted for inclusion in Presentations and other scholarship by an authorized administrator of RIT Scholar Works. For more information, please contact ritscholarworks@rit.edu.

2 A Color Scientist Looks at Video... Mark D. Fairchild RIT Munsell Color Science Laboratory 3rd International Workshop on Video Processing and Quality Metrics VPQM Scottsdale 2007 An Original Image

3 Which is Sharper? A B Which is Sharper? A B

4 Colorimetry of Imaging Systems Image Colorimetry Device Dependent (e.g. RGB, CMYK) Device Independent (e.g. XYZ, L*a*b*) Viewing-Conditions Independent (e.g. JCh)

5 Image Appearance Spatial, Temporal & Image Quality Questions Remain... Which degraded image is better? And by how much? Where is Video? In Reality... Device Dependent Appears Device Independent But Display Centric

6 Objectives in Color Reproduction Hunt s Objectives 1. Spectral 2. Colorimetric 3. Exact 4. Equivalent 5. Corresponding 6. Preferred

7 Fairchild s Levels 1. Color 2. Pleasing 3. Colorimetric 4. Color Appearance 5. Color Preference Where is Video? v' Hunt s Preferred Fairchild s Pleasing But Potential to Move Toward Preference Grass Skin Skin on Television Sky u'

8 Display-Centric Video From the moment of image capture... from C. Poynton... the colorimetry of the scene is often lost. And the video signal is encoded for some standard display.

9 A Problem? Not if Recognized or Artistic Intent Display Centric (Output Referred) Scene Centric (Scene Referred) Careful Camera Calibration AND Characterization could Bridge the Two What is YCBCR? Linear or Gamma Corrected? Which Transfer Function? What Primary Set? What Luminance Coefficients? Is It Even Luminance? (or Luma?) What Scaling & Quantization? Not to mention other spaces such as YUV?

10 Why Does it Work? Few, if any, displays match the standard? But displays are adjusted to look acceptable. Unknown and variable viewing conditions might negate many issues. Video Processing

11 Linear vs. Nonlinear Linear RGB to Y Transformation (Rec. 709) Y = Luminance Y = R G B Nonlinear R G B to Y Transformation (Rec. 709) Y! Y! Luminance (It s called luma.) Y = R G B R = R 0.45 (or similar) Linear vs. Nonlinear Linear Nonlinear R G B Y 0 R G B Y Gray Red

12 Orthogonal vs. Nonorthogonal Does the Luminance Dimension Include Color- Difference Variation? Do the Color-Difference Dimensions include Luminance Variation? Orthogonal Color Space = No to Both Orthogonal vs. Nonorthogonal Linear Nonlinear R G B Y CB CR 0 R G B Y CB CR Gray Darker Gray No Crosstalk for Gray

13 Orthogonal vs. Nonorthogonal Linear Nonlinear R G B Y -CB CR R G B Y -CB CR Red Darker Red 100% Crosstalk for Red... Y = -CB Orthogonal vs. Nonorthogonal CB = B - Y CR = R - Y Not a Problem for Encoding/Decoding Problematic with Differential Processing Shouldn t Assume Y Contains All the Luminance Information and CBCR None

14 Nonorthogonal Filtering Original: Luminance Only Blur Chroma More than Lightness: Orthogonal Space Blur Chroma More than Lightness: Nonorthogonal Space 4:1:0 Chroma Subsampling A: No Luminance Crosstalk B: 25% Luminance Crosstalk

15 4:1:0 Chroma Subsampling A: No Luminance Crosstalk B: 25% Luminance Crosstalk Song et al.! C C!!! $%&'()!"*+,!-./!01+2),!$(34!5)67!73!(%&879!78)!78%(:9!6%(079!+;:!0)23;:!:%4);0%3;,!! A!!! $%&'()!"*<,!.=>!?@A!01+2),!$(34!5)67!73!(%&879!?9!@9!+;:!A,!! A C B C B PCA3 PCA3!!! $%&'()!"*2,!=-B!01+2),!$(34!5)67!73!(%&879!=9!-9!+;:!B! PCA2 PCA2! PCA1 A PCA1!!! $%&'()!"*:,!@.<.(!01+2),!$3(4!5)67!73!(%&879!@9!.<9!+;:!.(,! A B B! For Y average =0.29! # # # " V 1 V 2 V 3 $!' $! X $ & # & # & & = #' ' & * # Y & & # % "' ' & # % " Z & %

16 Appearance vs. Tristimulus Values Even with Perfect Colorimetry Tristimulus Matching Requires Identical Viewing Conditions Color Appearance Matching Accounts for Changes in Viewing Conditions e.g. Philips Ambilight Display Adjustment & Characterization

17 Colorimetric vs. Preferred There is no color management in video. RGB always maps into an RGB device. New displays will make this more difficult. Accuracy and utility of user adjustments could be improved with a device-independent approach. e.g., Samsung My Color Control Gamuts: Chromaticity vs. Appearance Almost no information on appearance!

18 Gamuts: Chromaticity vs. Appearance Heckaman et al. Perceived gamut volume matches volume in color appearance space.

19 Heckaman et al. Perceived gamut volume can be manipulated drastically without changing the primary chromacities! Careful use of dynamic range is more important. Perceived Gamuts

20 Image Examples Image Examples

21 Summary Reality/Practice Display-Centric Capture (Scene Color Lost) Becomes Irreversible After Differential Processing of Luma/Color Difference Largely Unknown Displays & Environment

22 Ideal Theory HDR Colorimetric Capture Processed/Edited/Encoded in Perceptually Meaningful Spaces No Crosstalk or Nonlinearity Confusions Characterized Displays and Environment Possible Practical Improvements More Accurate Cameras & Metadata Process in Meaningful Spaces, not the Encoding Spaces More Consistent Displays and Ambient Adjustment through Characterization

23 Thank you for this opportunity... mcsl.rit.edu