Images and Formats. Dave Bancroft. Philips Broadcast Film Imaging

Transcription

1 1 Images and Formats Dave Bancroft Philips Broadcast Film Imaging

2 2 Objectives Survey what is happening with image representation as the broadcast television and movie industries converge Examine the impact of the choice of video versus data, according to: application image parameter Forecast where we re headed with image representation

3 3 Conclusion Data is more flexible than video, but costs more in processing overhead

4 Maybe there s more

5 5 Two Separate Industries FILM Shoot Process Edit/ Opticals Conform Release/ Project Shoot Air TELEVISION

6 6 Two Separate Industries FILM Shoot Process Edit/ Opticals Conform Release/ Project Shoot Edit/ Effects Play Air to Air TELEVISION

7 7 Two Separate Industries FILM Shoot Process Edit/ Opticals Conform Release/ Project Shoot Edit/ Effects Play to Air TELEVISION

8 8 Two Separate Industries FILM Shoot Process Edit/ Opticals Conform Release/ Project Shoot Edit/ Effects Play to Air TELEVISION

9 9 Conventional Telecine Film Scanning Section Video Processing Video Output 24/25 frames/sec (real time) Select TV standard & scan to video

10 10 Electronic Cinema - Technology Shift Content Acquisition and Editorial Film Video CGI Electronic Prep of Elements Digital Source Master (DSM) Digital Source Master Color Correction Layers Conversion Conversion Digital Cinema Distribution Master (DCDM) Home Video Product Pre Compression QC Distribution DTV,NTSC,PAL,DVD Digital Projector Data Centric Scanner (TK) Conversion Digital Archive multiple resolution profiles proposed Film Centric Traditional Production Negative Color Timed IP Dupe Negative Film Projector Release Print Answer Print Film Archive

11 Use HD scanning standards? 11

12 12 Conventional Telecine - HDTV Film Scanning Section Video Processing HD Video Output 24/25 frames/sec (real time) Scan at Select TV standard HDTV & scan to video 1920 x 1080

13 13 Conventional Use data representation Telecine - HDTV instead? Film Scanning Section Video Processing HD Video Output 24/25 frames/sec (real time) Scan at Select TV standard HDTV & scan to video 1920 x 1080

14 14 Adding Use data a Data representation Domain instead? Film Scanning Section Internal Data Domain Data-to- Video Conversion Video Processing Video Output as fast 24/25 frames/sec as (real possible time) Scan at HDTV scan 1920 to data x 1080

15 15 From Telecine to DataCine Film Scanning Section Internal Data Domain Data-to- Video Conversion Internal-to- External Formatting Video Interface Ext. Data Interface Video Output as fast as possible Scan to data Ext. Data Output Store Data Digital Film Master Standard

16 16 Digital Film Master File Format DPX File Header DS00023.DPX RGB 10 bit 4:3 Aspect Metadata 2048 X1556 Thumbnail Image Data SMPTE Standard 268M DPX (Digital Picture Exchange)

17 17 Digital Film Master File Format DS00023.DPX DS00024.DPX RGB 10 bit 4:3 Aspect Metadata RGB DS00025.DPX 10 bit 4:3 Aspect X1556 Metadata RGB DS00026.DPX 10 bit 4:3 Aspect X1556 Metadata RGB DS00027.DPX 10 bit 4:3 Aspect X1556 Metadata RGB DS00028.DPX 10 bit 4:3 Aspect RGB 10 bit 4:3 Aspect 2048 X1556 Metadata 2048 X1556 Metadata 2048 X1556 One DPX file per image frame

18 18 Disadvantages of data representation Elements of a data file or stream are stored in containers Each container has to be partly unpacked to read the processing instructions in the metadata Then fully unpacked to perform re-formatting necessary to make the data usable for processing or display e.g. re-aligning against a common event timeline ALL THIS TAKES TIME THAT WASN T NEEDED WITH VIDEO SO TODAY, DATA TRANSFERS ARE SLOW!

19 19 DPX Packing Fig. C.1 8-bit component(s): Annex C (informative) Data Packing Diagrams - Including Method A Filling bytes datum 3 datum 2 datum 1 datum 0 datum 7 datum 6 datum 5 datum bits 00 bits Fig. C.2 10-bit component(s): bytes d3 part datum 2 datum 1 datum 0 datum 6 part datum 5 datum 4 datum 3 part 16 datum 9 part datum 8 datum 7 datum 6 part... (pattern repeats every 160 bits) bits Fig. C.3 10-bit component(s) filled to 32-bit word boundaries, leading padding bits (Method A): bytes datum 2 datum 1 datum datum 5 datum 4 datum bits Fig. C.4 12-bit component(s): bytes datum 2 part datum 1 datum 0 d5 part datum 4 datum 3 datum 2 part datum 7 datum 6 datum 5 part Fig. C.5 Fig. C.6 12-bit component(s) filled to 16-bit word boundaries, leading padding bits (Method A): bytes datum datum datum datum bytes bit component(s): 0 datum 1 datum 0 4 datum 3 datum bits 00 bits 00

20 20 Disadvantages of data representation Cannot assume any redundancy in data more protection against errors needed for transit through interfaces FAR MORE error protection needed than for video for storage in noisy media, e.g. magnetic tape (but we re learning that from compressed video too)

21 21 Disadvantages of data representation Human factor: facility staff are used to real-time, interactive, WYSIWYG nature of video data transfers are difficult to monitor during operation (monitoring can slow them down) needs more of a batch mode way of working high res monitor displays have video inputs, not data

22 Issues by parameter

23 23 Issues by parameter Resolution Aspect Ratio Contrast Range & Transfer Characteristic Color Gamut Color Space

24 24 Spatial Resolution Vertical Horizontal (2.39:1) 1.85 Horizontal Bit Rate Mega- PELS DCDM 10D Gb/ sec 10.1 DCDM 5.5D Gb/ sec 5.7 DCDM 2.5D Gb/ sec 2.5 SMPTE DC28.2 DCDM Profiles (under consideration) SMPTE 274M Scanning Standard

25 25 Aspect Ratio Cinemascope 2.39:1 HDTV 16:9

26 26 Aspect Ratio image area lost 2.39:1 16:9 Center-slice, or pan/scan

27 27 Aspect Ratio Vertical resolution lost 16:9 2.39:1 Letterbox

28 28 Aspect Ratio Classic 1.37:1 HDTV 16:9

29 29 Aspect Ratio 1.37:1 16:9 image area lost Center-crop, or tilt/scan

30 30 Aspect Ratio 16:9 1.37:1 Horizontal resolution lost Pillarbox

31 31 Aspect Ratio DPX

32 32 Contrast Range/Transfer Characteristic V o 1 Gamma curve V o = V i 0.45 SMPTE 274M / ITU-R Rec Opto-electronic transfer characteristic (choice of one) Linear region: slope = Break point, V i = V i 1

33 33 Contrast Range/Transfer Characteristic Table 5A Transfer Characteristic Code 0 User defined 1 Printing density 2 Linear Transfer characteristic 3 Logarithmic [to be defined by SMPTE I23 Technology Committee, sub-group on Transfer Characteristics ] 4 Unspecified video 5 SMPTE 274M 6 ITU-R ITU-R system B or G (625) 8 ITU-R system M (525) 9 Composite video (NTSC); see SMPTE 170M 10 Composite video (PAL); see ITU-R Z (depth) linear 12 Z (depth) homogeneous (distance to screen and angle of view must also be specified in user-defined section) Reserved for future use SMPTE 268M DPX Options

34 34 Color Gamut X G X XR X (choice of one) v' sample D-Cinema projector gamut spectral locus print film gamut X X B u' Rec. 709 gamut gamut of naturally illuminated objects (Pointer colours) [sources R.W.G. Hunt, and Maier, Kennel & Bogdanowicz (Kodak)]

35 35 Color Gamut Table 5B Colorimetric Specification Code 1) 0 User defined 1 Printing density 2 Not applicable 3 Not applicable 4 Unspecified video 5 SMPTE 274M 6 ITU-R Colorimetric Specification 7 ITU-R system B or G (625) 8 ITU-R system M (525) 9 Composite video (NTSC); see SMPTE 170M 10 Composite video (PAL); see ITU-R Not applicable 12 Not applicable Reserved for future use 1) The codes are assigned to correspond to those in table 5A, except where there is no appropriate colorimetric specification. SMPTE 268M DPX Options

36 36 Color Space Video uses a color space containing: luma component at full bandwidth 2 x color difference components at reduced bandwidth components are matrixed after gamma, so you get failure of constant luminance This all comes as part of the video package If you want to use RGB video instead, you need 2 serial links If you want to use RGB in data, just say so (don t be shy)

37 37 Color Space Table 1 - Image Element Descriptors Value Components (and order in unpacked stream) 0 User defined (or unspecified single component) 1 Red (R) 2 Green (G) 3 Blue (B) 4 Alpha (matte) 6 Luma (Y) Note 1 7 Color Difference (CB, CR, subsampled by two) 8 Depth (Z) 9 Composite video Reserved for future single components 50 R,G,B Note 2 51 R,G,B, Alpha (A) Note 2 52 A, B, G, R Note Reserved for future RGB ++ formats 100 CB, Y, CR, Y (4:2:2) ---- based on SMPTE 125M 101 CB, Y, A, CR, Y, A (4:2:2:4) 102 CB, Y, CR (4:4:4) 103 CB, Y, CR, A (4:4:4:4) Reserved for future CBYCR ++ formats 150 User-defined 2-component element 151 User-defined 3-component element 152 User-defined 4-component element SMPTE 268M DPX Options

38 38 DPX Packing Fig. C.1 8-bit component(s): Annex C (informative) Data Packing Diagrams - Including Method A Filling bytes datum 3 datum 2 datum 1 datum 0 datum 7 datum 6 datum 5 datum bits 00 bits Fig. C.2 10-bit component(s): bytes d3 part datum 2 datum 1 datum 0 datum 6 part datum 5 datum 4 datum 3 part 16 datum 9 part datum 8 datum 7 datum 6 part... (pattern repeats every 160 bits) bits Fig. C.3 10-bit component(s) filled to 32-bit word boundaries, leading padding bits (Method A): bytes datum 2 datum 1 datum datum 5 datum 4 datum bits Fig. C.4 12-bit component(s): bytes datum 2 part datum 1 datum 0 d5 part datum 4 datum 3 datum 2 part datum 7 datum 6 datum 5 part Fig. C.5 Fig. C.6 12-bit component(s) filled to 16-bit word boundaries, leading padding bits (Method A): bytes datum datum datum datum bytes bit component(s): 0 datum 1 datum 0 4 datum 3 datum bits 00 bits 00

39 So what s it all really about then?

40 40 Implicit versus Explicit Metadata? Implicit metadata, hard-coded format DS00023.DPX RGB 10 bit 4:3 Aspect Metadata 2048 X1556? Explicit metadata, soft-coded format

41 41 Conclusion Data is more flexible than video, but costs more in processing overhead the disadvantage will disappear fairly rapidly data will take over in the front-end of post-production But video will persist as an economical, simple interface standard at key links in the production chain, especially capture and display because of consumer markets in receivers and camcorders

42 42 Conclusion Data is more flexible than video, but costs more in processing overhead the disadvantage will disappear fairly rapidly data will take over in the front-end of post-production But video will persist as an economical, simple interface standard at key links in the production chain, especially capture and display because of consumer markets in receivers and camcorders

43 43 Images and Formats Dave Bancroft Philips Broadcast Film Imaging