1 Images and Formats Dave Bancroft Philips Broadcast Film Imaging
2 Objectives Survey what is happening with image representation as the broadcast television and movie industries converge Examine the impact of the choice of video versus data, according to: application image parameter Forecast where we re headed with image representation
3 Conclusion Data is more flexible than video, but costs more in processing overhead
Maybe there s more...... 4
5 Two Separate Industries FILM Shoot Process Edit/ Opticals Conform Release/ Project Shoot Air TELEVISION
6 Two Separate Industries FILM Shoot Process Edit/ Opticals Conform Release/ Project Shoot Edit/ Effects Play Air to Air TELEVISION
7 Two Separate Industries FILM Shoot Process Edit/ Opticals Conform Release/ Project Shoot Edit/ Effects Play to Air TELEVISION
8 Two Separate Industries FILM Shoot Process Edit/ Opticals Conform Release/ Project Shoot Edit/ Effects Play to Air TELEVISION
9 Conventional Telecine Film Scanning Section Video Processing Video Output 24/25 frames/sec (real time) Select TV standard & scan to video
10 Electronic Cinema - Technology Shift Content Acquisition and Editorial Film Video CGI Electronic Prep of Elements Digital Source Master (DSM) Digital Source Master Color Correction Layers Conversion Conversion Digital Cinema Distribution Master (DCDM) Home Video Product Pre Compression QC Distribution DTV,NTSC,PAL,DVD Digital Projector Data Centric Scanner (TK) Conversion Digital Archive multiple resolution profiles proposed Film Centric Traditional Production Negative Color Timed IP Dupe Negative Film Projector Release Print Answer Print Film Archive
Use HD scanning standards? 11
12 Conventional Telecine - HDTV Film Scanning Section Video Processing HD Video Output 24/25 frames/sec (real time) Scan at Select TV standard HDTV & scan to video 1920 x 1080
13 Conventional Use data representation Telecine - HDTV instead? Film Scanning Section Video Processing HD Video Output 24/25 frames/sec (real time) Scan at Select TV standard HDTV & scan to video 1920 x 1080
14 Adding Use data a Data representation Domain instead? Film Scanning Section Internal Data Domain Data-to- Video Conversion Video Processing Video Output as fast 24/25 frames/sec as (real possible time) Scan at HDTV scan 1920 to data x 1080
15 From Telecine to DataCine Film Scanning Section Internal Data Domain Data-to- Video Conversion Internal-to- External Formatting Video Interface Ext. Data Interface Video Output as fast as possible Scan to data Ext. Data Output Store Data Digital Film Master Standard
16 Digital Film Master File Format DPX File Header DS00023.DPX 1023 +12 RGB 10 bit 4:3 Aspect Metadata 2048 X1556 Thumbnail Image Data SMPTE Standard 268M DPX (Digital Picture Exchange)
17 Digital Film Master File Format DS00023.DPX 1023 +12 DS00024.DPX 1023 +12 RGB 10 bit 4:3 Aspect Metadata RGB DS00025.DPX 10 bit 4:3 Aspect 1023 +12 2048 X1556 Metadata RGB DS00026.DPX 10 bit 4:3 Aspect 1023 +12 2048 X1556 Metadata RGB DS00027.DPX 10 bit 4:3 Aspect 1023 +12 2048 X1556 Metadata RGB DS00028.DPX 10 bit 4:3 Aspect 1023 +12 RGB 10 bit 4:3 Aspect 2048 X1556 Metadata 2048 X1556 Metadata 2048 X1556 One DPX file per image frame
18 Disadvantages of data representation Elements of a data file or stream are stored in containers Each container has to be partly unpacked to read the processing instructions in the metadata Then fully unpacked to perform re-formatting necessary to make the data usable for processing or display e.g. re-aligning against a common event timeline ALL THIS TAKES TIME THAT WASN T NEEDED WITH VIDEO SO TODAY, DATA TRANSFERS ARE SLOW!
19 DPX Packing Fig. C.1 8-bit component(s): Annex C (informative) Data Packing Diagrams - Including Method A Filling bytes 0 4 31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 datum 3 datum 2 datum 1 datum 0 datum 7 datum 6 datum 5 datum 4 15 14 13 12 11 10 09 08 07 06 05 04 03 02 01 bits 00 bits Fig. C.2 10-bit component(s): bytes 0 4 8 12 31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 d3 part datum 2 datum 1 datum 0 datum 6 part datum 5 datum 4 datum 3 part 16 datum 9 part datum 8 datum 7 datum 6 part... (pattern repeats every 160 bits) bits 15 14 13 12 11 10 09 08 07 06 05 04 03 02 01 00 Fig. C.3 10-bit component(s) filled to 32-bit word boundaries, leading padding bits (Method A): bytes 0 4 31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 14 datum 2 datum 1 datum 0 0 0 datum 5 datum 4 datum 3 0 0 13 12 11 10 09 08 07 06 05 04 03 02 01 00 bits Fig. C.4 12-bit component(s): bytes 0 4 8 31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 datum 2 part datum 1 datum 0 d5 part datum 4 datum 3 datum 2 part datum 7 datum 6 datum 5 part 14 13 12 11 10 09 08 07 06 05 04 03 02 01 00 Fig. C.5 Fig. C.6 12-bit component(s) filled to 16-bit word boundaries, leading padding bits (Method A): bytes 0 4 31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 14 datum 1 0 0 0 0 datum 0 0 0 0 0 datum 3 0 0 0 0 datum 2 0 0 0 0 bytes 31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 09 08 07 16-bit component(s): 0 datum 1 datum 0 4 datum 3 datum 2 13 12 11 10 09 08 07 06 06 05 05 04 04 03 03 02 02 01 01 bits 00 bits 00
20 Disadvantages of data representation Cannot assume any redundancy in data more protection against errors needed for transit through interfaces FAR MORE error protection needed than for video for storage in noisy media, e.g. magnetic tape (but we re learning that from compressed video too)
21 Disadvantages of data representation Human factor: facility staff are used to real-time, interactive, WYSIWYG nature of video data transfers are difficult to monitor during operation (monitoring can slow them down) needs more of a batch mode way of working high res monitor displays have video inputs, not data
Issues by parameter...... 22
23 Issues by parameter Resolution Aspect Ratio Contrast Range & Transfer Characteristic Color Gamut Color Space
24 Spatial Resolution Vertical Horizontal (2.39:1) 1.85 Horizontal Bit Rate Mega- PELS DCDM 10D 2048 4928 3808 8.7 Gb/ sec 10.1 DCDM 5.5D 1536 3680 2848 4.9 Gb/ sec 5.7 DCDM 2.5D 1024 2464 1920 2.2 Gb/ sec 2.5 SMPTE DC28.2 DCDM Profiles (under consideration) SMPTE 274M Scanning Standard
25 Aspect Ratio Cinemascope 2.39:1 HDTV 16:9
26 Aspect Ratio image area lost 2.39:1 16:9 Center-slice, or pan/scan
27 Aspect Ratio Vertical resolution lost 16:9 2.39:1 Letterbox
28 Aspect Ratio Classic 1.37:1 HDTV 16:9
29 Aspect Ratio 1.37:1 16:9 image area lost Center-crop, or tilt/scan
30 Aspect Ratio 16:9 1.37:1 Horizontal resolution lost Pillarbox
31 Aspect Ratio DPX
32 Contrast Range/Transfer Characteristic V o 1 Gamma curve V o = V i 0.45 SMPTE 274M / ITU-R Rec 709-4 Opto-electronic transfer characteristic (choice of one) Linear region: slope = 4.5 0 Break point, V i = 0.018 V i 1
33 Contrast Range/Transfer Characteristic Table 5A Transfer Characteristic Code 0 User defined 1 Printing density 2 Linear Transfer characteristic 3 Logarithmic [to be defined by SMPTE I23 Technology Committee, sub-group on Transfer Characteristics ] 4 Unspecified video 5 SMPTE 274M 6 ITU-R 709-4 7 ITU-R 601-5 system B or G (625) 8 ITU-R 601-5 system M (525) 9 Composite video (NTSC); see SMPTE 170M 10 Composite video (PAL); see ITU-R 624-4 11 Z (depth) linear 12 Z (depth) homogeneous (distance to screen and angle of view must also be specified in user-defined section) 13 254 Reserved for future use SMPTE 268M DPX Options
34 Color Gamut X G X XR X (choice of one) v' sample D-Cinema projector gamut spectral locus print film gamut X X B u' Rec. 709 gamut gamut of naturally illuminated objects (Pointer colours) [sources R.W.G. Hunt, and Maier, Kennel & Bogdanowicz (Kodak)]
35 Color Gamut Table 5B Colorimetric Specification Code 1) 0 User defined 1 Printing density 2 Not applicable 3 Not applicable 4 Unspecified video 5 SMPTE 274M 6 ITU-R 709-4 Colorimetric Specification 7 ITU-R 601-5 system B or G (625) 8 ITU-R 601-5 system M (525) 9 Composite video (NTSC); see SMPTE 170M 10 Composite video (PAL); see ITU-R 624-4 11 Not applicable 12 Not applicable 13 254 Reserved for future use 1) The codes are assigned to correspond to those in table 5A, except where there is no appropriate colorimetric specification. SMPTE 268M DPX Options
36 Color Space Video uses a color space containing: luma component at full bandwidth 2 x color difference components at reduced bandwidth components are matrixed after gamma, so you get failure of constant luminance This all comes as part of the video package If you want to use RGB video instead, you need 2 serial links If you want to use RGB in data, just say so (don t be shy)
37 Color Space Table 1 - Image Element Descriptors Value Components (and order in unpacked stream) 0 User defined (or unspecified single component) 1 Red (R) 2 Green (G) 3 Blue (B) 4 Alpha (matte) 6 Luma (Y) Note 1 7 Color Difference (CB, CR, subsampled by two) 8 Depth (Z) 9 Composite video 10 49 Reserved for future single components 50 R,G,B Note 2 51 R,G,B, Alpha (A) Note 2 52 A, B, G, R Note 3 53 99 Reserved for future RGB ++ formats 100 CB, Y, CR, Y (4:2:2) ---- based on SMPTE 125M 101 CB, Y, A, CR, Y, A (4:2:2:4) 102 CB, Y, CR (4:4:4) 103 CB, Y, CR, A (4:4:4:4) 104 149 Reserved for future CBYCR ++ formats 150 User-defined 2-component element 151 User-defined 3-component element 152 User-defined 4-component element SMPTE 268M DPX Options
38 DPX Packing Fig. C.1 8-bit component(s): Annex C (informative) Data Packing Diagrams - Including Method A Filling bytes 0 4 31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 datum 3 datum 2 datum 1 datum 0 datum 7 datum 6 datum 5 datum 4 15 14 13 12 11 10 09 08 07 06 05 04 03 02 01 bits 00 bits Fig. C.2 10-bit component(s): bytes 0 4 8 12 31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 d3 part datum 2 datum 1 datum 0 datum 6 part datum 5 datum 4 datum 3 part 16 datum 9 part datum 8 datum 7 datum 6 part... (pattern repeats every 160 bits) bits 15 14 13 12 11 10 09 08 07 06 05 04 03 02 01 00 Fig. C.3 10-bit component(s) filled to 32-bit word boundaries, leading padding bits (Method A): bytes 0 4 31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 14 datum 2 datum 1 datum 0 0 0 datum 5 datum 4 datum 3 0 0 13 12 11 10 09 08 07 06 05 04 03 02 01 00 bits Fig. C.4 12-bit component(s): bytes 0 4 8 31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 datum 2 part datum 1 datum 0 d5 part datum 4 datum 3 datum 2 part datum 7 datum 6 datum 5 part 14 13 12 11 10 09 08 07 06 05 04 03 02 01 00 Fig. C.5 Fig. C.6 12-bit component(s) filled to 16-bit word boundaries, leading padding bits (Method A): bytes 0 4 31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 14 datum 1 0 0 0 0 datum 0 0 0 0 0 datum 3 0 0 0 0 datum 2 0 0 0 0 bytes 31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 09 08 07 16-bit component(s): 0 datum 1 datum 0 4 datum 3 datum 2 13 12 11 10 09 08 07 06 06 05 05 04 04 03 03 02 02 01 01 bits 00 bits 00
So what s it all really about then?.... 39
40 Implicit versus Explicit Metadata? Implicit metadata, hard-coded format DS00023.DPX 1023 +12 RGB 10 bit 4:3 Aspect Metadata 2048 X1556? Explicit metadata, soft-coded format
41 Conclusion Data is more flexible than video, but costs more in processing overhead the disadvantage will disappear fairly rapidly data will take over in the front-end of post-production But video will persist as an economical, simple interface standard at key links in the production chain, especially capture and display because of consumer markets in receivers and camcorders
42 Conclusion Data is more flexible than video, but costs more in processing overhead the disadvantage will disappear fairly rapidly data will take over in the front-end of post-production But video will persist as an economical, simple interface standard at key links in the production chain, especially capture and display because of consumer markets in receivers and camcorders
43 Images and Formats Dave Bancroft Philips Broadcast Film Imaging