SMPTE SMPTE Education Webcast Series SMPTE ACES 1.0: Theory and Practice Welcome SMPTE Education Webcast Series is sponsored by: SMPTE Education Webcast Series Sponsors Thank you to our sponsors for their generous support: 2016 Powered by SMPTE www.smpte.org
SMPTE Your Host Joel E. Welch Director of Education SMPTE Today s Guest Speakers CTO Frame by Frame Digital Workflow Supervisor Founder, Kiwii Digital Solutions 2016 Powered by SMPTE www.smpte.org
SMPTE SMPTE Education Webcast Series SMPTE ACES 1.0: Theory and Practice Walter Arrighetti, PhD, CISSP CTO, Frame by Frame Francesco Giardiello Digital Workflow Supervisor Founder, Kiwii Digital Solutions Academy Color Encoding System Born in 2004 as IIF (Image Interchange Framework), then name was changed in 2012 to ACES Academy effort coveting scientists, engineers and artists from the industry Color Management for Media & Entertainment (film & animation) Encoding for color data, metadata, file and transport formats. System made up of several core components and SMPTE Standards targeting interoperability. 2016 Powered by SMPTE www.smpte.org
SMPTE Shot on ACES Up to 2016, over 100 theatrical & TV shows have used ACES, with similar figures in commercials & documentaries overall. Shot on ACES Up to 2016, over 100 theatrical & TV shows have used ACES, with similar figures in commercials & documentaries overall. 2016 Powered by SMPTE www.smpte.org
SMPTE Core Components and Reference Specifications: color encoding and metric specs file formats specs Color Transformations (in CTL language) Reference Implementation (open source code) Reference Images (+ calibration targets for film scanners/recorders) Documentation on software and tools ACES Central portal, acescentral.com SMPTE Standards family ST2065 AMPAS website, www.oscars.org/aces Official GitHub repository, https://github.com/ampas/aces-dev ACES Framework at-a-glance 2016 Powered by SMPTE www.smpte.org
SMPTE First Problem Solved by ACES Problem 1: Different Camera Sensitometry 2016 Powered by SMPTE www.smpte.org
SMPTE Before ACES: Robust and thorough Color Science required before principal photography Camera-matching 2016 Powered by SMPTE www.smpte.org
SMPTE Camera-matching Postproduction paradigm from Editorial 056 001 V C 04:14:37:24 04:14:38:17 00:03:20:13 00:03:21:06 057 GUERRILL V C 04:12:25:24 04:12:26:16 00:03:21:06 00:03:21:23 058 GUERRILL V C 03:13:18:20 03:13:19:12 00:03:21:23 00:03:22:15 059 GUERRILL V C 06:11:34:16 06:11:35:24 00:03:22:15 00:03:23:23 * FROM CLIP NAME: GUERRILLA EDL33.MOV * COMMENT: FINAL CUT PRO REEL: GUERRILLA EDL REPLACED BY: GUERRILL noise/grain mgmt raw processing VFX, CG, compositing localization titles & subtitles digital negs. 10~100 TB/movie Conforming Audio Sync 2016 Powered by SMPTE www.smpte.org
SMPTE Problem 2: Interoperability in post digital negs. 10 2 ~10 3 TB/movie noise/grain mgmt raw processing ACES2065-1 (AP0) VFX, CG, compositing ACEScg (AP1) localization titles & subtitles ACES2065-1 (AP0) Conforming ACEScc or ACEScct (AP1) Color Grading Problem 3: Plethora of Delivery Formats Every studio, broadcaster, (S)VoD provider, ad distributor, film archive has its own delivery / archival standards This rarely includes uniform color encoding Some SMPTE standards (IMF, DCDM) help, Yet no consistent colorimetry specs existed Need to future-proof archived content via an open standard for original looks to be easily reproducible years from now 2016 Powered by SMPTE www.smpte.org
SMPTE Product Partners & Logo program to guarantee interoperability Postproduction pipeline should be done with compatible tools as much as possible (Product Partners, but not limited to them) Product that receive the ACES Logo (in their respective categories) are guaranteed to be interoperable with each other Adherence to User Experience (UX) Guidelines facilitates less experienced / non-technical users (TB-2014-002) Internal processing in ACES color spaces. ACES Color Management from Editorial 056 001 V C 04:14:37:24 04:14:38:17 00:03:20:13 00:03:21:06 057 GUERRILL V C 04:12:25:24 04:12:26:16 00:03:21:06 00:03:21:23 058 GUERRILL V C 03:13:18:20 03:13:19:12 00:03:21:23 00:03:22:15 059 GUERRILL V C 06:11:34:16 06:11:35:24 00:03:22:15 00:03:23:23 * FROM CLIP NAME: GUERRILLA EDL33.MOV * COMMENT: FINAL CUT PRO REEL: GUERRILLA EDL REPLACED BY: GUERRILL 2016 Powered by SMPTE www.smpte.org
SMPTE Entering an ACES pipeline: Input Transform (IDT) Academy Procedure for creating Input Transforms (IDT, P-2013-001) Manfacturers can supply their IDTs to Academy for inclusion in the Reference Implementation as CTL files, and/or Product Partners can integrate IDTs in their logo ed software (e.g. SDKs). Entering an ACES pipeline: Input Transform (IDT) Academy Procedure for creating Input Transforms (IDT, P-2013-001) Manfacturers can supply their IDTs to Academy for inclusion in the Reference Implementation as CTL files, and/or Product Partners can integrate IDTs in their logo ed software (e.g. SDKs). 2016 Powered by SMPTE www.smpte.org
SMPTE Out of the ACES pipeline: Output Transforms ACES color spaces are never meant for viewing (as they are scene-referred). Every Output Transform is made up of: a common part, RRT to a «reference viewing environment» in display-referred space a device-dependent part, ODT to a specific color space/encoding/gamut and technology Output Transform may target diofferent display technologies / lighting environments / max luminance Out of the ACES pipeline: Output Transforms ACES color spaces are never meant for viewing (as they are scene-referred). Every Output Transform is made up of: a common part, RRT to a «reference viewing environment» in display-referred space a device-dependent part, ODT to a specific color space/encoding/gamut and technology Output Transform may target diofferent display technologies / lighting environments / max luminance 2016 Powered by SMPTE www.smpte.org
SMPTE ACES Color Spaces All based on RGB coloradditive model Scene-referred colorimetry 2 sets of color primaries / whitepoints: AP0 and AP1 3 different transfer characteristics: 1 linear, 2 «logarithmic» Due to different purposes & costrains ACES2065-1 main color space Central color space for ACES workflows (ST2065-1, TB-2014-004) Uses AP0 primaries Radiometrically linear («gamma-1.0») Uses 16-bit Floating-point codevalues Preferred storage encodings are: OpenEXR image with mandatory metadata and special flag (ST2065-4, TB-2014-006) Ideal for frame-per-file sequences MXF file container wrapping a sequence of ST2065-4 frames (ST2065-5) Ideal for single-file videos Elective format for any kinds of storage, as well as for long-term archival. CIE X Y Z color space (used for D-Cinema DCDM) is colorimetric, not scene-referred 2016 Powered by SMPTE www.smpte.org
SMPTE ACEScg for CG applications Conceived for modeling, compositing, matte painting, etc. (S-2014-004) Uses AP1 primaries for alignment of RGB operators from non-colormanaged applications Radiometrically linear («gamma-1.0») for lighting applications Represented in floating-point arithmetics Negative values are allowed as Outof-Gamut (OoG) codevalues, but not all applications behave well with them Designed as working-only space, not for storage ACEScc & ACEScct: film-style spaces for color-grading Conceived for color correction & grading operations (S-2014-003 and S-2016-001) Use AP1 primaries for more response continuity of grading operators. «Log.» characteristic emulates film-print tonecurves: ACEScc is a base-2 logarithm, ACEScct has smoother and higher «toe» for better emulation of film-look behaviour in the shadows. Represented in floating-point arithmetics Designed as working-only space, not for storage 2016 Powered by SMPTE www.smpte.org
SMPTE ACEScc & ACEScct: film-style spaces for color-grading Conceived for color correction & grading operations (S-2014-003 and S-2016-001) Use AP1 primaries for more response continuity of grading operators. «Log.» characteristic emulates film-print tonecurves: ACEScc is a base-2 logarithm, ACEScct has smoother and higher «toe» for better emulation of film-look behaviour in the shadows. Represented in floating-point arithmetics Designed as working-only space, not for storage ACESproxy for color transport integer-arithmetics channels Conceived for transport of images via devices with lower computing capabilities (S-2013-001) SDI cables & devices broadcast / reference monitors lower-precision LUT boxes Use AP1 primaries and the same tonecurve as ACEScc. Represented in integer arithmetics (10bits or 12bits per channel) Video-legal codevalue range Transport-only color space: meant for neither encoding, processing nor storing images No «integer»-analogue to ACEScct 2016 Powered by SMPTE www.smpte.org
SMPTE ACESproxy for color transport integer-arithmetics channels Conceived for transport of images via devices with lower computing capabilities (S-2013-001) SDI cables & devices broadcast / reference monitors lower-precision LUT boxes Use AP1 primaries and the same tonecurve as ACEScc. Represented in integer arithmetics (10bits or 12bits per channel) Video-legal codevalue range Transport-only color space: meant for neither encoding, processing nor storing images No «integer»-analogue to ACEScct Codevalue ranges compared 2016 Powered by SMPTE www.smpte.org
SMPTE ACES Color Pipeline from Editorial 056 001 V C 04:14:37:24 04:14:38:17 00:03:20:13 00:03:21:06 057 GUERRILL V C 04:12:25:24 04:12:26:16 00:03:21:06 00:03:21:23 058 GUERRILL V C 03:13:18:20 03:13:19:12 00:03:21:23 00:03:22:15 059 GUERRILL V C 06:11:34:16 06:11:35:24 00:03:22:15 00:03:23:23 * FROM CLIP NAME: GUERRILLA EDL33.MOV * COMMENT: FINAL CUT PRO REEL: GUERRILLA EDL REPLACED BY: GUERRILL LMT for Creative Color Operations Every «inter-aces» color mapping (thus IDT/RRT/ODT are not), applied to the whole image, that is not (just) a color-space conversion between the 5 ACES encodings (TB-2014-010) Creatively there can be more than one LMT applied to the same scene on top of each other (yet the stacking order does matter) Different scenes usually get different LMTs Examples of LMTs are (but not limited to): Local mathematical operations on the whole image (e.g. RGB-matrices, color curves,, but not convolution filters, ) Primary color grades On-set or near-set grades represented as Color Decision Lists (CDLs) Pre-defined LUTs/formulas emulating specific looks (monochrome, bleachbypass, ) 2016 Powered by SMPTE www.smpte.org
SMPTE srgb vs DCI X Y Z srgb vs ACES2065-1 2016 Powered by SMPTE www.smpte.org
SMPTE ACES2065-1 vs ACEScc Color Space comparison srgb reference: γ = 2.2 peak lum. = 100nits whitepoint = D65 2016 Powered by SMPTE www.smpte.org
SMPTE Color Space comparison Rec.2020 γ = 2.4 peak lum. = 1knits differences with: srgb Color Space comparison DCI P3 γ = 2.6 peak lum. = 48nits whitepoint = DCI differences with: srgb 2016 Powered by SMPTE www.smpte.org
SMPTE Color Space comparison DCI X Y Z γ = 2.6 peak lum. = 48nits Real colorimetric space Color Space comparison ACES2065-1 not meant for direct viewing 2016 Powered by SMPTE www.smpte.org
SMPTE Color Space comparison ACEScg working color space not meant for direct viewing subtraction from: ACES2065-1 Color Space comparison ACEScc working color space not meant for direct viewing subtraction from: ACES2065-1 2016 Powered by SMPTE www.smpte.org
SMPTE Color Space comparison ACEScct working color space not meant for direct viewing subtraction from: ACEScc Color Space comparison ACESproxy transport only not meant for direct viewing Video-legal levels not to be stored 2016 Powered by SMPTE www.smpte.org
SMPTE Education Webcast Series SMPTE Non-ACES color stack Original footage is usually in some «log» encoding (35mm film-density Log10 encoding?) This is sort of HDR where important re-exposure of the scene can still be made. Color correction layers are applied on top of this «log» space. Viewing LUT is applied before monitor / projector throughout the process Output LUT (e.g. a Print Film Emulation), is applied before final rendering to the delivery color space. viewing LUT 3D LUT ACES color stack LMT ACES2065 to ACEScc(t) 2016 Powered by SMPTE www.smpte.org
SMPTE Example from a real production On-Set / Near-Set: Cameras: ARRI Alexa (ARRIRAW), RED (R3D), GoPro (MPEG4 QuickTime), mobiles Color Spaces: ARRI Log.C + other camera-native s Reference monitors calibrated and aligned with the post lab s Miscellaneous, yet standard equipment (e.g. Pomfort LiveGrade) Postproduction / Digital Intermediate / Finishing: FilmLight Baselight Cross-calibrated theatre projector and broadcast monitors VFX Autodesk Maya 2017 for modeling and animation The Foundry Nuke X + Studio for compositing On-set / near-set Workflow 2016 Powered by SMPTE www.smpte.org
SMPTE Postproduction Workflow VFX Workflow 2016 Powered by SMPTE www.smpte.org
SMPTE Color Pipeline for the Tests SMPTE2065-4 OpenEXRs without CDL Rec.709 DPXs ACES2065 to ACEScc(t) Pre-shooting ACES tests Camera dept. shoots camera-raw files for all photography sets On-set color-grading using CDL (on top of ACEScc) Camera dept. sends to the lab: Camera-raw files CDLs SMPTE2065-4 renders (16bpc OpenEXR) of selected stills with just Input Transform applied with and without their CDLs Renders of the above stills + Output Transform to Rec.709 (10bpc DPX) «Post» lab processes the camera-raw with Baselight: Working c.s. is ACES2065-1; Grading c.s. is ACEScc CDLs are applied Renders are generated of the same EXRs and DPXs from camera dept. Photoshop comparison (in 32bpc environment) of the same frames 2016 Powered by SMPTE www.smpte.org
SMPTE Pre-shooting ACES tests results Pre-shooting ACES tests, zoomed 2016 Powered by SMPTE www.smpte.org
SMPTE Pre-shooting ACES tests, zoomed VFX ACES turnaround tests 2016 Powered by SMPTE www.smpte.org
SMPTE ACESclip: Clip-level metadata XML sidecar file describing all the color-related metadata of a still or moving picture (TB-2014-009) like: the «past» color transformation(s) it is baked with its current colorimetry the intended ouput /viewing environment in a word, its «color pedigree» Pairs also (and above all) «non-aces» files May contain other metadata (TimeCode, LMT, ) It is linked to the clip via UUIDs that must be honored Color Transformation Language It is a C-like programming language for color operations (S-2014-006) All ACES color transforms (IDTs, ODTs, LMT, ) are written and distributed as CTLs It is intended only for developers of ACES-compatible tools It is structured as a Color Management Library Academy-provided CTLs have a TransformID as UUID s float[3] restore_hue_dw3 (float pre_tone[3], float post_tone[3]) { int inds[3] = order3(pre_tone[0], pre_tone[1], pre_tone[2]); float orig_chroma = pre_tone[inds[0]] - pre_tone[inds[2]]; float hue_factor = ( pre_tone[inds[1]] - pre_tone[inds[2]])/orig_chroma; if(orig_chroma==0.) hue_factor = 0.; float new_chroma = post_tone[inds[0]] - post_tone[inds[2]]; float out[3]; out[inds[0]] = post_tone[inds[0]]; out[inds[1]] = hue_factor * new_chroma + post_tone[inds[2]]; out[inds[2]] = post_tone[inds[2]]; return out; } float moncurve_r(float y, float gamma, float offs) { //// Reverse monitor curve //// float x; const float yb = pow(offs * gamma / ((gamma-1.0)*(1.0+offs)), gamma); const float rs = pow((gamma-1.0)/offs,gamma-1.0)*pow((1.0+offs)/gamma,gamm if(y>=yb) x = (1.0+offs) * pow(y, 1.0/gamma) - offs; else x = y*rs; return x; } 2016 Powered by SMPTE www.smpte.org
SMPTE CommonLUT Format Comprehensive XML-based Color LUT file format (S-2014-006). It can accomodate many stacked color operations (processes): 1D or 3D Color LUT ASC CDL Color matrices Has metadata for input/output digital colorspace descriptions Still limited support by Product Partners (Autodesk Flame / Lustre, FilmLight Baselight / Truelight, Light Illusion LightSpace, ) ACES Framework at-a-glance 2016 Powered by SMPTE www.smpte.org
SMPTE Thank you! Questions CTO Frame by Frame Digital Workflow Supervisor Founder, Kiwii Digital Solutions 2016 Powered by SMPTE www.smpte.org
SMPTE SMPTE Education Webcast Series Sponsors Thank you to our sponsors for their generous support: 2016 Powered by SMPTE www.smpte.org