The Image Interchange Framework Demystified: What it is, and why it matters Andy Maltz Director Science and Technology Council Academy of Motion Picture Arts and Sciences
From urbandictionary.com Special Snowflake: A product created specifically for a customer or client who cannot or will not conform their requirements or needs to the general specifications causing said vendor to customize uniqueness to an established product thereby creating more work, time, energy, and effort to be expended on the vendor s employees' part
The Last Snowflake Workflow
Agenda What s changing Why the Academy is supporting IIF IIF overview, core technical features Demo room pointers Your call to action
This is a period of great change and instability Film is going away Digital motion picture cameras are getting better, and generating LOTS of data Cinema may not stay the highest quality for long: Brighter displays in the home Higher framerates coming 4K, UHDTV (wider color gamut than HDTV) Low cost technology lets anyone do nearly anything they want
Problems in Motion Picture Mastering It s a mess out there: Many image file formats and implementations Color management is slathered in secret sauces that make inter-facility interchange difficult Snowflake Workflows: good for HPA, bad for filmmakers 10-bit Cineon isn t good enough anymore Insufficient precision and dynamic range Based on an end-of-life technology (film) Extending to 16 bits won t fix the other problems
The Problems - 2 There is no defined Digital Source Master And therefore no digital archiving standard The traditional sources of industry leadership are absent 81% of a movie s total revenue is from non-theatrical distribution A well-designed, system-wide solution won t happen on its own.
Why the Academy? Mission since 1927: To Advance the Art and Sciences of Motion Pictures Image quality is important and shouldn t be so difficult to achieve The Academy is a neutral forum Represents the interests of the creators of motion pictures Takes on problems that are not commercially feasible Historical role in development of industry standards
Academy-led standards
Other Academy efforts: Digital Preservation
Just published (free copies outside) THE SCIENCE AND TECHNOLOGY COUNCIL
Back to the matter at hand Next-generation cinema is impossible without a nextgeneration imaging architecture This is complicated, and it won t happen all by itself
The Image Interchange Framework (IIF-ACES) An architecture for high fidelity motion picture imaging to support numerous workflows Digital Intermediate Visual Effects Animation Restoration Film and/or digital capture Scene- and output-referred methodologies
Key IIF components Standardized image encoding specifications: Colorimetric: 16 bit half-float, wide gamut, HDR Densitometric: 16 bit integer, fully specified metrics Standardized transforms and methodologies: Image input (digital and film) Rendering and display Standardized file formats Constrained OpenEXR Extended, fully specified DPX Straightforward adoption path
A Brief History of IIF 2004: Siggraph BOF: film scans are inconsistent, there needs to be better image interchange Academy File Format Project initiated 2005: problem study, requirements document written (42 pages long) Conclusion: color management is the toughest nut to crack Ed Giorgianni commissioned to write Digital Cinema color management whitepaper 2006: Project Committee begins designing, viewing images Follow-on whitepaper by Giorgianni Academy hires Senior Imaging Engineer
A Brief History of IIF 2 2007: Draft image encoding specifications published for comment and test Core transforms prototyped Proof-of-concept work launched Color Transform Language (CTL) published as SMPTE RDD-15 2008: IIF introduced at HPA, Call for Participation issued Color corrector implementations begin Container and metadata work begins IIF papers at SMPTE Fall Conference
A Brief History of IIF 3 2009: Working Groups continue working Academy Metadata Symposium IIF introduced at IBC Technical Sessions Draft encoding specifications finalized 2010: Transform development and testing continues SMPTE standards work initiated 2011: Industry trials begin First Designed With IIF In Mind products appear at NAB
A Brief History of IIF 4 2011 (cont d): Key transforms updated with results from trials Television discovers IIF: at least a dozen shows (pilots & series) IIF at NAB Justified panel Academy hires two more imaging engineers and builds state-of-the art imaging lab First 3 SMPTE standards elevated to Draft Publication SMPTE STANDARD Academy Color Encoding Specification (ACES) SMPTE ST 2065-1:2011 SMPTE STANDARD Academy Printing Density (APD) Spectral Responsivities, Reference Measurement Device, and Spectral Calculation SMPTE ST 2065-2:2011 SMPTE STANDARD Academy Density Exchange Encoding (ADX) Encoding Academy Printing Density (APD) Values SMPTE ST 2065-3:2011 Page 1 of 20 pages Page 1 of 20 pages Page 1 of 20 pages Warning This document is not a SMPTE Standard. It is distributed for review and comment. It is subject to change without notice and shall not be referred to as a SMPTE Standard. Recipients of this document are invited to submit, with their comments, notification of any relevant patent rights of which they are aware and to provide supporting documentation. Distribution does not constitute publication. Contents Page Foreword... 2! Intellectual Property... 2! Warning This document is not a SMPTE Standard. It is distributed for review and comment. It is subject to change without notice and shall not be referred to as a SMPTE Standard. Recipients of this document are invited to submit, with their comments, notification of any relevant patent rights of which they are aware and to provide supporting documentation. Distribution does not constitute publication. Contents Page Foreword... 2! Intellectual Property... 2! Warning This document is not a SMPTE Standard. It is distributed for review and comment. It is subject to change without notice and shall not be referred to as a SMPTE Standard. Recipients of this document are invited to submit, with their comments, notification of any relevant patent rights of which they are aware and to provide supporting documentation. Distribution does not constitute publication. Contents Page Foreword... 2! Intellectual Property... 2!
A Brief History of IIF 5 2012 (so far): Engineering nearing completion on key transforms Next round of industry trials in preparation Collaborative software development site (GitHub)
Industry-wide collaborative development More than 50 leading practitioners, engineers and scientists Leading equipment manufacturers Major post-production, VFX and animation facilities Major technical contributions from: Adobe, Arri, Autodesk, Dolby, FilmLight, Fujifilm, ILM, Kodak, Pixar, Sony Electronics and top color science and imaging experts
The IIF Eye Chart (Idealized System Block Diagram) ADX: Academy Density exchange ACES: Academy Color Encoding Specification RRT: Reference Rendering Transform RDT: Reference Display Transform IDT: Input Device Transform
Motion picture workflow: legacy on-set timing session digital acquisition source cam lut digital color correction release print film emulation rec 709 xform hd mon color correction session Full-range raw digital acquisition cam lut digital color correction release print film emulation DCDM xform dist master
Motion picture workflow - IIF on-set timing session digital acquisition source IDT ACES RRT hd ODT mon color correction session Full-range raw digital acquisition IDT ACES RRT RDT dist master
ACES: Key features Enhanced Precision Wide Color Gamut High Dynamic Range IMF-ready
Precision: why 10 bits isn t enough Steps visible Steps not visible
Exposure latitude of film negative exceeds 10 bit Cineon - that s why ACES is 16 bit half-float Beyond Video Conversion Range Video Conversion n Range
Wide gamut: ACES covers all visible colors ACES color primaries in perspective CIE x CIE y Red 0.73470 0.26530 Green 0.00000 1.00000 Blue 0.00010-0.07700
Color gamut is also increased with greater dynamic range
Normal exposure 3 2
Rec. 709 processing - 2 stops under 3 3
ACES 2 stops under 3 4
What the colorists say It s All There
What a famous Hollywood director says I want my whole cow. Not just the steak you serve me.
Why we render Film and digital sensors capture scene exposures (proportional to # of photons hitting the imaging plane) Unprocessed scene exposures don t look good on any display medium Rendering applies an image processing algorithm to scene exposure values to generate a pleasing image on a target display device or medium
Scene exposure vs. output referred Accurate Reproduction of Scene Colorimetry Image Rendered for Pleasing Reproduction 42
Scene exposure Printing Density
Scene exposure ACES 4 4
Rendered image ( Output-referred ) 4 5
Good rendering is hard to do Film manufacturers spent 70 years working on it Digital Intermediate facilities spent 15+ years on it (and they all do it differently) The IIF Project Committee spent 5 years working on a baseline renderer : the Reference Rendering Transform, or RRT 46
RRT Facts Designed to go several stops beyond film and today s digital displays (>1,000,000:1) Designed to make the colorist s job easier: a better starting point Designed to not get in the way of the creative vision Mathematically sound
Digital Cinema Projector r Video Display Print Film Dolby PRM Lab HDR Display IIF Theoretical Display Print Film Emulation (PFE)
Digital Cinema Projector r Video Display Print Film Dolby PRM Lab HDR Display IIF Theoretical Display RRT Release Candidate (February, 2012)
Mythbusting
IIF Myths It s not baked yet First standards are done Field trials taught us a lot, and they resulted in significant improvements Continued comprehensive, industry-wide testing is crucial It forces the Film Look Rendering is required All pleasing images have some things in common The RRT does not go to all the places PFEs go
IIF Myths - 2 IIF will never work in production Every trial project found some issues, and they all were resolved Issue: smaller, under-resourced VFX facilities may not be good early adopters IIF will never work for everything How do you prove a negative? We think it will Even if we re wrong, huge benefits for the projects it does work for
IIF Myths - 3 The Academy is dictating that you use IIF - Early SMPTE proponents include: ASC Paramount Pictures Sony Pictures Pixar Several manufacturers Visit the Demo Room
The cost of waiting SMPTE standards are publishing now Very difficult to change after publication The time to ask for changes is now
What s next RRT release candidate: March 1, 2012 Next round of industry trials Complete file container and base metadata standardization Support plan development
Demo Room: IIF in Real Life Dolby, Image Systems: High dynamic range display driven by IIFenabled color corrector FilmLight: Baselight with IIF support, matching multiple digital motion picture cameras Fujifilm: IIF-based on-set color management tool Image Essence: IIF software implementation
Try it out let us know what you think! For more information on The Image Interchange Framework please visit http://www.oscars.org/iff