UHD HDR Resource Kit

UHD HDR Resource Kit Production, Grading and Playout Workflows Authors: Joel Marsden, Sr. Video Producer at Harmonic and Executive Producer of NASA TV UHD Eric Gallier, VP Video Solutions at Harmonic Samuel Bilodeau, Post-Production Head at Mysterybox harmonicinc.com

Table of Contents Introduction... 2 UHD HDR Shooting and Production Workflow.... 2 File-Based Post Production and Grading in the UHD HDR Environment... 3 File-Based Playout Workflow for UHD HDR... 5 Conclusion... 6 Summary of Harmonic UHD HDR Solutions and Capabilities... 6 1

Introduction This UHD HDR Resource Kit provides an overview of all of the UHD HDR standards, including their status, readiness, and potential limitations and state of deployments. This technical guide will describe a real-world workflow for File-based content, including shooting, production, post-production, grading and playout. As the leader in video production and delivery solutions that provide exceptional video quality at low bitrates, up to UHD HDR, Harmonic has been actively shooting and producing HDR materials. Our ultimate goal is to provide the industry with a functional real-world workflow. One of the most recent projects we worked on involved shooting bike messengers in New York City. The video was specifically shot in HDR, and then multiple formats were created for quality comparison purposes. The HDR format greatly expands the available color space, increasing brightness and points of darkness. With the Rec. 709 color gamut standard, brightness is limited to 100 nits of output. HDR 10 brings this level up to 1000 nits. With the Dolby Vision proprietary HDR format, it s 4000 nits. We chose bike messengers as the central theme of this video production knowing that there would be action-packed, dynamic situations. For one week, Harmonic s video network specialists followed five different bike messengers around the city. We specifically selected times of the day when there is a wide spectrum of lighting. For instance, we wanted to capture the brightness enabled by HDR during sunrise and sunset on bridges and other iconic locations throughout NYC, while at the same time exposing all the details, even within deep shadows. One of the key characters in our video is Kurt Boone, a well-known messenger, published poet and NYC institution. UHD HDR File-Based Shooting and Production Workflow If you re a video content or service provider looking to create file-based UHD HDR video content, from a shooting standpoint, you don t need an HDR camera. Video capture can be done via the traditional way and in the traditional format. HDR comes into play during the postproduction and color grading process. However, it is of great advantage if the camera has the highest possible exposure latitude so that the maximum amount of information in the image is retained. Formats such as Red Raw or Slog3 are suggested to shoot HDR video. For our bike messenger shoot, we chose to use the RED Digital Cinema WEAPON camera with a DRAGON sensor in HDRx high dynamic range mode. RED raw files contain a massive amount of information that is optimal for color grading in any of the various dynamic range specifications. Note: We processed the bike messenger video into four formats: HDR 10, HLG, Dolby Vision SL-HDR1, and traditional SDR. Links to the finished files in all formats can be found at the end of this guide. 2

Collaborating with Brooklyn-based Mosomos Industries and using the RED camera in HDRx mode, we were able to capture two separate images at the same time. The normal exposure allows about 16 stops of latitude on a RED camera. HDRx extends the dynamic range up to six stops by simultaneously capturing two images of identical resolution and frame rate. The first image is a normally exposed track (A-track), while the second is an underexposed track (X-track) with an exposure value that reflects the additional stops of highlight protection. Let s say you re shooting a scene inside a building and outside is bright sunlight. Traditionally, if you re exposing a subject s face, with only 16 stops of latitude, it s possible that everything outside in the bright sunlight is outside of that range. Windows and exteriors will be clipped. It s the same situation when shooting a subject under a bridge in the shadows. If you re exposing their face, everything in the sky will be washed out. Clipping due to limited stops of latitude has always been a significant challenge for cinematographers. With next-generation cameras, such as the RED with HDRx, cinematographers can capture the basic 16 stops, and use the second exposure of the same shot to capture, for example, a bright sky or the face of a subject in the shadows under a bridge using an additional six stops. We used various techniques to shoot the bike messengers. Frequently our crew was static, capturing the bike riders as they passed by. In addition, we shot from bicycles and used an electric cargo bike to follow riders around NYC. The shoot was based on a method approach, with the production team essentially becoming bike messengers and following the subjects from one location to the next. We focused on shooting extreme lighting situations and notable locations, including the Manhattan skyline and Brooklyn Bridge at sunrise, sunset and night. The shoot took place in December, and there was a major snowstorm that hit the city. During the time-lapse sequence, you can see everything from the sunrise to the snowstorm, full day, sunset and night. That s exactly what HDR is meant to show an extreme range of situations and lighting. In this case, we used almost the entire dynamic range of exposure at 21.5 stops, with no clipping. Red HDRx Up to 21.5 stops dynamic range Records two separate video frames A and X can be mixed in post for best results A look at RED camera s proprietary HDRx menu, which allows two exposures simultaneously for a maximum of 21.5 stops of latitude. File-Based Post-production and Grading in the UHD HDR Environment All of the files from the bike messenger shoot were initially edited in Apple Final Cut 10 and then imported into a DaVinci Resolve editing system for color grading purposes. Color grading was performed by Mysterybox, a leading provider of film and video production services. Any type of color grading engine can be used for post-producing UHD HDR content; however, we at Harmonic and Mysterybox recommend using a system that is agnostic. It s important to utilize a color grading system that doesn t try to interpret what you re doing; rather, it does what you tell it to do. We brought the RED raw footage into the DaVinci Resolve system and used one of two structures for the initial grade: a custom node structure to isolate the A and X frames as separate clips, with the X overlaid on the A in the timeline, and a custom node structure to combine the A and X frames directly within the shot. Which structure we chose depended on the level of detail retained in the X frame. RAWs were interpreted using RedWideGamutRGB and Log3G10, with a custom lookup table (LUT) attached to the project to convert from the RedWideGamutRGB color space into the working color space of Rec 2020, and a custom output curve attached to spread the Log3G10 into the SMPTE ST2084 HDR Curve. 3

The footage was then graded as normal in HDR (SMPTE ST2084 Curve) and shots where the A and X frames were isolated as separate clips saw the A-frame graded first with a limited max brightness, then the X frame graded with an alpha channel to get the contents into the right brightness levels later. In half of these cases, these were directly blended in DaVinci and output as single combined clips, while the other shots saw the A and X frames rendered independently without alpha channels for blending later in After Effects. All footage was then rendered as 12-bit DPX files and round tripped through After Effects for stabilization, blending, and Mysterybox s proprietary footage enhancements designed to mitigate the noise and distortion introduced by the camera s Bayer Pattern. Intermediates were rendered from After Effects again as 12-bit DPX. We did a 12-bit uncompressed intermediate to preserve as much of that color detail as possible. The After Effects outputs were brought back into DaVinci and reassembled. Two sub sequences were created from this ST2084 (HDR10) master sequence for the HLG and SDR conversions. Onto the HLG sequence, a LUT was added to the first grading node of each shot to convert the ST2084 EOTF to the HLG EOTF, while custom tweaks were added to the subsequent node to optimize the look in HLG space. On the SDR sequence, a LUT added to the head of each shot transformed both the EOTF and the color primaries to Rec 709, again with individual tweaks applied to optimize contrast and colors for the medium. In the DaVinci system, Mysterybox flagged the shots where brightness and detail were being lost. Working with HDR, it s critical to adjust the colors, including high and contrast dynamics, to ensure they look accurate. During the bike messenger project, we emphasized pulling down the detail in the darks to use as much of the darks as possible. At the same time, we also aimed to bring up the bright range in certain scenes. One of our main goals was to retain all of the detail that HDR has to offer. As we went through and optimized media files, we looked at shots with clipping. In these cases, we would isolate the X frame, overlay it, and then do a luminous curve. All of the darks in the X frame would clip out and smooth into the previous normal exposed frame. We then brought the brightness of the X frame up so it blends in. These issues primarily occurred in scenes with details like stop lights, head lights, flashes on the street and ambulances. Working in HDR10 with the SMPTE ST 2084 (PQ) transfer function was the most beneficial because the PT curve gives you more dynamic range, especially for blacks. HLG clips are harder to grade during blacks because the bottom half of the image is designed to match SDR; therefore, they don t show quite as much detail and range. At the end of the color grading process, Mysterybox added a specific roll-off. Adding a roll-off allows you to compress the highlights in a controlled way. As you increase the brightness of the image it doesn t race through the highlights. Rather, it moves slowly through the highlights so you can retain control over the high and low lights in HDR, resulting in a smooth grading process. Live monitoring of the HDR grade in a full HDR environment is a must. We used Black Magic s Ultrastudio 4k Extreme connected to and BVM x300. HDMI 3 Thunderbolt Sony BVM x300 Ultrastudio 4K Extreme Mac Running Davinci Resolve 14 Studio Maintaining a closed ecosystem with only HDR compatible hardware components, in their correct settings, is fundamental for proper results. 4

File-Based Playout Workflow for UHD HDR For playout of the UHD HDR bike messenger video, we ingested the content simultaneously into Harmonic s Spectrum X media server and MediaGrid shared storage solution. Putting the UHD content on MediaGrid shared storage allowed our team to access the content directly without a MAM system. We stored the HDR content as an MXF OP1A wrapper file and then were able to edit directly on MediaGrid shared storage using editin-place software. The content was then transferred to a different Spectrum X media server, preserving the HDR video file throughout the workflow. Spectrum X can handle both SDR and HDR files. These can be, for example, HD SDR and UHD HDR. This is important because you do not always have all the files available natively in UHD HDR when you launch a new UHD HDR non-live 24x7 channel. How does it work? When the input file is HD only, Spectrum X first up-converts it on-the-fly in HD to a UHD BT.2020 uncompressed signal. When the input file is UHD, Spectrum X passes it through. All internal processing, including graphic branding, DVEs and effects, are then performed in BT.2020 color space. This applies to live inputs as well as files. HD inputs are up-converted to UHD and managed in BT.2020. The HDR metadata is a pass-through in Spectrum; therefore, it is not altered internally in the server. Spectrum X can air a UHD HDR non-live channel that combines native UHD HDR content like the bike messenger video file with traditional HD file content. This UHD HDR non-live channel is made available through a 4x 3G-SDI output. A simulcast HD (1080p BT.2020 or 1080i/720p BT.709) output that mirrors video and audio with or without unique graphics is also available through a HD-SDI output. Best practices for HDR color grading 1. Use a professional grade monitor. When it comes to color grading, there is no substitute for the light. It s imperative to use a professional color grade monitor. For our post-production and color grading process, we used Sony BVM X-300 monitors. Professional monitors provide you with an infinite amount more dynamic range. You can get away with using an HDR OLED consumer television if you know how to calibrate it properly. Make sure you have tested it and it s working as expected. 2. Add roll-off curve for brights. With a roll-off, you have more control over brightness. If you re not comfortable grading in ACES we recommend grading in BT.2020 with the PQ curve. ACES has gotten great results but it s inconsistent between applications. BT.2020 with the PQ curve is wide enough and contains sufficient color primaries, making it a great specification to start with. 3. Maintain intermediate files. When you render intermediate files, use 12 bits, if it is available. You will need that extra bit depth for HDR. Preserving it at 12 bits gives you the ability to use a derivative version without losing any detail. Don t try to push the image too far. You can quickly start to see banding issues that show up in HDR because it is such a clean image. Edit in Place Monitoring Acquire Ingest Control Ingest Direct Write File Access HDR Render Direct Read Playout Playout Control File Transfer HDR Render Central Storage File Access File Transfer Playout Encoding/ Transcoding Delivery Transcode Spectrum X is a pass through device for HDR If no effects, DVE or graphics are all applied, HDR data is unaffected File-based workflow for UHD HDR In the How-to Guide for HDR Live and Linear Workflows (available summer 2018), we will describe how Spectrum X handles live UHD HDR and/or HD content and can seamlessly switch between file and live UHD content. 5

Conclusion When shooting, take care not to clip highlights or crush dark areas. You want to shoot in a RAW or Slog3 format to retain the maximum bit depth and a neutral exposure on the footage. The more information the camera footage contains, the more HDR effects you will be able to pull out of it. Once in a color grading environment, make sure to keep your workflow contained in a closed HDR ecosystem both with the hardware as well as the video elements. Mixing SDR and HDR environments create uncontrollable errors that are hard to troubleshoot. During the color grading process, you can pull out all of the colors in the expanded BT 2020 color space and also set the brightness levels of the highlights (1000 nits in HDR10). When finishing, pay close attention to which specific HDR scheme you are grading to (i.e., HDR10, HLG or Dolby Vision). With finished video files make sure to include the metadata that describes MaxFALL and MaxCLL. MAXFALL Maximum Frame / Average Light Level MAXCLL Maximim Content / Light Level Sony BVM x300 Summary of Harmonic UHD HDR Solutions and Capabilities for File-Based Production Harmonic s Spectrum media servers and MediaGrid shared storage are perfect for producing UHD HDR file-based content, allowing users to scale systems while providing native codec and format support for SD and UHD operations. Video content and service providers can support UHD HDR and 1080p simulcasts to suit your production needs What makes the Spectrum media server and MediaGrid shared storage systems particularly well suited for UHD HDR file-based production is their rock-solid, industry-leading reliability. Spectrum is the most reliable server platform in the industry, assuring system availability in the most high-pressure environments. Bandwidth consistency sets MediaGrid apart from other shared storage systems, which significantly degrade in performance over time due to data fragmentation. The MediaGrid file system stripes data across all system servers, and clients then access the servers in parallel, ensuring reliable performance for any application, at any time. Finally, MediaGrid shared storage coupled with Harmonic s Spectrum media server enables video content and service providers to instantly share content, edits and metadata. Having parallel access for I/O and editing, along with integration with a wide range of media management tools, sets users up for a highly efficient and successful UHD HDR project. The bike messenger files in HDR 10, HLG, and traditional SDR formats are available to watch on YouTube. Watch, share with your friends and enjoy the amazing quality that UHD HDR provides. 2018 Harmonic Inc. All rights reserved. Harmonic, the Harmonic logo, VOS and Harmonic PURE Compression Engine are trademarks, registered trademarks or service marks of Harmonic in the United States and other countries. Other company, product and service names mentioned herein may be trademarks or service marks of their respective owners. All product and application features and specifications are subject to change at Harmonic s sole discretion at any time and without notice. 08.24.18 6