Technology Day Italy 4K Broadcast Chain David A. Smith February 2017 david.smith@rohde-schwarz.com
Overview ı UHD Resolutions WCG, HFR, and bits/pixel ı High Dynamic Range Principles Perceptual Quantisation and Hybrid Log Gamma ı IP in the Studio Sony Live, AIMS, ASPEN ı 4k/HDR Workflow and Broadcast Solutions David A. Smith : February 2017 2
ı With millions of pixels a few inches from your face Better be glad that the signal's not interlaced Don't you know it needs a big screen size Real wide Alright Beyond Resolution Ultra HDTV & HDR You love the spectral distribution Well you know It's just green and blue and red You think that it's a revolution Well you know It's just the latest hype instead David A. Smith : February 2017 3
EBU UHD Standards David A. Smith : February 2017 4
4k x 2k Ultra HDTV Format: UHD-1 HD: 1920 x 1080 DCI 4K: 4096 x 2160 UHD-1: 3840 x 2160 ı UHD-1: 3840 x 2160 4 x HD Resolution (8 Megapixels) ı DCI 4K: 4096 x 2160 Digital Cinema Initiative for equivalent resolution as 35mm Film David A. Smith : February 2017 5
UHD-1 and the WOW Factor ı Resolution not the only issue for better picture quality ı Improved Audio Greater immersion in sound field (3D and Object Based) ı Reduction of Temporal (Flicker/Judder) Effects Higher Frame Rates ı Increased Colour Space (Gamut) More bits/pixel to reduce contouring and banding effects More and brighter colours to improve realism David A. Smith : February 2017 6
More Than Just Resolution ı 4K on large TVs can appear similar to HD on smaller TVs Minimal WOW! Factor David A. Smith : February 2017 7
Theoretical Resolution: Screen Size v Viewing Distance David A. Smith : February 2017 8
(Reality) Limits of Visual Acuity ı Eye structure is NOT regular Eye has rods (B/W) and cones (Colour) Photoreceptors vary in size and shape across eye => One Pixel per Photoreceptor never applies ı Eye is Never Still (Saccadic Motion) Brain builds picture from multiple views => Higher equivalent resolution than camera Eye + Brain is NOT a Camera David A. Smith : February 2017 9
ı With millions of pixels a few inches from your face Better be glad that the signal's not interlaced Don't you know it needs a big screen size Real wide Alright Beyond Resolution: You love the spectral distribution Well you know It's just green and blue and red You think that it's a revolution Well you know It's just the latest hype instead HDR+ (>8bit + WCG + HFR + HDR) David A. Smith : February 2017 10
Higher Frame Rates (HFR) with UHD TV ı UHD will support frame rates up to 300fps 100fps in Europe and 120 in USA most likely (?) ı HFR Reduces: Judder on scrolling text/titles Motion blur => Arguably 2K@48fps is better than 4k@24fps David A. Smith : February 2017 11
Shutter Speeds: 60fps ı Faster Shutter speeds freezes motion better ı Slower shutter speeds better for low light applications Limitations for live sports and outside broadcast filming David A. Smith : February 2017 12
Increased Colour Depth with UHDTV l More bits to reduce contouring/banding l Rec2020: up to 12 Bits/pixel David A. Smith : February 2017 13
BT2020 Extended Color Space ı UHDTV: Rec 2020 HDTV: Rec 709 ı Wider Colour Gamut 8, 10 and 12 bit colour spaces ı 2015: UHDTVs were at best Rec709 ı 2017: UHDTVs now support subset of REC2020 Much better than Rec 709 Only professional monitors support full Rec2010 David A. Smith : February 2017 14
Wide Colour Gamut (WCG) Examples David A. Smith : February 2017 15
Expanded Colour Volume (Brightness) with BT2020 David A. Smith : February 2017 16
Real World Brightness 3000 Nits 80,000 Nits 150,000 Nits 15,000 Nits 10 Nits 1 Nits ı Extreme light and dark part of everyday experience ı Measured in candela/m2 ( nits ) David A. Smith : February 2017 17
Brightness Ranges David A. Smith : February 2017 18
SDR v HDR Brightness Range David A. Smith : February 2017 19
Different TVs technologies have different max brightness 100 Nits 100 Nits 400 Nits 1000 Nits 1400 Nits ı Gamma Curve needed to map to each display type Two main systems proposed for UHD: PQ and HLG Increasing display brightness David A. Smith : February 2017 20
Standard Dynamic Range (SDR) Gamma Curve (BT.1866) ı Standard CRT TVs max brightness around 100 Nits Mapped to 8 bits in CCIR601 ı Non-linear Response of Camera/CRT requires Gamma correction David A. Smith : February 2017 21
High Dynamic Range (HDR) Gamma Curves: l HDR: Opto-Electrical Transfer Functions (OETF) replace Gamma curves l Non-linear response of camera, panel and eye to brightness l Additional metadata (SEI) transmitted to identify exact EOTF to be used by TV David A. Smith : February 2017 22
HDR Levels: Different Content and Display Brightness David A. Smith : February 2017 23
Hybrid Log Gamma (HLG) EOTF ( Gamma ) Curve Scene Referred (as captured by camera) 120% Target ~1000 nits 100% Relative Standard (to display s capabilities) NO metadata David A. Smith : February 2017 24
Perceptual Quantiser (PQ) EOTF ( Gamma ) Curve Display Referred (as set in post production) Brightness ALWAYS mapped up to 10 00nits Absolute Standard (needs metadata) David A. Smith : February 2017 25
PQ: Dynamic or Static Metadata ı Metadata needed to convert picture for display Dolby Vision uses metadata for each frame or scene (not possible for live TV) HDR10 has one metadata set for entire film (Blu-ray) David A. Smith : February 2017 26
ITU-R BT.2100-0 July 2016: HLG and PQ ı HLG: Hybrid Log Gamma (BBC) ı PQ: Perceptual Quantisation ı Maps linear light to non-linear signal E = signal value for R, G, B at scene E = non-linear R, G, B signal ı Maps non-linear PQ to display light E = non-linear colour F = linear luminance Y = linear colour ı No Metadata Required ı Static or dynamic metadata required David A. Smith : February 2017 27
BT.709 v BT.2020 v BT.2100 David A. Smith : February 2017 28
SDR: Few Sources, Formats and Colour Spaces David A. Smith : February 2017 29
HDR: Multiple Sources, Colour Spaces and Display Formats David A. Smith : February 2017 30
Scene to Screen: Maximise Quality, Brightness, Colour Source: The Light Illusion David A. Smith : February 2017 31
4K/HDR over IP in the Studio David A. Smith : February 2017 32
Legacy Studio Formats David A. Smith : February 2017 33
Serial Digital Interconnect (SDI) ı SDI well established as the studio interconnect since 1982 SDI: 270Mbits/s up to 576i30 HD-SDI: 1.485Gbits/s up to 1080i30 3G-SDI: 2,97Gbits/s up to 1080p60 ı Reliable system carrying audio, SD/HD video and metadata on a single coax cable Single format and point to point cabling simplifies system installation and management ı Widespread availability of compatible routers, switchers, mixing desks etc. ı Future high speed SDI cabling in development for UHD and beyond: 6G-SDI, 12G-SDI, 24G-SDI, David A. Smith : February 2017 34
Why Change From SDI?? David A. Smith : February 2017 35
IP has many advantages over SDI ı Reduced costs standard-it solutions less cables ı Improved flexibility & agility UPnP (Universal Plug and Play) scalability, virtual resources remote production, non-central routing ı Multiple formats on a single network SD, HD, 4k, 8k MPEG-2, H.264, H265 (HEVC), J2K, AAC ı One cable for multiple A, V, and metadata bidirectional central sync over network David A. Smith : February 2017 36
IP Complexities ı Multiple formats and standards possible Combined audio & video Separate audio and video streams Compressed or uncompressed ı Identification and management of multiple sources Synchronisation across a network ı Bandwidth management Raw 4K bit rates > 10Gbit Intelligent routing of sources ı. David A. Smith : February 2017 37
IP Setup and Interoperability ı Single elementary ( Essence ) Streams OR ı Combined audio/video/metadata Streams ı Synchronisation across complete network ı What devices are in the network ı How are they found ı Quality of service across network ı Software Defined Networking (SDN) ı Interoperability of encoding Cloud or local Media Transport Timing Identity Discovery & registration Flow Control Flow Switching Compression David A. Smith : February 2017 38
Video/Audio over IP Formats David A. Smith : February 2017 39
IP Protocols Datagram Transport Specific David A. Smith : February 2017 40
UDP or TCP over IP Datagram Format Length (MTU) typically 1500 bytes IP UDP or TCP ı ı ı Internet Protocol Layer Contains Routing Information Source and Destination IP Addresses User Datagram Protocol: Simple send and forget system Port Number and payload No Acknowledgements or retries Transport Control Protocol: Connection based system Resend on packet loss possible but no FEC David A. Smith : February 2017 41
RTP/UDP/IP Datagram Format Length (MTU) typically 1500 bytes IP UDP RTP Sequence Number Time Stamp Source ID Payload ı ı ı Real Time Protocol used to carry e.g. video or audio Supports synchronisation and packet re-ordering RFC4175 for uncompressed video via RTP Up to 32k x 32k resolution! RFC2733 Defines Forward Error Correction for RTP packets David A. Smith : February 2017 42
AES67: Audio Over IP ı Transport: RTP/UDP/IP Unicast Multicast & IGMPv2 ı Encoding 16/24 bit linear 48 (44.1/96)kHz Channel Count: 1 8 (more possible) ı Synchronisation: IEEE 1588-2008 Media clock synchronous to PTP clock David A. Smith : February 2017 43
SMPTE 2022-2: Transport Stream over RTP/UDP/IP IP UDP RTP TS Packet TS Packet TS Packet TS Packet TS Packet TS Packet TS Packet ı ı ı RTP payload must contain complete 188 byte TS Packets Typically 7 TS Packets per RTP Packet Typically used for IPTV services David A. Smith : February 2017 44
Key Video over IP standards ı SMPTE 2022 2022-1 FEC for MPEG2 TS over IP 2022-2 CBR MPEG2 TS over IP 2022-3 VBR MPEG2 TS over IP 2022-4 Non-piecewise CVBR MPEG2 TS over IP 2022-5 FEC for High Bit Rate Media over IP IP UDP RTP Sequence Number Time Stamp Source ID Payload 2022-6 High Bit Rate Media (uncompressed HD-SDI) over IP (complete SDI stream with video, audio, ancillary data etc.) 2022-7 Seamless Switching SMPTE2022 streams David A. Smith : February 2017 45
IP in the Studio Sony Live IP AIMS ASPEN David A. Smith : February 2017 46
Core IP in the Studio Standards David A. Smith : February 2017 47
Three Competing Studio Systems David A. Smith : February 2017 48
Sony Live IP ı Based on SMPTE2022-6 SDI over IP (video, audio, data together) But separate video, audio and data essence option proposed ı SMPTE2059/IEE1588 for synchronisation ı Does not require special router hardware ı LLVC compression reduces network traffic 4Kp60 in <10GB Ethernet ı LLVC is proprietary codec from Sony Submitted to SMPTE for standardisation David A. Smith : February 2017 49
ASPEN Adaptive Sample Picture Encapsulation ı Based on SMPTE2022-2 (MPEG2 TS over IP) Uses PCR and PTS in transport stream for synchronisation ı Driven by Evertz but with some open aspects David A. Smith : February 2017 50
AIMS Alliance for IP Media Solutions ı Based on TR-03 and TR-04 from the Video Services Forum (VSF) with AES67 audio TR-03: Uncompressed elementary streams over IP using RFC4175 TR-04: video, audio and meta data using separate SMPTE2022-6 IP streams SMPTE2059/IEE1588 synchronisation David A. Smith : February 2017 51
Network Management: Registration and Discovery ı AMWA NMOS: Advanced Media Work Flow Association (AMWA) Network Media Open Specification (NMOS) Registration and Discovery Uses standard internet technology (DNS, TTP, JSON ) ı Video Services Forum (VSF) ı Joint Taskforce for Networked Media: Combining AMWA and VSF Drafting SMPTE2110 David A. Smith : February 2017 52
Converging Standards: SMPTE2110 (in Draft) David A. Smith : February 2017 53
Studio IP Network Video/Audio Storage/ Playout Network Manager & Timing Network Video/Audio Monitoring Video/Audio IP Input SDI to IP IP Switch (COTS) IPTV, OTT, Sat, DTT, Headend Video Audio Production Video/Audio Compression & Storage Distribution Package Creation David A. Smith : February 2017 54
R&S IP Studio Solutions SpycerCell SpycerStorage Software Defined Network Manager Prismon Video/Audio IP Input SDI to IP IP Switch (COTS) AVHE100 Venice 4K AVS100 Clipster David A. Smith : February 2017 55
4K/HDR Workflow David A. Smith : February 2017 56
Scene to Screen: Typical SDR/HDR Workflow David A. Smith : February 2017 57
HDR Workflow Issues: ı Framerate: 50p/60p becoming 100p/120p High Bandwidth Record/Play ı Colour Space/Volume Conversion P3, DCI, XYZ, Rec709, Rec2020 HDR10, PQ, HLG, Dolby Vision High Performance FP Processing Needed ı HDR Metadata (SEI) Static or Dynamic Must track with content from Scene to Screen David A. Smith : February 2017 58
Rohde & Schwarz HDR Solutions: Clipster IMF Interoperable Master Format David A. Smith : February 2017 59
R&S HDR Broadcast Solutions: Venice/Clipster and AVHE100 HDR or SDR Playout UHD/HEVC HDR Encoding R&S VENICE 4K R&S AVHE100 R&S TLU9 ı First SDI-based HDR/SDR signalization VENICE 4K / CLIPSTER to AVHE100 ı Frame by Frame Switching HDR David A. Smith : February 2017 60
Rohde & Schwarz: End to End HDR & SDR Broadcast Solutions Live Production HDR capable camera / converter box mastering 4 x 3G SDI with HDR Metadata Contribution 4Kp120 4:4:4 4 x 3G SDI with HDR Metadata HEVC UHD Encoding and Stat-Mux TS with SEI HDR Metadata Uplink Reception HDR capable decoder and display Colour Volume Conversion BT2100 PQ and HLG Clipster Venice 4K AVHE100 OTT IPTV SAT DTT David A. Smith : February 2017 61
Every Frame Counts Thank You! Rohde & Schwarz Caring for Your Content Every Pixel Counts David A. Smith : February 2017 62