Scalable Media Systems using SMPTE John Mailhot November 28, 2018 GV-EXPO

Scalable Media Systems using SMPTE 2110 John Mailhot November 28, 2018 SMPTE @ GV-EXPO

SMPTE 2110 is mostly finished and published!!! 2110-10: System Timing PUBLISHED 2110-20: Uncompressed Video PUBLISHED 2110-21: Traffic Shaping Video PUBLISHED 2110-22: Compressed Video Essence (DP Review) 2110-30: PCM Audio PUBLISHED 2110-31: AES3 Transparent Transport PUBLISHED 2110-40: Ancillary Data PUBLISHED 2022-8: Integration with ST 2022-6 (Pre-DP Review)

SMPTE 2110-X: Parts 2110-10: System Timing Model 2110-20: Uncompressed Video 2110-21: Traffic Shaping Uncompressed Video 2110-22: Compressed Video Essence 2110-30: PCM Audio 2110-31: AES3 Transparent Transport 2110-40: Ancillary Data 2022-8: Integration with ST 2022-6

ST 2110-10: How do the parts stay in sync? SDI was good in this regard the embedded audio and VANC were tightly bound to the video (from a timing perspective) In ST2110, Every Device Supports PTP for an exact time Reference Each Stream has RTP timestamps for Synchronization Senders mark each packet of video, audio, or ANC with an RTP Timestamp that indicates the sampling time (or equivalent) Receivers compare these timestamps in order to properly align the different essence parts to each other Users can Mix-and-Match any essence from any source!!!

Session Description (SDP) RFC4566 Each Stream has a set of metadata that tells the receiver how to interpret what is inside of it the receiver needs this info!! The SDP (RFC4566) tells the Receiver everything it needs to know Senders expose an SDP for every stream they make The control system (out of scope of 2110) conveys the SDP information to the receiver AMWA IS05 is the preferred way to do this

SMPTE 2110-X: Parts 2110-10: System Timing 2110-20: Uncompressed Video 2110-21: Traffic Shaping Uncompressed Video 2110-22: Compressed Video Essence 2110-30: PCM Audio 2110-31: AES3 Transparent Transport 2110-40: Ancillary Data 2022-8: Integration with ST 2022-6

ST2110-20: Uncompressed Video Only the Active image area is sent no blanking Supports image sizes up to 32k x 32k Supports Y Cb Cr, RGB, XYZ, I Ct Cp Supports 4:2:2/10, 4:2:2/12, 4:4:4/16, and more Supports HDR (PQ & HLG)

How Much Bandwidth was Saved? Scan Format 2022-6 (Gb/s) 2110-20 (Gb/s) difference 2160p @ 59.94 12282.2 10279.6-16.3% 1080p @ 59.94 3070.7 2570.1-16.3% 1080i @ 29.97 1535.4 1285.0-16.3% 720p @ 59.94 1535.4 1142.5-25.6% 2160p @ 50 12294.8 8754.9-30.3% 1080p @ 50 3074.1 2143.9-30.3% 1080i @ 25 1537.4 1071.9-30.3% 720p @ 50 1537.4 953.0-39.9%

Is ST2110-20 Ready for the Future? Image size can be up to 32K x 32K Frame Rate can be anything you want Can signal which HDR system (SDR, HLG, PQ, easy to add more)

2110-30: Important Facts Built On AES67 -- PCM Audio (only) Many things allowed but only a few required 48kHz sampling is required for all devices 1ms packet time is required for all devices 1..8 channels per stream is required for all devices 16 & 24 bit depth required for all devices Outside the required, must read specs carefully

A little more about channels/stream Send every channel separately? Lots of streams, more configuration, not typical Send bigger streams (2, 4, or 8 channels per) Switching in IP will switch all (2/4/8) channels Downstream sub-selecting makes this a bit better Giant stems up to 64 channels are possible Different Devices make different trade-offs Ask about the number of streams, not just channels

What about Non-PCM Audio? 2110-30 deals only with PCM audio 2110-31 provides bit-transparent AES3 pipes over IP Can handle non-pcm audio Can handle AES3 applications that use the user bits Can handle AES3 applications that use the C or V bits 2110-31 is always stereo (like AES3), can contain multiple AES3 in the same IP stream.

2110-40: Important Facts Over the years, lots of things have been put into the SDI Ancillary Data system Some are tightly related to the video signal Some are really separate essence Some are just along for the ride Audio is handled a better way don t use this method for audio IETF just published RFC 8331, which says how to wrap ANC in IP 2110-40 says how to use RFC 8331 in an ST2110 system

Break-Away Routing Ancillary Data? This is a capability we ve never had before What could you do with this kind of ability? Today we loop through a lot of VANC inserters Subtitles, Teletext, SCTE104, etc. Future the SDI (if you need it) is composed from the parts Subtitle Generators and Teletext systems can directly speak 2110-40

2110 GW 2110 GW VANC Data Routing Just Like Audio Break- Away? 2110-20 video 2110-20 video SDI 2110-30 audio Network 2110-30 audio SDI 2110-40 ancillary 2110-40 ancillary Captioner Downstream Triggers

What is ST 2022-8 (was called 2110-50 Briefly) SMPTE Codification of VSF TR-04 Describes how to use 2022-6 in a manner compatible with 2110 Constrains the packet timing of 2022-6 for network compatibility Requires the 2022-6 stream to be locked to PTP Defines the phase relationship between the 2022-6 stream and the 2110 world TR-04 is used in multiple projects around the world already

That s all nice, but how do I make a system out of it?

How is IP Television Routing Different from SDI? In SDI Routing, all the action happens in the SDI crosspoint frame The Control System tells the crosspoint what to switch where The Crosspoint Router switches everything internally The Receivers just eat whatever bits show up In IP Television Routing, the Receiver is involved in the switching The Control System tells the Receiver What Multicast Group & Port# to switch to (Main and Protect) The Technical Metadata of the new stream The Receiver is responsible for how it switches/transitions to it The Receiver asks the network for the new signal The network provides the new signal through packet forwarding logic

AMWA IS-04: How Devices join the System The system includes a Registry (or redundant registries) New (and old) devices find the registry, and register They also have to maintain their registration periodically The Control System can query the registry to find devices This also supports devices that move to different places on the network Cameras that move from studio to studio Set monitors, prompters, and other shared equipment

AMWA NMOS Connection Management IS-05 Controllers: things that make routing happen Know about the streams from the registration service Maintain the names and meanings of those streams Tell the Receivers what stream to take Act like a routing system to everything in the plant Senders and Receivers: things that make or eat streams Respond to IS-05 Connection Management API JNM - Imagine Communications VSF May 2018 24

What is involved in Switching a Signal in IP? Routing Control Systems Manage GROUPING of element signals Manage NAMES for groups of elementary signals (SOURCES) Manage NAMES for groups of elementary receivers (DESTINATIONS) Routing is: Connect SOURCE (group) to a DESTINATION (group) Confirm with a positive status (or not) But under the hood a lot is happening Human @ Panel Please take CAM 7 to MON 3 Routing Control System Hey Monitor 3, Switch your video to group 238.6.74.22:20000 with {width=1920, height=1080, } Monitor 3 IGMP Leave 237.44.5.3 IGMP Join 238.6.74.22 Network Monitor 3 Routing Control System Human @ Panel OPERATOR ACTION Multicast 238.6.74.22 starts Multicast 237.44.5.3 stops NEW VIDEO ON SCREEN I transitioned to 238.6.74.22:20000 MON 3 is getting CAM 7 OPERATOR STATUS

Can ST2110 Systems work without NMOS? Of course they can and do today The Control Systems have drivers for every different device they control Identifying any required new drivers, and writing them, is part of the project Always a danger of vendors updating a protocol and invalidating the driver Any Practical IP-Routing-Control System must handle all of: Non-NMOS devices (any 2110 RX or TX that doesn t support NMOS) Non-Managed streams (streams created from un-managed sources) Integration with Adjacent SDI routing Integration with other parts of the environment through routing protocols

At the Recent EBU Network Technology Seminar Several EBU member customers commented on the (lack of) vendor support for the AMWA IS04/05 specifications These customers also complained about the audio situation Every different vendor has a different preferred channels-per-stream Mappings at the endpoints are proprietary and messy There was a general call for a Full-Stack approach that combines ST2110, IS04/IS05, PTP, and adds enough details to make working systems An Editor was picked to pull this full-stack document together.

The Customer s View of the IP Universe

What Customer Problem are we Solving? Customer buys a new piece of 2110-x equipment What (all) do they need to do to get it running in their facility? Is it a reasonable set of things today? could it be? (yes)

What is a Full Stack in this context? The Full Stack, for a specific domain of use, describes the requirements on the devices, and the environment they live in, including standards and behaviors, so that A customer can take delivery of a new audio/video endpoint device, attach it to the network, and have a straightforward workflow leading to using the device

Domain of Use SMPTE ST 2110, AMWA IS-04/05, SMPTE ST 2059 were all written to be very flexible and cover a lot of user stories. Developing a full stack needs to be more specific to a type of use For the purposes of this document, the Domain of Use is: Engineered Facilities (Fixed or Mobile) with Engineered IP networks Producing, Packaging, or Delivering television or radio content Built around SMPTE ST 2110 and AMWA IS-04/05 technologies

Media Nodes, Environments, and Workflow Media Nodes Endpoint Devices which create/consume ST 2110 streams (senders, receivers) Environment The control and media networks that interconnect the Media Nodes The Network Services that are supplied on the networks Straightforward Workflow Human in the Loop workflow for new devices, in order to authenticate devices, assign names / identities / groupings to signals Should not require too much engineering knowledge to add a device

Standards and Behaviors Standards SMPTE ST 2110-10, -20, -21, -30, -31, and -40, 2059-1 and 2059-2 AMWA IS-04 and IS-05, IEEE 802.1AB, (LLDP), IEEE 802.1AX (LACP) Behaviors (How does a device ) Know its network host addresses, Gateway(s), and CIDR mask? Know if it is waking up in a system that they were part of, or a new one? Know the prevailing PTP domain and PTP system settings Find and use the IS-04 registry? Know what multicast transmit addresses to use?

I thought 2110 and IS-04/05 were finished? The Standards are done, and the constraints of how to use them can be documented Domain, Devices, Environment, and Workflow are pretty clear But What about those Behaviors? These are the core of the full stack definition for 2110 Media Systems Let s attack these in order and see which ones need work

1. How to Find your Network Details Endpoint Devices Need to know about their place in the network For Each Interface on Management and Media/Data Networks Host Address and CIDR mask Default Route (Gateway) DNS Server Information if needed DHCP (Dynamic Host Configuration Protocol) is typical for this It is well-known and works at scale It is well-supported in routers and network equipment and servers It is not inherently secure, but can be made fairly robust FS: Media Nodes Shall (by default) use DHCP on control and media nets

2: Fresh-Start vs- Re-Start Common practice in broadcast equipment is to power up and resume operation with the last stored settings Most Customers expect/demand this behavior This behavior is problematic for new equipment or rental gear The stored settings are probably not useful and might be harmful It would be very helpful if devices could reliably know that they are waking up in the same system that they were last used in (or not) If same system then use stored settings If new system then mute any multicast senders and wait for configuration FS: A System GUID (globally unique identifier) is compared against the previous stored value and defines a simple way to distribute the system GUID

3: Finding the Prevailing PTP Parameters Endpoint Devices need to know the local PTP Domain number Endpoint Devices with more than one Media Network port need to also know the PTP Announce Interval and Announce Timeout in order to make BMCA-like decisions across the two interfaces ST 2059 (or in general PTP) has no mechanism to distribute this FS: Defines a simple way to distribute this PTP information

4: Finding the IS-04 Registry & Timeout IS-04 specifies MDNS or DNS-SD for finding the registry MDNS is local-subnet specific (does not route) and impractical for most large network architectures DNS-SD via unicast DNS can work well and scale well Adds (redundant) DNS servers as critical infrastructure Makes every system requires local DNS servers FS: Media Nodes shall use unicast DNS-SD to find the registries

Global Information How to Distribute? FS: findable via DNS-SD, a new API structure: /x-nmos/system/v1.0/globals/ { "version": "1441700172:318426300", "caps": {}, "label": "Example System", "description": "System Global Information Example", "tags": {}, "id": "3b8be755-08ff-452b-b217-c9151eb21193", "registry_heartbeatinterval": 8, "ptp_logannounceinterval": -2, "ptp_announcereceipttimeout": 2, "ptp_domainnumber": 57, "syslogv2_server": "biglogger.ebu.ch" }

Behavior 5: Multicast Address Allocation Current practice is to manually allocate multicast addresses, preferably in some pattern related to the types of signals and identity of the sender MADCAP (RFC 2730) could possibly be used, but has some limitations IS-05 provides a mechanism for a controller to specify the transport parameters (multicast address) to senders, if the sender supports it FS: Media Nodes shall support the entire range of multicast addresses from 224.0.2.0 through 239.255.255.255 FS: All Senders and Receivers shall support configuration of their transport_params and master_enable through AMWA IS-05

Full Stack Document: Where are we now? The Full Stack proposal has been drafted by an ad-hoc drafting group under the AMWA IPR Guidelines, with representation from many of the JT-NM member organizations, including Broadcasters and Vendors. Interop Events will be held in the spring to ensure common understanding and implementation of these fine points Common approaches around these system design decision-points is critical in order to effectively build large systems

Thank You