Development trends in delivery of Live and VOD based services Thomas Kernen
Agenda Video codec evolution What next for 3DTV delivery? Audio loudness normalisation Beyond HD: Ultra High Definition Next generation time and sync for broadcast infrastructure 3
Video codec evolution
Codec Evolution Codec timeline: Standard ratification MPEG-2 / H.262 1994 MPEG-4 Part 2 / H.263 1999 AVC / H.264 2003 HEVC / H.265 / MPEG-H Part 2 2013 5
Codec Evolution HEVC Specification status ITU H.265 = MPEG HEVC (High Efficiency Video Coding) Developed by experts of the Joint Collaborative Team on Video Coding (JCT- VC) between ITU SG-16 VCEG and ISO MPEG WG11 Feb 2012 : Committee Draft: but only one profile (Main Profile) Jul 2012 : Draft International Standard (DIS) (Main, Main 10, Main Static) Jan 2013 : Final Draft International Standard (FDIS) Note: Parts of the specification will continue to evolve 6
Codec Evolution HEVC Main drivers Further bandwidth savings at all bitrates (Target is 2:1 over H.264/AVC) Ex: Enables expanding IPTV service delivery footprint for DSL based infrastructure Support for higher resolutions (8K by 4K and 4K x 2K) and frame rates Improve performance on mobile devices with HD display capabilities More integrated decode functions = less power/battery usage Launch of 1080p50/60 services to compete against package media (BluRay) Current services generally in 720p or 1080i Support for full resolution plano-stereoscopic 3DTV Current services are frame compatible (2 frames packed in single frame) Expected <10x more computational complexity (encode) and 2x-3x (decode) 720p30 software decode on ipad3 available today (in reference SW decoders) 7
Codec Evolution Classes of resolutions and bit rate points in Call for Proposal Class Resolution F. rate Rate 1 Rate 2 Rate 3 Rate 4 Rate 5 A 2560 x 1600 30 2.5 Mb/s 3.5 Mb/s 5 Mb/s 8 Mb/s 14 Mb/s B1 1920 x 1080 24 1 Mb/s 1.6 Mb/s 2.5 Mb/s 4 Mb/s 6 Mb/s B2 1920 x 1080 50 60 2 Mb/s 3 Mb/s 4.5 Mb/s 7 Mb/s 10 Mb/s C 832 x 480 30 60 384 Kb/s 512 Kb/s 768 Kb/s 1.2 Mb/s 2 Mb/s D 416 x 240 30 60 256 Kb/s 384 Kb/s 512 Kb/s 850 Kb/s 1.5 Mb/s E 1280 x 720 60 256 Kb/s 384 Kb/s 512 Kb/s 850 Kb/s 1.5 Mb/s 8
Where HEVC bandwidth savings stands HEVC Working Draft HM 8 Main Profile vs. JM 18.4 AVC High Profile (October 2012) Resolution Random Access Low Delay All Intra Class A 2560 x 1600 @30 36.9% 23.3% Class B 1920 x 1080 @24 39.5% 41.2% 22.6% Class C 832 x 480 @30/60 30.7% 32.7% 20.0% Class D 416 x 240 @30/60 28.7% 30.2% 16.7% Class E 1280 x 720 @60 43.2% 28.6% Average 34.3% 36.7% 21.9% Source: JCTVC-K0279 contribution (October 2012) 9
Codec Evolution HM 5.0 Subjective Testing (February 2012) Source: JCTVC-H1004 contribution (February 2012) 10
Summary of AVC vs. HEVC High Level View AVC HEVC 16 x 16 Macroblock size Coding Unit size 64x64 to 8x8 Various Inter partitions down to 4x4 Hierarchical quad-tree partitioning down to 8x8 9 Intra modes Up to 35 Intra modes 8x8 and 4x4 transform sizes 32x32, 16x16, 8x8 and 4x4 transform sizes 11
What next for 3DTV delivery?
3D Distribution Technologies Encoding and Transport Technology Networks Encoding Transport Advantages Drawbacks Dual Stream Primary Distribution AVC, MPEG-2 MPEG-2 TS 2 Video PES with same stream type Full Resolution Highest performance Requires dual decode Most bandwidth inefficient Synchronization issues 2D Compatible + enhancement Primary & Secondary Distribution MVC (Stereo profile) 2D + delta (TDVision) MVC over MPEG-2 TS Same stream type as AVC MVC specific descriptors Emphasis on backward compatibility Significant bandwidth Requirements: - MVC: ~1.7 x 2D Frame Compatible May be Primary Distribution Secondary Distribution AVC, MPEG-2 MPEG-2 TS Addition of MPEG-2 descriptor Compatible with existing infrastructure Can be deployed today Bandwidth efficient Dealing with half resolutions reduced video quality Typical bandwidth - ~1.1 x 2D no support for legacy STBs 13
3D Distribution Technologies Dual Stream Encoding Left Eye LE Time RE LE RE Right Eye LE LE RE RE LE RE 3D Frame Sequential Need to synch both views Provides full resolution quality but requires increased channel bandwidth and storage! 14
Frame Compatible Plano-Stereoscopic 3DTV Left Eye Right Eye Top & Bottom Side-by- Side Line Interleave Column Interleave Checkerboard Left Eye Right Eye Provides legacy channel compatibility but reduced picture resolution! 15
Existing standards Frame Compatible Plano-Stereoscopic 3DTV CableLabs OpenCable Content Encoding Profiles 3.0 Specification MPEG-2 or H.264/AVC encoding Top-and-Bottom for 720p and 1080p Side-by-Side for 1080i DVB TS 101 547 v1.1.1 H264/AVC encoding Top-and-Bottom for 720p and 1080p Side-by-Side for 720p, 1080i and 1080p Source coding information in TS 101 154 v1.10.1 16
Service Compatible Plano-Stereoscopic 3DTV Left Eye 2D Time Left Eye 2D 2D Right Eye 2D 2D Right Eye 2D 2D + Enhancement H.264/AVC Multi-view Coding (MVC) Can provide 2D playback compatibility in legacy devices! 17
Next step in 3DTV Service Compatible Plano-Stereoscopic 3DTV DVB Steering Board Approved October 2011 DVB-3DTV Phase 2a Allow 2D and 3D versions of a program in a single video signal 2D derived from left or right image of stereo pair Required work in DVB and MPEG Work completed in June 2012 Published as DVB TS 101 547-3 v1.1.1 in November 2012 Source coding information in TS 101 154 v1.11.1 18
Frame Compatible Compatible Frame Compatible packs 2 pictures into one frame Loss of 50% resolution per image Code 2 nd view into primary image Target approx. 25% extra bandwidth New generation receivers could decode the 2 nd view. Legacy receivers would ignore the additional content Roadmap: MPEG Call for Proposals in July 2012 ITU/MPEG group set up: JCT-3V H.264/AVC and HEVC models under study Working Draft for Multi Resolution Frame Compatible (MFC) planned for completion by end of 2013 19
Next stage in 3DTV Summary Dual stream mostly used in contribution networks and primary distribution Frame compatible is the de facto standard today Service compatible specification published late 2012 by DVB With extra MPEG-2 TS syntax elements in MPEG MPEG MFC work started late 2012 HEVC may help with improving 3D encoding and distribution 20
Audio loudness normalisation
Audio loudness normalisation Current state of audio across multiple services 22
Audio loudness normalisation Target state of audio across multiple services 23
Audio loudness normalisation All sources to be treated according to Tech 3344 26
Audio loudness normalisation EBU Measurements according to ITU-R BS.1770-2 27
Audio loudness normalisation EBU Measurements: target level -23 LUFS 28
Audio loudness normalisation EBU Level adjustments 29
Audio loudness normalisation 2 audio receiver systems are defined in EBU Tech 3343 System A MPEG-1 LII Dolby Digital or Dolby Digital Plus System B MPEG-1 LII HE-AAC (and AAC-LC) Receivers can be System A, System B or both A and B 2 Output modes TV & Stereo amplifier: -23 LUFS AV Amplifier: -31 LUFS 30
Audio loudness normalisation EBU: Receiver audio paths in System A 31
Audio loudness normalisation EBU: Receiver audio paths in System B 32
Audio loudness normalisation Adoption time line France: January 1 st 2012 Live programmes by June 2012, all by end 2012 Switzerland: End of Feburary 2012 Linked with switch to HD Belgium: Mid August 2012 French speaking stations only. Dutch speaking planned early 2013 Germany + Austria: End of August 2012 Linked to IFA trade show UK: Digital Production Partnership standards updated in October 2012 To be enforced in 2013 33
Audio loudness normalisation The real world goal 34
Audio loudness normalisation References ITU BS.1770-2 (BS.1770-3 published, not currently used in R128) EBU R128 Loudness Recommendation EBU Tech 3341 Metering specification EBU Tech 3342 Loudness Range descriptor EBU Tech 3343 Production Guidelines EBU Tech 3344 Distribution Guidelines 35
Beyond HD: Ultra High Definition
Why Ultra High Definition? History of High Definition Early work to improve SD systems: Baird 600 line colour system in 1940 French 819 lines (1949-1983) Initial HD systems: NHK Color in 1972 (1125 lines) MUSE (1125 lines) in Japan with commercial broadcasting from 1994 HD-MAC trials from 1990 to 1993 (1152 lines) Current HD systems: ATSC approved in December 1996, official public HD service launch in 1998 Commercial DVB broadcasts started on January 1 st 2004 (Euro1080 channel) 37
Basics of Ultra High Definition Higher spatial resolution Better immersion with larger field of view Shorter ideal viewing distance than HD Higher frame rates for better motion portrayal Benefit for larger displays (50 and above) Comparable to high end cinema experience Next sales cycle for displays manufactures To be introduced with new services Source: ITU 38
Ultra High Definition systems 2 levels defined Ultra HD-1: 3840x2160 24, 50, 60, 120Hz Near term evolution: 4 times the resolution of current High Definition Ultra HD-2: 7680x4320 24, 50, 60, 120Hz Long term evolution with ongoing trials, 16 times the resolution of current HD Improved colorimetry for more realistic colours 10 and 12-bit depth Progressive scanning mode only No support for interlaced legacy (for now) 39
Higher frame rates Content genre dependent Provides an impression of higher resolution Most beneficial to higher resolutions In broadcast world, higher than 50/60Hz New issues with conversions between rates? Further research being conducted in this space Source: BBC 40
How to get 4K content? Theatrical productions: 35mm cinema print (scanned and converted to 4K format) 4K Digital Cinema production, editing and play out in cinemas Note: 4K Digital Cinema is 4096x2160 Broadcast productions: Trials for shooting live content Requires new production trucks and photography style Distribution: Rely on new HEVC encoding standard. Trials with H.264/AVC exist HDMI currently only supports up to 4K@30fps 41
4K and 8K Production Uncompressed delivery 2160p50/60 (10-bit 4:2:0 & 4:2:2) 12 Gbit/s 2160p50/60 (12-bit 4:2:0, 4:2:2, 4:4:4) 24 Gbit/s 4320p50/60 (10-bit 4:2:0 & 4:2:2) 48 Gbit/s 4320p50/60 (12-bit 4:2:0, 4:2:2, 4:4:4) 96 Gbit/s ITU BT.2020 (production standard) additionally supports 120Hz Even higher rates may be required 42
Ultra High Definition Summary Higher spatial resolution, frame rates and wider field of view More realistic and immersive experience More accurate colour representation Production and distribution chains need to be upgraded New video codec to distribute content due to high bandwidth requirements New displays and an update to HDMI specification to deliver to end point Distribution standards (DVB) need to update specifications First permanent 24/7 demo channel on air in Europe Japan plans first commercial broadcasts for July 2016 (FIFA World Cup) Expect to see more news in the near future 43
Next generation time and sync for broadcast infrastructure
Broadcast production Content is either live (news, sports) or recorded Outdoor broadcast (OB) trucks/vans - to capture events in locations without pre-installed equipment, or connecting to 3 rd party equipment Small(ish) stationary studios or TV stations - to connect to stationary equipment or OB truck TV plant/station - Shot live in studio (news cast, morning show), play out to air live or pre-recorded content Content shot at source is typically in an uncompressed format running over SDI (Serial Digital Interface) 45
Traditional view of how production elements are connected VTR Video Tape Recorder. Original equipment which gets replaced by digital servers but the name stays for now. *Excerpt from RFT_270208 46
Synchronisation in studio Studios are very tightly synchronised For analogue composite signal the alignment is 0.5ns For digital signal the alignment is +/-1.5us All studios use frame synchronisers to align cameras Frame synchronisers compensate cable length differences between cameras Adapt input rate from cameras to the common output rate by removing some frames if input rate is faster or replaying the same frame if input rate is slower Frame synchronisers are often built into video multiplexers Synchronisation within studio must happen within seconds Important to provide means for eliminating packet delay variation (PDV) and asymmetry effects within studio network 47
Benefit of precise synchronisation Having all cameras running at the same frequency and same time would provide means to eliminate frame drops/replays Having common time at all cameras would allow simple time stamping without a mandatory link to the number of frames, if required. Common time would simplify conversion between different video systems (ex. 50/60 Hz). Common time would simplify storage and replay of video content 48
Inter-studio connections Some major events locations such as large stadiums are hardwired to TV stations or production facilities via optical fiber If location of broadcasting is relatively random then the transmission from OB truck to TV station could be microwave or satellite. Follow the information flow. News happens everywhere Connections between local TV stations and headquarters most often use optical fiber 49
So why the need for change? Cost, complexity of infrastructure Multiple distributions required Inflexibility have to pull cables everywhere Many single points of failure Analogue susceptibilities (legacy) Shift in technology 50
Transition to a new network Some cameras, video tape recorders (VTR) / digital servers and phase synchronizers are Ethernet ready Each camera and microphone have individual connections to the mixer Massive amount of cabling is done in each studio Each camera control unit (CCU) has video cable, synchronisation cable and sometimes separate Continuous Time code. May also include return video. The audio is digitized in the audio mixer. The mixer receives Digital Audio Reference Signal (DARS) via separate cable IP based connections for all cables could be replaced by a single one Distribute data and timing synchronisation over the same connection Most audio and video mixers are still not network ready 51
Follow in the footsteps of others IEEE 1588 IEEE 1588-2008 Design industry specific profile to adapt to requirements: ITU-T Telecom profile, IEEE SmartGrid profile 52
SMPTE TC-33TS20 Work Group Create Broadcast IEEE 1588 profile Goal is synchronisation of systems and essence in both digital and analogue forms over networked and streaming transports The reasons for a new standard: Current reference signals are about 30 years old. Based on color black and time labeling and require a dedicated infrastructure No support for multi frequency standards, i.e. needs to have different color black for different frequencies Not easy to sync audio and video... Digital packet based timing distribution in multi-standard media creation and production environments allows overcome these limitations. Support arbitrary frame rates, ms, µs etc. New sync signal not tied to old analogue technology Independent of any specific audio/video standard Ethernet transport (and possibly others) Sync also carries global time, i.e. no need for separate LTC feed to camera 53
Example of a production environment Based on IEEE 1588 PTP monitors PTP slave PTP BC or TC CCU 2-3 Switches microphones BC boundary clock TC transparent clock GM grandmaster GNSS Global Navigation Satellite System, ex. GPS/GLONASS/Galileo CCU Camera Control Unit GNSS PTP GM PTP BC or TC with slave Video mixer with delay adjustment Content to/from other studio 5-10 One studio potentially Switches may be master to another PTP BC or TC Servers Special effects, CC, commercial PTP BC or TC with slave PTP BC or TC Audio mixer/ digitizer Live transmission Nearline Server PTP BC or TC with slave PTP slave Playout Server PTP BC or TC with slave PTP GM can be collocated with some of central piece of equipment such as servers, video mixers, etc Ethernet connection Physical clock connection Analog audio connection Compressed stream/ end customer. No timing Encoder distribution is required PTP slave Due to potentially high number of switches the on-path support such as BC or TC in the switches would be required 54
Example of a pre-standard implementation 55
Breakout Summary H.265/HEVC/MPEG-H part 2 is the next step in the video codec evolution MPEG Frame Compatible is the next stage in 3DTV delivery Audio loudness normalisation to be enforced across Europe Ultra High Definition will enable a more immersive experience Broadcast production industry working on next generation time and sync 56
Call to Action Visit the Cisco Campus at the World of Solutions to experience the following demos/solutions in action: Cisco - Future of TV Meet the Engineer Discuss your project s challenges at the Technical Solutions Clinics 57
58