High Efficiency Video Coding (HEVC): Applications on Broadcast Platforms

Transcription

1 High Efficiency Video Coding (HEVC): Applications on Broadcast Platforms Mohammad Ghanbari, Professor, PhD, LFIEEE, School of Electrical and Computer Engineering, University of Tehran Video Networking Laboratory, VNL.ut.ac.ir

2 Fundamental question H.264/AVC/MPEG-4 was primarily designed for low quality (QSIF) pictures at very low bit rates but due to its high compression efficiency it is used for HD H.265/HEVC was primarily designed for high quality (UHD) pictures at high bit rates Will H.265/HEVC replace H.264/AVC for compression of HD pictures?

3 Overview UHD video and the necessity for High Efficiency Video Coding (HEVC) Overview of High Efficiency Video Coding standard Compression performance Global experiences in different broadcast platforms 4K/HEVC products and specifications Conclusions

4 UHD video and the HEVC Context and Performance Tools Research Tips & Conclusion 4K Video and its applications 2/56

5 UHD video and the HEVC Video is the Killer Network Application Consumer Web Video Traffic By Geography (PB per month) 12,000 10,000 8,000 6,000 4, North America Latin America Europe Asia ME & Africa 2, Web Video Traffic Walled Video Traffic P2P delivering high-quality video to TV-scale mass audiences will require greater bandwidth, scalability and control than the existing Internet infrastructure is capable of delivering. -- Heavy Reading, August /56

6 UHD video and the HEVC Evolving Digital Media: Need for High Bit Rates Source: Gbps 500 Mbps Gbps 250 Mbs - 6 Gbps 250 Mbps Gbps 200 Mbps - 3 Gbps 20 Mbps Gbps 5-25 Mbps More 8K 4K 2 SHD 4K 2K 2 2K HD 2 HDTV HDV UHDTV Stereo 4K (future) SHD (Quad HD) Digital Cinema Stereo HD HDTV Consumer HD

7 UHD video and the HEVC Why More Resolution is Better? HDTV (2K) More resolution allows closer viewing of larger images Closer viewing of larger image increases viewing angle º Increased viewing angle produces stronger emotional response UHDTV(8K) Picture Height 3840 SHD (4K) Picture Height 100º 1.5 Picture Height 60º

8 UHD video and the HEVC London Olympics 2012 Implement and demonstrate experimental services for real-time 8K video format Capture and multicast 8K content in specific public locations around the UK

9 UHD video and the HEVC Applications Video conference/virtual Presence Simulation & Training Digital Cinema Monitoring/Surveillance Multimedia Production

10 Overview of High Efficiency Video Coding

11 UHD video and the HEVC Standard codecs ITU-T Standard H.261 (Version 1) H.261 (Version 2) H.263 H.263+ H Joint ITU-T/MPEG Standards H.262/MPEG-2 H.264/MPEG-4 AVC H.265/HEVC MPEG Standard MPEG-1 MPEG-4 (Version 1) MPEG-4 (Version 2)

12 UHD video and the HEVC Video Coding Standardization HEVC Context: High Efficiency Video Coding (H.265/HEVC) is a joint standard of ISO/MPEG and ITU-T Successor of H.264/AVC HEVC has become International Standard in January 2013 Goal was to achieve a compression gain over 50% over AVC HEVC IS early 2013 HEVC products CFP 33% gain Intl.Standard 50% gain Offline S/W enc. S/W dec. VOD 3D/MV/Scalability/Range extensions Live S/W enc. H/W dec. Catch up - WebTV - IPTV Live H/W enc. IPTV-Broadcast Extension products What will become possible with 50% video rate reduction? More customers on the same network infrastructure (most of IP traffic is video) Deployment of OTT services New formats (Ultra HD 4K, 8K, 3D ) Large shift in IPTV eligibility (SD and HD)

13 UHD Context video and Performance the HEVC Advances in standard codecs from ITU-T and/or ISO-MPEG Name of 2D video coding standard Name of extension Date Associated profiles Main applications SDO Compression x2 MPEG Main profile Video broadcast (SDTV, HDTV) and storage (DVD) AVC MPEG4-part 10 ITU-T H Baseline Low complexity (mobile, PMP) Main, High Video broadcast (SDTV, HDTV), IPTV ISO-MPEG ITU-T ISO-MPEG ITU-T Compression x2 SVC 2007 Scalable baseline Scalable high MVC 2008 MultiView High Mobile, video-conferencing Video broadcast Video broadcast ISO-MPEG ITU-T ISO-MPEG ITU-T HEVC MPEG-H ITU-T H Main Main10 Still picture Broadcast, storage, videoconferencing, mobile, IPTV (SD ISO-MPEG ITU-T SHVC, MV- HEVC 2014 Scalable/Multi view Main Mobile, video-conferencing ISO-MPEG ITU-T 3D-HEVC 2014 Multiview/mult idepth 3D Main Broadcast ISO-MPEG ITU-T

14 Block Diagram of an HEVC Encoder Sub-division into Coding Tree Blocks Coding Quadtree With Coding block Tools Sub-division into Coding Tree Units Input video signal

15 HEVC Tools: Picture partitioning Typically 64 pixels Prediction Units (PU) 8 possible shapes for a 2Nx2N CU Prediction parameters for each PU Tools Coding Tree Unit (CTU) Coding Unit (CU) Intra only uses these partitions Asymmetric Motion Partitions, (AMP) can be disabled at sequence level Nx2N 2NxN 2NxnU 2NxnD nlx2n nrx2n Transform Units (TU) Quadtree shape The PUs in a CU are all intra or all inter

16 HEVC Tools: Picture partitioning Tools Blue lines: Inter-frame mode Red lines: Intra-frame mode

17 HEVC Tools: Picture partitioning Tools H.264/AVC H.265/HEVC

18 HEVC Tools: Intra-frame Coding Tools Mode 0: Planar Bottom-right pixel is interpolated, then bottom row and right column, then inner pixels Mode 1: DC Mode 2 34: 33 Angular modes Some modes restricted to certain PU sizes

19 Motion estimation

20 Transformation matrices of: 4 4, 8 8, 16 16, and BUT 2D can N/2 x 2N Transformation

21 Parallel Processing Tiling Wavefront Parallel Processing (WPP)

22 Differences (HEVC vc AVC) H.265/HEVC H.264/AVC مولفه واحد درخت کدگذاري با ابعاد ماکروبلاك با ابعاد ساختار تصویر مربعی مستطیلی متقارن مستطیلی نامتقارن مربعی مستطیلی متقارن بلوك هاي تخمین اندازه بلوك هاي تخمین حرکت 4 4 و و 4 4 اندازه بلوك هاي تبدیل + حالت 33 حالت جهت دار + حالت Planar حالت + DC 3 یا 8 حالت جهت دار تخمین درون قابی DC

23 Differences (HEVC vc AVC) مولفه H.265/HEVC H.264/AVC Advanced motion vector prediction, merge, skip, direct نیم پیکسل 8 نقطه اي ویک چهارم پیکسل 7 نقطه اي 4 Spatial median, temporal co-located, skip, direct نیم پیکسل 6 نقطه اي و یک چهارم پیکسل دوخطی تخمین حرکت درون یابی پیکسل هاي Luma درون یابی پیکسل هاي Chroma دوخطی اي نقطه (bilinear) کدگذاري آنتروپی CABAC CABAC, CAVLC فیلتر ضد بلاکی شدن با 4 شدت فیلتر ضد بلاکی شدن با 3 شدت Sample Adaptive Offset (SAO) Loop Filtering

24 Similarities Very similar Video Coding Layer (VCL) Network Abstraction Layer (NAL) Multiple Reference Prediction 52 level quantisation (each 6 Qp unit doubles the step size) Rate-distortion Optimization Deblocking Filter Slightly similar/different Variable Block size Motion Estimation (but non-quadratic) Intra Sample Prediction (but 35 instead of 9 And 4) Entropy Coding (but CABAC only) Transformation (but variable size instead of 4 4 or 8 8) Zing-Zag scanning of DCT Coef. (but in different directions) H.265/HEVC is an enhanced version of H.264/AVC

25 Compression Performance

26 Video Sequences

27 HEVC vs. AVC, visual comparison Using HEVC and AVC in a fixed bitrate could effectively result in visual quality improvement The example shows a same region from a 1080p video encoded at 1Mbps by the AVC and HEVC both by the best available encoders

28 Interlaced vs progressive video Progressive Interlaced

29 Bitrate-PSNR For Interlaced and Progressive video in HEVC Based on video content Interlaced can decrease or increase video bitrate Interlaced has better result in River and Beauty video Progressive has better result in Sky, Pedestrian, Old28, Old30 Old 28 means: video Old with QP = 28 And Old30 means: video Old with QP = 30

30 Bitrate-PSNR For Interlaced video in HEVC and AVC HEVC works better even for Interlaced video in comparison to AVC video.

31 Bitrate-PSNR 1920x1080 video HM and JM YSNR [db] HM-Beauty JM-Beauty HM-Pedestrian JM-Pedestrian HM-Sky JM-Sky Bitrate [kb/s]

32 Bitrate-PSNR 896x504 video (near SD) HM and JM YSNR [db] HM-Beauty JM-Beauty HM-Pedestrian JM-Pedestrian HM-Sky JM-Sky Bitrate [kb/s]

33 Average Bitrate saving in using HM instead of JM HEVC not only works well for high resolutions, but also it outperforms AVC in lower resolutions.

34 The Global Experiences in Different Platforms

35 The Global Experiences in Different Platforms Satellite video broadcast Terrestrial video broadcast Cable video broadcast Video over the Internet

36 Satellite Broadcast TV quality, compression & satellite transmission standards: past, present and the future

37 Satellite Broadcast Transition from DVB-S to DVB-S2 and DVB-S2 extension Ongoing projects in different countries: Japan The United States France Canada South East Asia Spain UK Germany Russia India

38 Satellite Broadcast cont. Japan: NexTV s Channel 4K is already on air via satellite in pilot phase, and it will be lunched by Advanced narrowband transmission method with modulation/demodulation method based on 32PSK It can be received by a 45cm-diameter parabolic antenna at home. The United States MaxLinear and STMicroelectronics s project France: Ateme s project with cooperation of EUTELSAT for live coverage of French Open Tennis (Rolland Garros) 2014 via satellite.

39 Terrestrial Broadcast Broadcasting standard: DVB series (European-based standard): Both European and non-european countries (including Iran) are using DVB-T standard. Countries with ongoing 4K/HEVC related projects are: UK, France, Germany, Czech Republic, Sweden etc. ATSC series: Mostly used in the North America and Pacific countries. Countries with ongoing 4K/HEVC related projects: The United States, Canada etc. ISDB series: A Japanese standard mostly used in East Asia and South America. Countries with ongoing 4K/HEVC related projects: Japan, Brazil

40 Terrestrial Broadcast (DVB) DVB-T to DVB-T2 transition: Faster transmission, larger pictures, more channels Concurrently use of DVB-T2+HEVC improvement in transmission rate: instead of DVB+T/AVC would cause in more 30% in signal compression 40% in transmission rate Total improvement of 70% percent could result in concurrent transmission of 7 HD channels on a single digital multiplex. Achievements: saving in the transmission cost + get free more TV frequencies AVC HEVC 30% save in compression Transmission cost DVB-T DVB-T2 40% save in transmission cost Free TV frequencies

41 Terrestrial Broadcast (DVB) cont. Ongoing Projects: UK: Cooperation of BBC, Arqvia and NEC in a project for DTT transmission. Live terrestrial broadcast of the World Cup 2014 was a part of this project. France: French channel NRJ with cooperation of Envivio and TDF successfully broadcasted French Open Tennis (Rolland Garros) 2014 and another live event. Germany: Successfully replaced DVB-T with DVB-T2 in 2013 and now Germany is working on replacing AVC with HEVC by Czech Republic: České Radiokomunikace with HEVC is transmitting 10 HD channels (alternatively 3 UHD channels or 40 SD channels) on a multiplex. Sweden: Broadcast of UHD channels in cooperation with Teracom, Boxer TV and Broadcom.

42 Terrestrial Broadcast (ISDB) Some projects using ISDB-T standard: Cooperation of Kansai Digital, Elemental Technologies, Toshiba and Fuji in Japan: A big project for terrestrial broadcast of 4k/HEVC content in Japan. Japan Marathon 2014: UHF in-door antenna, 64QAM modulation, 14.5 Mbps Cooperation of Brazilian Globo Grupo with Japanese NEC and BBC in two projects 4K60fps video broadcast with hardware codecs by NEC (Main10 profile, Level 5.1, Main tire) Live terrestrial broadcast of 24 matches from 2014 World Cup

43 Cable Broadcast Japan: Japanese government released the next generation TV implementation plane in June K via cable project s detail: 4K with HEVC Mbps 8K with HEVC Mbps

44 Cable Broadcast cont. The United States Comcast (the largest cable broadcaster worldwide) in cooperation with Samsung has started the Xfinity project. UHD bandwidth: Mbps Using DOCSIS 3.0 standard (Data Over Cable Service Interface Specification) South Korea: Officially, the first industrial project in UHD broadcast over cable (with cooperation of French Ateme and South Korean HomeChoice companies). The content will be accessible via the UMAX free cable channel.

45 Video over the Internet The United States: Ultraflix and Netflix have started serving UHD VoD (Video on-demand) content in cooperation with different participants, including Footprint Media, Redbull Media House etc. MatrixStream has introduced the MatrixCloud OTT platform for 4K/HEVC VoD services. Turkey: IPTV system with the Turkish receiver Air 7410 (using Broadcom s MBC7241) Streaming a single 4K-30fps channel or two HD-60fps OTT broadcast with cooperation of French Ateme and Octashape South Korea: Korean SK Broadband company with more than 1.2 million subscribers, has started UHD services via IPTV for mobile devices.

46 Pioneer Countries in HEVC Countries Terrestrial Satellite Cable Internet Japan X X X United States X X X France X X X Germany X X X UK X X X Czech Republic X Sweden X Brazil X Canada X Spain X India X Russia X South Korea X Turkey X

47 Video Archive (AVC_Intra vs JPEG2000) For long AVC/Intra and JPEG 2000 have been competing for video archive, but there has been no winner

48 Video Archive (AVC_Intra vs HEVC_intra)

49 Products and Specifications The existing HEVC/4K related products can be divided into 2 groups: Hardware products Encoder & Decoder (i.e. Codecs) Accessories (e.g. Display, Set-top-boxes etc.) Software products Encoder & Decoder (i.e. Codecs) Toolkits (e.g. SDKs)

50 Hardware products Allegro: AL-HEVC-E, AL-HEVC-D: Encoder/decoder with Main and Main10 profiles and level 5.1, capable of real-time 4Kx2K video decoding. Multicore processor Al2000: Encoder/transcoder capable of conversion between MPEG-2, AVC and HEVC compressed format, without decoding the streams. It supports MPEG-DASH standard and its equivalents (e.g. Apple HLS, Microsoft Live Smooth Streaming etc.) MediaExcel: HERO-5000E and HERO-5000D: Two hardware codec products optimized for point-to-point video transmission over IP and Asynchronous Serial Interface (ASI).

51 Hardware products Ericsson: SVP 5500 encoder: Ideally suited for mobile applications, it allows Mobile operators and Telcos to take the lead in delivery of high quality video to smart phones and tablets ViXS: XCode 6400 SoC: is the World s First System-on-Chip (SoC) with native support for 10-bit HEVC and UHD 4K. Broadcom: BCM7251 SoC: is a SoC for multi-hd/uhd IP set-top boxes. It also incorporates Dual Display, a feature that enables presentation of two simultaneous video channels from the same STB via independent HDMI outputs and remote control devices. Qualcomm: Snapdragon 805: the UHD processor, is designed to enable end-to-end UHD mobile user experiences, including capturing and playing back UHD video at home and on-device, as well as more realistic gaming, and up to 55MP digital photos, along with outstanding battery life for premium smartphones and tablets.

52 Software products Allegro: AL-3200: Software file-to-file and transcoder, capable of conversion between MPEG-2, AVC and HEVC compressed formats. It exploits Allegro DTV Cluster Manager product which is a task management software for operators to handle repetitive packet transmission issues and task assignment. GENOVA: Software encoder and transcoder for VoD applications using cloud computing approach. Envivio: Muse Live: Software encoder and transcoder with two different service packs (multiscreen and IPTV). Its main application is for real-time (live) encode and broadcast video networking. Muse On-Demand: Software encoder and transcoder for VoD applications in multiscreen environments.

53 Software products MainConcept and DivX: HEVC SDK 4.1: A Software Development Kit optimized for developers wanting to support HEVC in their own software or applications. It offers comprehensive encoding and decoding support for the 4:2:0 8 bit Main and Main10 profiles, including support for BT.2020 enhanced color gamut. TotalCode Serie TotalCode Studio: Professional desktop encoding for digital distribution and over the top services TotalCode Enterprise: Fast, efficient, and cost effective server-distributed transcoding solution for high volume multi-screen content preparation and delivery TotalCode Engine: a powerful standalone command-line transcoding solution for content preparation houses and post production studios, which provides streamlined workflow automation for your in-house production environment.

54 Conclusions (Concept and Platforms) H265/HEVC is similar but is an enhanced version of H264/AVC All coding elements are either the same or improved version But they are not compatible Transcoding is easier HEVC is currently used for 4K video, but it performs equally well in lower resolution video HEVC is tried in all platforms: Satellite Terrestrial Cable Internet (OTT) and Intranet (IPTV) It is very likely to be used in DVB-T2, DVB-S2 and DVB-C2

55 Conclusions (Trials and Manufacturers) It is been tested in almost all developed countries (e.g.): Great Briton, Germany, France, USA, Japan, Korea, South Korea, Spain, India, Russia, Turkey, China, etc It will start with 4K video for sport programs, but will eventually be used in HD for other programs IPTV and OTT (On Demand) will be the main platforms, with largely UHD quality There are numerous Hardware and Software manufacturers in all over the world For both Progressive and Interlaced Video HEVC is superior to AVC HEVC is more complex than AVC Strength: HEVC is more compression efficient than AVC Weakness: It is more complex than AVC, but the technology will evolve

56 Conclusions (Laboratory tests) HEVC is better than AVC in various picture formats: 40 percent more efficient in HD 30 percent more efficient in SD 20 percent more efficient in QSIF (HEVC is a real threat) HEVC in I-P-B mode is about 40% better than AVC HEVC is a better candidate for transmission and Broadcasting HEVC for I-pictures is about 30% more efficient than AVC HEVC is a good candidate for Archive (Archive study group should take it seriously) Like its predecessors, H264/AVC /MPEG4 and H262/MPEG2 it will have great impact on Broadcasting industry It can be used in all chains of Broadcasting

57 Final remarks According to Frost & Sullivan vice president Dan Rayburn: The initial HEVC rollouts will occur : in closed-loop solutions like enterprise video conferencing, ultra HD services in the far east, and low-bandwidth video-on-demand services worldwide, due to the potential cost savings associated with deploying lower bandwidth HEVC video. Rayburn expects : satellite direct to home (DTH) service providers to start rolling out HEVC gear in the timeframe, with some pilot digital terrestrial television (DTT) channels expected in 2015.

58 Final remarks Overall, however, he concludes: While certain applications will embrace HEVC much sooner than the norm and HEVC encoding and decoding cores should mature by 2014, we expect it will be around 2017 before a comprehensive ecosystem of first-generation HEVC-enabled products will come to market. Furthermore, we expect AVC to remain in widespread use even in 2018, although it will definitely be considered a commodity technology at that point -- much as MPEG-2 is today.