Low Power Design of the Next-Generation High Efficiency Video Coding

Transcription

1 Low Power Design of the Next-Generation High Efficiency Video Coding Authors: Muhammad Shafique, Jörg Henkel CES Chair for Embedded Systems

2 Outline Introduction to the High Efficiency Video Coding (HEVC) HEVC Analysis complexity, memory access, thermal Power-Efficient HEVC System Design Conclusion 2

3 Normalized Memory BW. [GB/s] High Efficiency Video Coding (HEVC) Ultra-HD (or supervision) million pixels per frame By 2017: 80% 90% global internet traffic New video compression standards/techniques required JCT-VC s High Efficiency Video Coding (HEVC) ~2 compression efficiency compared to H.264 Full 30fps 1 second 712 Mbits 1 hour 2.4 Tbits Time Bitrate (a) 3 1.4E E E E E E E+11 HEVC H.264/AVC (b) Basketball Kimono PeopleOnStreet 0 HD720 1 HD K 3 3

4 Challenges for Developing HEVC-based Multimedia Systems Challenges & Requirements Compute Complexity Content-Awareness, HW-SW Collaboration, Many-core Systems Power Efficiency Accelerator Design, Content-Awareness, Power- Gating Thermal Management Thermal Analysis, Configurations, Content-Adaptive Parallelization Workload Balancing, Arch.-Awareness, Power Budgeting Video Memory Memory Hierarchy Design, Content-Aware 4

5 HEVC Overview: Encoding Flow Input Video in CTUs + Transform and Quatization Inverse Transform and Quantization Recursive TU Size Reduction Intra Prediction Recursive CU/PU Size Reduction Inter Prediction Bitstream Headers CABAC Entropy Coder Decoded Picture Buffer Deblocking and SAO Filter Output Reconstructed Video Output Bitstream 5

6 HEVC Overview: Slices and Tiles Slice 0 Slice 2 Slice 3 Slice 1 Tile 0 Tile 1 Tile 2 Tile 3 Tile 4 Tile 5 GOP 0 GOP 0 F 0 F 0 F M-1 T 0 T 1 T 0 T 1 T K-1 T K-1 Core 0 f 0 Core 1 f 1 Core K-1 f K-1 HEVC Parallel Encoding 6

7 HEVC Overview: Tree-Block Structure CTU CTU 0 CTU 1 CTU Example PU Configuration Example CU Configuration Tested TU Configurations 7

8 CTU Distribution 8

9 HEVC Overview: Intra and Inter Prediction HEVC Intra Prediction HEVC Inter Prediction Vertical Angular Predictors Horizontal Angular Predictors 0: Planar 1: DC log2 2 2 M i 0 2 i N i log2 3 2 M i 13 2 i 0 HEVC-Intra: ~2.56 more mode decisions than H.264 HEVC-Inter: ~2.2 more complex than H.264 9

10 HEVC Overview: Motion Estimation Block Matching (BM) or Motion Estimation (ME) Compression by searching temporal neighbors High energy/time, high compression efficiency (H.264-Inter, HEVC-Inter) Reference Frame Current Frame Residue Frame Motion Vector Best Matching Current Block Search Window Previous Frame 10 Current Frame

11 HEVC Overview: Search Data Fetching High leakage High dynamic External Memory (DRAM) High bus power External Memory Bus Very high leakage On-Chip Memory (SRAM) Current Frame A memory subsystem with low power consumption and high efficiency is Current Block required Search Window Block Matching Reference Frame 11

12 Outline Introduction to the High Efficiency Video Coding (HEVC) HEVC Analysis complexity, memory access, thermal Power-Efficient HEVC System Design Conclusion 12

13 Percentage Area HEVC Analysis: Computational Complexity CU/PU Partitioning Large partitions for low-variance and homogeneous image areas and vice-versa High Variance Low Early PU size prediction may provide Regions Variance significant reduction in computational Regions and energy requirements Smooth texture (due to larger QP or resolution) is usually captured by larger sized PUs BasketballDrill ParkScene PeopleOnStreet

14 HEVC Analysis: CTU Distribution 14

15 Percentage Utilization HEVC Analysis: Memory Accesses Memory Access for Motion Estimation Memory accesses of HEVC 3.86 of H.264 Most of the on-chip memory is wasted (leakage power) 100% 75% 50% H.264 HEVC (a) Maximum (b) 25% Only a part of the full search window is utilized 0% Median Adapting the search window size at run-time provides Minimum increased potential for leakage power savings 75 % 25 % Keiba BasketballDrill RaceHorses KristenAndSara 15

16 Using a thermal camera setup Linux Ubuntu kernel Voltage supply IR Camera A bottom view Water-cooling unit to cool down the thermoelectric device Thermal pad CPU chip Thermal map Water heat sink Thermoelectric device Copper plate Peltier Based Cooling Intel Atom 45nm dual-core processor (1.8 GHz) Src: Intel DIAS Pyroview thermal camera operates at 50Hz with spatial resolution of 50 µm Copyright: Chair for Embedded Systems (CES), Karlsruhe Institute of Technology (KIT), Germany 16

17 Temperature Measurements for HEVC vs. 22QP] Temp max.: 55.0 C Temp min.: 36.0 C Temp avg.: 53.0 C DATE 14 Temp max.: 53.0 C Temp min.: 35.0 C Temp avg.: 49.0 C Copyright: Chair for Embedded Systems (CES), Karlsruhe Institute of Technology (KIT), Germany 17

18 HEVC Analysis: Temperature Temperature ( C) So What is Required? Interplay between Software and Hardware needs 45 Keiba (1.8 GHz) 40 Basketball (1.35 GHz) Time (sec) to be explored for power/energy optimization 62 ºC 56 ºC 50 ºC 44 ºC Temperature ( C) Keiba Basketball Time (sec) 1. Optimized Algorithms for Fast Intra- and Inter- Prediction 2. Energy-Efficient Hardware Accelerators 3. Energy-Efficient Video Memory Heirarchy 4. Content-Adaptive Power Management Frequency Dependence Content Dependence 62 ºC 60 ºC 58 ºC 56 ºC 54 ºC 52 ºC 50 ºC 48 ºC 46 ºC 44 ºC 18

19 Outline Introduction to the High Efficiency Video Coding (HEVC) HEVC Analysis complexity, memory access, thermal Power-Efficient HEVC System Design Conclusion 19

20 Power Efficient HEVC Design: Hardware Architecture HEVC Software Layer Application Driven Adaptive Power/ Thermal Manager Video Tile Formation HEVC Encoding Intra/Inter Energy to Quality Tradeoff Complexity Reduction Scheme Data Analysis and Statistics Adaptive Workload Budgeting CT CT R R CT CT R R CT CT R R HEVC Hardware Processing Architecture Feedback Monitors to Software CT R CT R... CT R Battery Off-chip DRAM 20

21 Analysis and Statistics 2000 PDF Frequency 8x8 16x16 32x32 64x64 Variance PDF Frequency 8x8 16x16 32x32 64x64 Distortion Variance Distortion Parameter Value SAD SSE SATD Kbps Max. CU Depth Search Range Variance and Motion based Classification AMP

22 Complexity Reduction: PU Size Estimation CTU variance computation at 4 4 v 1 n 1 x 1 n i x i 0 2 HEVC CTU Compressor Recursive 4 neighbors merge PU Map (PUM) PU Map Above (PUMA) v c CombineVariances v if v v OR v v i, i {1,2,3,4} c Th i, i {1,2,3,4} Th MergeBlocks v Th 4 1 log 2 ln 2 v QP Rayleigh CDF Analysis Empirical Analysis H µ v = Mean of variance curve Δ = CDF threshold (0.8) H = Size of PU to combine 22

23 Normalized Time Time Savings and Video Quality Results Sequence Class Size BD-PSNR BD-Rate Traffic A 4K BasketballDrive B 1080p BasketballDrill Traffic BasketballDrive BasketballDrill C BQSquare WVGA RaceHorses Johnny Basketball BQSquare Drive D Drill WQVGA DrillText RaceHorses D WQVGA Johnny E 720p BasketballDrillText F WVGA

24 Time [msec] Tile Mapping and Parallelization Cores CPUs Max freq. f max Frame Rate f p Core 0 Output Core 1 Core 2 Core 3... Core K-2 Core K-1 Workload is not equal for tiles Workload (per core) Tile Formation and Maximum Workload Tile Estimator 0 Tile 1 Tile 3 Tile Video 4 Input Frequency (per CPU) Workload Allocator Monitoring Unit Threshold Generator Workload Adaptation Total Intra Angles (θ) Frame Offline Tuning Workload Manager Core Frequency Selector Intra Mode Prediction User bit-rate tolerance n 24

25 HEVC Thermal Management Application-Driven DTM Extract Motion Intensity HEVC Encoder Application driven DTM Frequency scaling Execute HM Core0 Sensor Core1 Sensor T current > T critical NO YES 25

26 Temperature (ºC) HEVC Thermal Management 56 ºC 54 ºC 52 ºC 50 ºC 48 ºC 46 ºC 44 ºC 42 ºC 40 ºC 38 ºC 60 Max Average Min No DTM DTM 54ºC DTM 50ºC DTM 46ºC 0 PSNR (db) Bit rate (kbps) No DTM Our 54 C Our 50 C Our 46 C 26

27 Peak temperature (ºC) HEVC Thermal Management No DTM DTM 54ºC DTM 50ºC DTM 46ºC # Frames Peak temperature (ºC) No DTM DTM 54ºC DTM 50ºC DTM 46ºC # Frames Keiba BasketballDrill 27

28 Power Efficient HEVC Design: Hardware Architecture CT CT R R CT CT R R CT CT R R HEVC Hardware Processing Architecture Feedback Monitors to Software CT R CT R... CT R Battery 28

29 Hardware Accelerators M CTU Row HW 0 HW 1 HW Intra 2 HW 8 PPC 2N/8 Predictor Number of datapaths in parallel Legend: Slice LUTs (luma) Slice LUTs (chroma) Slice registers (luma) Slice registers (chroma) Occupied Slices (luma) Occupied Slices (chroma) 29

30 AMBER: Memory Subsystem External Memory holds the current frame High density, low read and write power On-chip SRAM memory (FIFO) holds only the current block External Memory (Current Frame) External Memory Controller High read and write speed and low dynamic write power Hides latencies from HEVC engine MRAM Buffers (N Reference Frames) Reference Write Master Current Read Master - Reads current frame data - Writes SRAM Buffer -Low write amount -Fast Write On-chip Current Data (Block) SRAM SRAM Block FIFO Reference Read Master - Reads reference frames - Low latency read Block Matching Engine HEVC Encoder (Transform loop) Power Control HEVC Video Compression Control 30

31 AMBER: MRAM Reference Buffers One MRAM buffer holds a full reference frame Each column (sector) of reference buffer is power-gated Reference read and write masters read and write data to the MRAM buffer Reference Write Master H MRAM Reference Buffers W W Reference F 1 H Reference F N Row Buffer SRAM FIFO MRAM Power Gate Control Reference Read Master HEVC Encoder Block Matching 31

32 AMBER: Reference Buffer Power Management Observation: Not all of the search window is used Block matching algorithm accesses only a small percentage of reference buffer sectors Power-gate unused sectors Reduce leakage s CU s 1 x min s 1 s 2 Block Matching Turned OFF Turned ON s CUPrediction s of Unused Sectors is based on: 2 x max 1. Self-Organizing Map 2. Content Properties 32

33 Power [W] Power Consumption (4 reference frames) Search Window AMBER Keiba China Speed Four People Basketball Drive People Keiba ChinaSpeed FourPeople BasketballDrive People 832x x x x x1600 Increasing the number of reference frames improves the power consumption of the AMBER system compared to the search window approach 33

34 Conclusion Comprehensive analysis of HEVC Architecture, power, thermal and complexity Challenges posed by HEVC Architectural (memory, reconfiguration, accelerators) Power/thermal (power-gating, configuration control) Complexity (parallelization, many-core, workload balancing) Both Hardware and Software need to be optimized while leveraging the application-specific knowledge Our approach Adaptive complexity management Video tiling, workload budgeting, CU/PU partitioning Power and thermal aware HEVC configuration Hybrid video memory hierarchy with content-driven power-gating 34

35 ces265: Multi-threaded HEVC Encoder Open-source C++ based Multithreading via pthread API One thread of ces faster than HM-9.2 Tile Formation and Workload Curtailing Slice Compressor Sniper many-core x86 simulator HEVC-Intra Encoder s Top GOP Compressor Workload Queue Tile Compressor Threads Workload Manager System Configuration YUV Read Write Encoder Statistics CTU Compressor Workload Allocator Proposed HEVC Intra Encoder Simulator statistics McPAT power simulator Power statistics Web Download 35

36 Acknowledgement Muhammad Usman Karim Khan Daniel Palomino Claudio M. Diniz Felipe Sampaio 36

37 Thank you! Questions? Web: Download: 37