AV1 Update. Thomas Daede October 5, Mozilla & The Xiph.Org Foundation

Transcription

1 AV1 Update Thomas Daede October 5, 2017

2 Who are we? 2 Joint effort by lots of companies to develop a royalty-free video codec for the web

3 Current Status Planning soft bitstream freeze by the end of the month! Lots of decisions made Some tools still have work remaining IPR analysis ongoing 3

4 How AV1 Works 4

5 Overview New high-level syntax Easily parseable sequence header, frame header, tile header, etc. New adaptive multisymbol entropy coding More block sizes Prediction blocks from 128x128 down to 4x4 Transforms from 32x32 down to 4x4 Includes 1:2 and 2:1 rectangular transforms (4x8, 8x4, ) More transform types More references Up to 7 per frame (out of a store of 8) More prediction modes 5 Includes rectangular blocks with 1:2 and 2:1 (4x8, 8x4, etc.) as well as 1:4 and 4:1 ratios (4x16, 16x4, etc.) Both intra and inter More in-loop filtering

6 r e v o c t n ( : a c s i I th l l a Overview New high-level syntax Easily parseable sequence header, frame header, tile header, etc. New adaptive multisymbol entropy coding More block sizes Prediction blocks from 128x128 down to 4x4 Transforms from 32x32 down to 4x4 Includes 1:2 and 2:1 rectangular transforms (4x8, 8x4, ) More transform types More references Up to 7 per frame (out of a store of 8) More prediction modes 6 Includes rectangular blocks with 1:2 and 2:1 (4x8, 8x4, etc.) as well as 1:4 and 4:1 ratios (4x16, 16x4, etc.) Both intra and inter More in-loop filtering

7 Overview take 2 New features and supported use cases Highlight recent additions Integration and deployment 7 Javascript!

8 High-level Syntax Sequence Header Frame Header Tile Group 8 Tile Group

9 High-level Syntax Assists in easy parsing of bitstream Used to define packing into containers 9 Matroska & WebM ISOBMFF (MP4) RTP (WebRTC) Extensibility

10 Colors and HDR Colorspace, color matrix, transfer function can now be encoded directly in the bitstream 10 Chroma siting and levels too

11 Intra Prediction 11

12 Intra Prediction UV Mode Selection Example ( CFL_PRED 17% DC_PRED 44.36% TM_PRED 7.98% SMOOTH_PRED 4.85% Ohashi0806shield.y4m QP = 55 12

13 Intra Prediction Modes More directional modes 8 main directions plus delta for up to 56 directions Not all available at smaller block sizes Smooth modes Smoothly interpolate between values in left column (resp. above row) and last value in above row (resp. left column) Paeth predictor mode Palette mode 13 Color index map with up to 8 colors

14 Other Intra Prediction Enhancements Blend neighbor pixels before prediction Edge extension 14 Strength depends on prediction angle (relative to border orientation) and block size If pixels from one neighboring block unavailable, extend from an adjacent neighboring block edge Chroma from Luma

15 Chroma from Luma 15

16 Chroma from Luma Step 1: Compute AC Contribution Subsample Average 202 Reconstructed Luma Pixels 16

17 Chroma from Luma Step 2: Scale Chroma Planes αcb = αcr= Scaled Values 17

18 Chroma from Luma Step 3: Add Chroma DC_PRED Scaled Values Chroma DC_PRED 18 CfL Prediction

19 CFL in Action Chroma DC_PRED CFL_PRED Scaling factors (-0.25, 0.125) 19

20 Awesome for Gaming 20

21 Inter Prediction 21

22 Motion Vector Coding 22 Each frame has a list of 7 previous frames to reference (out of a pool of 8) Construct list of top 4 MVs for given reference/reference pair from neighboring areas

23 Compound Modes Inter-Inter Compound Segment 23 Pixel weight depends on difference between prediction pixels Inter-Intra gradual weighting Smoothly blends from inter to intra prediction Wedge codebook (Inter-Inter or Inter-Intra)

24 Global Motion Defines up to a 6-parameter affine model for the whole frame (translation, rotation, scaling) Blocks can signal to either use the global motion vector or code a motion vector like normal 24 If global motion isn t used, default is 0,0

25 Warped Motion Use neighboring blocks to define same motion model within a block Decomposed into two shears with limited range 25 Similar complexity to subpel interpolation

26 Loop Filtering 26

27 Constrained Directional Enhancement Filter 27 Combines Daala s directional deringing filter and Thor s Constrained Low-Pass Filter (CLPF)

28 28 Single-pass design Both filters applied simultaneously Fewer line buffers in hardware compared to a simple combination

29 Loop Restoration 29 Two filter choices per superblock Separable Wiener filter with explicitly coded coefficients Self-guided filter Runs in a separate pass after CDEF Showed best metrics of any approach tested Uses deblocking filter output outside of superblock boundaries to minimize line buffers

30 Metrics 30

31 Metrics Relative Bitrate at Equivalent Quality % % PSNR PSNR HVS SSIM MS SSIM CIEDE % 50.00% x265 v1.9 placebo 31 VP9 (8/31) Follow link for AWCY details AV1 (8/26) Goal

32 Complexity AV1 gets most of its compression gains by adding more tools and more options More partition and transform sizes More prediction modes with more parameters More transform types Searching them all in the encoder is slow Decoder also slower, but not as much 32 About 200x slower than VP9 right now Hard to give precise numbers in unoptimized state

33 Demo 33

34 Tools (the utility kind) 34 AreWeCompressedYet AOM Analyzer Subjective viewing

35 AreWeCompressedYet 35

36 AOM Analyzer 36

37 AOM Analyzer Tabs 37

38 Subjective testing 38

39 Code Gerrit code review AOM Analyzer Specification 39 AreWeCompressedYet

40 Questions? 40