Conver'ng SD and HD Content to 4K Resolu'on: Tradi'onal Up- Conversion Is Not Enough. Jed Deame February 22, 2013

Transcription

1 Conver'ng SD and HD Content to 4K Resolu'on: Tradi'onal Up- Conversion Is Not Enough Jed Deame February 22, 2013

2 Problem Statement 4k Overview & Interfaces 4k Data Rates Upconversion Basics De- interlacing Scaling Hardware ImplementaAons Summary

3 Problem Statement 4k (UHD) displays are entering the market Professional Consumer Most content is not 4k 1080i/p is 1/4 of 4k res 720p is 1/8 of 4k res 480i/p is 1/20 of 4k res Challenges: Larger Displays ArAfacts Amplified Lots of pixels to invent 3

4 4k Interfaces I/O Interfaces Quad/Octal HD- SDI (292M) Dual/Quad 3G SDI (ST425-3/5) Other MulA- Link Interfaces (32NF40 AHG) SMPTE Gbps HDMI 1.4 a/b, Coming Soon: HDMI 2.0 HDBaseT (4k HDMI Video/Audio + Ethernet + RS232 + IR + Power) DisplayPort 1.2 4

5 Data Rates DVI Single Link / HDMI 1.3 => (HD) HD Video (Y/C) = 1920x1080 x60 fps = MHz (x10) = GHz (x2) = 2.97 GHz HD Computer (RGB) = 1920x1080 x60 fps = MHz (x10) = GHz (x3) = 4.5 GHz WUXGA (RGB) = 1920x1200 x60 fps = 154 MHz (x10) = 1.54 GHz (x3) = 4.62 GHz Max: 165 MHz x 10 = 1.65 GHz x 3 = 4.95 GB/s 5

6 Data Rates DVI Dual Link / HDMI 1.4 [a/b] => (UHD) 4k = 9 MPix = 4096x2160 x24 fps = 284 MHz (x10) = 2.84 GHz (x3) = 8.5 GHz (GB/s) Quad HD = 8 MPix = 3840x2160 x30 fps = 339 MHz (x10) = 3.39 GHz (x3) = GHz (GB/s) Quad HD (RB) = 8 MPix = 3840x2160 x30 fps = 263 MHz (x10) = 2.63 GHz (x3) = 8 GHz (GB/s) Max: 340 MHz x 10 = 3.4 GHz x 3 = 10.2 GB/s 6

7 Data Rates DisplayPort / HDMI 2.0 => (UHD@60) Quad HD@60 = 3840x2160 x60 fps = 712 MHz (x10) = 7.12 GHz (x3) = 21.4 GB/s Max: 540 MHz x 4 = 21.6 GB/s, With Reduced Blanking, 10-bit at 60 fps possible at 16 GB/s 7

8 Building a 4k Canvas 4k MulAViewing 4K Display Sources Up to 8 DVI and HDMI inputs Input grouping for 4K sources 640x480 to 1920x x1152, 720p, 1080i, 1080p 4K Source PC with quad-head graphics card DVI Single or dual link Grouped Input 2K Sources SuperView 4K 4 outputs Single-link: up to 1920x1200 and 2048x1080 Dual-link: up to 2560x1600 Note: Images not to scale 8

9 4k Format Conversion Format Conversion Process: Input Video Signal (serial digital) De-Interlace Input (I-P Conversion) Scale (Resize) Colorspace Convert/Re-sampling Processing Options Proc Amp Detail Enhance Noise Reduce Output Converted Video Signal (serial digital) 9

10 I- P Conversion WHY De- interlace? Displays are progressive scan naavely interlace not an opaon Odd and even fields spaaally shioed due to camera or scene moaon De- Interlacing prior to scaling enables image resizing to be done on a spaaally coherent frame 10

11 I- P Conversion Basic Methods Non- MoAon AdapAve VerAcal (Field) InterpolaAon (aka Bob or Field to Frame ) Temporal InterpolaAon (aka Weave ) VerAcal Temporal InterpolaAon MoAon AdapAve Frame Based Pixel Based MoAon Compensated 2D Pixel Based 11

12 I- P Conversion At the Pixel Level 12

13 I- P Conversion At the Pixel Level 13

14 I- P Conversion MoAon AdapAve 14

15 I- P Conversion MoAon AdapAve 15

16 I- P Conversion Low Angle InterpolaAon 16

17 I- P Conversion Low Angle InterpolaAon 17

18 I- P Conversion MoAon AdapAve w/low angle 18

19 Improved Low Angle InterpolaAon After Before 19

20 Improved Low Angle InterpolaAon 20

21 Improved Low Angle InterpolaAon Cube Vision FPGA Best ASIC 21

22 Image Scaling and Enhancement How do Scalers Work? 22

23 Image Scaling and Enhancement Types of Scalers: Bi- linear Polyphase Finite Impulse Response (FIR) Filter 1D (Separable) 2D (Non- Separable) 23

24 Image Scaling and Enhancement What drives Scaler Performance? Number of Taps (Neighboring pixels) Number of Phases (Sub- pixel Accuracy) The Algorithm (Flat Frequency and Phase) 24

25 Image Scaling and Enhancement 2- Tap Bi- linear Filter 1. Soo with Jaggies 2. Aliasing 3. Uneven response 4. No Ringing Image Credit: AMD 25

26 Image Scaling and Enhancement 6- Tap, Flat Frequency 1. Smooth, Bright, no Jaggies 2. Aliasing 3. Beser response 4. Ringing on edges Image Credit: AMD 26

27 Image Scaling and Enhancement 6- Tap, Flat Phase 1. Very Smooth, no Jaggies 2. Minimal Aliasing 3. Clean but soo 4. Freq steps visible Image Credit: AMD 27

28 Image Scaling and Enhancement Image Enhancement Goal is to improve the perceived detail in the image ExaggeraAng the differences between light and dark (contrast) can produce what appears to be a sharper image Can cause ringing or halo s if not done properly 28

29 Silicon ImplementaAons ApplicaAon Specific Integrated Circuits (ASICs) HQV (Teranex/Silicon OpAx/IDT) EOL VXP Limited support, EOL Concerns, Cannot support 4k Consumer IC s Low- Med Performance, not supported for B cast Market Field Programmable Gate Arrays (FPGAs) The only pracacal soluaon for Broadcast/Pro Video Requires Intellectual Property (IP) Takes many years to develop (high risk) - 3 rd Party IP readily available 29

30 Silicon ImplementaAons FPGA Chip suppliers Xilinx Altera Lavce FPGA De- interlacing & Scaling IP Suppliers Xilinx IP ( - Simple MoAon AdapAve Altera VIP ( - Improved MoAon AdapAve Cube Vision Technologies ( - Broadcast Quality, Exceeds ASIC Performance 30

31 What is possible in an FPGA? Format Conversion Process: ü Input Video Signal (serial digital) ü De-Interlace Input (I-P Conversion) ü Scale (Resize) ü Colorspace Convert ü Processing Options ü Proc Amp ü Detail Enhance ü Noise Reduce ü Output Converted Video Signal (serial digital) 31

32 FPGA ImplementaAon Format Converter 1080i/p HD-SDI FPGA 4096x2160 (1-4 links) DDR3 32

33 FPGA ImplementaAon 4k Format Converter Reference Design 1080i/p HD-SDI CSC SDI Serdes De- interlace FPGA Scale & Enhance Noise Reduce CSC/ Proc SDI Serdes 4096x2160 Quad 3G SDI 1-4 HDMI 10G SDI DisplayPort μp Memory Controller 1x DDR3 Power Supplies 33

34 FPGA 4k True System on a Chip Integrated SERDES Input EQ Output link training Low cost 13 Gbps Tranceivers, up to 28 Gbps+ High Speed Memory Interfaces (60 GBps w/x32 DDR3) Integrated GHz CPU Cores Low cost, small footprint Easy Reconfigurability MulAple suppliers PlenAful IP Blocks 34

35 FPGA 4k Scaling beyond HD 1 Scaler is beser than 4 (avoid edge effects) EliminaAng the Jaggies Need high quality De- interlacing with advanced Low Angle InterpolaAon Managing the Device Size/Cost Requires highly efficient algorithms Managing the 20 GBps Bandwidth Requires efficient memory usage 35

36 Summary How to properly Convert to 4k? Ø High Quality De- interlacing (with Advanced Low Angle InterpolaAon) Ø High Quality Integrated Scaling (with Flat Frequency & Phase Response) Ø Efficient Algorithms Ø Efficient use of Memory FPGA s can provide cost effecave soluaons for 4k video processing 4k Format Conversion 4k Noise ReducAon 4k MulA- Viewing 8k and beyond 36

37 Thank You! Jed Deame