Conver'ng SD and HD Content to 4K Resolu'on: Tradi'onal Up- Conversion Is Not Enough Jed Deame February 22, 2013
Problem Statement 4k Overview & Interfaces 4k Data Rates Upconversion Basics De- interlacing Scaling Hardware ImplementaAons Summary
Problem Statement 4k (UHD) displays are entering the market Professional Consumer Most content is not 4k 1080i/p is 1/4 of 4k res 720p is 1/8 of 4k res 480i/p is 1/20 of 4k res Challenges: Larger Displays ArAfacts Amplified Lots of pixels to invent 3
4k Interfaces I/O Interfaces Quad/Octal HD- SDI (292M) Dual/Quad 3G SDI (ST425-3/5) Other MulA- Link Interfaces (32NF40 AHG) SMPTE 2036-3 10Gbps HDMI 1.4 a/b, Coming Soon: HDMI 2.0 HDBaseT (4k HDMI Video/Audio + Ethernet + RS232 + IR + Power) DisplayPort 1.2 4
Data Rates DVI Single Link / HDMI 1.3 => (HD) HD Video (Y/C) = 1920x1080 x60 fps = 148.5 MHz (x10) = 1.485 GHz (x2) = 2.97 GHz HD Computer (RGB) = 1920x1080 x60 fps = 148.5 MHz (x10) = 1.485 GHz (x3) = 4.5 GHz WUXGA (RGB) = 1920x1200 x60 fps = 154 MHz (x10) = 1.54 GHz (x3) = 4.62 GHz Max: 165 MHz x 10 = 1.65 GHz x 3 = 4.95 GB/s 5
Data Rates DVI Dual Link / HDMI 1.4 [a/b] => (UHD) 4k = 9 MPix = 4096x2160 x24 fps = 284 MHz (x10) = 2.84 GHz (x3) = 8.5 GHz (GB/s) Quad HD = 8 MPix = 3840x2160 x30 fps = 339 MHz (x10) = 3.39 GHz (x3) = 10.17 GHz (GB/s) Quad HD (RB) = 8 MPix = 3840x2160 x30 fps = 263 MHz (x10) = 2.63 GHz (x3) = 8 GHz (GB/s) Max: 340 MHz x 10 = 3.4 GHz x 3 = 10.2 GB/s 6
Data Rates DisplayPort / HDMI 2.0 => (UHD@60) Quad HD@60 = 3840x2160 x60 fps = 712 MHz (x10) = 7.12 GHz (x3) = 21.4 GB/s Max: 540 MHz x 4 = 21.6 GB/s, With Reduced Blanking, 10-bit at 60 fps possible at 16 GB/s 7
Building a 4k Canvas 4k MulAViewing 4K Display Sources Up to 8 DVI and HDMI inputs Input grouping for 4K sources 640x480 to 1920x1200 2048x1152, 720p, 1080i, 1080p 4K Source PC with quad-head graphics card DVI Single or dual link Grouped Input 2K Sources SuperView 4K 4 outputs Single-link: up to 1920x1200 and 2048x1080 Dual-link: up to 2560x1600 Note: Images not to scale 8
4k Format Conversion Format Conversion Process: Input Video Signal (serial digital) De-Interlace Input (I-P Conversion) Scale (Resize) Colorspace Convert/Re-sampling Processing Options Proc Amp Detail Enhance Noise Reduce Output Converted Video Signal (serial digital) 9
I- P Conversion WHY De- interlace? Displays are progressive scan naavely interlace not an opaon Odd and even fields spaaally shioed due to camera or scene moaon De- Interlacing prior to scaling enables image resizing to be done on a spaaally coherent frame 10
I- P Conversion Basic Methods Non- MoAon AdapAve VerAcal (Field) InterpolaAon (aka Bob or Field to Frame ) Temporal InterpolaAon (aka Weave ) VerAcal Temporal InterpolaAon MoAon AdapAve Frame Based Pixel Based MoAon Compensated 2D Pixel Based 11
I- P Conversion At the Pixel Level 12
I- P Conversion At the Pixel Level 13
I- P Conversion MoAon AdapAve 14
I- P Conversion MoAon AdapAve 15
I- P Conversion Low Angle InterpolaAon 16
I- P Conversion Low Angle InterpolaAon 17
I- P Conversion MoAon AdapAve w/low angle 18
Improved Low Angle InterpolaAon After Before 19
Improved Low Angle InterpolaAon 20
Improved Low Angle InterpolaAon Cube Vision FPGA Best ASIC 21
Image Scaling and Enhancement How do Scalers Work? 22
Image Scaling and Enhancement Types of Scalers: Bi- linear Polyphase Finite Impulse Response (FIR) Filter 1D (Separable) 2D (Non- Separable) 23
Image Scaling and Enhancement What drives Scaler Performance? Number of Taps (Neighboring pixels) Number of Phases (Sub- pixel Accuracy) The Algorithm (Flat Frequency and Phase) 24
Image Scaling and Enhancement 2- Tap Bi- linear Filter 1. Soo with Jaggies 2. Aliasing 3. Uneven response 4. No Ringing Image Credit: AMD 25
Image Scaling and Enhancement 6- Tap, Flat Frequency 1. Smooth, Bright, no Jaggies 2. Aliasing 3. Beser response 4. Ringing on edges Image Credit: AMD 26
Image Scaling and Enhancement 6- Tap, Flat Phase 1. Very Smooth, no Jaggies 2. Minimal Aliasing 3. Clean but soo 4. Freq steps visible Image Credit: AMD 27
Image Scaling and Enhancement Image Enhancement Goal is to improve the perceived detail in the image ExaggeraAng the differences between light and dark (contrast) can produce what appears to be a sharper image Can cause ringing or halo s if not done properly 28
Silicon ImplementaAons ApplicaAon Specific Integrated Circuits (ASICs) HQV (Teranex/Silicon OpAx/IDT) EOL VXP Limited support, EOL Concerns, Cannot support 4k Consumer IC s Low- Med Performance, not supported for B cast Market Field Programmable Gate Arrays (FPGAs) The only pracacal soluaon for Broadcast/Pro Video Requires Intellectual Property (IP) Takes many years to develop (high risk) - 3 rd Party IP readily available 29
Silicon ImplementaAons FPGA Chip suppliers Xilinx Altera Lavce FPGA De- interlacing & Scaling IP Suppliers Xilinx IP (www.xilinx.com) - Simple MoAon AdapAve Altera VIP (www.altera.com) - Improved MoAon AdapAve Cube Vision Technologies (www.cubevisiontech.com) - Broadcast Quality, Exceeds ASIC Performance 30
What is possible in an FPGA? Format Conversion Process: ü Input Video Signal (serial digital) ü De-Interlace Input (I-P Conversion) ü Scale (Resize) ü Colorspace Convert ü Processing Options ü Proc Amp ü Detail Enhance ü Noise Reduce ü Output Converted Video Signal (serial digital) 31
FPGA ImplementaAon Format Converter 1080i/p HD-SDI FPGA 4096x2160 (1-4 links) DDR3 32
FPGA ImplementaAon 4k Format Converter Reference Design 1080i/p HD-SDI CSC SDI Serdes De- interlace FPGA Scale & Enhance Noise Reduce CSC/ Proc SDI Serdes 4096x2160 Quad 3G SDI 1-4 HDMI 10G SDI DisplayPort μp Memory Controller 1x DDR3 Power Supplies 33
FPGA Benefits @ 4k True System on a Chip Integrated SERDES Input EQ Output link training Low cost 13 Gbps Tranceivers, up to 28 Gbps+ High Speed Memory Interfaces (60 GBps w/x32 DDR3) Integrated GHz CPU Cores Low cost, small footprint Easy Reconfigurability MulAple suppliers PlenAful IP Blocks 34
FPGA Challenges @ 4k Scaling beyond HD 1 Scaler is beser than 4 (avoid edge effects) EliminaAng the Jaggies Need high quality De- interlacing with advanced Low Angle InterpolaAon Managing the Device Size/Cost Requires highly efficient algorithms Managing the 20 GBps Bandwidth Requires efficient memory usage 35
Summary How to properly Convert to 4k? Ø High Quality De- interlacing (with Advanced Low Angle InterpolaAon) Ø High Quality Integrated Scaling (with Flat Frequency & Phase Response) Ø Efficient Algorithms Ø Efficient use of Memory FPGA s can provide cost effecave soluaons for 4k video processing 4k Format Conversion 4k Noise ReducAon 4k MulA- Viewing 8k and beyond 36
Thank You! Jed Deame 407-310- 4312 Jed@CubeVisionTech.com