VIDEO 2D SCALER. User Guide. 10/2014 Capital Microelectronics, Inc. China

Transcription

1 VIDEO 2D SCALER User Guide 10/2014 Capital Microelectronics, Inc. China

2 Contents Contents Introduction Function Description Overview Function I/O Description I/O timing Register configure Usage guide Getting started Video_2d_scaler configure Video_2d_scaler application note Resource usage and performance analysis Generate File Directory Structure

3 1 Introduction The purpose of this document is to describe the user guide of CME s video_2d_scaler. The scaler is used to resize video. The video_2d_scaler IP is used to change output video size with same frame rate as input. It converts video signals from one display resolution to another. Typically, scalers are used to convert a signal from a lower resolution (such as 480p standard definition) to a higher resolution (such as 1080p high definition), a process known as "up conversion" or "upscaling" (by contrast, converting from high to low resolution is known as "down conversion" or "downscaling"). Video scalers are typically used in consumer electronics devices such as digital televisions, medical imaging, video surveillance, video conferencing,video game consoles, and DVD or Blu-ray disc players. The scalers are often combined with other video processing devices or algorithms to create a video processor that improves the apparent definition of video signals. The CME video_2d_scaler has following features: (1) Supports video up and down scaling. (2) Supports YCbCr 444, RGB video format, data width can be configured from 8 bits to 10 bits. (3) Supports standard video scaler upto 1920x1080, such as 480P, 576P, 720P, 1080P and other PC signal. (4) Supports configurable video scaler algorithm, including nearest,, lanczos or user defined. (5) Supports max 4x4 tapes poly-phase filter with 256x4 orders coefficients. (6) Configurable scaler factor, scaler initial phase. (7) Vertical and horizontal scaler share with same filter coefficients rom. (8) CME s EMB is used as video data and coefficients buffer, no external memory is needed. Device family support: CME-M5, CME-M7, CME-HR3 3

4 2 Function Description 2.1 Overview The scaler uses pixels interpolation for resize. Shown as Figure 2-1, it is scaler pixel input and output diagram. The horizontal direction is video columns and the vertical direction is video rows. The scaler output pixels are space interpolated by scaler input pixels. scaler output pixel scaler input pixel Figure 2-1 diagram of video scaler input and output pixel We assume the space distance of input two continuing pixels or lines is 1, the space distance of output two continuing pixels or lines is calculated as Equation 2-1 and Equation 2-2. scaler houtstep = scaler in hw 1 scaler outhw 1 Equation 2-1 horizontal scaler space step calculation scaler voutstep = scaler invw 1 scaler out vw 1 Equation 2-2 vertical scaler space step calculation Here, scaler houtstep is horizontal scaler output pixel space distance, scaler inhw is horizontal scaler input pixel width, scaler outhw is horizontal scaler output pixel width, scaler voutstep is vertical scaler output line space distance, scaler invw is vertical scaler input line width, scaler outvw is vertical scaler output line width. Scaling with nearest neighbor algorithm 4

5 neighbor is a simple method of multivariate interpolation. The nearest neighbor algorithm selects the value of the nearest point and does not consider the values of neighboring points at all, yielding a piecewise-constant interpolant. This algorithm generates visible jagged edges in the high frequency part of output image. But it requires no DSP blocks, and uses fewer logic elements than the other methods. The interpolation value is calculated as Equation 2-3. scaler outpixel (i, j) = scaler inpixel (m, n) Equation 2-3 nearest neighbor interpolation calculation Here, scaler outpixel (i, j) is scaler output pixel value, i range is(0, scaler outvw ), j range is(0, scaler outhw ). scaler inpixel (m, n) is scaler input pixel value, m range is(0, scaler invw ), n range is(0, scaler inhw ) and i j m = integer [ scaler vout step ], n = integer [ scaler hout step ] 0 0 Shown as Figure 2-2, it is method diagram of nearest neighbor interpolation. For example, we have scaler hout step = 0.8, scaler = 0.7, i = 0, j = 0, so m = 1, n = 1, the interpolation pixel value voutstep scaler outpixel (0,0) = scaler inpixel (1,1). 0_0 0_1 1_0 1_1 Figure 2-2 method diagram of nearest neighbor interpolation Scaling with algorithm The algorithm performs linear interpolation first in one direction using two pixels, and then again in the other direction using two pixels. It has good performance than nearest neighbor especially on image edge part, but it also lost some high frequency information on image edge. The interpolation value is calculated by 2x2 taps interpolation as Equation 2-4. scaler outpixel (i, j) = (scaler inpixel (m, n) (1 W(m)) + scaler inpixel (m + 1, n) W(m)) (1 W(n)) + (scaler inpixel (m, n + 1) (1 W(m)) + scaler inpixel (m + 1, n + 1) W(m)) W(n) Equation 2-4 interpolation calculation Here, scaler outpixel (i, j) is scaler output pixel value, i range is(0, scaler outvw ), j range is(0, scaler outhw ). scaler inpixel (m, n) is scaler input pixel value, m range is(0, scaler invw ), n range is(0, scaler inhw ) and 5

6 i j m = integer [ scaler vout ], n = integer [ scaler step hout ] step 0 0 i j W(m) = fraction [ scaler vout ], W(n) = fraction [ scaler step hout ] step 0 0 Shown as Figure 2-3, it is method diagram of interpolation. For example, we have scaler hout step = 0.8, scaler vout = 0.7, i = 1, j = 1, so m = 1, n = 1, W(m) = 0.7, W(n) = 0.8 the interpolation pixel step value scaler outpixel (1,1) = ((scaler inpixel (1,1) scaler inpixel (2,1) 0.7)) ((scaler inpixel (1,2) scaler inpixel (2,2) 0.7)) 0.8 0_0 0_1 1_0 1_1 Figure 2-3 method diagram of interpolation Scaling with lanczos algorithm The lanczos algorithm performs better performance than linear interpolation. The module uses 4x4 taps for vertical and horizontal direction interpolation. The interpolation of this algorithm is shown as l=2 k=2 scaler outpixel (i, j) = ( scaler inpixel (m + k, n + l) LUT(W(m), k + 1) ) LUT(W(n), l + 1) l= 1 k= 1 Equation 2-5 lanczos interpolation calculation Here, scaler outpixel (i, j) is scaler output pixel value, i range is(0, scaler outvw ), j range is(0, scaler outhw ). scaler inpixel (m, n) is scaler input pixel value, m range is(0, scaler invw ), n range is(0, scaler inhw ) and i m = integer [ 0 scaler vout ], n = integer [ scaler step 0 hout ], step i W(m) = fraction [ 0 scaler vout ], W(n) = fraction [ scaler step hout step LUT(W(m),0)~LUT(W(m),3) is lanczos vertical 4 taps look up table coefficients. LUT(W(n),0)~LUT(W(n),3) is lanczos horizontal 4 taps look up table coefficients. j j 0 ] Shown as Figure 2-4, it is method diagram of lanczos interpolation. For example, we have scaler hout step = 0.8, scaler vout = 0.7, i = 1, j = 1, so m = 1, n = 1, W(m) = 0.4, W(n) = 0.6 the interpolation pixel step l=2 k=2 value scaler outpixel (1,1) = l= 1 ( k= 1 scaler inpixel (m + k, n + l) LUT(0.4, k + 1) ) LUT(0.6, l + 1) 6

7 0_0 0_1 0_2 0_3 1_0 1_1 1_2 1_3 2_0 2_1 2_2 2_3 3_0 3_1 3_2 3_3 Figure 2-4 method diagram of lanczos interpolation 2.2 Function The system block diagram of using video_2d_scaler IP is shown as in Figure 2-5. Video frontend The function of video frontend modules converts input video format, De-interlacing, De-noising and some other same function modules. The modules output standard video, such as 480P, 576P, 720P, etc. Video 2D scaler The function of video 2D scaler module is to convert input video size to display video size, such as from 480P to 720P, 720P to 1080P, 1080P to 720P, etc. Video backend The function of video backend modules processes video to get good performance, convert video format for display and other same function modules. CME FPGA Video input source Video frontend Video data Video 2D scaler Video data Video Backend Video output for display Figure 2-5 system block diagram of using video_2d_scaler IP 7

8 The functional block of video_2d_scaler IP is as Figure 2-6. Video_2d_scaler Scaler_ctrl rom_rd_addr Scaler_coeff_rom Horizontal coefficient frame_valid_start Line_buffer_rd_en Line_buffer_rd_addr Vertical coefficient Video input Video_pre_down_scaler Pre_scaler_dat Line_buffer_dat Vertical_scaler _dat Dat_line_buffer Ver_scaler Hor_scaler Video output Figure 2-6 functional block diagram of video_2d_scaler The detail function description of each module is as following: Video_pre_down_scaler The module function is as following: (1) Down scaling the input horizontal size less than or equal to 1024 if input horizontal size bigger than (2) Down scaling the input horizontal size to less 2 times scaler output horizontal size. For example, if input horizontal size is 1920, but final output scaler horizontal size is 320, the pre-down scaler module output horizontal size is 640. (3) Down scaling the input vertical size to less 2 times scaler output vertical size. For example, if input vertical size is 1080, but final output scaler vertical size is 480, the pre-down scaler module output vertical size is 540. (4) When non-down scaling input mode, the module bypass the down scaler function and directly output the pixel data. Dat_line_buffer The module function is as following: (1) Input pixel data is directly written into line buffer one by one as line time sequence. (2) Input line_buffer_en is used to select which line buffer is valid at current vertical scaling. (3) Input line_buffer_addr is used to select which data in line buffer is valid at current horizontal scaling. (4) Generate frame valid start to scaler_ctrl module. Scaler_ctrl The module function is as following: (1) When detects line valid start signal is valid, start to generate scaler output line counter and line valid. (2) When detects frame valid start signal is valid, start to generate read line buffer enable and line 8

9 buffer address signal. (3) Generates reading coefficient rom address for vertical and horizontal scaling. Scaler_coeff_rom The module function is as following: (1) Generate vertical scaler coefficients. (2) Generate horizontal scaler coefficients. Ver_scaler The module function is as following: (1) Vertical video scaler interpolation calculation (2) Configured different scaler mode. (3) Scaler overflow and underflow control. Her_scaler The module function is as following: (1) Horizontal video scaler interpolation calculation (2) Configured different scaler mode. (3) Scalar overflow and underflow control. 2.3 I/O Description Shown as in Table 2-1, it is the pin description of video_2d_scaler. It has three parts connection, first part is connected with the system interfaces; second part is connected with input video interfaces; third part is connected with output video interfaces. Table 2-1 the pin of video_2d_decoder top module Name Type Size Description System Interface i_sc_inclk I 1 Scaler input data clock I_sc_outclk I 1 Scaler output data clock i_rstn I 1 System reset, low is active Input video interface I_frame_vld I 1 Input video frame valid signal I_dat_vld I 1 Input video data valid signal I_dat I 8*3-10*3 Input video data, RGB/YUV 444. Binary offset format input Output video interface o_frame_vld O 1 output video frame valid signal o_dat_vld O 1 output video data valid signal o_dat O 8*3-10*3 output video data, RGB/YUV 444. Binary offset format output 9

10 o_wr_dat_ready O 1 1 b1, ready for input write data 2.4 I/O timing Video scaler data input port timing is shown as Figure 2-7. Figure 2-7 timing of video_2d_scaler input signal Video scaler data output port timing is shown as Figure 2-8. Figure 2-8 timing of video_2d_scaler output signal Video scaler input and output signals timing relation is shown as Figure 2-9. Figure 2-9 relation timing of video_2d_scaler input and output signals Video scaler input data valid and output wr_data_ready signal relation is shown as Figure Figure 2-10 relation timing of video_2d_scaler input data valid and output wr_data_ready 10

11 2.5 Register configure The parameter of video 2d decoder is configured in IP GUI. The IP wizard calculates some parameters according user input video size, scaler mode and clock frequency. Shown as Table 2-2, it is detail description about parameters used in video 2d scaler top module. Table 2-2 the parameter of video_2d_decoder top module Parameter name Range Default value Description DATA_WIDTH Input and output data width, the data has three paths(rgb/ycbcr). VSC_MODE Vertical scaling method select: 1:nearest 2:, vertical filter taps 2 4:lanczos, vertical filter taps 4(in this case, coefficients can be set by user) HSC_MODE Horizontal scaling method select: 1:nearest 2:, horizontal filter taps 2 4:lanczos, horizontal filter taps 4(in this case, coefficients can be set by user) IN_HSC_WID Input video horizontal valid pixel number one line TH IN_VSC_WID Input video vertical valid line number one frame TH IN_HSC_TOTA Input video horizontal total pixel number one line L_WIDTH IN_CLK_FRE (0,200) 65Mhz Input video frequency OUT_HSC_WI output video horizontal valid pixel number one line DTH OUT_VSC_WI output video vertical valid line number one frame DTH OUT_HSC_TO TAL_WIDTH output video horizontal total pixel number one line, it is calculated by wizard automatically. it is calculated as following: RD_LINE_START = VSC_MODE ==2? 3 : VSC_MODE==4? 4 :2; OUT_HSC_TOTAL_WIDTH=((IN_HSC_TOTAL_WIDTH*PDV_SCALER _FACTOR)*(PDV_SCALER_WIDTH-RD_LINE_START)*OUT_CLK_FR E)/((OUT_VSC_WIDTH-RD_LINE_START)* IN_CLK_FRE) OUT_CLK_FR E (0,200) 108Mhz output video frequency 11

12 3 Usage guide 3.1 Getting started The video_2d_scaler IP is available by using CME s Primace wizard manager tool. User can use wizard manager tool to generate scaler IP and re-configure the IP s according parameters. (1) Open wizard manager tool in Primace If the IP is first created, then choose create a new design, else if the IP is re-configured, then choose edit an existing design. Detail see Figure 3-1. Figure 3-1 Open wizard manager (2) Choose video 2d scaler IP in IP cores list User can choose video 2D scaler IP in IP cores list, then click Next button. Current device name is shown to user. Detail see Figure 3-2. Figure 3-2 choose video 2d scaler in wizard manager 12

13 3.2 Video_2d_scaler configure (1) Edit IP name and path User can rename IP name in module name blank box, select generation IP directly path. Detail see Figure 3-3. Figure 3-3 Edit video_2d_scaler name and path in wizard manager (2) Edit IP parameter User can edit IP parameter as list in Table 2-2 the parameter of video_2d_decoder top module. Detail description of configuration is shown in Table 3-1 and Figure 3-4. Table 3-1 Description of configuration parameter Configuration parameter Description Input/output width Input or output data width can be selected from 8bits to 10 bits in each YCbCr or RGB Horizontal scaling mode 3 types horizontal scaling method can be selected: : the method use non DSP and use least resource but poor video performance. Bilinear: the method use 2 horizontal pixels interpolation and has better video performance than nearest. Lanczos : the method use 4 horizontal pixels non-linear interpolation and has better video performance than. vertical scaling mode 3 types vertical scaling method can be selected: : the method use non DSP and use least resource but has poor video performance. 13

14 Horizontal scaling phase vertical scaling phase Input frequency Input valid width Input total width Input valid height output frequency output valid width output total width output valid height Bilinear: the method use 2 vertical pixels interpolation and has better video performance than nearest. Lanczos: the method use 4 vertical pixels non-linear interpolation and has better video performance than. Horizontal_value/2^16, range is 0~1, horizontal scaling initial phase Vertical_value/2^16, range is 0~1, vertical scaling initial phase Input pixel frequency Input horizontal valid pixel number Input horizontal total pixel number Input vertical valid line number output pixel frequency, for input and output continuing valid signal, the output clock must be set with proper value output horizontal valid pixel number output horizontal total pixel number output vertical valid line number Figure 3-4 Edit video_2d_scaler parameter in wizard manager 3.3 Video_2d_scaler application note (1) Resource limited For HR3 and HR2, it has no DSP resource, so the scaler just can be used with nearest mode. 14

15 For M5, the maximum 18x18 DSP number is 16. So the scaler can be used in horizontal with lanczos and vertical with mode. For down-scaling application, more EMB is used than up-scaling, so user must note the EMB limited in M5 device. For M7, the DSP and EMB number are enough for each scaling module using. (2) Input and output standard timing The video scaler output timing is affected by configured input video total pixel,input line number, input and output clock frequency, output total pixel number and input video timing. If user expects a standard video timing output and the scaler output can t get this requirement, user can add a FIFO after video scaler output. The scaler can receive non-continuing line video input, but output may be also non-continuing line video. (3) Parameter configure For input and output clock frequency, user must set with FPGA clock real value. Input horizontal total pixel number is a real value when input video line is continuing. But when input video line is non-continuing, user can give total pixel number of one line with current input video average line pixel number of total lines. (4) Lanczos coefficients User can use standard or your own defined lanczos filter coeffcients for scaling. Because the scaler lanczos mode interpolation uses 4x4 taps, the precision of interpolation position is 1/256, so the total number of filter coefficients stages is 256x4. We use poly phase method to calculate scaling. So the lanczos confidents files structure is as following table: Table 3-2 description of lanczos coefficients Line number Coeff3 Coeff2 Coeff1 Coeff0 Note 0 9 h00 9 hff 9 h00 9 h00 Coeff3 is interpolated pixel left pixel coefficient Coeff2 is interpolated pixel coefficient h00 9 h01 9 hff 9 h1ff Coeff1 is interpolated pixel right 1 pixel coefficient Coeff0 is interpolated pixel right 2 pixels coefficient Coefficients format is two s complement, 1bit signed + 8 bits The sum of each 4 same phase filter coefficients must be normalize to 1. So the normalization of filter coefficient can be referenced as following Matlab code. 15

16 Figure 3-5 Matlab reference for the generating lanczos coefficients 16

17 4 Resource usage and performance analysis Resource usage and performance of the video 2d scaler IP on Primace is shown in following Table 4-1,Table 4-2 and Table 4-3. Table 4-1 resources and performance in M7A12N0F484C7 Input video size Output video size Scaling method 720x x720 (480P) (720P) x 720x x720 (480P) (720P) x 720x x720 Lanczos (480P) (720P) x lanczos 1280x x1080 (720P) (1080P) x 1280x x1080 (720P) (1080P) x 1280x x1080 Lanczos (720P) (1080P) x lanczos 1280x x480 (720P) (480P) x 1280x x480 (720P) (480P) x 1280x x480 Lanczos (720P) (480P) x lanczos 1920x x720 (1080P) (720P) x 1920x x720 (1080P) (720P) x 1920x x720 Lanczos (1080P) (720P) x Pixel width Coefficients Register LUT4s EMB (5K) MAC (12x9) Fmax (Mhz)

18 lanczos Table 4-2 resources and performance in M5C06N0F256C7 Input video size Output video size Scaling method 720x x720 (480P) (720P) x 720x x720 (480P) (720P) x 720x x720 Lanczos (480P) (720P) x 1280x x1080 (720P) (1080P) x 1280x x1080 (720P) (1080P) x 1280x x1080 Lanczos (720P) (1080P) x 1280x x480 (720P) (480P) x 1280x x480 (720P) (480P) x 1280x x480 Lanczos (720P) (480P) x 1920x x720 (1080P) (720P) x 1920x x720 (1080P) (720P) x 1920x x720 Lanczos (1080P) (720P) x Pixel width Coefficients Register LUT4s EMB (5K) MAC (12x9) Fmax (Mhz) Table 4-3 resources and performance in HR03PN0C192C7 18

19 Input Output Scaling Pixel Coefficients Register LUT4s EMB Fmax video size video size method width (5K) (Mhz) 720x x (480P) (720P) x 1280x x (720P) (480P) x 19

20 5 Generate File Directory Structure The video_2d_scaler IP wizard generated file includes: source files (src), simulation files(sim) and example design files. The detailed design directory structure is as below. Project src outputs ip_core system_top.v (define by user) Video_2d_scaler_v1 src sim doc example Cme_ip_video_2d_scaler_ v1.v Data_line_buffer.v Hor_scaler.v Pre_down_scaler.v Scaler_coeff_rom.v Scaler_ctrl.v Ver_scaler.v M7_lib/ M5_lib/ HR3_lib/ src_vp *.vp (Protected RTL) m7 Color_bar_tb_m7.v Color_bar_tb_m7.do Color_bar_tb_m7.f m5 sim_src *.v ( simulation RTL) Color_bar_tb_m5.v Color_bar_tb_m5.do Color_bar_tb_m5.f CME_video_2d_scaler_ user_guide_en01.pdf CME_video_2d_scaler_example _user_guide_en01.pdf m7 M7_example.zip hr3 hr3_example.zip sim_src *.v ( simulation RTL) = directory = source RTL code = simulation related files = documentation hr3 Color_bar_tb_hr3.v Color_bar_tb_hr3.do Color_bar_tb_hr3.f sim_src *.v ( simulation RTL) Figure 5-1 IP wizard generated file directory structure 20

21 Table 5-1 Generated File Directory structure Directory src\ Ip_core\ \video_2d_scaler_v1 \doc\cme_video_2d_scaler_user_guide_en01.doc Description Directory for project source code, including IP wizard generate code. The directory specially for all IPs Directory for video_2d_scaler User guide for video_2d_scaler \src IP Design RTL \cme_ip_video_2d_scaler_v1.v The top of video_2d_scaler IP (Encrypted) \Data_line_buffer.v The data line buffer of video_2d_scaler (Encrypted) \hor_scaler.v The horizontal scaling (Encrypted) \pre_down_scaler.v The pre-down scaling(encrypted) \scaler_coeff_rom.v The coefficients rom of scaling (Encrypted) \scaler_ctrl.v The scaler controller (Encrypted) \ver_scaler.v The vertical scaling (Encrypted) \m7_lib\ Scaling sub-modules library for M7 \m5_lib\ Scaling sub-modules library for M5 \hr3_lib\ Scaling sub-modules library for HR3 \sim Design functional verification \src_vp Encrypted source code for modelsim simulation \m7 Simulation files directory for M7 device Color_bar_tb_m7.v Simulation top file Color_bar_tb_m7.f File list of simulation related file Color_bar_tb_m7.do For modelsim simulation tcl file \sim_src Files for simulation \m5 Simulation files directory for M5 device Color_bar_tb_m5.v Simulation top file Color_bar_tb_m5.f File list of simulation related file Color_bar_tb_m5.do For modelsim simulation tcl file \sim_src Files for simulation \hr3 Simulation files directory for HR 21

22 Color_bar_tb_hr3.v Color_bar_tb_hr3.f Color_bar_tb_hr3.do \sim_src device Simulation top file File list of simulation related file For modelsim simulation tcl file Files for simulation \example \m7 \m7_example.zip \hr3 \hr3_example.zip Video 2d scaler primace project Video 2d scaler primace project CME_video_2d_scaler_example_guide_EN01.pdf User guide of example design 22

23 Revision History Revision Date Author Comments IP Group create first version 23