VHDL test bench for digital image processing systems using a new image format

VHDL test bench for digital image processing systems using a new image format A. Zuloaga, J. L. Martín, U. Bidarte, J. A. Ezquerra Department of Electronics and Telecommunications, University of the Basque Country Key words: Abstract: VHDL, test bench, image processing, The design of complex electronic circuits such as image processing circuits require new specific simulation and modelling tools, in order to reduce the development time. To simulate image-processing models described in VHDL an application specific test bench is needed. In this work a VHDL test bench was designed specifically for image-processing applications. It was necessary to define a new image file format with special characteristics to be used with VHDL and to be configurable to use in image applications with different specifications. 1. INTRODUCTION At the present time, many designers are carrying out several research projects whose objective is the development of specific circuits for the static and dynamic image processing. These developments are realised in devices such as FPGAs or ASIC, usually by mean of standard hardware description languages such as VHDL [1]. VHDL allows the use a single language throughout all the process of design. Nevertheless, only this is not enough to do a successfully project. Complex algorithms used to process images and image sequences make necessary to do simulations of his operation to verify the fulfilment of the specifications under which it has been designed [2][3][4]. To test systems that process images it is necessary to feed the model with a complete set of signals and data. Current commercial VHDL synthesis and simulation tools are equipped with utilities for create stimulus signals, but they have limited

utility in applications that require a big amount of data such as image processors. One of the most useful advantages of VHDL is its capacity to be used in the design of test benches that generate the required signals to stimulate the system under test. In order to accelerate the design, adjust and test cycle a good test bench must be automated and easy to attach to the design. This is accomplished with a modular and highly flexible test bench [5]. A big number of designs are currently implemented on the newest FPGAs in order to take advantage of their programming flexibility using incircuit programming characteristic [6][7]. FPGA vendors supply their own VHDL development systems, such as Altera s Max Plus II [8] or Cypress Warp3 [9], but many times these systems not support test bench capabilities. To simulate complex systems is necessary other VHDL simulators such as Model Sim from Model Technology [10]. The VHDL can read or write files from disk [1]. With this feature it is possible to design test benches that read the test data from disk, generate stimulus signals to the VHDL test module and write results to disk. Unfortunately, VHDL only is capable to read and write ASCII character files and it is not possible to read images in standard formats such as bitmap directly from disk. In order to save this problem it was defined a new image format to be used with the test bench described here or another new test bench that will be created for image processing applications. This image format will be called hex-image. Image processing applications for real-time usually works in a pipeline form [4][5]. These systems require a constant flow of data at their inputs and generate a constant flow of data at their outputs. The test bench system must be capable to drive those flows of data and process results to be useful for the developers. 2. TEST BENCH SYSTEM Basically, four blocks or subsystems compose the image processing test bench system here proposed. These subsystems are the Visor program, the camera VHDL module, the monitor VHDL module and the device under test described or instantiated in VHDL. The signal flow through the subsystems can be observed at figure 1. In first place, images or image sequence is passed to the Visor program to be converted from bitmap or SIF (Standard Interchange Format) to the hex-image format. Image in this format is used by camera module to generate the signals that stimulate the device under test. Signals generated by the device under test feed the monitor module that generates the hex file to be visualised or interpreted by the Visor program.

Bitmap or SIF VISOR (converter) CAMERA (camera.vhd) DEVICE UNDER TEST MONITOR (monitor.vhd) VISOR (viewer) AAF02 3B7E2 3F234 12E3B 12E3A 086D5 430B4 A250C Bitmap or SIF Figure 1. Block diagram of the image processing test bench. Information produced by image processing systems may be diverse, but Visor program was designed for two types of image like information: image sequences in strict sense and optical flow vector field sequences. The optical flow is the vector field that represents the apparent motion of brightness pattern in an image sequence [11]. As any vector field, in each point of the space there is a vector that is represented by two quantities: horizontal and vertical magnitude. Pipeline image-processing systems that generates information of optical flow must generate two streams of data in a image like format to represent the horizontal and vertical magnitudes. Test bench for these applications will be organised as is shown in figure 2. Bitmap or SIF MONITOR (monitor.vhd) VISOR (viewer) VISOR (converter) CAMERA (camera.vhd) DEVICE UNDER TEST (OPTICAL FLOW ESTIMATOR) MONITOR (monitor.vhd) AAF02 3B7E2 3F234 12E3B 6D48B F5D36 12B7E C561B Figure 2. Block diagram of test bench for optical flow estimation. In this architecture it is necessary to define previously a continuous-dataflow format for image transmission. This format is basically a digital version of TV signal. Image is codified in 256 grey levels and as shown in figure 3 is

composed by the following signals: eight-bit grey-level data bus (vdat), clock (clk), horizontal synchronism signal (n_hs), vertical synchronism signal (n_vs) and data valid signal (dv). Figure 3. Signals of the serial digital format for image transmission. 3. HEX-IMAGE FILE The hardware description language VHDL does not allow to process bitmap files or similar types, since it was developed to carry out readings and writings of files with ASCII characters. For that reason, it is necessary to define a special format where only ASCII characters will be used. The most obvious and simple way to represent binary information with ASCII characters is the hex format. Hex characters are quickly and easily converted to binary format by VHDL, although it requires twice more bytes than bitmap to codify the same image. In the hex-image format here proposed, two characters HEX-ASCII represent brightens level of each pixel of the image. It means that can be represented 256 levels of grey per pixel. The brightest level will be represented by FF characters, and darkest by 00 characters. Additionally, it is necessary to codify in the same file the video synchronism signals. The code of a hex-image example file can be observed in figure 4. To simplify the example each image of the sequence has only 16 lines and 16 pixels by line. As was stated above, the grey level of each pixel is represented with two characters in the file. The horizontal synchronism of each line is codified with Carriage Return and Line Feed characters. For UNIX machines it is necessary to consider that the line end is codified with only the Line Feed character. On the other hand, the vertical synchronism of each line is codified with an asterisk. This file contains only information required for signal generation and does not contain information about image dimensions or similar.

Brightness of each pixel is represented by two HEX-ASCII characters. FFFEFDFAF4F07772FFFEFDFAF4F07772 FFFEFDFAF4F0FEFDFAF4F077727772FF F07772FFFEFDFAF4F0FFFEFDFAF47772 FFFEFDFFFEFDFAF4F0FFFDFAF4F04772 F4F0F07772FF7772FFFEFDFAF4F0D774 F6FDFAF4F0F07772FFFEFDFAF4F01772 72F4F07772FFFEFDF4F07772FFFEFD72 FFFEFDFAF4F0F4F0F07772FFFEFDAE72 FFFEFDFAF4F07772FFFEFDFAF4F07772 FFFEFDFAF4F0F4F0F07772FFFEFD2312 F4F0F07772FFFD72FFFEFDFAF4F07772 FFFCFDFAF4F07772FFFEFDFAF4F07772 FFFCFDFAF4F0F4F0F07772FF77FFFD12 E2DEFDFAF4F07772FFFEFDFAF07772EE FFEEFDFAF4F07772FFFEFDFAF4E47772 FFAEAAF93F307071F3FE45F534F02433 * EEEEEDFAF4F07772FFFEFDFAF4F0777A FFEEFDFAF4F0FEFDFAF4F077727772FF F07E72FFFEFDFAF4F0FFFEFDFAF4777A FFF8FDFFF8F8F6D4D0FFFDFAF4F04772 F4F0F07772FF7772FDFEFDFAF4F0D774 F6FDFAF4F0F07772FFFEFDFAF4F0177A 72F4F27772FFFEFDF4F07772FFFEFD72 FFFEFDFAF4F0F4F0FD7772FFFEFDAE73 FF4EFDFAF4F07772FFFEFDFAF4F07772 FFFEFDFAF4F0F4F0F07772FFFEFD2314 F430F07772FFFD72FFFEFDFAF4F07772 FFFCFDFAF4F07772FFFEFDFAF4F07775 FFFCFDFAF4F0F4F0F07772FF77FFFD12 E23EFDFAF4F07772FFFEFDFAF07772E2 FFEEFDFAF4F07772FFFEFDFAF4E47772 FFAEAAF93F307071F3FE45F534F02432 * 72F4F27772FFFEFDF4F07772FFFEFD72 72F4F27772FFFEFDF4F07772FFFEFD72 FFF8FDFFF8F8F6D4D0FFFDFAF4F04772 F4F0F07772FF7772FDFEFDFAF4F0D774 F6FDFAF4F0F07772FFFEFDFAF4F0177A Line end means horizontal sync It is composed by two characters: carriage return and line feed An asterisk means vertical sync A block between two asterisks is an image frame of the sequence Figure 4. Example of a hex-image file. For optical flow generator systems, two hex-image files are generated: one with horizontal magnitudes and one with vertical magnitudes. But the magnitude is a signed number and it can not be represented in a satisfactory form in image like files if it is not transformed. Adopted transformation is a shift by 128. Then, zero magnitude will be represented by the hex number 80, biggest vector (127) to the right or top will be represented by FF and biggest vector to left or bottom by 01. 4. CONVERTING AND VIEWING IMAGE FILES To work with the hex-image format it was necessary to develop a converter-viewer program to be used in standard windows-based computers. That program was called Visor and has been developed with Microsoft Visual BASIC language.

Objectives of Visor program are: convert files from bitmap or SIF format to hex-image format, visualise hex-image, bitmap or SIF format files and calculate some parameters of images. In figure 5 a sample view of the Visor program is shown. This program works only with grey level images. Figure 5. Visor program showing a bitmap file and hex-image file. Standard Interchange Format (SIF) is a format for exchanging video images of 240 lines by with 352 pixels for NTSC (60 fields/s), and 288 lines by 352 pixels for PAL and SECAM (50 fields/s). Each frame of the image sequence is stored in a single file. In this format the grey level and colour information is included but the Visor program reads only the grey level of the file. The hex-image format can include many images in the same file to be used as an image sequence. The visor program can package many images from bitmap or SIF files in a single hex-image file. There are two ways to do this, using many times the same figure as a static image flowing from a camera or many different images as a real camera. As a viewer, the Visor program can display the bitmap, SIF and heximage files. Last one can be viewed in normal form or as a vector field. Additionally, the Visor program has some general utilities to calculate the average grey level, the variance and the histogram graph of the images. All these options make easy the visual verification of results obtained from simulation of VHDL described systems under test.

5. VHDL TEST BENCH MODULES The VHDL part of the test bench is made of basically by two modules: the camera module and the monitor module. Camera module simulates a black and white video digital camera. The input images are codified in a hex-image format. As was mentioned above, the camera generates the digital signals usually required for video digital processing: video data, horizontal synchronism, vertical synchronism, data validation and data sampling signal. Figure 6 shows the VHDL code of the camera.vhd file. The input data is stored in a file named inputvideo.hex that can be changed passing it as a generic parameter. Also the name and location of this file, as well as the clock period and the horizontal and vertical synchronism lengths can be easily modifiable because they are defined as generic parameters. The generic ths represents the number of clock cycles that lasts the horizontal synchronism, tvs the number of clock cycles for the vertical synchronism and tlin the number of clock cycles that the horizontal synchronism is in high state during the vertical synchronism. The camera reads the file and generates the five digital signals named above. In the time diagram shown in figure 3 the relationship between all these signals and constants can be viewed. The camera has been designed with two processes. The first one generates the clock signal. The second one reads the input file and generates the digital signals. Every clock falling edge, the end of file is checked. If the end is not yet arrived, the first character is read. If the first character is a Carriage Return, the second one (a Line Feed) is read, and the horizontal synchronism counter (cnt1) is initialised to ths. If an asterisk is detected, the vertical synchronism counter (cnt2) is initialised to tvs and the horizontal synchronism counter for the vertical synchronism periods (cnt3) is initialised to tlin. If no synchronism is detected, a new data character is read. These two ASCII characters are converted first to hexadecimal values and then to std_logic_vector (7 downto 0). The data valid (dv) signal is high when data characters are detected and low in any other case. The monitor module simulates a video digital monitor. The images arrive to the monitor in the digital format generated by the camera and this video sequence is storage in the specific hexadecimal format used by the testbench.

ENTITY camera IS GENERIC( CONSTANT filec: IN STRING := "videoinput.hex" ; -- Hex-video file name CONSTANT tclk: IN TIME := 100 ns ; -- Clock speed CONSTANT ths: IN INTEGER := 3 ; -- Horizontal sync time in clock cycles CONSTANT tvs: IN INTEGER := 29 ; -- Vertical sync time in clock cycles CONSTANT tlin: IN INTEGER := 8 ); -- Line time vert. sync generation in clock cycles PORT( SIGNAL vdat: OUT STD_LOGIC_VECTOR (7 DOWNTO 0) ; -- Data SIGNAL clk: OUT STD_LOGIC ; -- Clock SIGNAL n_hs: OUT STD_LOGIC ; -- Horizontal sync SIGNAL n_vs: OUT STD_LOGIC ; -- Vertical sync SIGNAL dv: OUT STD_LOGIC ); -- Data valid END camera; ------------------------------------------------------------------------------- ARCHITECTURE behav OF camera IS SIGNAL clkx: STD_LOGIC := '0'; FILE image: hex_image_file IS IN filec; -- File open ---- Clock generator ------------------------------------------------------------------------- pclk: PROCESS WAIT FOR tclk; clkx <= NOT(clkx); clk <= clkx; END PROCESS; ---- Camera simulation ----------------------------------------------------------------------- pcamera: PROCESS (clkx) VARIABLE cnt1: INTEGER := 0 ; -- horizontal sync counter VARIABLE cnt2: INTEGER := 0 ; -- vertical sync counter VARIABLE cnt3: INTEGER := 0 ; -- horizontal sync counter inside vertical sync M IF clkx = '0' THEN IF cnt1=0 AND cnt2=0 THEN -- Data process time, no vertical or horizontal syncs IF not (ENDFILE (image)) THEN READ (image,chr1); -- First character is read IF (chr1 = CR) THEN -- if CR-LF detected, horizontal sync must be generated READ (image,chr2); -- Second carachter is read. It must be Line Feed vdat <= (OTHERS => '0'); -- Data is cleared cnt1 := ths; -- Horizontal sync time n_hs <= '0'; -- Horizontal sync n_vs <= '1'; -- Vertical sync dv <= '0'; -- Data no valid ELSIF chr1 = '*' THEN -- if '*' detected, vertical sync must be generated vdat <= (OTHERS => '0'); -- Data is cleared cnt2 := tvs; -- Vertical sync time cnt3 := tlin; -- Line time for vertical sync n_vs <= '0'; -- Vertical sync dv <= '0'; -- Data no valid ELSE -- Hex data chr2hex(chr1,hex_nib); -- First character is converted to STD_LOGIC hex2std(hex_nib,idat(7 DOWNTO 4)); READ (image,chr2); -- Second carachter is read and converted to STD_LOGIC chr2hex(chr2,hex_nib); hex2std(hex_nib,idat(3 DOWNTO 0)); vdat <= idat; -- Data out n_hs <= '1'; -- No horizontal sync n_vs <= '1'; -- No vertical sync dv <= '1'; -- Data valid ELSE -- If end of file is reached ASSERT (false) REPORT "File end reached..."; -- Console message n_hs <= '1'; -- No horizontal sync n_vs <= '0'; -- Vertical sync dv <= '0'; -- Data no valid IF cnt1>0 THEN -- Horizontal sync generation n_hs <= '0'; n_vs <= '1'; cnt1 := cnt1-1; -- Clock count for horizontal sync time ELSE n_hs <= '1'; IF cnt2>0 THEN n_vs <= '0'; cnt2 := cnt2-1; cnt3 := cnt3-1; ELSE n_vs <= '1'; IF cnt3=0 AND cnt2>0 THEN n_hs <= '0'; cnt1 := ths-1; cnt3 := tlin+ths; END PROCESS; END behav; -- Vertical sync generation -- Clock count for vertical sync time -- Clock count for horizontal sync time inside vertical sync -- Horizontal sync generation inside vertical sync Figure 6. VHDL code for camera module.

As it can be seen in the figure 7, three processes compose the monitor module. The first process converts to ASCII and writes the eight bits of the signal in the output file. It is activated by the rising edge of the clock. The second process writes the characters Carriage Return and Line Feed every horizontal synchronism falling edge if the vertical synchronism signal is high. The third one writes an asterisk every vertical synchronism falling edge. ENTITY monitor IS GENERIC( CONSTANT file_name: IN STRING := "videooutput.hex" ) ; -- Target Hex-video file name PORT( SIGNAL vdat: IN STD_LOGIC_VECTOR (7 DOWNTO 0) ; -- Data input SIGNAL clk: IN STD_LOGIC := '0' ; -- Clock input SIGNAL n_hs: IN STD_LOGIC := '1' ; -- Horizontal sync input SIGNAL n_vs: IN STD_LOGIC := '1' ; -- Vertical sync input SIGNAL dv: IN STD_LOGIC := '0') ; -- Data valid input signal END monitor; ------------------------------------------------------------------------------- ARCHITECTURE behav OF monitor IS FILE imagen: hex_image_file IS OUT filem; -- File open pm1: PROCESS (clk) ---- Data valid writing process VARIABLE nib: STD_LOGIC_VECTOR (3 DOWNTO 0); VARIABLE hex_nib: hex; VARIABLE chr: CHARACTER; IF clk='1' THEN -- Signals are sampled at rising edge IF dv = '1' THEN -- if the data is valid nib := vdat(7 DOWNTO 4); -- data is converted to hex and written into file std2hex(nib,hex_nib); hex2chr(hex_nib,chr); WRITE(file_name,chr); std2hex(vdat(3 DOWNTO 0),hex_nib); hex2chr(hex_nib,chr); WRITE(file_name,chr); END PROCESS; pm2: PROCESS (n_hs) ---- Horizontal sync pulse IF (n_hs'event AND n_hs='0' AND n_vs='1') THEN -- CR-LF are written into file WRITE(file_name,CR); WRITE(file_name,LF); END PROCESS; pm3: PROCESS (n_vs) ---- Vertical sync pulse IF (n_vs'event AND n_vs='0') THEN -- an asterisc is written into file WRITE(file_name,'*'); END PROCESS; END; Figure 7. VHDL code for camera module. 6. CONCLUSIONS AND FUTURE WORK The simulation of VHDL model provides the capability of verifying its functional validity before it will be manufactured, however the test bench components and the Device Under Test (DUT) must be designed carefully to obtain reliable results. Test bench here proposed has been probed extensively using the simulator Model Sim. It has been used to develop several image-processing

systems such as image enhancers, optical flow estimators, and morphological processors. VHDL provides a large number of ways to write test benches, but unfortunately, some synthesisers can not support all possibilities of the language. Therefore, it is necessary to use simulator tools to validate the system under development and after that to proceed to synthesise with the adequate tool. The hex-image file here described was used only with grey level images and vector files, but the standard can be extended to be used with colour images RGB or YUV. Future developments are aimed to use this format with minor changes. 7. ACKNOWLEDGEMENTS This work has been carried out by the Electronic Design Group of the University of the Basque Country and has been supported by the Education, Universities and Research Department of the Basque Government in the framework of PI 96/91 and PI-1998-41 research projects. 8. REFERENCES [1] IEEE Standard VHDL Language Reference Manual, Std 1076-1993, IEEE, NY, 1993. [2] S. Mueller, Validate image-processing device models with a VHDL test environment, Electronic design, October 1997. [3] A. Zuloaga, U. Bidarte, J. L. Martin, J. Ezquerra, "Optical flow estimator using VHDL for implementation in FPGA," Proceedings XIII design of circuits and systems conference, pp. 36-41, November 1998. [4] A. Zuloaga, J. L. Martin, J. Ezquerra, "Hardware architecture for optical flow estimation in real time," Proceedings of 1998 IEEE International Conference on Image Processing, vol. 3, pp. 972-976, October 1998. [5] L. Entrena, J. Espejo, E. Olías, Building testbenches under specification changes, Proceedings of 1997 VHDL user s forum in Europe, pp 73-82, April 1997. [6] P. Merino, J. López, M. Jacome, Designing dynamically reconfigurable systems: a high level approach, Proceedings XIII design of circuits and systems conference, pp. 458-463, November 1998. [7] T. Ramirez, E. de la Torre, Y. Torroja, J. Uceda, A virtual prototype of an FPGA-based rapid prototyping system, Proceedings XIII design of circuits and systems conference, pp. 102-107, November 1998. [8] Altera Max Plus II VHDL, Altera Corporation. 1994. [9] Cypress Data Book CD-ROM, Cypress Semiconductor. 1998. [10] Model Sim PE/PLUS User s Manual. Model technology, 1997. [11] B. Horn, Robot vision, Mc Graw - Hill. Massachusetts, 1986.