Motion-JPEG2000 Stream Scaling for Multi-Resolution Video Transmission

Similar documents
Comparative Study of JPEG2000 and H.264/AVC FRExt I Frame Coding on High-Definition Video Sequences

INF5080 Multimedia Coding and Transmission Vårsemester 2005, Ifi, UiO. Wavelet Coding & JPEG Wolfgang Leister.

JPEG2000: An Introduction Part II

OBJECT-BASED IMAGE COMPRESSION WITH SIMULTANEOUS SPATIAL AND SNR SCALABILITY SUPPORT FOR MULTICASTING OVER HETEROGENEOUS NETWORKS

Intra-frame JPEG-2000 vs. Inter-frame Compression Comparison: The benefits and trade-offs for very high quality, high resolution sequences

Research Topic. Error Concealment Techniques in H.264/AVC for Wireless Video Transmission in Mobile Networks

COMP 249 Advanced Distributed Systems Multimedia Networking. Video Compression Standards

INTRA-FRAME WAVELET VIDEO CODING

Error Resilient Video Coding Using Unequally Protected Key Pictures

CODING EFFICIENCY IMPROVEMENT FOR SVC BROADCAST IN THE CONTEXT OF THE EMERGING DVB STANDARDIZATION

Optimal Interleaving for Robust Wireless JPEG 2000 Images and Video Transmission

Fast MBAFF/PAFF Motion Estimation and Mode Decision Scheme for H.264

MULTI-STATE VIDEO CODING WITH SIDE INFORMATION. Sila Ekmekci Flierl, Thomas Sikora

Module 8 VIDEO CODING STANDARDS. Version 2 ECE IIT, Kharagpur

Motion Video Compression

Lecture 2 Video Formation and Representation

PAPER Parameter Embedding in Motion-JPEG2000 through ROI for Variable-Coefficient Invertible Deinterlacing

Comparative Analysis of Wavelet Transform and Wavelet Packet Transform for Image Compression at Decomposition Level 2

Project Proposal: Sub pixel motion estimation for side information generation in Wyner- Ziv decoder.

Ch. 1: Audio/Image/Video Fundamentals Multimedia Systems. School of Electrical Engineering and Computer Science Oregon State University

Modeling and Optimization of a Systematic Lossy Error Protection System based on H.264/AVC Redundant Slices

Video Over Mobile Networks

Content storage architectures

An Overview of Video Coding Algorithms

Bit Rate Control for Video Transmission Over Wireless Networks

Error concealment techniques in H.264 video transmission over wireless networks

Applications of Digital Image Processing XXIV, Andrew G. Tescher, Editor, Proceedings of SPIE Vol (2001) 2001 SPIE X/01/$15.

Contents. xv xxi xxiii xxiv. 1 Introduction 1 References 4

Constant Bit Rate for Video Streaming Over Packet Switching Networks

Lecture 1: Introduction & Image and Video Coding Techniques (I)

Schemes for Wireless JPEG2000

Performance evaluation of Motion-JPEG2000 in comparison with H.264/AVC operated in pure intra coding mode

Performance Comparison of JPEG2000 and H.264/AVC High Profile Intra Frame Coding on HD Video Sequences

A Novel Approach towards Video Compression for Mobile Internet using Transform Domain Technique

OL_H264e HDTV H.264/AVC Baseline Video Encoder Rev 1.0. General Description. Applications. Features

Copyright 2005 IEEE. Reprinted from IEEE Transactions on Circuits and Systems for Video Technology, 2005; 15 (6):

Systematic Lossy Error Protection of Video based on H.264/AVC Redundant Slices

OL_H264MCLD Multi-Channel HDTV H.264/AVC Limited Baseline Video Decoder V1.0. General Description. Applications. Features

Reduced complexity MPEG2 video post-processing for HD display

Study of AVS China Part 7 for Mobile Applications. By Jay Mehta EE 5359 Multimedia Processing Spring 2010

Joint Optimization of Source-Channel Video Coding Using the H.264/AVC encoder and FEC Codes. Digital Signal and Image Processing Lab

EMBEDDED ZEROTREE WAVELET CODING WITH JOINT HUFFMAN AND ARITHMETIC CODING

The H.26L Video Coding Project

Chapter 2 Video Coding Standards and Video Formats

Selective Intra Prediction Mode Decision for H.264/AVC Encoders

Video compression principles. Color Space Conversion. Sub-sampling of Chrominance Information. Video: moving pictures and the terms frame and

Multimedia Communications. Image and Video compression

A video signal consists of a time sequence of images. Typical frame rates are 24, 25, 30, 50 and 60 images per seconds.

Adaptive Key Frame Selection for Efficient Video Coding

ATSC Standard: Video Watermark Emission (A/335)

Skip Length and Inter-Starvation Distance as a Combined Metric to Assess the Quality of Transmitted Video

Chapter 2 Introduction to

PAL uncompressed. 768x576 pixels per frame. 31 MB per second 1.85 GB per minute. x 3 bytes per pixel (24 bit colour) x 25 frames per second

ELEC 691X/498X Broadcast Signal Transmission Fall 2015

Embedding Multilevel Image Encryption in the LAR Codec

HEVC: Future Video Encoding Landscape

University of Bristol - Explore Bristol Research. Peer reviewed version. Link to published version (if available): /ISCAS.2005.

INTERNATIONAL JOURNAL OF ELECTRONICS AND COMMUNICATION ENGINEERING & TECHNOLOGY (IJECET)

COMPRESSION OF DICOM IMAGES BASED ON WAVELETS AND SPIHT FOR TELEMEDICINE APPLICATIONS

Robust Transmission of H.264/AVC Video using 64-QAM and unequal error protection

Multimedia Systems. Part 13. Mahdi Vasighi

Video Codec Requirements and Evaluation Methodology

The Development of a Synthetic Colour Test Image for Subjective and Objective Quality Assessment of Digital Codecs

MPEG-2. ISO/IEC (or ITU-T H.262)

Video Transmission. Thomas Wiegand: Digital Image Communication Video Transmission 1. Transmission of Hybrid Coded Video. Channel Encoder.

Colour Reproduction Performance of JPEG and JPEG2000 Codecs

Overview: Video Coding Standards

Digital Cinema Delivery using Frequency Multiplexed DVB T Signals

Compressed-Sensing-Enabled Video Streaming for Wireless Multimedia Sensor Networks Abstract:

Impact of scan conversion methods on the performance of scalable. video coding. E. Dubois, N. Baaziz and M. Matta. INRS-Telecommunications

OPEN STANDARD GIGABIT ETHERNET LOW LATENCY VIDEO DISTRIBUTION ARCHITECTURE

Motion Re-estimation for MPEG-2 to MPEG-4 Simple Profile Transcoding. Abstract. I. Introduction

1. INTRODUCTION. Index Terms Video Transcoding, Video Streaming, Frame skipping, Interpolation frame, Decoder, Encoder.

Understanding Compression Technologies for HD and Megapixel Surveillance

The H.263+ Video Coding Standard: Complexity and Performance

Real Time PQoS Enhancement of IP Multimedia Services Over Fading and Noisy DVB-T Channel

Chapter 10 Basic Video Compression Techniques

MPEGTool: An X Window Based MPEG Encoder and Statistics Tool 1

STUDY OF AVS CHINA PART 7 JIBEN PROFILE FOR MOBILE APPLICATIONS

Performance Evaluation of Error Resilience Techniques in H.264/AVC Standard

AN IMPROVED ERROR CONCEALMENT STRATEGY DRIVEN BY SCENE MOTION PROPERTIES FOR H.264/AVC DECODERS

Line-Adaptive Color Transforms for Lossless Frame Memory Compression

Feasibility Study of Stochastic Streaming with 4K UHD Video Traces

Improved H.264 /AVC video broadcast /multicast

Speeding up Dirac s Entropy Coder

Audio and Video II. Video signal +Color systems Motion estimation Video compression standards +H.261 +MPEG-1, MPEG-2, MPEG-4, MPEG- 7, and MPEG-21

Systematic Lossy Forward Error Protection for Error-Resilient Digital Video Broadcasting

ABSTRACT ERROR CONCEALMENT TECHNIQUES IN H.264/AVC, FOR VIDEO TRANSMISSION OVER WIRELESS NETWORK. Vineeth Shetty Kolkeri, M.S.

FAST SPATIAL AND TEMPORAL CORRELATION-BASED REFERENCE PICTURE SELECTION

ATSC Candidate Standard: Video Watermark Emission (A/335)

Visual Communications and Image Processing 2002, C.-C. Jay Kuo, Editor, Proceedings of SPIE Vol (2002) 2002 SPIE X/02/$15.

HEVC/H.265 CODEC SYSTEM AND TRANSMISSION EXPERIMENTS AIMED AT 8K BROADCASTING

PERCEPTUAL QUALITY OF H.264/AVC DEBLOCKING FILTER

A parallel HEVC encoder scheme based on Multi-core platform Shu Jun1,2,3,a, Hu Dong1,2,3,b

Express Letters. A Novel Four-Step Search Algorithm for Fast Block Motion Estimation

Color Image Compression Using Colorization Based On Coding Technique

Analysis of Packet Loss for Compressed Video: Does Burst-Length Matter?

Module 8 VIDEO CODING STANDARDS. Version 2 ECE IIT, Kharagpur

Advanced Computer Networks

How Does H.264 Work? SALIENT SYSTEMS WHITE PAPER. Understanding video compression with a focus on H.264

Transcription:

Motion-JPEG2 Stream Scaling for Multi-Resolution Video Transmission Rong-Yu Qiao CSIRO ICT Centre PO Box 76, Epping NSW 171 Australia +61-2-9372-4491 Rong-Yu.Qiao@csiro.au Michael H. Lee CSIRO ICT Centre PO Box 76, Epping NSW 171 Australia +61-2-9372-4135 Michael.Lee@csiro.au Keith Bengston CSIRO ICT Centre PO Box 76, Epping NSW 171 Australia +61-2-9372-4424 Keith.Bengston@csiro.au Abstract In this paper, we present a real-time video system for the transmission of Motion-JPEG2 codestreams with multiple resolutions. To achieve this multi-resolution video in real-time, a software tool, named stream scaler, was newly developed. It directly looks at each packet s location in a codestream before deciding whether it is accepted as a new packet for a scaled codestream. The SOP marker is used for identifying the packet. The stream scaler can control the resolution levels and quality layers of Motion- JPEG2 codestreams to be adapted to end-user device requirements and to different bandwidth capacities. The computational complexity of the stream scaler was measured to be very low and suits real-time processing. We have demonstrated a Motion-JPEG2 video system with the stream scaler, which delivers videos with three different resolutions: HD, SD and CIF, in real-time. Keywords Broadband multimedia, Motion-JPEG2, scalable video delivery, HD video codec, real-time processing. INTRODUCTION The recent development of video processing technologies and the wide availability of network access enable the many powerful applications of real-time video. The Motion-JPEG2 standard [1] is offering a number of compelling features. It is an extension of JPEG2 [2] for video processing. As Motion-JPEG2 is an intraframebased coding technique, its computational complexity could be lower than those of other video coding standards, such as MPEG-2 and H.264, which use motion compensation techniques to remove interframe redundancies. Some studies have analytically proven that Motion- JPEG2 has a great potential for video processing [3],[4]. It was further reported that Motion-JPEG2 is superior to MPEG-2 at higher bitrates than 7 Mbps [5]. The Motion-JPEG2 codec could be a primary solution to delivery applications of HD (High Definition) or greater video signals, and there have been a number of successful results on the hardware implementation of HD video based on Motion-JPEG2 [6]-[8]. Digital Cinema Initiatives have selected Motion-JPEG2 as the core compression method for their specification in Digital Cinema applications which require a very high picture resolution of up to 496x3112 pixels [9],[1]. In JPEG2, there are a number of useful features, which include high compression performance, good perceptual quality, error resilience, lossless compression, and regionof-interest (ROI) coding. Among these, abilities to scale resolution and quality without recompression are probably the most attractive features in JPEG2 [11]. These features offer a great flexibility to manage compressed picture data. This paper presents a real-time video system for the transmission of Motion-JPEG2 codestreams with multiple resolutions. To achieve this multi-resolution video in real-time, a software tool (called stream scaler) was newly developed. It controls the scalability levels and layers of the codestreams to be adapted to requirements of the end-user devices with different resolutions and further to different bandwidth capacities of the network. The proposed tool is useful for the multicast applications of Motion-JPEG2 video. The rest of the paper is organized as follows. It first briefly introduces the JPEG2/Motion-JPEG2 codestream, and provides a related work about the scalability and realtime streaming of Motion-JPEG2 video. In the next section, it presents the design of the stream scaler including the algorithm of the proposed SOP-based stream scaler. It then describes experimental results on stream scaling of Motion-JPEG2. Finally, conclusions are given in the last section. BACKGROUND JPEG2/Motion-JPEG2 Codestream There are a number of excellent review papers which describe the detailed structure of the JPEG2 (JP2) codestream [12]-[15]. In this section, we briefly introduce the JP2 codestream structure and its relationship with a Motion-JPEG2 (MJ2 or MJP2) file or stream. A JPEG2 codestream is structured as a main header followed by a sequence of tile-streams of which each

consists of a tile-part header and a tile-part body. The codestream is terminated by a two byte marker, EOC (End of codestream). The main header contains information on a number of different properties of the signal, including the SOC (Start of codestream) marker, the image and tile size, the region of interest, and the quantization component. Parameters specified in marker segments of the main header serve as defaults for the entire codestream. Each tile-part header begins with the SOT (Start of tilepart) marker segment, and ends with the SOD (Start of data) marker. Optional marker segments can appear between SOT and SOD marker segments. Figure 1 shows a JP2 codestream structure. The Motion-JPEG2 standard [1] focuses on the file format and it does not specify the transmission format for the network. Figure 2 depicts the file structure of Motion- JPEG2 with a number of JP2 codestreams. To encapsulate objects in the file, MJP2 Header includes some boxes, such as the file type compatibility box ftyp and the media data box mdat. Movie Box is given as a presentation meta-data wrapper at the end of the MJ2 file. For real-time streaming, however, we may not need such MJ2 file boxes but only the sequence of raw JP2 codestreams. Figure 2. Motion-JPEG2 file (stream) structure. Figure 2 shows that the packet 1 is the fundamental unit of codestream organization. Each packet can be described as one quality increment for a certain resolution level in a particular spatial location (or precinct). It contains the incremental contributions to that quality layer from all code-blocks within the spatial location [11]. Figure 1. JPEG2 codestream structure. Every marker is two bytes long. The first byte consists of a single xff byte. The second byte denotes the specific marker and can have any value in the ranges x4f to x6f and x9 to x93. A marker segment includes a marker and associated parameters, called marker parameters. In every marker segment, the first two bytes after the marker are an unsigned big-endian integer value that denotes the length in bytes of the marker parameters. Related Work There have been some studies on scalability for Motion- JPEG2 transmission. Qui and Yu [16] used the quality scalability to deal with the network congestion in transmitting Motion-JPEG2 codestreams. The proposed scheme can selectively drop packets to achieve quality adaptation, and offers better video quality scalability when network congestion occurs. Wee and Apostolopoulos [17] presented a secure scalable streaming system based on Motion-JPEG2. In the system, each packet, which contained nine scalable segments with 3 resolutions and 3 quality layers, was encrypted with an encryption technique, such as Data Encryption Standard (DES) or Advanced Encryption Standard (AES). Itakura et al. [18] developed a real-time scalable video system based on Motion- JPEG2. Signaling parameters in RTSP (Real Time 1 In this paper, the term packet is defined in JPEG2 and is differentiated from the network packet.

Streaming Protocol) were extended to define a scalability header for setting the range of each resolution and quality layer in the proposed system. For the real-time streaming of Motion-JPEG2 video, Futemma et al. [19] has specified payload formats based on the Real-time Transport Protocol (RTP). The document [19] is an on-going standard effort submitted to the Internet Engineering Task Force (IETF). There have also been a number of studies on Motion-JPEG2 video transmission over wired [18],[2] and wireless [21],[22] networks. STREAM SCALING OF MOTION-JPEG2 Design of Stream Scaler The design goal of the stream scaler was to control picture resolutions as well as data rates of Motion-JPEG2 video. We have initially designed it for MJ2 video files [23], and later extended it to real-time MJ2 video streaming applications [24]. We have also successfully tested the stream scaler for error-resilient video over the Internet [25]. codestreams from the original without a decompression/recompression process. It copies the parameters of resolution and quality layer, and processes tiles and further code-blocks from input to output MJ2 codestreams. As this parsing procedure looks at all details of the codestream, we call it a deep parsing method in this paper. Figure 4 summarizes the algorithm of the stream scaler. For each tile of the current frame, the coding parameters and contents of the input tile are parsed and copied to the relevant output tiles, and the contents of code-blocks are selectively copied from input to output code-blocks according to resolution level parameters. Figure 3 shows how the designed stream scaler plays a role for the multiple scaled outputs of MJ2 codestreams. The input codestream, coded with certain resolution levels (R) and quality layers (L), is first parsed and then scaled down by selectively dropping packets. Figure 3. Motion-JPEG2 stream scaling for multiresolution video outputs. In this process, the level parameter of spatial scalability determines which wavelet subbands are discarded to achieve a scaled picture resolution. The layer parameter of quality scalability can further reduce overall data rates but picture quality is reduced. In the example of Figure 3, the data rate of the each scaled codestream is approximately a quarter of that of the codestream with one level higher resolution. We wrote a software prototype of the stream scaler using some functions of the Kakadu software package, which is a comprehensive software toolkit for JPEG2-based developments [26]. The stream scaler can create new Figure 4. Stream scaling for frame i. For example, the contents of the high frequency subbands (HL R-1, LH R-1, & HH R-1 ) for resolution (R-1) will be discarded with discard_level=1, where R is the maximum number of the picture resolutions. To test the performance of the stream scaler, a scalable MJ2 file was generated by encoding a video sequence with 72x576 pixels and the RGB colour format at 24bpp. It also contains a total of 9 frames at 25 frames per second. Using the Kakadu MJ2 encoder in the lossy mode with the 9/7-tap irreversible filter kernel, the video sequence was encoded at

the maximum rate of 28.5Mbps with 5 resolution levels and 8 quality layers. The stream scaler, which runs on a PC with 3.GHz Pentium4 CPU, generated 3 down-scaled versions from the input MJ2 file. This initial test showed that the overall speed of the stream scaler was reasonably fast, however, real-time processing could hardly be achieved at higher than resolution 4 (36x288) of the scaled codestream. To further control the data rates of the output codestreams, the quality layer parameter were set to layers=8 for resolution 5, layers=4 for resolution 4 and layers=2 for resolution 3, respectively. The bitrate of each scaled codestream is now about a quarter of that of the codestream with one level higher resolution and is approximately proportional to its spatial resolution. Table 1 shows the test results of the output MJ2 codestreams generated by the stream scaler prototype. Table 1. Test results of the stream scaler prototype with a deep parsing method Output resolution Bitrate Processing time 72x576 28.5Mbps 95msec/frame 36x288 7.4Mbps 47msec/frame 18x144 1.8Mbps 34msec/frame SOP-Based Stream Scaler As we found in the previous section, the stream scaler with a deep parsing method requires some computational power and may not fully satisfy real-time applications of video over the network unless it is built using a dedicated hardware. We have developed a simple technique to minimize the parsing procedure for stream scaling of Motion-JPEG2. This stream scaler directly looks at each packet s location in a codestream before deciding whether it is accepted as a new packet for a scaled codestream. In the developed technique, the SOP (Start of Packet: xff91) marker is used for identifying each packet. The SOP marker segment is located before the packet, and was originally defined as a useful means for error resilient decoding in JPEG2 [11]. As it keeps the packet sequence number i SOP, the presence of subsequent SOP marker segments enables resynchronization at a packet boundary when errors occur. Figure 5 shows an example of packet ordering for CPRL (component-position-resolution-layer progression [2]) in a typical JPEG2 codestream with no tiling and three colour components: Y, Cb, and Cr. The CPRL progression is one of five progression orders defined in JPEG2, of which the others are LRCP (layer-resolution-componentposition), RLCP (resolution-layer-component-position), RPCL (resolution-position-component-layer), and PCRL (position-component-resolution-layer). The CPRL progression is primarily progressive by component. In the case of the YCbCr colour format, all packets from the Y component precede all packets from the Cb component, and all packets from the Cb component precede all packets from the Cr component. For a given tile t, the CPRL progression can be constructed by the groups of packets using four nested loops, as follows: for each c for each p for each j for each k include G t,c,p,j,k The indices used in these loops are: component c, position (or precinct) p, resolution j, and layer k. G t,c,p,j,k denotes a packet from the given tile t. Figure 5. CPRL packet ordering in a JPEG2 codestream with three colour components: Y, Cb, and Cr.

In Figure 5, each packet is represented by the equation packet i ( j, k), where the packet sequence number i, resolution j, and quality layer k are given as (j 1) x L + k 1 : for Y-component i = (R + j 1) x L + k 1 : for Cb-component (2R + j 1) x L + k 1 : for Cr-component j = 1, 2,, R, and k = 1, 2,, L. R and L are the maximum numbers of resolutions and layers, respectively. In the codestream, there are a total of (3R x L) packets, of which R groups contribute to resolutions and L groups contribute to quality layers. To achieve a certain scaled resolution and quality, therefore, we simply discard the packets that belong to higher than targeted values of J and K. For example, the following groups of packets will be discarded to achieve resolution J=3 and quality layer K=4 in a codestream with the maximum numbers of resolutions R=6 and layers L=8: packet i (4,*), packet i (5,*), packet i (6,*) ; packet i (*,5), packet i (*,6), packet i (*,7), packet i (*,8). The simple algorithm of the SOP-based stream scaler is given in Figure 6. Read/write HEADER; i = i scal = ; while (i < 3R x L - 1) { } If (j > J OR k > K) { } else { } i ++; Discard PACKET i (j,k); PACKET iscal (j,k); i scal ++; Write EOC; // J & K: target resolution & layer // new packet sequence number Figure 6. Algorithm of the SOP-based stream scaler. (1) EXPERIMENTAL RESULTS SD Stream Scaling For the real-time transmission test of video over the IP network, the stream scaler was implemented to work with Analog Devices Motion-JPEG2 codec (two ADV22- SD PCI cards) [27], as shown in Figure 7. Scaler1 and Scaler2, which represent the stream scaler with a deep parsing method and the SOP-based stream scaler, respectively, were separately tested for comparison. Two PCs with 3.GHz Pentium4 CPUs were used as the sender and receiver in this test, respectively. Figure 7. Motion-JPEG2 stream scaling test system. Encoding conditions for the test are given as follows: Input video format: SD 72x576 / 5Hz (interlaced) Colour format: YCbCr 4:2:2 Max. bitrate: 27.5Mbps Wavelet filter: (9,7) irreversible Tiling: no (one tile per picture) Code-block size: 64x32 Resolution levels: 5 Quality layers: 8 Figure 8 shows the bitrates of scaled outputs from the stream scaler. Each bitrate was calculated as the average value of 1 frames for each resolution and layer. The bitrate plots are identical for both Scaler1 and Scaler2. These bitrate plots are useful as we can estimate a target rate by controlling the resolutions and layers of the stream scaler. While an output codestream is scaled for the required resolution of the end-user device, its bitrate can be adjusted according to the available bandwidth capacity of the network. As we mentioned in the previous sections, the computational complexity of Scaler1 is relatively high while that of Scaler2 is very low. Figures 9(a) and (b) show the processing speeds of Scaler1 and Scaler2 for the full ranges of resolution and layer, respectively. It is clearly seen that Scaler2 is overall about 1 times faster than Scaler1. In real-time, Scaler1 can process an output codestream with the frame size of up to 36x288 (resolution 4) at layer 3, which is indicated as 19.4msec. per field in Figure 9(a). There should be no problem for the real-time processing of

Scaler2 throughout the full ranges of resolution and layer as shown in Figure 9(b), where the highest processing time is marked as only 4.msec. per field for the codestream with the frame size of 72x576 (resolution 5) at layer 8. layers=5 for 48x27, respectively. The bitrate of each scaled codestream is approximately a quarter of that of the codestream with one level higher resolution. Processing times are relatively higher than those of the SD video test but they are still small enough for real-time processing. bitrate [Mbps] 3 25 2 15 1 resolution1 resolution2 resolution3 resolution4 resolution5 Table 2. Test results of Motion-JPEG2 stream scaling for multi-resolution outputs from HD video Output resolution Bitrate Processing time 192x18 8.2Mbps 6.4msec/field 96x54 21.3Mbps 2.5msec/field 48x27 5.5Mbps 1.3msec/field 5 1 2 3 4 5 6 7 8 layer Figure 8. Bitrates of scaled outputs from the stream scaler (resolution1: 45x36, resolution2: 9x72, resolution3: 18x144, resolution4: 36x288, resolution5: 72x576). The computational complexities of Scaler1 and Scaler2 are further compared in Figure 1. Each plot depicts the processing times of the stream scaler (Scaler1 or Scaler2) for the output codestreams with a total of 5 resolutions at layer 8. HD Stream Scaling Finally, we have built a Motion-JPEG2 HD video system with the stream scaler for multi-resolution videos, as shown in Figure 11. Two Analog Devices ADV22-HD PCI cards were used as the Motion-JPEG2 encoder and decoder for HD video with the resolution of 192x18. The encoding conditions are the same as those of the stream scaler test system for SD (Standard Definition) video in Figure 7, except for the following factors: Input video format: HD 192x18 / 6Hz (interlaced) Bitrate: Variable As the ADV22HD PCI card does not support the display of other than the HD resolution, we wrote a display application based on Windows DirectX. Its configuration is shown in Figure 12. The decoder we developed is fast enough to decode codestreams in real-time with resolutions other than the standard HD. We implemented only Scaler2 in this HD system. Table 2 summarizes the test results of Motion-JPEG2 stream scaling for multi-resolution outputs from HD video. The quality layer parameters were set to layers=8 for the full resolution of 192x18, layers=6 for 96x54, and The scaled output resolutions of 96x54 and 48x27 are horizontally larger than standard video formats, such as SD (72x562) and CIF (Common Intermediate Format: 352x288), but the DirectShow video renderer can flexibly adjust these over-sized videos to such standard format sizes. processing time [msec/field] processing time [msec/field] 4 3 2 1 5 4 3 2 1 resolution1 resolution2 resolution3 resolution4 resolution5 1 2 3 4 5 6 7 8 resolution1 resolution2 resolution3 resolution4 resolution5 layer (a) 1 2 3 4 5 6 7 8 layer (b) Figure 9. Computational complexities: (a) Scaler1, (b) Scaler2.

Figure 13 shows visual results for the decoded videos of the three resolutions: 192x18, 96x54, and 48x27. Figures 13(d) and (e) also show resized video output windows from 96x54 to the SD format and from 48x27 to the CIF format, respectively. processing time [msec/field] 4 3 2 1 Scaler1 Scaler2 45x36 9x72 18x144 36x288 72x576 resolution @ layer8 Figure 1. Comparison of speeds between Scaler1 and Scaler2. Figure 11. Motion-JPEG2 HD video system with the stream scaler for multi-resolution video transmission. Figure 12. DirectShow filter graph configuration for decoding Motion-JPEG2 codestreams. To measure objective qualities of the decoded frames, we use the PSNR (Peak Signal-to-Noise Ratio) defined as 2 PSNR = 1log1 (255 / MSE) [db] (2) where MSE is the mean-square error between the original and reconstructed frames and is given as the following equation: 1 M 1 N 1 2 MSE = { x( i, j) y( i, j)} (3) MN i= j= where x(i,j) and y(i,j) are the original and reconstructed frame pictures, respectively. M and N are horizontal and vertical sizes of the picture. This measure may not be indicative of the actual subjective quality, but the PSNR can be used as a rough indicator of the picture quality. The measured PSNR values were varied as 41.6dB for the resolution of 192x18, 38.39dB for 96x54, and 3.37dB for 48x27, respectively. However, the visual qualities of the three scaled videos were equally very good with few artifacts as shown in Figure 13. In our preliminary tests on the visual quality of Motion-JPEG2 video, it was indicated that no degradation is visible at layers=5 or higher. Some perceptual distortions start to appear from layers=4 and the full frame is degraded at layers=1. CONCLUSIONS This paper presented a software tool, named stream scaler, which can control the resolution levels and quality layers of Motion-JPEG2 codestreams to be adapted to different bandwidth capacities and to end-user device requirements. We first attempted to design the stream scaler based on a deep parsing method which copies the parameters of resolution and quality layer, and processes tiles and further code-blocks from input to output codestreams. Although no decompression/recompression process is involved for creating new output codestreams from the original, the deep parsing method shows some limitation for real-time applications as its computational complexity is relatively high. We further developed a SOP-based technique for the stream scaler, which greatly minimizes the parsing procedure in a codestream. The SOP-based stream scaler runs fast and suits real-time applications. However, this method has a drawback. It relies on the SOP marker to identify each packet in a codestream, and does not work for the codestream without the SOP maker that is optionally given during the encoding process. Nevertheless, the SOP marker is a useful means for error resilient decoding, and is good to be always included in the codestream. It requires a very few additional bits. We have built a Motion-JPEG2 HD video system with the stream scaler for multi-resolution videos, which fully runs in real-time. The system is ready for multi-resolution video applications using various devices, such as HD & SD displays and a wireless device (PDA or mobile phone).

(a) (b) (d) (c) (e) Figure 13. Visual results for the decoded videos of the three resolutions: (a) 192x18: PSNR=41.6dB, (b) 96x54: PSNR=38.39dB, (c) 48x27: PSNR=3.37dB, (d) SD-resized ActiveMovie Window (72x562) of (b), (e) CIF-resized ActiveMovie Window (352x288) of (c).

REFERENCES [1] ITU-T Recommendation T.82 ISO/IEC 15444-3, Information Technology - JPEG2 Image Coding System Part 3: Motion JPEG2, Jan. 25. [2] ITU-T Recommendation T.8 ISO/IEC 15444-1, Information Technology - JPEG2 Image Coding System: Core Coding System, Aug. 22. [3] Yu, W., Qiu, R., and Fritts, J., Advantages of Motion- JPEG2 in Video Processing, Proc. SPIE, Visual Communications and Image Processing, vol.4671, pp.635-645, 22. [4] Marpe, D., George, V., Cycon, H.L., and Barthel, K.U., Performance Evaluation of Motion-JPEG2 in Comparison with H.264/AVC operated in Pure Intracoding Mode, Proc. SPIE, Wavelet Applications in Industrial Processing, vol.5266, pp.129-137, 24. [5] Bucklen, B., JPEG2: The Compression Solution for HDTV Distribution throughout the Home, DisplaySearch HDTV Conference 25, Los Angeles, USA, Aug. 25. [6] NEC, HDTV Real-Time Encoder/Decoder PC Board Conforming to JPEG2, Broadcast Technology 15, Summer 23. [7] Fößel, S., Föttinger, G., and Mohr, J., Motion JPEG2 for High Quality Video Systems, IEEE Trans. Consumer Electronics, vol.49, no.4, pp.787-791, Nov. 23. [8] Yamauchi, H., Okada, S., Taketa, K., Matsuda, Y., Mori, T., Watanabe, T., Matsuo, Y., and Matsushita, Y., 144x18 Pixel, 3 Frames per Second Motion- JPEG 2 Codec for HD-Movie Transmission, IEEE Trans. Solid-State Circuits, vol.4, no.1, pp.331-341, Jan. 25. [9] Barda, J., Press Release of the 33 rd WG1 Redmond Meeting, 24-7-19/23: JPEG2 Gains Market Shares, Goes for Digital Cinema, ISO/IEC JTC 1/SC 29/WG 1 N3329, July 24. [1] Digital Cinema: http://www.dcinema.fraunhofer.de/ [11] Taubman, D.S., and Marcellin, M.W., JPEG2: Image Compression Fundamentals, Standards and Practice, Kluwer Academic Publishers, Boston, 22. [12] Chrysafis, C., Taubman, D., and Drukarev, A., Overview of JPEG 2, Proc. Image Processing Image Quality Image Capture Systems Conference at IS&T/PICS, pp.333-338, April 1999. [13] Marcellin, M.W., Gormish, M.J., Bilgin, A., and Boliek, M.P., An Overview of JPEG2, Proc. IEEE Data Compression Conference, pp.523-541, 2. [14] Christopoulos, C., Skodras, A., and Ebrahimi, T., The JPEG2 Still Image Coding System: An Overview, IEEE Trans. Consumer Electronics, vol.46, no.4, pp.113-1127, Nov. 21. [15] Rabbani, M., and Josh, R., An Overview of the JPEG2 Still Image Compression Standard, Signal Processing: Image Communication Journal, vol.17, pp.3-48, 22. [16] Qiu, R., and Yu, W., An Efficient Quality Scalable Motion-JPEG2 Transmission Scheme, Technical Report WUCS-1-37, Washington University, Department of Computer Science, Nov. 21. [17] Wee, S., and Apostolopoulos, J., Secure Scalable Streaming and Secure Transcoding with JPEG2, Proc. IEEE International Conference on Image Processing, Sep. 23. [18] Itakura, E., Futemma, S., Wang, G., and Yamane, K., JPEG2 Based Real-Time Scalable Video Communication System over the Internet, Proc. IEEE Consumer Communications & Networking Conference, pp.539-543, Jan. 25. [19] Futemma, S., Leung, A., and Itakura, E., RTP Payload Format for JPEG2 Video Streams, draftietf-avt-rtp-jpeg2-1.txt, IETF draft, Feb. 26. [2] Qiu, R., Yu, W., and Fritts, J., Motion-JPEG2 Video Transmission over Active Networks, Proc. Image and Video Communications and Processing Conference at IS&T/SPIE Electronic Imaging 23, Jan. 23. [21] Frescura, F., Giorni, M., Feci, C., and Cacopardi, S., JPEG2 and MJPEG2 Transmission in 82.11 Wireless Local Area Networks, IEEE Trans. Consumer Electronics, vol.49, no.4, pp.861-871, Nov. 23. [22] Guo, Z., Nishikawa, Y., Omaki, R.Y., Onoye, T., and Shirakawa, I., A low-complexity FEC assignment scheme for Motion-JPEG2 over wireless network, IEEE Trans. Consumer Electronics, vol.52, no.1, pp.81-86, Feb. 26. [23] Lee, M.H., Scalability Manager Software for Motion- JPEG2 Codestreams, Technical Report 5/223, CSIRO ICT Centre, Dec. 25. [24] Lee, M.H., and Senanayake, B., Real-time Stream Scaling of Motion-JPEG2 Video, Technical Report 6/19, CSIRO ICT Centre, May 26. [25] Lee, M.H., Qiao, R.-Y., and Bengston, K., Error- Resilient Scalable Video over the Internet, Proc. Australian Telecommunication Networks and Applications Conference, pp.42-424, Dec. 26. [26] Kakadu software: http://www.kakadusoftware.com/ [27] Analog Devices ADV22 - JPEG2 Video Codec: http://www.analog.com/adv22