Minimax Disappointment Video Broadcasting DSP Seminar Spring 2001 Leiming R. Qian and Douglas L. Jones http://www.ifp.uiuc.edu/ lqian
Seminar Outline 1. Motivation and Introduction 2. Background Knowledge 3. Simulation Systems Configuration 4. Results and Conclusions
Motivation 1. Multiple users receiving the same transmission under different channel situations. 2. Different class of users can be mapped to different received SNR values. 3. Need to effectively and meaningfully evaluate the performance of a broadcasting system. 4. Traditional weighted averaging schemes either penalize certain user class or are too ad-hoc.
Minimax Disappointment Criterion 1. Denote the ith user s optimal performance as P i, obtained by applying JSCC to this user alone. 2. Denote each user s actual received performance as p i, p i is usually smaller than P i. 3. J i = P i p i is the ith user s disappointment value, J = max(j i ) is the broadcasting performance. 4. We set out to minimize J, the maximum of all users disappointment value.
Minimax Disappointment Criterion (MD) 1. Fairness: no class of users is penalized more than the others. 2. Individuality: users under different channels do expect different performance. 3. Tractability: system optimization falls in the Minimax optimization framework. 4. MD is not limited to JSCC, it can be a universal broadcasting performance criterion.
Background Knowledge 1. Joint source-channel coding (JSCC) 2. Video source coding a. Video source coders b. Rate Control 3. Channel coders
Joint Source-Channel Coding 1. Information separation theorem is valid under the impractical assumption of point-to-point transmission and allowing infinite delay (long keywords). 2. True JSCC involves the co-design of source and channel codes, we focus on the more general joint source-channel matching. (JSCM) 2. Joint source-channel matching improves performance by optimizing limited system resources (bandwidth, power, etc) between source and channel.
Joint Source-Channel Matching 90 55 10 1 80 50 Distortion 70 60 50 40 30 20 End to End PSNR 45 40 35 30 P3 JSCM P2 P1 Bit Error Probability 10 2 10 3 10 4 10 25 0 0 1 2 3 4 5 6 7 8 Source Rate in bits x 10 5 20 1 2 3 4 5 6 7 8 9 10 Channel SNR 10 5 0.4 0.45 0.5 0.55 0.6 0.65 0.7 0.75 RCPC Channel Rate Fixed-parameter systems (p1, p2, p3) perform poorly under unfavorable channel situations. Adapting to the varying channel situation, JSCC jointly allocates source and channel rates. JSCC always yields the optimal coding parameters, which are located on the convex hull.
Video Source Coders 1. Intra-frame coding (Motion-JPEG, Motion-SPIHT): low efficiency, random-frame accessibility. 2. Inter-frame coding: exploiting frame dependency a. 3D waveform coding (3D-DCT, 3D-SPIHT). b. Motion-Compensated coding (MPEG, H.26x). c. Object-Knowledge based model coding.
RD-Curve: SPIHT 3 100 continuous frames for Football sequence coded using Motion SPIHT 2.5 2 PSNR (db) 1.5 1 0.5 0 5 4.5 4 3.5 3 2.5 2 1.5 1 0.5 0 Source Rate in bit/pixel (log scale)
RD-Curve: Motion-SPIHT 32 R D curve for Football sequence encoded using Motion SPIHT PSNR (db) 30 28 26 24 22 20 18 16 14 0 0.2 0.4 0.6 0.8 1 Source Rate (bit/pixel)
RD-Curve: 3D-SPIHT 35 30 Football sequence initially coded in 1 bit/pixel using 3D SPIHT non truncated truncated original PSNR (db) 25 20 15 10 5 0 0.2 0.4 0.6 0.8 1 Source rate (bit/pixel)
RD-Curve: H.263 35 R D curve for Football sequence encoded using H.263 PSNR (db) 34 33 32 31 30 29 28 27 26 25 0 0.2 0.4 0.6 0.8 1 Source rate (bit/pixel)
Video Transmission Issues Delay Constraint: in video transmission, only a small, finite delay is allowed. Rate Control: finite decoding buffer size and a constant playback rate requirement. Finite-size decoding buffer Buffer Vacancy Next Frame Constant Decode Flow Source Symbols Protection Symbols Current Frame Leftover from previous frame Constant-rate Playback Buffer Occupancy
Channel Coders 1. Reed-Solomon (RS): block codes with good ability to correct channel burst errors, defined by (n,k,t), can correct t errors. 2. Rate-Compatible Punctured Convolutional Codes (RCPC): convolutional codes obtained by puncturing a low-rate code, allowing transmission in incremental redundancy.
Optimization Toward Single User Class 1. Obtain best end-to-end performance within the constraints of limited channel capacity, delay and rate-control requirement, with regard to current user s channel situation. 2. Achieve a certain level of system generality so JSCM can be applied to various standard sourcechannel coder combinations.
General System Diagram Raw Video Frames Scene Change Detection Source Coding Channel Coding To Transmission Source Rate Channel Rate Source Model Rate-Distortion Curve Estimation Joint End-to-End Distortion Optimization Channel Rate-Error Curve Precalculation
Simulation Results (MJPEG-RS) 32 Motion JPEG and RS coder, Football Sequence, GOP size 20 Average PSNR 30 28 26 24 22 20 18 Optimal Protection 16 Little Fixed Protection Medium Fixed Protection Overabundant Fixed Protection 14 2 3 4 5 6 7 8 9 10 Eb/N0
Self-adaptive Suboptimal System Raw Video Frames H.263 Encoder Channel BER Feedback Coded Inter Frame MQUANT Joint Source-Channel Matching Loop, Optimize for every data packet Buffer Occupany Estimator To Transmission Coded Inter Frame Puncture Rate Coded Inter Frame Coded Intra Frame Packetizing and CRC Encoding Rate-Compatible Puncturing (RCPC) Coder H.263 and RCPC Coder
Simulation Results (H.263-RCPC) 55 H.263 and RCPC System 50 45 Rate 8/9 Rate 8/16 Rate 8/24 Optimal 2% Optimal 5% Optimal 0.1% Average Frame PSNR (db) 40 35 30 25 20 1 2 3 4 5 6 7 8 9 10 Channel Situation E /N (db) b 0
MD Broadcast Simulation Parameters 1. Fast-motion football sequence in QCIF format. 2. Five user classes with channel SNRs spread out evenly in the range of 1dB to 9dB. 3. RCPC encoder with 9 available puncture rates. 4. Limited channel capacity of approximately 1 bit/pixel. 5. We tested both the case of limited channel capacity and the case of limited transmission power.
Broadcasting Simulation System (I) 1 Bit/Pixel SPIHT Compressed Video Frame Divided into Equal Size Packets C1 C2 Group into 9 User Classes C9 C1 P1 C2 P2 9 Level RCPC Protection C9 P9 Motion-SPIHT and RCPC Coder
Simulation Results: MD = 0.6703dB 28 27 26 25 Motion SPIHT and RCPC: Minimax Disappointment Optimized JSCC Optimized SNR = 1dB SNR = 3dB SNR = 5dB SNR = 7dB SNR = 9dB MD Optimized PSNR (db) 24 23 22 21 20 19 1 2 3 4 5 6 7 8 9 Channel SNR (db)
Broadcasting Simulation System (II) 1. Layered-H.263 as source encoder, RCPC as the channel coder. 2. Each layer is an independent H.263 stream protected with different RCPC rates. 3. Each user class decodes the layer which gives best decoding performance. 4. Inefficient because of retransmission of coarse information, however each layer coder is efficient.
Simulation Results: MD = 2.4dB 34 32 30 28 Layered H.263 and RCPC: Minimax Disappointment Optimized JSCC Optimized SNR = 1dB SNR = 3dB SNR = 5dB SNR = 7dB SNR = 9dB MD Optimized PSNR (db) 26 24 22 20 18 16 1 2 3 4 5 6 7 8 9 Channel SNR (db)
Broadcasting Simulation System (III) Transmission Energy Per Bit 3D-SPIHT Compressed Video Bitstream Adjust the transmission power for each bit in a 3D-SPIHT stream under a total power constraint.
Simulation Results: MD = 0.9831dB 35 30 3D SPIHT and Bit Power Adjusting: Minimax Disappointment Optimized JSCC Optimal SNR = 1dB SNR = 3dB SNR = 5dB SNR = 7dB SNR = 9dB MD Optimized PSNR (db) 25 20 15 1 2 3 4 5 6 7 8 9 Channel SNR (db)
Discussion on Optimization 1. Constrained, nonlinear, discrete multi-variable optimization problem, non-trivial to solve. 2. Optimizing for each user class can use gradientdescent algorithms and penalty function approach. 3. We use the first method of successive approximation to solve the minimax problem, convergence guaranteed with convex cost function.
Extension: Network Multicasting 1. Multicasting sends the same packets simultaneously to all receivers: bandwidth-efficient. 2. Multicasting is undesirable when users are connected to the network at heterogeneous rates. 3. One popular approach is receiver-driven layered multicast. 4. Similar to broadcasting, JSCC and minimax disappointment can benefit network multicasting.
Conclusion 1. Minimax disappointment can serve as a fair and meaningful broadcasting/multicasting criterion. 2. With layered service, multiple user classes can have the same level of performance degradation. 3. We need a progressive video encoder to achieve good performance with our MD criterion.