TEKTRONIX SYMPOSIUM CONVERGENCE Video Services Telecommunications Mobile Communications Page 1
Modern Television System Seminar Topics Video and Compression Standards MPEG-2 Compression MPEG-2 System Testing in Compressed Systems Traditional test methods Picture quality assessment MPEG-2 protocol analysis Page 2
Modern Television System Program Production Program Compression Other Programs Program Demultiplexing Transmission Multi-Program Multiplexing Program Decompression Program Display or Reuse Page 3
Video Production Formats Analog Composite Video Parallel Digital Composite C a m e r a R G B M a t r i x R-Y B-Y Y Composite Encoder A to D Conv Serializer Serial Digital Composite (144 Mb/Sec) (177 Mb/Sec) Component Analog Video (RGB) Y B-Y R-Y 6.75 MHz Sample Component Analog Video (R-Y, B-Y, Y) 13.5 MHz Sample A to D Conv A to D Conv A to D Conv 10 bits 4xFsc Sample (PAL=17.7 MHz) (NTSC=14.4 MHz) Multiplexed 27 Mwords/Sec...Y / R-Y / Y / B-Y / Y... 10 bits Parallel Parallel Digital Component Clock X10 270 MHz clock Serializer AES/EBU Digital Audio Serial Digital Component 270 Mb/Sec Rec 601 Page 4
Video Compression Standards JPEG, still images (Joint Photographics Experts Group) M-JPEG; motion JPEG, not a standard, generally proprietary H.261 (px64), video conferencing px64 kb/s (p=1, 2,. 32) H.263, video conferencing, emphasis on low bitrates MPEG-1, CD-ROM and multimedia (Motion Picture Experts Group) ETSI 300 174, Broadcast distribution and contribution MPEG-2, Broadcast entertainment/contribution and DVD Non-DCT methods Wavelets, Fractal, DPCM Lossless (e.g., special JPEG mode) MPEG-4, very low bitrate coding (possibly wavelets) Page 5
Typical Video Data Rates 10-bit Rec 601 270 Mbps 8-bit Rec 601 216 Mbps 8-bit Rec 601 (active only) 167 Mbps Digital Betacam ~90 Mbps MPEG-2 4:2:2P@ML 15-50 Mbps MPEG-2 MP@ML 1.5-15 Mbps MPEG-1 constrain. param. 0.5-1.8 Mbps H.261 videoconferencing 64 kbps - 1.5 Mbps H.263 videoconferencing 4 kbps - 0.5 Mbps Page 6
MPEG-2 Applications RF Transmission DVB-S Digital Video Broadcasting - Satellite DVB-C Cable, DVB-T Terrestrial broadcast ENG Electronic (satellite) news gathering Broadband Network Contribution quality programs Video on demand Storage Media DVD Digital Versatile Disk Video servers Intra-studio Point-to-point (being developed by SMPTE) Networking (being specified by EBU/SMPTE) Page 7
MPEG-2 Standards Documents MPEG-2 ISO/IEC 13818 Part 1 Systems Part 2 Video Part 3 Audio Part 4 Conformance testing (for 1, 2 and 3) Part 5 Software simulation Part 6 System extensions - DSM-CC (Digital Storage Media - Command & Control) Part 7 Audio extension - NBC (non backward compatible) Part 9 System extension RTI (real time interface) Part 10 Conformance extension - DSM-CC Page 8
Standards Organizations ITU Place des Nations 1211 Geneve 20, Switzerland Ph: 41 22 730 6003 http://www.itu.ch SMPTE 595 West Hartsdale Ave White Plains, NY 10607 Ph: 914-761-1100 http:// www.smpte.org ISO Case Postale 56 1 rue de Varembe 1211 Geneve 20, Switzerland Phone: +41 22 749 01 11 http://www.iso.ch Documents for sale Global Engineering 15 Inverness Way East Englewood, CO 80112 Ph: 800-854-7179 Page 9
Modern Television System Seminar Topics Video and Compression Standards MPEG-2 Compression MPEG-2 System Testing in Compressed Systems Traditional test methods Picture quality assessment MPEG-2 protocol analysis Page 10
MPEG-2 Video Compression Pre-processing Clean-up pictures and prepare video samples Temporal Compression (IntER-frame) Compresses the data from multiple frames Spatial Compression (IntRA-frame) Compresses the data within one frame (Similar to JPEG) Rate Control Constant bitrate Constant (or nearly so) quality Page 11
Important Points about MPEG Only specifics bitstream syntax and decoding Encoding algorithms are not defined Open to invention and generally proprietary Future improvements are compatible with all decoders Asymmetric Compression Encoder is very complex Encoder contains a decoder model Decoder definition emphasizes low complexity (cost) Page 12
Required Pre-Processing Decode from composite to component Produce correct picture size Reduce 10-bit samples to 8-bit samples Convert to 4:2:0 sampling (entertainment quality) Optional Noise reduction Other picture clean up Page 13
Picture Sizes Rec 601/656 525/30/2:1 720 x 486 Rec 601/656 625/25/2:1 720 x 576 MPEG-2 30 fps (quasi-std) 704 x 480 MPEG-2 422P@ML 30 fps 720 x 512 MPEG-2 422P@ML 25 fps 720 x 608 SIF (30fps, 25 fps) 352 x 240,288 CIF (always 30 fps) 352 x 240 QSIF (30fps, 25 fps) 176 x 128,144 QCIF (always 30 fps) 176 x 144 Page 14
Rec 601 Component Video Sampling (313) (314) Page 15
4:2:0 Chroma Sub-Sampling 4:2:2 Rec 601 4:1:1 1 Luminance sample Y 2 Chrominance samples Cb, Cr 4:2:0 Page 16
Redundancies Spatial Redundancies Redundant information in the horizontal and vertical picture dimensions. Data that is similar or repeats itself in picture areas which are close to one another. Temporal Redundancies Redundant data over a given time. Data that is similar or repeats itself from moment to moment, even if its location in the picture area changes. Page 17
Discrete Cosine Transform (Definition) Encoder: The NxN two dimensional DCT is defined as: Decoder: Page 18
DCT Function 8 X 8 pixel Blocks are converted from the spatial domain to the spatial frequency domain. Transformed blocks are numerically represented as 8 X 8 DCT coefficients. DCT coefficients are more suitable for bit rate reduction techniques. The transform process does not result in bit rate reduction. Page 19
DCT Example 720 Pixels 480 Lines (Pixels) 8x8 Pixels 223 191 159 128 98 72 39 16 223 191 159 128 98 72 39 16 223 191 159 128 98 72 39 16 223 191 159 128 98 72 39 16 223 191 159 128 98 72 39 16 223 191 159 128 98 72 39 16 223 191 159 128 98 72 39 16 223 191 159 128 98 72 39 16 43.8-40 0-4.1 0-1.1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Picture Sample Values DCT Coefficients Page 20
Picture of the DCT Coefficients V H Horizontal spatial frequency waveforms Page 21
Quantization Divides each DCT coefficient by a frequency-dependent value and truncates the results to an integer. Many of the resulting integers are zero or small values (e.g., 1, 2, 3, 12, 13,.) Quantization coefficients can be tailored to complement limitations of the human visual system Quantization causes information to be irretrievably lost. Reconstructed pixels usually differ in value from the original Page 22
7842 199 448 362 342 112 31 22 980 12 23 16 13 4 1 0 198 151 181 264 59 37 14 3 12 9 8 11 2 1 0 0 142 291 218 87 27 88 27 12 7 13 8 3 0 2 0 1 111 133 159 119 58 65 36 2 49 85 217 50 8 3 14 12 58 120 60 40 41 11 2 1 Divide by Quant Matrix Divide by Quant Scale 5 6 6 4 2 1 0 0 2 3 8 1 0 0 0 0 2 4 2 1 1 0 0 0 30 121 61 22 30 1 0 1 1 4 2 1 0 0 0 0 22 28 2 33 24 51 44 81 0 0 1 0 0 0 0 0 Input DCT Coefficients (a more complex block) Output DCT Coefficients Value for display only not actual results 8 16 19 22 26 27 29 34 16 16 22 24 27 29 34 37 19 22 26 27 29 34 34 38 22 22 26 27 29 34 37 40 22 26 27 29 32 35 40 48 26 27 29 32 35 40 48 58 26 27 29 34 38 48 56 69 27 29 35 38 46 56 69 83 Quant Matrix Values Value used corresponds to the coefficient location Code Linear Non-Linear Quant Scale Quant Scale 1 2 1 8 16 8 16 32 24 20 40 40 24 48 56 28 56 88 31 62 112 Quant Scale Values Not all code values are shown One value used for complete 8x8 block
Processing Sequences Zigzag or Classic (nominally for frames) Alternate (nominally for fields) Page 24
Entropy Coding Run length coding uses a special code for repeating values (e.g., 13 0s, 5 1s, 4 2s ) Variable length coding uses shorter code words for more probable symbols (like Morse code) Symbol A B C D E F Probability 0.5 0.25 0.125 0.0625 0.03125 0.03125 Code Word 0 10 110 1110 11110 11111 Page 25
INTRA-Frame Coding Rate Control Quantizing Data Full Bitrate 10-bit Data Convert 4:2:2 to 8-bit 4:2:0 DCT Quantize Entropy Coding Buffer Compressed Data Information lost Data reduced No Loss No Data reduced Data reduced (information lost) Data reduced (no loss) Quantizing Reduce the number of bits for each coefficient. Give preference to certain coefficients. Reduction can differ for each coefficient. Entropy Coding Variable Length Coding Use short words for most frequent values (like Morse Code) Run Length Coding Send a unique code word instead of strings of zeros Page 26
Redundancies Spatial Redundancies Redundant information in the horizontal and vertical picture dimensions. Data that is similar or repeats itself in picture areas which are close to one another. Temporal Redundancies Redundant data over a given time. Data that is similar or repeats itself from moment to moment, even if its location in the picture area changes. Page 27
Temporal Redundancies Frame to Frame redundancies New location same data New data uncovered Page 28
Motion Estimation Macro Block 16x16 Pixels Motion Vector Search Range Frame N Frame N + 1 Page 29
INTER-Frame Coding (Forward Prediction) Video in Present Frame Predicted Present Frame Motion Vectors Subtract/Pass Motion Compensation Fixed Store (previous frame) SUM Difference Frame Much less information if the prediction is good Previous Frame or Reconstructed Present Frame to use in the next Prediction Motion Estimation Motion Vectors Page 30
P-pictures only Rate Control Quantizing Data Video in Subtract DCT Q RLC VLC MUX Motion Vectors Predicted Present Frame Motion Compensation Motion Estimation Fixed Store Q -1 DCT -1 SUM Group of Pictures Control Page 31 Motion Vectors Buffer
Group of Pictures Bi-directional Prediction 0 1 2 3 4 5 6 7 N = Distance between anchors I B M = Distance between I pictures B P B B B P I Forward Prediction I pictures: Inter-coding only P pictures: Contain forward motion compensation B pictures: Contain forward, backward & bi-directional motion compensation Page 32
15/3 Frame Sequence I B B P B B P B B P B B P B B I Page 33
Time Sequence of Pictures Page 34
MPEG-1 Design focused on non-interlaced SIF (352x240) Application was media storage e.g., CD-ROM Uses most of the H.261 techniques Introduced the concept of B-frames Trick modes are supported Fast search Reverse, etc Used in early DTV testing Page 35
MPEG-2 MPEG-2 = MPEG-1 + interlace tools + Profiles & Levels New field/frame prediction and DCT modes for interlace Quantization with greater range and adaptivity New intra-frame VLCs (variable length codes) New adaptive coefficient VLCs Scalability extensions; Spatial, SNR, Temporal System layer for multiple program transport streams Audio extended to 5-channel sound Page 36
MPEG-2 Profiles and Levels HIGH 4:2:0 4:2:0. 4:2:2 1920 x 1152 1920 x 1152 80 Mb/s 100 Mb/s I, P, B I, P, B HIGH-1440 4:2:0 4:2:0 4:2:0, 4:2:2 1440 x 1152 1440 x 1152 1440 x 1152 60 Mb/s 60 Mb/s 80 Mb/s I, P, B I, P, B I, P, B MAIN 4:2:0 4:2:0 4:2:0 4:2:0, 4:2:2 760 x 576 720 x 576 720 x 576 720 x 576 15 Mb/s 15 Mb/s 15 Mb/s 20 Mb/s I, P I, P, B I, P, B I, P, B LOW 4:2:0 4:2:0 352 x 288 352 x 288 4 Mb/s 4 Mb/s I, P, B I, P, B LEVEL PROFILE SIMPLE MAIN SNR SPATIAL HIGH Page 37
MPEG-2 4:2:2 Profile HIGH 4:2:0 1920 x 1152 80 Mb/s I, P, B 4:2:2 1920 x 1088 300 Mb/s I, P, B 4:2:0, 4:2:2 1920 x 1152 100 Mb/s I, P, B HIGH-1440 4:2:0 1440 x 1152 60 Mb/s I, P, B 4:2:0 1440 x 1152 60 Mb/s I, P, B 4:2:0, 4:2:2 1440 x 1152 80 Mb/s I, P, B MAIN 4:2:0 720 x 576 15 Mb/s I, P 4:2:0 720 x 576 15 Mb/s I, P, B 4:2:2 720 x 608 50 Mb/s I, P, B 4:2:0 720 x 576 15 Mb/s I, P, B 4:2:0, 4:2:2 720 x 576 20 Mb/s I, P, B LOW 4:2:0 352 x 288 4 Mb/s I, P, B 4:2:0 352 x 288 4 Mb/s I, P, B LEVEL PROFILE SIMPLE MAIN 4:2:2 PROFILE SNR SPATIAL HIGH Page 38
MPEG-2 4:2:2 Profile for Production Quality Better Chroma Resolution than MP@ML Higher quality (bit rate) than MP@ML Good multi-generation performance Flexibility Short GOPs for editability Capability to pass all active video, some vertical info Economy Storage costs Transmission costs Compatibility Page 39
How Good is MPEG-2 4:2:2? The MPEG committee has conducted subjective assessment tests to verify the performance of the MPEG-2 4:2:2 profile. Tests demonstrated that, with proper choices of data rate and GOP structure, MPEG-2 4:2:2 can meet professional requirements. Page 40
Achieving Quality 50 Mb/s Higher Quality Bit Rate 30 Mb/s Lower Quality 20 Mb/s I IB IBBP GOP Structure Page 41
Application Examples News and Acquisition 18 Mb/s IB GOP Structure Distribution 20 Mb/s IBBP GOP Structure Fully Compliant MPEG-2 4:2:2 Decoders Archive 30 Mb/s IB GOP Structure Post-Production 50 Mb/s I only GOP Structure Page 42
If you can t hear it, don t send it Psychoacoustic Models Pre-Masking Post-Masking Simultaneous Masking Data Structures MPEG Audio Page 43
Temporal Masking sound pressure Premasking Postmasking time Page 44
sound pressure Simultaneous Masking 1 khz sinewave Threshold in quiet Masking threshold 20 Hz 1 khz 20 khz Page 45
MPEG Audio Encoder Filterbank 32 Subbands Scaler and Quantizer MUX 512 Point FFT Masking Thresholds Dynamic Bit and Scale Factor Allocator and Coder Audio Frame input PCM samples 384 for Layer 1 3 * 384 = 1152 for Layer 2 Page 46
Audio Time Frame 12 x 32 Samples 12 Sections of 32 Samples 32 Samples 32 Samples 32 Samples 32 Samples Filterbank 32 Subbands 32 Samples = 0.66 msec (@ 48 khz) 1 MPEG audio layer 1 frame = 8 msec of audio 1 MPEG audio layer 2 frame = 24 msec Page 47
Layer I Frame Structure Header CRC Bit Allocation Scalefactors 384 PCM Audio Input Samples Duration 8 msec @ 48 khz Subband Samples Anc Data GR0 GR1 GR2 GR11 20 Bit System 12 Bit Sync Optional 4 bit linear 6 bit linear 0 1 2 31 Unspecified Length Page 48
Modern Television System Seminar Topics Video and Compression Standards MPEG-2 Compression MPEG-2 System Testing in Compressed Systems Traditional test methods Picture quality assessment MPEG-2 protocol analysis Page 49
MPEG System Video encoder Audio encoder Elementary Streams System encoder PES Syntax Transport Stream System decoder PES Syntax Elementary Streams Video decoder Audio decoder Not Standardized MPEG-2 Standards Page 50
MPEG-2 System Mux Video Data Audio Data Video Encoder Audio Encoder Packetizer Packetizer Video PES Audio PES Program Stream MUX Program Stream (DVD) Elementary Stream Data Transport Stream MUX Single Program Transport Stream Page 51
Transport Stream Formation Page 52
Multi-program Transport Stream Page 53
Transport Packets 188 Bytes Header Payload Packet Packet Packet Packet Packet Packet Packet Packet The Transport Stream (TS) is a continuous data stream in 188 byte packets containing format (syntax) information and payload data Page 54
188 Bytes Transport Packet Header Header Payload Minimum 4-byte header Sync Byte 8 Transport Error Indicator 1 Start Indicator 1 Transport Priority 1 PID 13 Scrambling Control 2 Adaptation Field Control 2 Continuity Counter 4 Adaption Field Payload Adaptation Field Length 8 Discontinuity Indicator 1 Random Access Indicator 1 Elem Stream Priority Indicator 1 5 Flags 5 Optional Fields Stuffing Bytes PCR 48 OPCR 48 Splice Countdown 8 Transport Private Data Adaption Field Extension Page 55
Program Clock Model PCR clock frequency generator Video Encoder PCR clock phase generator PCR: encoder stamps departure time of packet PCR: arrival time of packet PCR clock recovered Video Decoder Display PES Syntax System MUX System DEMUX PES Syntax Audio Encoder Audio Decoder variable delay = e(n) constant trans delay = C trans variable delay = d(n) constant total delay = C total Page 56
Reference Clock Synchronization 27 MHz Clock Video In Video Encoder Elementary Stream Transport Stream Formation 188 byte packets PCR = X n bits PCR = X plus the time of exactly n bits Transport Stream Decoder PCR Compare Low Pass Filter Load 27 MHz Clock Local PCR 27 MHz Xtal VCO Receiver 27 MHz clock Page 57
Decoding the Transport Stream Program Specific Information (PSI) Program Association Table (PAT) PID = 0, must be present in every transport stream Program Map Table (PMT) PID values assigned by transmission system (DVB, ATSC, etc.) Conditional Access Table (CAT) PID = 1 Network Information Table (NIT) PID values assigned by transmission system DVB considers this part of System Information (SI) Null Packets PID = 8191 (1FFF hex = 13 1 s binary ) Page 58
PSI Example Program Association Table (PID 0) Program 0 16 Program 1 22 Program 3 33...... Network Information Table Private Network Data Stream 1 Video 54 Stream 2 Audio 48 Stream 3 Audio 49......... Stream k Data 66......... Program k 55...... Program Map Tables Stream 1 Video 19 Stream 2 Audio 81 Stream 3 Audio 82......... Stream k Data 88......... Conditional Access Table (PID 1) Conditional Access Data PAT 0 Prog 1 MAP Prog 3 MAP EMM Transport Stream Prog 1 Audio 1 Prog 3 Audio 2 Prog 3 Video 1 Prog 3 Video 1 Prog 1 Video 1 Prog 3 Audio 1 Prog 3 Video 1 22 33 1 48 82 19 19 54 81 19 Page 59
DVB System MPEG-2 Transport Streams Service Information (SI) in addition to MPEG-2 (PSI) SI includes NIT - Network Information SDT - Service Description EIT - Event Information TDT - Time and Date BAT - Bouquet Association RST - Running Status ST - Stuffing Tables DVB Systems provide: Common Scrambling systems A common Conditional Access Interface Facilities for reverse channel operation Page 60
DVB Channel Coding Provides error correction over the channel ( 1 in 10-4 on channel to 1 in 10-11 on Transport Stream ) Outer Coding - energy dispersal and RS Inner Coding - interleaving and viterbi Not used for cable transmission 204 Bytes per packet 204 Bytes per packet 204 Bytes per packet Transport Stream Packet 188 Bytes Reed Solomon Coding 16 Bytes Transport Stream Packet 188 Bytes Reed Solomon Coding 16 Bytes Transport Stream Packet 188 Bytes Reed Solomon Coding 16 Bytes Page 61
Modern Television System Seminar Topics Video and Compression Standards MPEG-2 Compression MPEG-2 System Testing in Compressed Systems Traditional test methods Picture quality assessment MPEG-2 protocol analysis Page 62
Video Testing Layers Signal Standardized Video Signals Studio Connections Test Function Video Quality MPEG-2 Transport Stream Program Compression Transmission Channel Formatting INTRA-Facility Connections Protocol Analysis SDH/ATM or Modulated RF INTER-Facility Connections Page 63 Transmission Channel Analysis
Signal vs Picture Quality Analog and digital video systems are linear Superposition applies Results are time invariant and signal independent Test signals can be substituted for program material Testing in the vertical interval is equivalent to full-field tests Static test signals are sufficient (Indirect measurement) Compression video systems are non-linear Test signals are easily/accurately compressed Picture quality is a function of; data rate, picture complexity and encoding algorithm capabilities Test with complex motion sequences (Direct measurement) Page 64
Modern Television System Seminar Topics Video and Compression Standards MPEG-2 Compression MPEG-2 System Testing in Compressed Systems Traditional test methods Picture quality assessment MPEG-2 protocol analysis Page 65
Traditional Test Methods A significant portion of the modern television system is analog or full bandwidth digital video These signals will continue to be used in the foreseeable future Input picture quality to the compression system must be maintained Page 66
Analog Video Testing Camera RGB Composite Encoder PAL NTSC Transmission or Operation PAL NTSC Record or Display Functional Layers VIDEO Operational Monitoring Technical Measurements SYNCHRONIZING WAVEFORMS Technical Measurements PHYSICAL LAYER (COAX) Technical Measurements Waveform Monitors and Measurement Sets TDRs Page 67
Analog Tests Amplitude Rise/fall times Bandwidth Group delay Sig/Noise ratio Non-linearities Color gamut Diff Phase/Gain No one test will do it all Page 68
Digital Video Testing Camera RGB Composite Encoder PAL NTSC Transmission or Operation PAL NTSC Record or Display Decode A/D Encode D/A Studio Rec 601/656 Interconnect, Rec 601/656 Operation Functional Layers VIDEO SIGNAL CODING (Rec 601) DIGITAL FORMATTING (Rec 656) DIGITAL WAVEFORM (Rec 656) PHYSICAL LAYER (COAX/FIBER) Waveform Monitors, Measurement Sets with Analog or Digital Capabilities TDRs, OTDRs Page 69
A/D and D/A Converters More Analog Tests Converters add distortions to the signal Measure SNR on a shallow ramp Differential gain/phase Full ramp for analog measurements Shallow ramp for digital measurements Ringing on digitally generated signals Page 70
Rec 601 Digital Measurements Digital coding Levels, excluded values Formatting Synchronization data, Embedded audio Error Detection Waveform (eye pattern) Amplitude, risetime, overshoot Jitter by eye pattern or demodulation Headroom (cable length) Page 71
Modern Television System Seminar Topics Video and Compression Standards MPEG-2 Compression MPEG-2 System Testing in Compressed Systems Traditional test methods Picture quality assessment MPEG-2 protocol analysis Page 72
Factors Affecting Video Quality in a Compression System Quality of the input video Amplitude, dc level, bandwidth, ringing, jitter Noise, composite/component decoding artifacts Prefiltering to eliminate the above problems Nature of the input video Picture spatial and temporal complexity Page 73
Factors Affecting Video Quality Encoding parameters used Profile/level, field/frame, output data rate, GOP Encoding algorithm Speed required, hardware vs software Multiple pass (iteration of parameters) Algorithm design Quantizing table selection Use of motion vectors, search range Page 74
Compression Impairments Blocking: appearance of underlying block structure Error Blocks: a form of block distortion One or more blocks bear no resemblance to the current or previous scene and often contrast greatly with the adjacent blocks Edge busyness: distortion concentrated at edges of objects Characterized by temporal and spatial features Mosquito noise: edge busyness associated with movement Characterized by moving artifacts or blotchy noise patterns superimposed over the objects Quantization noise: snow or salt & pepper Similar to random noise but not uniform over the image Blurring: distortion of the entire image, Characterized by reduced sharpness of edges and spatial details Jerkiness: smooth, continuous motion now perceived as a series of distinct images Page 75
Blocking Blurring Page 76
Picture Quality Measurements We can not actually measure picture quality We can measure picture degradation comparison to reference Subjective measurements ITU-R BT.500 has been updated Further work to extend subjective methods Objective measurements are most useful if they have good correlation with subjective results Page 77
Subjective Tests (Human Viewer Trials) Strengths Produces valid results in conventional and digital television system applications Provides a scalar result Mean Objective Score Weaknesses Requires meticulous setup and control Needs lots of participants Is time-consuming Works well over a wide range of video (and still image) applications Subjective tests are only applicable for development purposes. They do not lend themselves to operational monitoring, production line testing or troubleshooting. Page 78
Quest for Objective Measurements Intuition has led many developers to the same starting point: Picture quality is related to the differences between the original and impaired scenes. A measurement of the magnitude of these differences is somehow related to picture quality Therefore, construct a device which indicates the magnitude of these differences. There are two approaches to objective measurements Feature Extraction Picture Differencing Page 79
Feature Extraction Reference Picture Processing System Degraded Picture Feature Extraction Low Bandwidth Data Feature Difference Feature Extraction Results Page 80
Picture Differencing Reference Picture Processing System Degraded Picture Image Processing Picture Data Picture Data Difference Picture Data Image Processing Results Page 81
Mean Squared Error Prediction of quality derived from the result of computing the mean of the squares of the differences As the result approaches zero, the more identical are the original and copy. Conversely, as the result grows, the more different is the copy from the original. Peak Signal to Noise Ratio is a variation of MSE: 255 2 PSNR=10 log 10 MSE 2 Page 82
But MSE (and PSNR) is Easily Fooled! MSE = 27.10 MSE = 21.26 Page 83
Objective Picture Measurements ANSI T1.801.03 Peak Sig/noise Average gain Offset level Spatial shift Spatial info Temporal info Added/lost; spatial frequencies, motion/edge energy Radial average of spatial frequencies Insufficient for comparison of systems Useful for yesterday/today comparisons Unfortunately most systems are not constant Bit rate changes Concatenation of different coding systems Page 84
Human Vision System A number of picture assessment methods have been proposed based on the human vision model A robust metric of image quality Independent of nature of the video material Independent of the type of impairments Independent of the compression system Principles of HVS models Contrast sensitivity Spatio-temporal response Color perception Page 85
The Tektronix/Sarnoff Method In a review of 32 human visual models All but one of the models can be regarded as simplified versions of the Lubin model. Dr. Lubin at Sarnoff Labs Extended the work to cover picture quality Temporal chroma models have been added JND Image Quality Metric Tektronix/Sarnoff cooperative product development Page 86
Just Noticeable Differences Central features Approximates the optics of the eye and retinal structure Filters that decompose image into subbands Directional filters Psycho-visually valid calibration curves Pooling process to combine local results JND scale At a value of 1 JND, 3 out of 4 can detect a difference Values above 1 have more noticeable differences Values below 1 have less noticeable differences Page 87
impaired image JND Algorithm optics oriented responses sampling transducer distance gain control...... JND map contrast pyramid Identical Process JND value reference image Page 88
Model Output: JND Maps Reference Image JND Map Degraded Image Local magnitude of JND Map indicates probability of seeing a difference between two images at that point Average of JND Map indicates overall magnitude of visible differences Page 89
JND vs. MSE MSE = 27.10 Average JNDs = 0.75 MSE = 21.26 Average JNDs = 2.52 Page 90
Objective Measurement Operation Close matching of presentation Chroma/luma gain and dc level Spatial and temporal alignment Test motion sequences Difficult but not killer Several different program types Program material not okay (for now) Compute power/time for matching Material often not difficult Page 91
Basic Concept Encoder Transport Decoder Play test sequences 5+ seconds Perform automatic measurements 2 seconds Test Sequence Source Reference Sequence Source Picture Quality Meter Page 92
Use of Calibration Stripes Original Compare original and transported Transported Special markings used to derive gain/level/shift. Page 93
Modern Television System Seminar Topics Video and Compression Standards MPEG-2 Compression MPEG-2 System Testing in Compressed Systems Traditional test methods Picture quality assessment MPEG-2 protocol analysis Page 94
MPEG Protocol Analysis MPEG Transport Stream Hardware I/O Interface Real-time Firmware Display key errors Hard-disk Storage Generation and Analysis Software Display header and Timing Information Tektronix MTS 100 provides creation, generation and in-depth analysis of MPEG-2 transport streams Real-time analysis and display of key errors is planned Page 95
MTS Analysis Analyzes transport and/or PES packets Syntax (structural) Extracts PES packets Windows NT Application MPEG Transport Stream Analysis Program MPEG TS Data Files on the NT Disk Elementary streams Hierarchical view PSI (program specific information) Programs Channels MTS Data Files in the Data Store System Multiple Hardware Output Types Windows NT Application Control MPEG TS Page 96
MTS Generator ES Data Files are supplied by Tektronix or the user The ES Data Files can be on disk or CD ROM MTS Data Files are built using multiplexing software Formatted ES User defined configuration Application of MPEG rules MTS outputs the transport stream to the device under test ES Data Files on the NT Disk CD ROM MTS Data Files in the Data Store System Windows NT Application Control MTS Formatter and Multiplexer Multiple Hardware Output Types Windows NT Application Control MPEG TS User Defined Configure File Page 97
Transport Stream Hierarchical Display Page 98
Transport Packet Field Description Page 99
Transport Packet Data Analysis Page 100
Transport Packet Hex Display Page 101
Program Association Table Analysis Page 102
PCR number 21-30 PCR Data Analysis 0.804640 0.985496 1.166352 PCR number 23 PCR Base : 79659 units PCR Extension : 93 units PCR Value (PCRV) : 0.885103444 sec Interpolated PCR (PCRI) : 0.885104096 sec PCRI - PCRV : 0.000000652 sec Previous PCR Arrived Since : 0.040156800 sec Packet Number : 5886 Packet Number of PID 50 : 2279 Page 103
Time Stamp Analysis Page 104
System Target Decoder An abstract model for deciding legality of streams and decoders (This is really control on encoder operation.) Based on passing of streams through cascading buffers Overflow of any buffer implies illegality of stream Sometimes underflow is allowed; sometimes it isn t. Page 105
System Target Decoder Simulation Page 106
Automatic Analysis Selectable functions Error tables available for quick access to data Page 107
Service Information Selection of SI data Page 108
Service Information Interpreted Decoded Service Description Table Page 109
Sarnoff Compliance Bitstreams STRM100 A visual inspection of Video Decoder Operation No digital interface or capture equipment required Ease of use: pass-fail testing, rapid fault isolation Look for gray embossed VERIFY without artifacts One parameter or technique at a time Loopable Page 110
Sarnoff Compliance Bitstreams: How They Work Tests without B-pictures IPPP... PP PP... P PP... PP Verify (60 to 90 frames) -- does not change image Test section (1 or more frame) Start Title (operator convenience only) Tests with B-pictures IPPP... PP PBBBB... BBBBP Test and Verify (60 to 90 frames) -- the test runs continuously, each picture should be verify Start Title (operator convenience only) Page 111
Sarnoff Compliance Bitstream Tests Different picture types Motion vector range DC differential range Macroblock pattern range Macroblock type range AC run/level test Macroblock address range Dynamic GOP structure Dynamic slice size Macroblock & zero stuffing Postprocessing Zone plate Color bars Ramp Downloadable quant matrices Different bitrates Dynamic picture size 2:3 pulldown insertion VBV buffer sizes Mixed MPEG-1 & MPEG-2 Alternate scan pattern Alternate AC run/level VLC Frame/field coding modes Frame/field prediction modes Non-linear quantization scale Pan and Scan Page 112
Summary Modern television systems are much more than DTV Cameras and displays will continue to be analog Program production uses analog and full bandwidth digital MPEG-2 will be the dominant compression method Entertainment quality video to the home Satellite, Terrestrial, Cable Digital Versatile Disk (DVD) Video servers Contribution quality, reusable video for studios Page 113
Summary MPEG only specifies bitstream syntax and decoding A non-symmetrical system Complex encoders, Simple (inexpensive) decoders Encoding can improve and be compatible with today s decoders Increasing compute power for complex algorithms Manufacturers can compete with proprietary algorithms 4:2:2 profile provides contribution quality Decoders must be compliant for flexibility of application MPEG-2 System for multi-program transmission Page 114
Summary System testing requires a layered approach Signal/Picture quality MPEG-2 system protocol Transmission channel Signal quality testing uses traditional methods Indirect measurement, Static test signals, vertical interval Picture quality testing is a complete new paradigm Dynamic, complex test scenes utilize the complete channel Reliable objective measurements use human visual model Picture differencing methods provide best results Page 115
Summary MPEG-2 system protocol testing Generation of valid (and invalid) test signals is important In-depth testing provides complete data analysis off-line Real-time testing provides limited analysis of all packets Tektronix equipment for video and protocol testing TSG422 Digital Video Test Signal Generator SPG422 Digital Video Sync Pulse Generator TG2000 Multi-format Test Signal Generator 1700 series Composite/Component Waveform Monitors WFM600 Series Digital Video Monitors VM700T Video Signal Measurement Set MTS100 MPEG Transport Stream Analyzer/Generator Page 116
Thank You for Your Attendance... providing 50 years of technical innovation and leadership. Page 117
References Page 118
References Page 119