CS A490 Digital Media and Interactive Systems

Transcription

1 CS A490 Digital Media and Interactive Systems Lecture 8 Review of Digital Video Encoding/Decoding and Transport October 7, 2013 Sam Siewert

2 MT Review Scheduling Taxonomy and Architecture Traditional CPU SMP vs. NUMA Uni-processor (Peaked in 2000, Going Away ) Multi-core, Many-core Fixed Prio vs. Dynamic Prio. Scheduling Offload and Vector Processing Flynn s Taxonomy Single/Multiple Instruction x Single/Multiple Data SIMD (Vector Processing) SPMD (GP-GPU), Single Program, Multiple Data Sam Siewert 2

3 MT Review RT Theory, Pthreads Fundamentals of Real-Time Theory and Analysis RT Correctness Before Deadline and Correct Result Best Effort, Hard RT, Isochronal RT, and Soft RT Utility Curves for each of the above CPU, I/O, and memory resource space (CPU-bound, I/O-bound, memory/cpu trade-offs) Basic Timing Diagrams RM Least Upper Bound Lehoczky, Shah, Ding If deadline(s) are met over longest period (or better yet, LCM) from C.I., then system is feasible Fixed priority, preemptive, run-to-completion scheduling Services and interference (interrupts and higher priority services) Necessary and Sufficient feasibility means? Soft Real-Time Compared to Hard Real-Time Linux NPTL (Native POSIX Threads Library) Scheduling Classes (FIFO, RR, OTHER), 1-to-1 Pthread/Kernel Mapping pthread_create pthread_join pthread_detach Sam Siewert 3

4 MT Review SRT Dynamic Scheduling HRT - Rate Monotonic Theory RM priority assignment policy (optimality is?) Liu and Layland Assumptions C = WCET T=D Critical Instant Others? RM LUB Feasibility test is? Why is it Most Often NOT used for Digital Media? SRT - Dynamic Priority Theories EDF Least Laxity Timing Diagrams Harmonic vs. Non-harmonic Service Requests Overlapping Requests for Same Service (T D) Common in On-Demand Sam Siewert 4

5 MT Review DM Encoding Digital Sampling for Encoding Color Models (RGB Cube vs. HSV) Luminance, Chrominance RGB YCrCb Digital Audio PCM (Pulse Code Modulation), A-to-D Resolution and Frequency Audio Playback: DACs, Speakers, Channels Digital Video Encoding Pixel (POINT) - Encoding and Color Sub-sampling Frame (XY Pixel Map) - Resolution, AR, Frame Rate Sequence of Frames (Group of Pictures) Digital Media Systems Encoding, Transport, Decoding Digital Cable (Quadrature Amplitude Modulation), HFC Networks IPTV (Packet Switched MPEG TS encapsulated in UDP or RTP) ATSC OTA 8 level Vestigal Sideband Modulation, 6MhZ channel, Mbps, Reed-Solomon Error Correction MPEG Basics and Standards Order of Operators for Encoding (Compression) Program Streams Transport Streams MPTS (PID, PSI data, Video Elementary, Audio Elementary) Sam Siewert 5

6 MPEG-2: Order Of Operators #1 #2C #2B #2A #3 #1: POINT (Pixel) Encoding #2 A-C: Macro-Block Lossy Intra-Frame Compression #3: Motion-Based Compression in Group of Pictures Sam Siewert 6

7 DCT Fundamentals Small Losses Due to DCT, idct Numerical Truncation Large Losses Due to H.O.T. Truncation Sam Siewert 7

8 Step #2A: Macro-block Discrete Cosine Transform 8x8 Pixel Block Macro-block SD NTSC 720x480 (90x60 Macro-blocks), 3:2 Aspect Ratio HD x720 (160x90 Macro-blocks), 16:9 AR HD x1080 (240x135 Macro-blocks), 16:9 AR Sam Siewert 8

9 Step #2B: Macro-block Quantization (Lossy) Apply Weighting and Scaling 8x8 to DCT Produces Lots of Repeated Values (and Zeros) Compared to Original Sam Siewert 9

10 Decode Process for #2A-B Sam Siewert 10

11 Details on Q8 Weighting and Scaling Q=8 Quantization is DCT / (Weighting[i,j] x 8) 1) Example Macroblock DCT MPEG DCT Weighting W 2) Example DCT (use cvdct to Compute) Q8 = DCT[Ii,j] / (Weight[i,j] x 8) 3) q=8 from DCT / (Weighting x q=8) trunca Sam Siewert 11

12 Details on Q8 Weighting and Scaling Inverse Q=8 Inverse Quantization is Q8 * (Weighting[i,j] x 8) MPEG DCT Weighting W 5) Recovered Macroblock (cvdct INVERSE) idct 3) q=8 from DCT / (Weighting x q=8) trunca DCT recovered = Q8 x (Weight[i,j] x 8) 4) q=8 INVERSE is q8 * Weighting * Sam Siewert 12

13 MT Review DV Encoding MPEG Encoding Order of Operations Pixel/Color Encoding in YUV (e.g. YCrCb 4:2:2) Macro block encoding (DCT, Quantization, Zigzag, Huffman) Group of Pictures (I-Frame, B-Frame, P-Frame) Deeper Dive in and : Transport Streams for Video & Audio Container for Program Streams (188 Byte Packets) Multiplexed Video and Audio Elementary Streams PSI Program Specific Information System Clock (PCT, PTS/DTS) : Elementary Video Stream Encode/Decode Video DCT Macro-blocks Color Format GoP (I-Frame, B-Frame, P-Frame) Motion Compensation and Vector Quantization Sam Siewert 13

14 MPEG2 vs MPEG4 Differences Between MPEG-2 and MPEG-4 MPEG-2 Leverages MPEG-1 (Motion Picture Experts Group 1988) Widely Used for Digital Video Digital Cable TV, DVD Transport Stream designed for Broadcast (Lossy, No Beginning or End of Stream) MPEG-4 Leverages MPEG-2 Standards for Program/Transport, Encode/Decode Better Compression Rates (Macroblock), MPEG-4 Part-10 (H.264), e.g. Blu-Ray Extensions for Digital Rights Management Advanced Audio Encoding Becoming More Widely Deployed for HD and Because of Lower Bit-Rate Transport Streams Sam Siewert 14

15 MT Review - MPEG2 Transport Stream vs. Program Stream PCR, PTS/DTS What are they for? Multi-program Transport Streams Sam Siewert 15