Lecture 18: Exam Review The Digital World of Multimedia Prof. Mari Ostendorf
Announcements HW5 due today, Lab5 due next week Lab4: Printer should be working soon. Exam: Friday, Feb 22 Review in class today Sample exam solutions posted tonight Note: previous exam did not cover error coding or watermarking, but this year s might Ostendorf office hours: Thurs 1:30-3 Fri 9:30-12
What will the exam cover? Key concepts in the class so far Digital signal = a vector/matrix of numbers Frequency content of signals Analog-to-digital: sampling & quantization Filtering More on each topic to come! Compression Watermarking, error coding for both sounds and images (gray & color) Material in lectures, labs, HW
What will the questions be like? Similar to homework and quizzes Multiple question types: conceptual (e.g. what type of filter is the floor of your apartment?) pictorial (which image has more high frequency content?), possibly audio something involving equations simple calculations (sample rate, compression factor, bits/image, ); calculators not required simple MATLAB commands (see sample exam)
More on the exam No calculators, cell phones, etc. Open notes (but you are advised to have a summary sheet); no books Exam starts and ends promptly Be sure to JUSTIFY YOUR ANSWERS! No credit for no work (e.g. one word answers) Partial credit for reasonable justification even if the answer is wrong
Digital signal = a collection of numbers Time signal (speech, audio, bird sounds, ) Vector: X(n) n=1,,t Grayscale image Matrix: X(i,j) i=1,,m j=1,,n Color image Matrix x 3: X(i,j,k) i=1,,m j=1,,n k=1,2,3 (one matrix each for R, G & B)
Numbers are easy to manipulate! Signals are can be generated from Weighted sums of sinusoids (or other base signals) Concatenating different signals Math operations can change signals: Mixing (Z=aX+bY) Envelope scaling (Z=XY) Echo/shadow Z=X + ax shifted Time reversal (Z(n)=X(N-n)) (same idea works for mirror images) Filtering (see subsequent slide)
Frequency content of signals Audio signal Image
Varying Frequency Content Audio signal: spectrogram (time slices) Image: Block DCT (space blocks)
Analog-to-Digital Conversion Two key steps: Time/space sampling Amplitude quantization Very important concept: aliasing Shannon s sampling theorem for time signals: sampling rate Fs > 2B where B=bandwidth of analog signal to be sampled When Fs<2B, you get aliasing: high frequencies sound/appear lower, mix with actual low frequencies Same basic idea holds for images
Aliasing Time Signals Fs=samples per second Images Pixels per inch downsample Single tone sounds lower frequency Singing or speech sounds muffled
Filtering Linear scaling of the frequency content of signals Important types of filters: Low pass filter: Mainly low frequencies left Smoothing effect, blurring for images High pass filter: Mainly high frequencies left Emphasizes noise (speckles in an image), highlights edges or abrupt changes Can build other filters from these two Bandpass filter: keep middle frequencies only Bandstop filter: keep high and low, but not middle
Examples of Filtering Sound signals original filtered Images original filtered
Compression smaller Why compress? faster General steps Compress/ Encode Storage or Communication Decompress/ Decode Two types of compression Lossless: No change to sound/image when recovered Takes advantage of redundancy & imbalanced distributions Examples: entropy coding, run-length coding Lossy Some changes to the sound/image, hopefully subtle Takes advantage of human perception (esp. frequency domain) Examples: MP3 (audio files), JPEG (images) Cost: more computing
Watermarking, etc. Embed a low bit rate signal in a high bit rate signal (i.e. not fully compressed) Invisible watermarking (or information hiding) takes advantage of the same perception tricks as compression ADDS bits to signal compared to compression (replace vs. omit low order bits) Issues: robustness, security,.
Error Coding ADD bits to the signal to help detect and sometimes correct errors Examples: Parity code (detection only) Repetition code (correct errors by majority vote)