Fourier Transforms 1D - PDF Free Download

Fourier Transforms 1D 3D Image Processing Torsten Möller

Overview Recap Function representations shift-invariant spaces linear, time-invariant (LTI) systems complex numbers Fourier Transforms Transform pairs properties Convolution Theorem Understanding sampling in the Fourier space 2

What is a (digital) image? An image is made of pixels (=picture elements) the coordinate values are discretized Laurent Condat / Torsten Möller 3

Quantization The pixel values are quantized: they belong to a discrete set of values, generally represented by integers between 0 and N-1 coded on 8 bits 255 possible val. coded on 4 bits Laurent Condat / Torsten Möller coded on 2 bits 4 possible val. 4

2D lattices An image is defined on a lattice. The most common is the Cartesian (a.k.a square) lattice. But other lattices exist and have interesting properties. Laurent Condat / Torsten Möller 5

Image quality Common defaults in images: blur (motion blur, out of focus blur...) ringing aliasing (staircasing, Moiré patterns) Laurent Condat / Torsten Möller 6

Image quality Low-frequency Moiré artifacts appear when high-frequency content is sampled in an incorrect way. Laurent Condat / Torsten Möller 7

Image quality We need to be able to measure the difference between two images, for instance an original image and a degraded one. Classical difference measures between two images I 1 and I 2 : mean absolute error: MAE = 1 P W P H WH k x =1 k y =1 I 1[k x,k y ] I 2 [k x,k y ] mean square error: MSE = 1 P W P H WH k x =1 k y =1 I 1[k x,k y ] I 2 [k x,k y ] 2 255 2 peak signal to noise ratio (db): PSNR = 10 log 10 MSE There exist much more sophisticated quality measures and difference measures for images (SSIM...) Laurent Condat / Torsten Möller 8

Image enhancement Intensity Transformations image negatives log transforms gamma (power-law) transforms contrast stretching intensity-level slicing bit-plane slicing 9

Histogram equalization Idea -- stretch histogram non-uniformally such that final histogram is a uniform distribution p s (s) =p r (r) dr ds s = T (r) = Z r 0 p r (w)dw 10

Histogram matching original matched equalization 11

Local histogram equalization 12

Mechanics of filtering Correlation: w(x, y)? f(x, y) = ax bx w(s, t) f(x s, y t) Convolution: w(x, y) f(x, y) = s= a ax t= b bx w(s, t) f(x s, y t) s= a t= b 13

Median filtering (denoising) 14

Sharpening enhance / highlight transition in intensity how to find transition? unsharp masking / highboost filtering first / second order derivatives in 1D multi-d: gradient magnitude Laplacian 15

How to represent a function? on a computer, can only store a bunch of numbers, not a continuous function! brute-force idea: sampling 17

What is sampling? (mathematically speaking) modeled through an impulse not really a function, but a distribution: Z 1 1 (t) = 1 if t =0 (t)dt =1 0 if t 6= 0 18

The sifting property picking a value off from f: more general: Z 1 1 Z 1 1 f(t) (t)dt = f(0) f(t) (t t 0 )dt = f(t 0 ) 19

The impulse train pick up multiple values of f at once: s T (t) = 1X 1 (t n T ) 20

What is sampling? f(t) s T (t) = 1X f(n T ) (t n T ) 1 1X 1 f[n] (t n T ) 21

Sampling: step by step 1. break function into pieces & measure 22

Sampling: step by step 1. break function into pieces & measure 2. reconstruct function from measurements = 23

Sampling: step by step 1. break function into pieces & measure 2. reconstruct function from measurements = 24

Sampling: step by step f(t) = 25

Sampling: step by step Z 1 1 f(t) (t t 0 )dt (t t 0 ) f(t) = 26

Sampling: step by step Z 1 1 f(t) (t t 0 )dt (t t 0 ) f(t) Z 1 1 f(t) (t (t 0 T ))dt (t (t 0 T )) = 27

Sampling: step by step Z 1 1 f(t) (t t 0 )dt (t t 0 ) f(t) Z 1 1 = Z 1 1 f(t) (t (t 0 T ))dt f(t) (t (t 0 2 T ))dt (t (t 0 T )) (t (t 0 2 T )) 28

Sampling: step by step Z 1 1 f(t) (t t 0 )dt (t t 0 ) f(t) Z 1 1 = Z 1 1 f(t) (t (t 0 T ))dt f(t) (t (t 0 2 T ))dt (t (t 0 T )) (t (t 0 2 T )) Z 1 1 f(t) (t (t 0 3 T ))dt (t (t 0 3 T )) 29

Sampling: step by step f(t) Z 1 1 Z 1 1 = Z 1 1 f(t) (t t 0 )dt (t t 0 ) f(t) (t (t 0 T ))dt f(t) (t (t 0 2 T ))dt (t (t 0 T )) (t (t 0 2 T )) f[0] (t t 0 ) Z 1 1 f(t) (t (t 0 3 T ))dt (t (t 0 3 T )) 30

Sampling: step by step f(t) Z 1 1 Z 1 1 = Z 1 1 f(t) (t t 0 )dt (t t 0 ) f(t) (t (t 0 T ))dt f(t) (t (t 0 2 T ))dt (t (t 0 T )) (t (t 0 2 T )) f[0] (t t 0 ) f[1] (t (t 0 T )) Z 1 1 f(t) (t (t 0 3 T ))dt (t (t 0 3 T )) 31

Sampling: step by step f(t) Z 1 1 Z 1 1 = Z 1 1 Z 1 1 f(t) (t t 0 )dt (t t 0 ) f(t) (t (t 0 T ))dt f(t) (t (t 0 2 T ))dt f(t) (t (t 0 3 T ))dt (t (t 0 T )) (t (t 0 2 T )) (t (t 0 3 T )) f[0] (t t 0 ) f[1] (t (t 0 T )) f[2] (t (t 0 2 T )) 32

Sampling: step by step f(t) Z 1 1 Z 1 1 1 Z 1 1 f(t) (t t 0 )dt (t t 0 ) f(t) (t (t 0 T ))dt f(t) (t (t 0 2 T ))dt f(t) (t (t 0 3 T ))dt (t (t 0 T )) = Z 1 (t (t 0 2 T )) (t (t 0 3 T )) f[0] (t t 0 ) f[1] (t (t 0 T )) f[2] (t (t 0 2 T )) f[3] (t (t 0 3 T )) 33

A more general view on sampling in summary: f(t) = f(t) = more generally: f(t) = f(t) = 1X n= 1 1X n= 1 1X n= 1 1X n= 1 Z 1 1 f(t) (t n T )dt f[n] (t n T ) Z 1 1 f(t) (t n T )dt c[n] (t n T ) (t n T ) (t n T ) 34

A more general view on sampling in summary: f(t) = f(t) = 1X n= 1 Z 1 1 even more general: f(t) = f(t) = 1X n= 1 1X n= 1 1X n= 1 f(t) (t n T )dt f[n] (t n T ) Z 1 1 f(t) (t n T )dt c[n] (t n T ) (t n T ) (t n T ) 35

Sampling: generalization I = 36

Sampling: generalization I (t) = 37

Sampling: generalization II (t) (t) = 38

A more general view on sampling in summary: f(t) f(t) 1X n= 1 1X n= 1 Z 1 1 f(t) (t n T )dt c[n] (t n T ) (t n T ) ѱ also known as a point-spread function ; common for image acquisition Φ also known as the reconstruction function 39

Shift-invariant spaces All functions f that can be represented as f(t) = considered being part of a shift-invariant space V(Φ) Function f(φ) is characterized by coefficients c[n] positions nδt we can store this in a computer! Remaining questions: 1X n= 1 What is a good Φ and ѱ? c[n] (t n T ) 40

What are systems? Engineers like to figure out stuff in boxes (ever heard of reverse engineering? Example: Radio incoming: radio waves outcoming: sound http://stevenhmagician.blogspot.co.at/ 2013/02/general-information-of-radiowave.html http://www.soundctrl.com/blog/wpcontent/uploads/2013/07/old_radio.jpg http://www.picsandsticksmusic.com 42

What are systems? Engineers like to figure out stuff in boxes (ever heard of reverse engineering? Example: Radio incoming: radio waves outcoming: sound r(t) s(t) http://www.soundctrl.com/blog/wpcontent/uploads/2013/07/old_radio.jpg 43

Time-invariant system Example: Radio if the same radio waves come in tomorrow, it ll produce the same sound! r(t) s(t) http://www.soundctrl.com/blog/wpcontent/uploads/2013/07/old_radio.jpg 44

Time-invariant system Example: Radio if the same radio waves come in tomorrow, it ll produce the same sound! r(t t 0 ) s(t t 0 ) http://www.soundctrl.com/blog/wpcontent/uploads/2013/07/old_radio.jpg 45

Linear system Example: Radio When two stations are sending at the same frequency, then you hear both at once! r 1 (t) s 1 (t) r 2 (t) s 2 (t) r 1 (t)r 2 (t) s 1 (t)s 2 (t) http://www.soundctrl.com/blog/wpcontent/uploads/2013/07/old_radio.jpg 46

LTI systems How to characterize such systems? All we need to know is how the system response to an impulse! Black Box System (x x 0 ) (x x 0 ) 47

LTI systems Sidenote If response to the impulse is finite (bounded) = FIR infinite (not bounded) = IIR Black Box System (x x 0 ) (x x 0 ) 48

LTI systems (t) (t) Black Box System 49

LTI systems (t) 1X f s (t) = n= 1 f[n] (t n T ) Black Box System 50

LTI systems f s (t) = 1X n= 1 f[n] (t n T ) f r (t) = Black Box System Also known as a 1X convolution! n= 1 f[n] (t n T ) 51

LTI s for sinecosine Let s plug in sine and cosine waves and see what happens to them in an LTI system! Why sine cosine? carry amplitude and frequency hence, they are closer to our radio waves! however, they are just special! First -- need to learn about complex numbers! 52

Complex numbers most general: C = xjy components of C: Re(C) = x Im(C) = y Im C w y Re x Also expressed in terms of frequency (angle) and amplitude: C = C (cos! j sin!)= C e j! C = p x 2 y 2 54

Conjugate complex numbers complex: C = xjy Im conjugate: C* = x-jy C w x y -y Re 55

LTI systems f s (t) = 1X n= 1 f[n] (t n T ) Black Box System f r (t) = 1X n= 1 f[n] (t n T ) 56

LTI systems f s (t) = 1X n= 1 f[n] (t n T ) Let s try f r (t) = Black Box System f[n] =e j!n 1X f[n] (t n T ) n= 1 57

Steps of the algebra I f r (t) = f r (k T )= f r [k] = f r [k] = 1X n= 1 1X n= 1 1X m= 1 1X m= 1 f[n] (t n T ) f[n] ((k n) T ) f[k m] (m T ) e (k m)j! T (m T ) 58

Steps of the algebra II 1X f r [k] = e (k m)j! T (m T ) m= 1 1 f r [k] =e kj! T X e mj! T (m T ) f r [k] =f s [k] (!) m= 1 59

LTI systems sinecosine are eigenfunctions of an LTI system (a convolution) The eigenvalue popping out is the Fourier Transform of the LTI Black Box System e j!n T e j!n T (!) (!) = 1X e mj! T (m T ) m= 1 60

What is a Fourier Transform? an eigenvalue of an LTI? Huh? Let s go back to representation of functions: f(t) f(t) 1X n= 1 1X n= 1 Z 1 1 What is a good Φ/ѱ? f(t) (t n T )dt c[n] (t n T ) (t n T ) Current choices were focused on a spatial location! (also known as spatial domain) 62

Sampling: Spatial Domain Rep (t) (t) = 63

What is a Fourier Transform? w2 w1 f 1 f 2 w4 w3 f 3 f 4 http://mopred.blogspot.co.at/2012/04/discrete-cosine-transform.html 64

What is a Fourier Transform? Let s go back to (spatial) representation of functions: f(t) f(t) c n = 1 T f(t) Z T/2 1X 1X n= 1 1X n= 1 Fourier series into Frequency Domain: n= 1 T/2 Z 1 1 f(t)e j 2 n T c n e j 2 n T t f(t) (t n T )dt c[n] (t n T ) t dt (t n T ) 65

Leap of faith? we can represent ANY function through a sum of sine cosine! Fourier 1807/1822 not very intuitive, even for the mathematical elite of the time 66

Some intuition 67

There are 4 Fourier Transforms! Recall Fourier series: f(t) is periodic with period T! General Fourier Transform requires no periodicity: F (!) = f(t) = Z 1 1 Z 1 1 c n = 1 T f(t) Z T/2 1X n= 1 T/2 f(t)e j 2 n T c n e j 2 n T f(t)e j2!t dt F (!)e j2!t dt t t 68

DFT the most important one Discrete Fourier Transform (DFT) requires periodicity in both transform pairs F m = M 1 X n=0 f n = 1 M f n e j2 mn/m M 1 X m=0 F m e j2 mn/m 69

All Fourier Transforms Spatial Domain Frequency Domain FT FS Fourier Series DFT Discrete FT DTFT Discrete Time FT f(t) = f(t) = Z 1 1 F (!)e j2!t dt continuous f n = 1 M 1X n= 1 MX 1 m=0 c n e j 2 n T t continuous periodic f n = 1 2 Z F m e j2 mn/m discrete periodic discrete F (!) = c n = 1 T F m = F (!)e j!n d! F (!) = Z 1 1 Z T/2 MX 1 n=0 T/2 f(t)e j2!t dt continuous discrete f n e j2 mn/m 1X n= 1 f(t)e j 2 n T t dt discrete periodic f n e j!n d! continuous periodic 70

A little more intuition 71

Visualization of the spectrum Generally, we look at the amplitude of an image transform; hence we take a logarithmic scale to represent the values as gray-values. high frequencies original image Laurent Condat / Torsten Möller its amplitude spectrum low frequencies 72

Interpretation The lines correspond to discontinuities, with perpendicular orientation. The horizontal and vertical lines come from the implicit periodic boundary conditions. For natural images, most of the information is concentrated in the low-frequency region. Low frequencies correspond to the slowly varying components, whereas high frequencies correspond to fast gray level changes (edges...) The diagonal line results from the discontinuity induced by the hat. Laurent Condat / Torsten Möller 73

Interpretation The small lines correspond to the windows and flags. Laurent Condat / Torsten Möller 75

Some Examples Since the image content is periodic, the spectrum is discrete. If the period of the signal increases, the distance between the frequencies decreases, and vice versa. Laurent Condat / Torsten Möller 76

Some Examples The 2D periodicity of the image induces the 2D periodicity of the FT Since the image content is periodic, the spectrum is discrete. Laurent Condat / Torsten Möller 77

Texture = periodic pattern The Fourier spectrum is well suited for describing the directionality of textures Laurent Condat / Torsten Möller 78

A little more intuition 79

A little more intuition 80

A little more intuition 81

Not very intuitive 82

What happens to an impulse? it is basically a constant! c n = 1 T Z T/2 T/2 (t)e j 2 n T t c n = 1 T e0 c n = 1 T 84

What about a shifted impulse? the shifts remain as frequencies c n = 1 T Z T/2 T/2 (t t 0 )e j 2 n T t c n = 1 T e j 2 n T t 0 85

What happens to an impulse train? Impulse train is periodic apply Fourier series, will not do the math here, see book: s T (t) = S T (!) = 1 T 1X (t n T ) 1 1X n (! T ) n= 1 distance between impulses grows inversely 86

What happens to a box? it is the well-known sinc function sinc(t) = sin t t 87

LTI systems f s (t) = 1X n= 1 f[n] (t n T ) Black Box System f r (t) = 1X n= 1 f[n] (t n T ) 89

What is the Fourier Transform of a convolution? doing this in the continuous domain: f r (t) =f (t) = Z 1 1 Z applez F r (!) = f( )h(t )d e j2!t dt Z applez = f( ) h(t )e j2!t dt d Z = f( ) H(!)e j2! d Z = H(!) f( )e j2! d f( )h(t )d = H(!)F (!) 91

What is the Fourier Transform of a convolution? convolution == multiplication: f (t) () F (!)H(!) multiplication == convolution: f(t) (t) () F (!) H(!) 92

What was a convolution again? in summary: f(t) f(t) 1X n= 1 1X n= 1 Z 1 1 f(t) (t n T )dt c[n] (t n T ) (t n T ) ѱ also known as a point-spread function ; common for image acquisition Φ also known as the reconstruction function 94

What was a convolution again? (t) (t) = = 95

Convolution the movie from wikipedia: 96

What was sampling again? f(t) s T (t) = 1X f(n T ) (t n T ) 1 1X 1 f[n] (t n T ) 97

Sampling in the Fourier Domain f(t)s T (t) () F (!) S T (!) F (!) S T (!) =F (!) 1 T = 1 T 1X n= 1 1X n= 1 F (! (! n T ) n T ) 98

Sampling in the Fourier Domain 1 T > 2! max 99

How do you properly recover? H(!) () Sinc(t) f(t) =f s (t) Sinc(t) 1X t n T = f[n]sinc n T n= 1 100

Aliasing 101

Aliasing 102

Aliasing 103

Next time repeating all this for 2D (designing) filters in the Fourier Domain 104