Compuer ercepual Vision and Sensor II Summer 4 Augmened Compuing Compuer ercepual Vision and Sensor II Summer 4 Augmened Compuing Compuer ercepual Vision and Sensor II Summer 4 Augmened Compuing Compuer ercepual Vision and Sensor II Summer 4 Augmened Compuing Compuer ercepual Vision and Sensor II Summer 4 Augmened Compuing Compuer ercepual Vision and Sensor II Summer 4 Augmened Compuing Course Ouline Compuer Vision II Lecure 8 Tracking wih Linear Dnamic Models.5.4 Single-Objec Tracking Background modeling Templae based racking Color based racking Conour based racking Tracking b online classificaion Tracking-b-deecion Basian Leibe RWTH Aachen hp://www.vision.rwh-aachen.de leibe@vision.rwh-aachen.de Baesian Filering Kalman filer aricle filer Muli-Objec Tracking Ariculaed Tracking Image source: Helmu Grabner Disne/ixar Recap: Tracking-b-Deecion Recap: Sliding-Window Objec Deecion Fleshing ou his pipeline a bi more we need o:. Obain raining daa. Define feaures 3. Define classifier Training examples Main ideas Appl a generic objec deecor o find objecs of a cerain class Based on he deecions exrac objec appearance models Link deecions ino rajecories Feaure exracion Car/non-car Classifier 3 Slide credi: Krisen Grauman 4 Recap: Objec Deecor Design Recap: Hisograms of Oriened Gradiens (HOG In pracice he classifier ofen deermines he design. Tpes of feaures Speedup sraegies We ll look a 3 sae-of-he-ar deecor designs Based on SVMs Las lecure Holisic objec represenaion Localized gradien orienaions [............] Objec/Non-objec Linear SVM Collec HOGs over deecion window Conras normalize over overlapping spaial cells Based on Boosing Las lecure Weighed voe in spaial & orienaion cells Based on Random Foress osponed o a laer slo... Compue gradiens Gamma compression Image Window 5 Slide adaped from Navnee Dalal 6
Compuer ercepual Vision and Sensor II Summer 4 Augmened Compuing Compuer ercepual Vision and Sensor II Summer 4 Augmened Compuing Compuer ercepual Vision and Sensor II Summer 4 Augmened Compuing Compuer ercepual Vision and Sensor II Summer 4 Augmened Compuing Compuer ercepual Vision and Sensor II Summer 4 Augmened Compuing Compuer ercepual Vision and Sensor II Summer 4 Augmened Compuing Recap: Deformable ar-based Model (DM Recap: DM Hpohesis Score Score of filer: do produc of filer wih HOG feaures underneah i Score of objec hpohesis is sum of filer scores minus deformaion coss Muliscale model capures feaures a wo resoluions Slide credi: edro Felzenszwalb 7 [Felzenszwalb McAlliser Ramanan CVR 8] Slide credi: edro Felzenszwalb 8 [Felzenszwalb McAlliser Ramanan CVR 8] Recap: Inegral Channel Feaures Recap: Inegral Channel Feaures Generalizaion of Haar Wavele idea from Viola-Jones Insead of onl considering inensiies also ake ino accoun oher feaure channels (gradien orienaions color exure. Sill efficienl represened as inegral images. Generalize also block compuaion s order feaures: Sum of pixels in recangular region. nd -order feaures: Haar-like difference of sum-over-blocks Generalized Haar: More complex combinaions of weighed recangles. Dollar Z. Tu. erona S. Belongie. Inegral Channel Feaures BMVC 9. 9 Hisograms Compued b evaluaing local sums on quanized images. Recap: VerFas Deecor Recap: VerFas Deecor Idea : Inver he relaion Idea : Reduce raining ime b feaure inerpolaion model 5 image scales 5 models image scale 5 models image scale 5 models image scale R. Benenson M. Mahias R. Timofe L. Van Gool. edesrian Deecion a Frames per Second CVR. Slide credi: Rodrigo Benenson Shown o be possible for Inegral Channel feaures. Dollár S. Belongie erona. The Fases edesrian Deecor in he Wes BMVC. Slide adaped from Rodrigo Benenson
Compuer ercepual Vision and Sensor II Summer 4 Augmened Compuing Compuer ercepual Vision and Sensor II Summer 4 Augmened Compuing Compuer ercepual Vision and Sensor II Summer 4 Augmened Compuing Compuer ercepual Vision and Sensor II Summer 4 Augmened Compuing Compuer ercepual Vision and Sensor II Summer 4 Augmened Compuing Compuer ercepual Vision and Sensor II Summer 4 Augmened Compuing Recap: VerFas Classifier Consrucion Elemens of Tracking + - + - + - + - + - + - score = w h + w h + +w N h N Ensemble of shor rees learned b AdaBoos Slide credi: Rodrigo Benenson 3 Deecion Daa associaion redicion Deecion Las lecure Where are candidae objecs? Daa associaion Which deecion corresponds o which objec? redicion Toda s opic Where will he racked objec be in he nex ime sep? 4 Toda: Tracking wih Linear Dnamic Models Topics of This Lecure Tracking wih Dnamics Deecion vs. Tracking Tracking as probabilisic inference redicion and Correcion Linear Dnamic Models Zero veloci model Consan veloci model Consan acceleraion model The Kalman Filer Kalman filer for D sae General Kalman filer Limiaions 5 Figure from Forsh & once 6 Tracking wih Dnamics General Model for Tracking Ke idea Given a model of expeced moion predic where objecs will occur in nex frame even before seeing he image. Represenaion The moving objec of ineres is characerized b an underling sae. Sae gives rise o measuremens or observaions Y. Goals Resric search for he objec Improved esimaes since measuremen noise is reduced b rajecor smoohness. Assumpion: coninuous moion paerns Camera is no moving insanl o new viewpoin. Objecs do no disappear and reappear in differen places. Gradual change in pose beween camera and scene. A each ime he sae changes o and we ge a new observaion Y. Y Y Y Slide adaped from S. Lazebnik K. Grauman 7 Slide credi: Svelana Lazebnik 8 3
Compuer ercepual Vision and Sensor II Summer 4 Augmened Compuing Compuer ercepual Vision and Sensor II Summer 4 Augmened Compuing Compuer ercepual Vision and Sensor II Summer 4 Augmened Compuing Compuer ercepual Vision and Sensor II Summer 4 Augmened Compuing Compuer ercepual Vision and Sensor II Summer 4 Augmened Compuing Compuer ercepual Vision and Sensor II Summer 4 Augmened Compuing Sae vs. Observaion Tracking as Inference Inference problem The hidden sae consiss of he rue parameers we care abou denoed. The measuremen is our nois observaion ha resuls from he underling sae denoed Y. A each ime sep sae changes (from - o and we ge a new observaion Y. Hidden sae : parameers of ineres Measuremen: wha we ge o direcl observe Our goal: recover mos likel sae given All observaions seen so far. Knowledge abou dnamics of sae ransiions. Slide credi: Krisen Grauman 9 Slide credi: Krisen Grauman Seps of Tracking Simplifing Assumpions redicion: Wha is he nex sae of he objec given pas measuremens? Y Y Correcion: Compue an updaed esimae of he sae from predicion and measuremens. Y Y Y Onl he immediae pas maers Dnamics model Measuremens depend onl on he curren sae Y Y Y Y Tracking can be seen as he process of propagaing he poserior disribuion of sae given measuremens across ime. Observaion model Slide credi: Svelana Lazebnik Y Y Y Tracking as Inducion Tracking as Inducion Base case: Assume we have iniial prior ha predics sae in absence of an evidence: ( Base case: Assume we have iniial prior ha predics sae in absence of an evidence: ( A he firs frame correc his given he value of Y = ( ( ( Y ( ( ( A he firs frame correc his given he value of Y = Given correced esimae for frame : redic for frame + Correc for frame + oserior prob. of sae given measuremen Likelihood of measuremen rior of he sae predic correc Slide credi: Svelana Lazebnik 3 Slide credi: Svelana Lazebnik 4 4
5 ercepual and Sensor Augmened Compuing Compuer Vision II Summer 4 Inducion Sep: redicion redicion involves represening given 5 d d d Law of oal probabili Slide credi: Svelana Lazebnik A A B db ercepual and Sensor Augmened Compuing Compuer Vision II Summer 4 Inducion Sep: redicion redicion involves represening given 6 d d d Slide credi: Svelana Lazebnik Condiioning on A B A B B ercepual and Sensor Augmened Compuing Compuer Vision II Summer 4 Inducion Sep: redicion redicion involves represening given 7 d d d Slide credi: Svelana Lazebnik Independence assumpion ercepual and Sensor Augmened Compuing Compuer Vision II Summer 4 Inducion Sep: Correcion Correcion involves compuing given prediced value 8 d Baes rule B A A A B B Slide credi: Svelana Lazebnik ercepual and Sensor Augmened Compuing Compuer Vision II Summer 4 Inducion Sep: Correcion Correcion involves compuing given prediced value 9 d Independence assumpion (observaion depends onl on sae Slide credi: Svelana Lazebnik ercepual and Sensor Augmened Compuing Compuer Vision II Summer 4 Inducion Sep: Correcion Correcion involves compuing given prediced value 3 d Slide credi: Svelana Lazebnik Condiioning on
Compuer ercepual Vision and Sensor II Summer 4 Augmened Compuing Compuer ercepual Vision and Sensor II Summer 4 Augmened Compuing Compuer ercepual Vision and Sensor II Summer 4 Augmened Compuing Compuer ercepual Vision and Sensor II Summer 4 Augmened Compuing Compuer ercepual Vision and Sensor II Summer 4 Augmened Compuing Compuer ercepual Vision and Sensor II Summer 4 Augmened Compuing Summar: redicion and Correcion Summar: redicion and Correcion redicion: redicion: d d Dnamics model Correced esimae from previous sep Dnamics model Correced esimae from previous sep Correcion: Observaion model rediced esimae d Slide credi: Svelana Lazebnik 3 Slide credi: Svelana Lazebnik 3 Topics of This Lecure Tracking wih Dnamics Deecion vs. Tracking Tracking as probabilisic inference redicion and Correcion Linear Dnamic Models Zero veloci model Consan veloci model Consan acceleraion model The Kalman Filer Kalman filer for D sae General Kalman filer Limiaions Noaion Reminder x ~ N( μ Σ Random variable wih Gaussian probabili disribuion ha has he mean vecor ¹ and covariance marix. x and ¹ are d-dimensional is d d. d= d= If x is D we jus have one parameer: he variance ¾ 33 Slide credi: Krisen Grauman 34 Linear Dnamic Models Example: Randoml Drifing oins Dnamics model Sae undergoes linear ranformaion D plus Gaussian noise n nn n Observaion model x ~ N D x d Measuremen is linearl ransformed sae plus Gaussian noise ~ N M x m Consider a saionar objec wih sae as posiion. osiion is consan onl moion due o random noise erm. x p p p Sae evoluion is described b ideni marix D=I x D x noise Ip noise m mn n Slide credi: S. Lazebnik K. Grauman 35 Slide credi: Krisen Grauman 36 6
Compuer ercepual Vision and Sensor II Summer 4 Augmened Compuing Compuer ercepual Vision and Sensor II Summer 4 Augmened Compuing Compuer ercepual Vision and Sensor II Summer 4 Augmened Compuing Compuer ercepual Vision and Sensor II Summer 4 Augmened Compuing Compuer ercepual Vision and Sensor II Summer 4 Augmened Compuing Compuer ercepual Vision and Sensor II Summer 4 Augmened Compuing Example: Consan Veloci (D oins Example: Consan Veloci (D oins Measuremens Sae vecor: posiion p and veloci v p p p ( v x v v v p x D x noise noise v (greek leers denoe noise erms Saes Measuremen is posiion onl p Mx noise v noise ime Slide credi: Krisen Grauman 37 Figure from Forsh & once Slide credi: S. Lazebnik K. Grauman 38 Example: Consan Acceleraion (D oins Example: Consan Acceleraion (D oins Sae vecor: posiion p veloci v and acceleraion a. p x v a p p v v a a ( v ( a (greek leers denoe noise erms Slide credi: Krisen Grauman 4 Figure from Forsh & once p x D x noise v noise a Measuremen is posiion onl p Mx noise a Slide credi: S. Lazebnik K. Grauman v noise 4 Example: General Moion Models Topics of This Lecure Assuming we have differenial equaions for he moion E.g. for (undampened periodic moion of a pendulum d p p d Subsiue variables o ransform his ino linear ssem dp d p p p p p3 d d Then we have p x p p 3 p p ( p p p ( p 3 p p 3 D Tracking wih Dnamics Deecion vs. Tracking Tracking as probabilisic inference redicion and Correcion Linear Dnamic Models Zero veloci model Consan veloci model Consan acceleraion model The Kalman Filer Kalman filer for D sae General Kalman filer Limiaions 43 44 7
Compuer ercepual Vision and Sensor II Summer 4 Augmened Compuing Compuer ercepual Vision and Sensor II Summer 4 Augmened Compuing Compuer ercepual Vision and Sensor II Summer 4 Augmened Compuing Compuer ercepual Vision and Sensor II Summer 4 Augmened Compuing Compuer ercepual Vision and Sensor II Summer 4 Augmened Compuing Compuer ercepual Vision and Sensor II Summer 4 Augmened Compuing The Kalman Filer Kalman filer Mehod for racking linear dnamical models in Gaussian noise The prediced/correced sae disribuions are Gaussian You onl need o mainain he mean and covariance. The calculaions are eas (all he inegrals can be done in closed form. The Kalman Filer Know correced sae from previous ime sep and all measuremens up o he curren one redic disribuion over nex sae. Mean and sd. dev. of prediced sae: Time updae ( redic Receive measuremen Time advances: ++ Know predicion of sae and nex measuremen Updae disribuion over curren sae. Measuremen updae ( Correc Mean and sd. dev. of correced sae: Slide credi: Svelana Lazebnik 45 Slide credi: Krisen Grauman 46 Kalman Filer for D Sae ropagaion of Gaussian densiies Wan o represen and updae x N ( x N ( Shifing he mean Baesian combinaion Increasing he variance 47 Slide credi: Svelana Lazebnik D Kalman Filer: redicion D Kalman Filer: Correcion Have linear dnamic model defining prediced sae evoluion wih noise ~ N dx d Wan o esimae prediced disribuion for nex sae N ( Updae he mean: d Updae he variance: ( Slide credi: Krisen Grauman ( d d for derivaions see F& Chaper 7.3 49 Have linear model defining he mapping of sae o measuremens: Y N Wan o esimae correced disribuion given laes measuremen: N ( Updae he mean: Updae he variance: m ( m m ( 5 Slide credi: Krisen Grauman Derivaions: F& Chaper 7.3 ~ mx m m ( m ( m m ( 8
Compuer ercepual Vision and Sensor II Summer 4 Augmened Compuing Compuer ercepual Vision and Sensor II Summer 4 Augmened Compuing Compuer ercepual Vision and Sensor II Summer 4 Augmened Compuing Compuer ercepual Vision and Sensor II Summer 4 Augmened Compuing Compuer ercepual Vision and Sensor II Summer 4 Augmened Compuing Compuer ercepual Vision and Sensor II Summer 4 Augmened Compuing posiion redicion vs. Correcion Recall: Consan Veloci Example m ( ( m m ( m m ( m m ( Wha if here is no predicion uncerain ( The measuremen is ignored! (? Wha if here is no measuremen uncerain ( m? ( Slide credi: Krisen Grauman m The predicion is ignored! 5 measuremens sae Slide credi: Krisen Grauman ime Sae is D: posiion + veloci Measuremen is D: posiion 5 Figure from Forsh & once Consan Veloci Model Consan Veloci Model o sae o sae x measuremen x measuremen * prediced mean esimae * prediced mean esimae + correced mean esimae + correced mean esimae bars: variance esimaes before and afer measuremens bars: variance esimaes before and afer measuremens Slide credi: Krisen Grauman 53 Figure from Forsh & once Slide credi: Krisen Grauman 54 Figure from Forsh & once Consan Veloci Model Consan Veloci Model o sae o sae x measuremen x measuremen * prediced mean esimae * prediced mean esimae + correced mean esimae + correced mean esimae bars: variance esimaes before and afer measuremens bars: variance esimaes before and afer measuremens Slide credi: Krisen Grauman 55 Figure from Forsh & once Slide credi: Krisen Grauman 56 Figure from Forsh & once 9
Compuer ercepual Vision and Sensor II Summer 4 Augmened Compuing Compuer ercepual Vision and Sensor II Summer 4 Augmened Compuing Compuer ercepual Vision and Sensor II Summer 4 Augmened Compuing Compuer ercepual Vision and Sensor II Summer 4 Augmened Compuing Compuer ercepual Vision and Sensor II Summer 4 Augmened Compuing Kalman Filer: General Case (>dim Summar: Kalman Filer Wha if sae vecors have more han one dimension? REDICT CORRECT T T x D x K M M M m residual T D D x x K d Mx I KM ros: Gaussian densiies everwhere Simple updaes compac and efficien Ver esablished mehod ver well undersood Cons: Unimodal disribuion onl single hpohesis Resriced class of moions defined b linear model More weigh on residual when measuremen error covariance approaches. for derivaions see F& Chaper 7.3 Slide credi: Krisen Grauman Less weigh on residual as a priori esimae error covariance approaches. 57 Slide adaped from Svelana Lazebnik 58 Wh Is This A Resricion? Man ineresing cases don have linear dnamics E.g. pedesrians walking Ball Example: Wha Goes Wrong Here? Assuming consan acceleraion model redicion E.g. a ball bouncing 59 redicion is oo far from rue posiion o compensae ossible soluion: Keep muliple models Keep muliple differen moion models in parallel I.e. would check for bouncing a each ime sep redicion redicion 3 redicion 4 redicion 5 Correc predicion 6 References and Furher Reading A ver good inroducion o racking wih linear dnamic models and Kalman filers can be found in Chaper 7 of D. Forsh J. once Compuer Vision A Modern Approach. renice Hall 3 6