CS 1699: Intro to Computer Vision. Introduction. Prof. Adriana Kovashka University of Pittsburgh September 1, 2015

Transcription

1 CS 1699: Intro to Computer Vision Introduction Prof. Adriana Kovashka University of Pittsburgh September 1, 2015

2 Course Info Course website: Instructor: Adriana Kovashka Please use "CS1699" at the beginning of your Subject Office: Sennott Square 5325 Office hours: Tuesday and Thursday, 4pm-5pm Grader: Nils Murrugarra-Llerena

3 Textbooks Computer Vision: Algorithms and Applications by Richard Szeliski Visual Object Recognition by Kristen Grauman and Bastian Leibe

4 Course Goals To learn about the basic computer vision tasks and approaches To get experience with some computer vision techniques To learn absolute basics of machine learning To think critically about vision approaches, and to see connections between works and potential for improvement

5 Plan for Today Introductions What is computer vision? Why do we care? What are the challenges? Course structure and policies Overview of topics

6 Introductions

7 Introductions What is your name? What is your department, major and year? What one thing outside of school are you passionate about? What do you hope to get out of this class? What do you plan to do when you graduate?

8 Computer Vision

9 What is computer vision? Done? "We see with our brains, not with our eyes (Oliver Sacks and others) Kristen Grauman (adapted)

10 What is Computer Vision? Automatic understanding of images and video Computing properties of the 3D world from visual data (measurement) Algorithms and representations to allow a machine to recognize objects, people, scenes, and activities (perception and interpretation) Algorithms to mine, search, and interact with visual data (search and organization) Kristen Grauman

11 Vision for measurement Real-time stereo Structure from motion Multi-view stereo for community photo collections NASA Mars Rover Pollefeys et al. Goesele et al. Kristen Grauman Slide credit: L. Lazebnik

12 Vision for perception, interpretation The Wicked Twister ride Lake Erie sky water Ferris wheel amusement park Cedar Point tree ride 12 E Objects Activities Scenes Locations Text / writing Faces Gestures Motions Emotions ride tree people waiting in line people sitting on ride Kristen Grauman deck tree bench tree carousel umbrellas pedestrians maxair

13 Visual search, organization Query Image or video archives Relevant content Kristen Grauman

14 Related disciplines Graphics Image processing Artificial intelligence Computer vision Algorithms Machine learning Cognitive science Kristen Grauman

15 Vision and graphics Images Vision Model Graphics Inverse problems: analysis and synthesis. Kristen Grauman

16 Why vision? As image sources multiply, so do applications Relieve humans of boring, easy tasks Human-computer interaction Perception for robotics / autonomous agents Organize and give access to visual content Kristen Grauman

17 Why vision? Images and video are everywhere! Personal photo albums Movies, news, sports Lana Lazebnik Surveillance and security Medical and scientific images

18 Faces and digital cameras Camera waits for everyone to smile to take a photo [Canon] Setting camera focus via face detection Kristen Grauman

19 Devi Parikh Face recognition

20 Linking to info with a mobile device Situated search Yeh et al., MIT kooaba MSR Lincoln Kristen Grauman

21 Exploring photo collections Snavely et al. Kristen Grauman

22 Special visual effects The Matrix What Dreams May Come Mocap for Pirates of the Carribean, Industrial Light and Magic Source: S. Seitz Kristen Grauman

23 Yong Jae Lee Interactive systems

24 Video-based interfaces Human joystick NewsBreaker Live Assistive technology systems Camera Mouse Boston College Kristen Grauman

25 Vision for medical & neuroimages fmri data Golland et al. Image guided surgery MIT AI Vision Group Kristen Grauman

26 Safety & security Navigation, driver safety Monitoring pool (Poseidon) Kristen Grauman Pedestrian detection MERL, Viola et al. Surveillance

27 Kristen Grauman Obstacles?

28 Kristen Grauman What the computer gets

29 Why is vision difficult? Ill-posed problem: real world much more complex than what we can measure in images 3D 2D Impossible to literally invert image formation process Kristen Grauman

30 Challenges: many nuisance parameters Illumination Object pose Clutter Occlusions Intra-class appearance Viewpoint Kristen Grauman

31 Challenges: intra-class variation slide credit: Fei-Fei, Fergus & Torralba

32 Challenges: importance of context slide credit: Fei-Fei, Fergus & Torralba

33 Challenges: complexity Thousands to millions of pixels in an image 3,000-30,000 human recognizable object categories 30+ degrees of freedom in the pose of articulated objects (humans) Billions of images indexed by Google Image Search 18 billion+ prints produced from digital camera images in million camera phones sold in 2005 About half of the cerebral cortex in primates is devoted to processing visual information [Felleman and van Essen 1991] Kristen Grauman

34 Challenges: Limited supervision Less More Kristen Grauman

35 Ok, clearly the vision problem is deep and challenging time to give up? Active research area with exciting progress! Kristen Grauman

36 Datasets today ImageNet: 22k categories, 14mil images Microsoft COCO: 70 categories, 300k images PASCAL: 20 categories, 12k images SUN: 5k categories, 130k images

37 Some Visual Recognition Problems

38 Recognition: What is this?

39 Recognition: What is this? building street balcony truck carriage horse table person person car

40 Detection: Where are the cars?

41 Activity: What is this person doing?

42 Scene: Is this an indoor scene?

43 Instance: Which city? Which building?

44 Joo et al., Visual Persuasion: Inferring Communicative Intents of Images, CVPR 2014 Visual Persuasion

45 Pirsiavash et al., Assessing the Quality of Actions, ECCV 2014 Evaluating Action Quality

46 Gatys et al., A Neural Algorithm of Artistic Style, ArXiv 2015 Transferring Art Style

47 Answering Visual Questions Antol et al., VQA: Visual Question Answering, ArXiv 2015

48 Course Structure and Policies

49 Course Components 50% homework (5 problem sets) 20% midterm exam 20% final exam 10% in-class participation

50 Course Schedule

51 Homework Submission We will use CourseWeb Navigate to the CourseWeb page for CS1699, click on "Assignments" and the corresponding HW # Attach a zip file with your written responses and code Name the file as YourFirstName_YourLastName.zip or YourFirstName_YourLastName.tar Homework is due at 11:59pm on the due Grades will appear on CourseWeb

52 Exams One mid-term and one final exam The final exam will focus on the latter half of the course Exams will be preceded by review sessions (if our schedule allows it) Exam is tentatively scheduled for Monday, December 12pm

53 Participation 10% of grade will be based on attendance and participation Two free absences; let me know and explain beyond that Answer questions asked by instructor and others Ask meaningful questions Bring in relevant articles about recent developments in computer vision Feedback is welcome!

54 Late Policy You get 3 "free" late days, i.e., you can submit homework a total of 3 days late. For example, you can submit one problem set 12 hours late, and another 60 hours late. Once you've used up your free late days, you will incur a penalty of 25% from the total project credit possible for each late day. A late day is anything from 1 minute to 24 hours.

55 Collaboration Policy You will work individually. The work you turn in must be your own work. You can discuss the problem sets with your classmates, but do not look at their code. You cannot use posted solutions, search for code on the internet or use Matlab's implementations of something you are asked to write. When in doubt, ask the instructor! Plagiarism will cause you to fail the class and receive disciplinary penalty.

56 Disabilities If you have a disability for which you are or may be requesting an accommodation, you are encouraged to contact both your instructor and Disability Resources and Services (DRS), 140 William Pitt Union, (412) , drsrecep@pitt.edu, (412) for P3 ASL users, as early as possible in the term. DRS will verify your disability and determine reasonable accommodations for this course.

57 Medical Conditions If you have a medical condition which will prevent you from doing a certain assignment or coming to class, you must inform the instructor of this before the deadline. You must then submit documentation of your condition within a week of the assignment deadline.

58 Questions?

59 Overview of Topics

60 Features and filters Transforming and describing images; textures, colors, edges Kristen Grauman

61 Features and filters Detecting repeatable features Describing images with local statistics

62 Indexing and search Matching features and regions across images Kristen Grauman

63 Grouping and fitting Clustering, segmentation, fitting; what parts belong together? Kristen Grauman [fig from Shi et al]

64 How does light in 3d world project to form 2d images? Image formation Kristen Grauman

65 Multiple views Multi-view geometry, matching, invariant features, stereo vision Lowe Hartley and Zisserman Fei-Fei Li Kristen Grauman

66 Visual recognition Recognizing objects and categories, learning techniques Kristen Grauman

67 Object detection Detecting novel instances of objects Classifying regions as one of several categories

68 Attribute-based search Describing the high-level properties of objects Searching for objects with relative attributes

69 Crowdsourcing annotations Using non-expert labelers to collect data Actively requesting labels

70 Deep learning AlexNet Google s Inceptionism

71 Motion and tracking Tracking objects, video analysis, low level motion, optical flow Tomas Izo Kristen Grauman

72 Pose and actions Automatically annotating a human s pose Recognizing actions in first-person video

73 Next Time Matlab tutorial HW1 out