How can robots perceive the world and their own movements so that they accomplish navigation and manipulation tasks? In this module, we will study how images and videos acquired by cameras mounted on robots are transformed into representations like features and optical flow. Such 2D representations allow us then to extract 3D information about where the camera is and in which direction the robot moves. You will come to understand how grasping objects is facilitated by the computation of 3D posing of objects and navigation can be accomplished by visual odometry and landmark-based localization.
This course is part of the Robotics Specialization
Robotics: Perception
Offered By
About this Course
46,246 recent views
Learner Career Outcomes
40%
started a new career after completing these courses
29%
got a tangible career benefit from this course
Skills you will gain
Computer VisionEstimationRandom Sample Consensus (Ransac)Geometry
Learner Career Outcomes
40%
started a new career after completing these courses
29%
got a tangible career benefit from this course
Shareable Certificate
Earn a Certificate upon completion
100% online
Start instantly and learn at your own schedule.
Course 4 of 6 in the
Flexible deadlines
Reset deadlines in accordance to your schedule.
Intermediate Level
Approx. 29 hours to complete
Suggested: 4 weeks of study, 3-5 hours/week
English
Subtitles: English
Syllabus - What you will learn from this course
7 hours to complete
Geometry of Image Formation
Welcome to Robotics: Perception! We will begin this course with a tutorial on the standard camera models used in computer vision. These models allow us to understand, in a geometric fashion, how light from a scene enters a camera and projects onto a 2D image. By defining these models mathematically, we will be able understand exactly how a point in 3D corresponds to a point in the image and how an image will change as we move a camera in a 3D environment. In the later modules, we will be able to use this information to perform complex perception tasks such as reconstructing 3D scenes from video.
7 hours to complete
15 videos (Total 180 min), 1 reading, 9 quizzes
15 videos
Introduction10m
Camera Modeling10m
Single View Geometry14m
More on Perspective Projection8m
Glimpse on Vanishing Points10m
Perspective Projection I14m
Perspective Projection II14m
Point-Line Duality8m
Rotations and Translations18m
Pinhole Camera Model10m
Focal Length and Dolly Zoom Effect8m
Intrinsic Camera Parameter13m
3D World to First Person Transformation13m
How to Compute Intrinsics from Vanishing Points12m
Camera Calibration11m
1 reading
Setting up MATLAB10m
8 practice exercises
Introduction14m
Vanishing Points10m
Perspective Projection10m
Rotations and Translations14m
Dolly Zoom4m
Feeling of Camera Motion6m
How to Compute Intrinsics from Vanishing Points4m
Camera Calibration6m
5 hours to complete
Projective Transformations
Now that we have a good camera model, we will explore the geometry of perspective projections in depth. We will find that this projection is the cause of the main challenge in perception, as we lose a dimension that we can no longer directly observe. In this module, we will learn about several properties of projective transformations in depth, such as vanishing points, which allow us to infer complex information beyond our basic camera model.
5 hours to complete
5 videos (Total 69 min)
5 videos
Compute Projective Transformations13m
Projective Transformations and Vanishing Points6m
Cross Ratios and Single View Metrology13m
Two View Soccer Metrology11m
4 practice exercises
Homogeneous Coordinates10m
Projective Transformations8m
Vanishing Points8m
Cross Ratios and Single View Metrology8m
6 hours to complete
Pose Estimation
In this module we will be learning about feature extraction and pose estimation from two images. We will learn how to find the most salient parts of an image and track them across multiple frames (i.e. in a video sequence). We will then learn how to use features to find the position of the camera with respect to another reference frame on a plane using Homographies. We will also learn about how to make these techniques more robust, using least squares to hand noisy feature points or RANSAC to remove completely erroneous feature points.
6 hours to complete
8 videos (Total 126 min)
8 videos
Singular Value Decomposition30m
RANSAC: Random Sample Consensus I13m
Where am I? Part 116m
Where am I? Part 213m
Pose from 3D Point Correspondences: The Procrustes Problem9m
Pose from Projective Transformations8m
Pose from Point Correspondences P3P10m
5 practice exercises
Visual Features12m
Singular Value Decomposition16m
RANSAC6m
3D-3D Pose2m
Pose Estimation8m
8 hours to complete
Multi-View Geometry
Now we will use what we learned from two view geometry and extend it to sequences of images, such as a video. We will explain the fundamental geometric constraints between point features in images, the Epipolar constraint, and learn how to use it to extract the relative poses between multiple frames. We will finish by combining all this information together for the application of Structure from Motion, where we will compute the trajectory of a camera and a map throughout many frames and refine our estimates using Bundle adjustment.
8 hours to complete
14 videos (Total 221 min)
14 videos
Epipolar Geometry II14m
Epipolar Geometry III24m
RANSAC: Random Sample Consensus II6m
Nonlinear Least Squares I3m
Nonlinear Least Squares II6m
Nonlinear Least Squares III13m
Optical Flow: 2D Point Correspondences19m
3D Velocities from Optical Flow16m
3D Motion and Structure from Multiple Views18m
Visual Odometry19m
Bundle Adjustment I17m
Bundle Adjustment II18m
Bundle Adjustment III17m
4 practice exercises
Epipolar Geometry24m
Nonlinear Least Squares12m
3D Velocities from Optical Flow6m
Bundle Adjustment10m
Reviews
4.4
TOP REVIEWS FROM ROBOTICS: PERCEPTION
by ASNov 8, 2019
The content is quite useful but the teaching can be improved upon through shorter videos and more animations instead of hand gestures (or static images) to explain mathematical derivations.
by SGJun 16, 2017
The content is not very easy to understand because the lecture speaks very fast and the document is not very sufficient. But in all, the content is good, help me with my research.
by SKMar 31, 2018
Outstanding Course! I could always count on Prof.Jianbo to crunch some of the most complex and confusing parts of the course into a much easier understandable language.
by DYMar 27, 2020
The concepts were explained very well and clearly. The last week content seemed a bit complicated to follow, but it was not unsolvable. I enjoyed the course. Thank you!
by LLFeb 26, 2017
The 4th week content is hard to follow than the previous three. It would be better if more detailed math and examples are provided in the 4th week.
by LRDec 31, 2018
This is quite challenging course. So far, this is the course with the largest amount of material, I wish the class will be split into two courses.
by RSJun 18, 2016
Excellent organization and presentation of the course material, and very prompt responses from the teaching staff on the message boards.
by GGJun 17, 2016
Very good course. The only thing that I would suggest is an higher precence of moderators in the forum. It would be very appreciated.
by RKFeb 23, 2018
Course is nicely organized and helps even a novice without much in depth knowledge of image processing to understand the concepts
by JDNov 29, 2016
Awesome material! I think this is the one course of the specialization that had the appropriate amount of work for the timeline.
by SZOct 25, 2018
Subtitles are generated by machines I think. Very many subtitles are wrong. It is very unfriendly to non-English speaking users.
by JSJan 5, 2018
Extremely fast-paced course that gives a great overview of Perception but leaves a lot of things unexplained or without proofs.
by JBNov 14, 2016
It was a very good course, the only thing is the time, I think that was to short in order to cover all the topics more deeply.
by RAAug 20, 2017
Very interesting and useful course. Professors give a lot of information. However, some explanations are not very clear.
by JMJun 25, 2018
For Computer Vision enthusiast who wants to learn about Multiple View Geometry, this is the best beginner course
by SVOct 16, 2019
It is hard course, thoroughly enjoyed it. Lessons on how to effectively use vanishing points was very useful.
by TODec 9, 2016
Lots of good content, good explanations, and good pictures to explain things. I enjoyed the assignments too
by HBMar 1, 2019
One of the most usefful courses I have taken by the coursera. Thank you for useful materail covered here.
by MHJul 9, 2019
very useful course. However it needs some supplementary materials in math. also more solved examples.
by ANMay 12, 2016
Solid Material as an introductory course and gives glimpse on the new horizons on computer vision.
Offered by
University of Pennsylvania
The University of Pennsylvania (commonly referred to as Penn) is a private university, located in Philadelphia, Pennsylvania, United States. A member of the Ivy League, Penn is the fourth-oldest institution of higher education in the United States, and considers itself to be the first university in the United States with both undergraduate and graduate studies.
About the Robotics Specialization
The Introduction to Robotics Specialization introduces you to the concepts of robot flight and movement, how robots perceive their environment, and how they adjust their movements to avoid obstacles, navigate difficult terrains and accomplish complex tasks such as construction and disaster recovery. You will be exposed to real world examples of how robots have been applied in disaster situations, how they have made advances in human health care and what their future capabilities will be. The courses build towards a capstone in which you will learn how to program a robot to perform a variety of movements such as flying and grasping objects.
Frequently Asked Questions
When will I have access to the lectures and assignments?
Once you enroll for a Certificate, you’ll have access to all videos, quizzes, and programming assignments (if applicable). Peer review assignments can only be submitted and reviewed once your session has begun. If you choose to explore the course without purchasing, you may not be able to access certain assignments.
What will I get if I subscribe to this Specialization?
When you enroll in the course, you get access to all of the courses in the Specialization, and you earn a certificate when you complete the work. Your electronic Certificate will be added to your Accomplishments page - from there, you can print your Certificate or add it to your LinkedIn profile. If you only want to read and view the course content, you can audit the course for free.
What is the refund policy?
Is financial aid available?
More questions? Visit the Learner Help Center.