About this course: How can robots perceive the world and their own movements so that they accomplish navigation and manipulation tasks? In this module, we will study how images and videos acquired by cameras mounted on robots are transformed into representations like features and optical flow. Such 2D representations allow us then to extract 3D information about where the camera is and in which direction the robot moves. You will come to understand how grasping objects is facilitated by the computation of 3D posing of objects and navigation can be accomplished by visual odometry and landmark-based localization.
Created by: University of Pennsylvania
Taught by: Kostas Daniilidis, Professor of Computer and Information Science
School of Engineering and Applied Science
Taught by: Jianbo Shi, Professor of Computer and Information Science
School of Engineering and Applied Science
| Basic Info | Course 4 of 6 in the Robotics Specialization |
| Level | Intermediate |
| Commitment | 4 weeks of study, 3-5 hours/week |
| Language | English |
| How To Pass | Pass all graded assignments to complete the course. |
| User Ratings |
Syllabus
WEEK 1
Geometry of Image Formation
Welcome to Robotics: Perception! We will begin this course with a tutorial on the standard camera models used in computer vision. These models allow us to understand, in a geometric fashion, how light from a scene enters a camera and projects onto a 2D image. By defining these models mathematically, we will be able understand exactly how a point in 3D corresponds to a point in the image and how an image will change as we move a camera in a 3D environment. In the later modules, we will be able to use this information to perform complex perception tasks such as reconstructing 3D scenes from video.
15 videos, 1 reading
- Video: Introduction
- Video: Camera Modeling
- Video: Single View Geometry
- Video: More on Perspective Projection
- Video: Glimpse on Vanishing Points
- Video: Perspective Projection I
- Video: Perspective Projection II
- Video: Point-Line Duality
- Video: Rotations and Translations
- Video: Pinhole Camera Model
- Video: Focal Length and Dolly Zoom Effect
- Video: Intrinsic Camera Parameter
- Video: 3D World to First Person Transformation
- Video: How to Compute Intrinsics from Vanishing Points
- Video: Camera Calibration
- Reading: Setting up MATLAB
Graded: Introduction
Graded: Vanishing Points
Graded: Perspective Projection
Graded: Rotations and Translations
Graded: Dolly Zoom
Graded: Feeling of Camera Motion
Graded: How to Compute Intrinsics from Vanishing Points
Graded: Camera Calibration
Graded: Dolly Zoom
WEEK 2
Projective Transformations
Now that we have a good camera model, we will explore the geometry of perspective projections in depth. We will find that this projection is the cause of the main challenge in perception, as we lose a dimension that we can no longer directly observe. In this module, we will learn about several properties of projective transformations in depth, such as vanishing points, which allow us to infer complex information beyond our basic camera model.
5 videos
- Video: Vanishing Points; How to Compute Camera Orientation
- Video: Compute Projective Transformations
- Video: Projective Transformations and Vanishing Points
- Video: Cross Ratios and Single View Metrology
- Video: Two View Soccer Metrology
Graded: Homogeneous Coordinates
Graded: Projective Transformations
Graded: Vanishing Points
Graded: Cross Ratios and Single View Metrology
Graded: Image Projection using Homographies
WEEK 3
Pose Estimation
In this module we will be learning about feature extraction and pose estimation from two images. We will learn how to find the most salient parts of an image and track them across multiple frames (i.e. in a video sequence). We will then learn how to use features to find the position of the camera with respect to another reference frame on a plane using Homographies. We will also learn about how to make these techniques more robust, using least squares to hand noisy feature points or RANSAC to remove completely erroneous feature points.
8 videos
- Video: Visual Features
- Video: Singular Value Decomposition
- Video: RANSAC: Random Sample Consensus I
- Video: Where am I? Part 1
- Video: Where am I? Part 2
- Video: Pose from 3D Point Correspondences: The Procrustes Problem
- Video: Pose from Projective Transformations
- Video: Pose from Point Correspondences P3P
Graded: Visual Features
Graded: Singular Value Decomposition
Graded: RANSAC
Graded: 3D-3D Pose
Graded: Pose Estimation
Graded: Image Projection
WEEK 4
Multi-View Geometry
Now we will use what we learned from two view geometry and extend it to sequences of images, such as a video. We will explain the fundamental geometric constraints between point features in images, the Epipolar constraint, and learn how to use it to extract the relative poses between multiple frames. We will finish by combining all this information together for the application of Structure from Motion, where we will compute the trajectory of a camera and a map throughout many frames and refine our estimates using Bundle adjustment.
14 videos
- Video: Epipolar Geometry I
- Video: Epipolar Geometry II
- Video: Epipolar Geometry III
- Video: RANSAC: Random Sample Consensus II
- Video: Nonlinear Least Squares I
- Video: Nonlinear Least Squares II
- Video: Nonlinear Least Squares III
- Video: Optical Flow: 2D Point Correspondences
- Video: 3D Velocities from Optical Flow
- Video: 3D Motion and Structure from Multiple Views
- Video: Visual Odometry
- Video: Bundle Adjustment I
- Video: Bundle Adjustment II
- Video: Bundle Adjustment III
Graded: Epipolar Geometry
Graded: Nonlinear Least Squares
Graded: 3D Velocities from Optical Flow
Graded: Bundle Adjustment
Graded: Structure from Motion
FAQs
How It Works
Coursework
Each course is like an interactive textbook, featuring pre-recorded videos, quizzes and projects.
Help from Your Peers
Connect with thousands of other learners and debate ideas, discuss course material, and get help mastering concepts.
Certificates
Earn official recognition for your work, and share your success with friends, colleagues, and employers.
Creators
University of Pennsylvania
The University of Pennsylvania (commonly referred to as Penn) is a private university, located in Philadelphia, Pennsylvania, United States. A member of the Ivy League, Penn is the fourth-oldest institution of higher education in the United States, and considers itself to be the first university in the United States with both undergraduate and graduate studies.
Pricing
| Audit | Purchase Course | |
|---|---|---|
| Access to course materials | Available | Available |
| Access to graded materials | Not available | Available |
| Receive a final grade | Not available | Available |
| Earn a shareable Course Certificate | Not available | Available |
Ratings and Reviews
Great lectures! I felt a little confused at the beginning , but everything makes sense by the end of class.
This course is interesting and very thorough. Some concepts of robot perception are explained in detail, with a focus on perception based on 2D vision. The videos are clear and there is a great number of quizzes and Matlab programming to improve your practical understanding of the topic. Be warned, though, that this course takes longer than 4 weeks in fact due to the numerous and long lectures.
The course is very important for any student / engineer working in the field of robotics. It gives a lot of detailed information about the background needed as well as some hands-on experience with the basic tools in computer vision. A very good point is connecting what we study in the course with some real applications.
Coursera provides universal access to the world’s best education, partnering with top universities and organizations to offer courses online.
© 2017 Coursera Inc. All rights reserved.
Excellent course!! Especially Bundle Adjustment was covered in good details.