About this course: How can robots perceive the world and their own movements so that they accomplish navigation and manipulation tasks? In this module, we will study how images and videos acquired by cameras mounted on robots are transformed into representations like features and optical flow. Such 2D representations allow us then to extract 3D information about where the camera is and in which direction the robot moves. You will come to understand how grasping objects is facilitated by the computation of 3D posing of objects and navigation can be accomplished by visual odometry and landmark-based localization.
Created by: University of Pennsylvania
Taught by: Kostas Daniilidis, Professor of Computer and Information Science
School of Engineering and Applied Science
Taught by: Jianbo Shi, Professor of Computer and Information Science
School of Engineering and Applied Science
| Basic Info | Course 4 of 6 in the Robotics Specialization |
| Level | Intermediate |
| Commitment | 4 weeks of study, 3-5 hours/week |
| Language | English |
| How To Pass | Pass all graded assignments to complete the course. |
| User Ratings |
Syllabus
WEEK 1
Geometry of Image Formation
Welcome to Robotics: Perception! We will begin this course with a tutorial on the standard camera models used in computer vision. These models allow us to understand, in a geometric fashion, how light from a scene enters a camera and projects onto a 2D image. By defining these models mathematically, we will be able understand exactly how a point in 3D corresponds to a point in the image and how an image will change as we move a camera in a 3D environment. In the later modules, we will be able to use this information to perform complex perception tasks such as reconstructing 3D scenes from video.
15 videos, 1 reading
- Video: Introduction
- Video: Camera Modeling
- Video: Single View Geometry
- Video: More on Perspective Projection
- Video: Glimpse on Vanishing Points
- Video: Perspective Projection I
- Video: Perspective Projection II
- Video: Point-Line Duality
- Video: Rotations and Translations
- Video: Pinhole Camera Model
- Video: Focal Length and Dolly Zoom Effect
- Video: Intrinsic Camera Parameter
- Video: 3D World to First Person Transformation
- Video: How to Compute Intrinsics from Vanishing Points
- Video: Camera Calibration
- 阅读: Setting up MATLAB
Graded: Introduction
Graded: Vanishing Points
Graded: Perspective Projection
Graded: Rotations and Translations
Graded: Dolly Zoom
Graded: Feeling of Camera Motion
Graded: How to Compute Intrinsics from Vanishing Points
Graded: Camera Calibration
Graded: Dolly Zoom
WEEK 2
Projective Transformations
Now that we have a good camera model, we will explore the geometry of perspective projections in depth. We will find that this projection is the cause of the main challenge in perception, as we lose a dimension that we can no longer directly observe. In this module, we will learn about several properties of projective transformations in depth, such as vanishing points, which allow us to infer complex information beyond our basic camera model.
5 videos
- Video: Vanishing Points; How to Compute Camera Orientation
- Video: Compute Projective Transformations
- Video: Projective Transformations and Vanishing Points
- Video: Cross Ratios and Single View Metrology
- Video: Two View Soccer Metrology
Graded: Homogeneous Coordinates
Graded: Projective Transformations
Graded: Vanishing Points
Graded: Cross Ratios and Single View Metrology
Graded: Image Projection using Homographies
WEEK 3
Pose Estimation
In this module we will be learning about feature extraction and pose estimation from two images. We will learn how to find the most salient parts of an image and track them across multiple frames (i.e. in a video sequence). We will then learn how to use features to find the position of the camera with respect to another reference frame on a plane using Homographies. We will also learn about how to make these techniques more robust, using least squares to hand noisy feature points or RANSAC to remove completely erroneous feature points.
8 videos
- Video: Visual Features
- Video: Singular Value Decomposition
- Video: RANSAC: Random Sample Consensus I
- Video: Where am I? Part 1
- Video: Where am I? Part 2
- Video: Pose from 3D Point Correspondences: The Procrustes Problem
- Video: Pose from Projective Transformations
- Video: Pose from Point Correspondences P3P
Graded: Visual Features
Graded: Singular Value Decomposition
Graded: RANSAC
Graded: 3D-3D Pose
Graded: Pose Estimation
Graded: Image Projection
WEEK 4
Multi-View Geometry
Now we will use what we learned from two view geometry and extend it to sequences of images, such as a video. We will explain the fundamental geometric constraints between point features in images, the Epipolar constraint, and learn how to use it to extract the relative poses between multiple frames. We will finish by combining all this information together for the application of Structure from Motion, where we will compute the trajectory of a camera and a map throughout many frames and refine our estimates using Bundle adjustment.
14 videos
- Video: Epipolar Geometry I
- Video: Epipolar Geometry II
- Video: Epipolar Geometry III
- Video: RANSAC: Random Sample Consensus II
- Video: Nonlinear Least Squares I
- Video: Nonlinear Least Squares II
- Video: Nonlinear Least Squares III
- Video: Optical Flow: 2D Point Correspondences
- Video: 3D Velocities from Optical Flow
- Video: 3D Motion and Structure from Multiple Views
- Video: Visual Odometry
- Video: Bundle Adjustment I
- Video: Bundle Adjustment II
- Video: Bundle Adjustment III
Graded: Epipolar Geometry
Graded: Nonlinear Least Squares
Graded: 3D Velocities from Optical Flow
Graded: Bundle Adjustment
Graded: Structure from Motion
FAQs
How It Works
课程作业
每门课程都像是一本互动的教科书，具有预先录制的视频、测验和项目。
来自同学的帮助
与其他成千上万的学生相联系，对想法进行辩论，讨论课程材料，并寻求帮助来掌握概念。
证书
获得正式认证的作业，并与朋友、同事和雇主分享您的成功。
Creators
University of Pennsylvania
The University of Pennsylvania (commonly referred to as Penn) is a private university, located in Philadelphia, Pennsylvania, United States. A member of the Ivy League, Penn is the fourth-oldest institution of higher education in the United States, and considers itself to be the first university in the United States with both undergraduate and graduate studies.
Pricing
| Purchase Course | |
|---|---|
| Access to course materials | Available |
| Access to graded materials | Available |
| Receive a final grade | Available |
| Earn a shareable Course Certificate | Available |
Ratings and Reviews
Outstanding Course! I could always count on Prof.Jianbo to crunch some of the most complex and confusing parts of the course into a much easier understandable language.
Excellent Course, at the beginning I was a beginner in this topic and now I learned a lot about computer vision and visual perception. That is a fundamental part of mobil robot localization and planning. As a feedback, I only recomend more numerical examples, this would help to understand more quickly the topics.
Course is nicely organized and helps even a novice without much in depth knowledge of image processing to understand the concepts
Coursera provides universal access to the world’s best education,
partnering with top universities and organizations to offer courses online.
© 2018 Coursera Inc. All rights reserved.
肖
It's a wonderful course! I got lots of view in 3D vision field.