[SOLVED] IJCV 2004]

Structure from motion

What is Point Cloud
A collection of Un-ordered points with Geometry: expressed as [x, y, z]

Copyright By Assignmentchef assignmentchef

Color Attributes: [r g b], or [y u v]
Additional info: normal, timestamp, etc.
Key difference from mesh: no order info

Point Cloud Capture
Passive: Camera array stereo depth sensor
Active: LiDAR, mmWave, TOF sensors

Readings Szeliski, Chapter 7.1 7.4

Shape From X
Recovery of 3D (shape) from one or two 2D images

Structure from motion
Given many images, how can we
a) figure out where they were all taken from? b) build a 3D model of the scene?
This is (roughly) the structure from motion problem

Applications
Object Recognition Robotics
Computer Graphics Image Retrieval
Localization https://www.youtube.com/watch?v=p16frKJLVi0

Structure from motion
Reconstruction (side)
Input: images with points in correspondence
pi,j = (ui,j,vi,j)
structure: 3D location xi for each point pi motion: camera parameters Rj , tj
Objectivefunction:minimizereprojectionerror

Two scenarios Suppose we know 3D points
and have matches between these points and an image
How can we compute the camera parameters?
Suppose we have known camera parameters,
each of which observes a point
How can we compute the 3D location of that point?

Structure from motion
SfM solves both of these problems at once
A kind of chicken-and-egg problem (but solvable)

First step: how to get correspondence? Feature detection and matching

Feature detection Detect features using SIFT [Lowe, IJCV 2004]

Feature detection Detect features using SIFT [Lowe, IJCV 2004]

Feature matching Match features between each pair of images

Correspondence estimation
Link up pairwise matches to form connected components of matches across several images
Image 1 Image 2 Image 3 Image 4

Structure from motion
minimize g(R, T, X)

Re-projection Error

Problem size
What are the variables need to be solved? R t P
Trevi Fountain collection 466 input photos
+ > 100,000 3D points
= very large optimization problem

Constraints vs #Unknowns
argmin(uj f(K,R ,T ,P))2 +(vj g(K,R ,T ,P))2
{P},K,{R },{T } j=1 i=1 ijj
M camera poses
2MN point constraints

Structure from motion
Minimize sum of squared reprojection errors:
predicted observed
image location image location is point i visible in image j ?
Minimizing this function is called bundle adjustment
e.g. Levenberg-Marquardt
indicator variable:

SfM applications
3D modeling
Robot navigation and mapmaking
Visual effects (Match moving)
https://www.youtube.com/watch?v=RdYWp70P_kY

CS: assignmentchef QQ: 1823890830 Email: [email protected]