Figure 1: (a) You will use the vanishing points in a single image to calibrate your cameras. (b) You will use multiple images of checkerboard patterns to calibrate your camera.

You will calibrate your camera (focal length, principal points, and lens distortion parameters).

Write-up:

• (Single Image Calibration) Take a single image where you can estimate three vanishing points v_{X inf}, v_{Y inf}, and v_{Z inf} as shown in Figure 3(c). Compute f, p_x, and p_y using the vanishing points.
• (Multiple Image Calibration) Print out the checkerboard pattern as shown in Figure 1(b) (http://www-users.cs.umn.edu/~hspark/CSci5980/code/pattern.pdf). Derive f, p_x, and p_y based on homographies from multiple images (at least 5 images). Compute the intrinsic parameters by solving the least squares system.
• (MATLAB Toolbox Calibration) Use the existing MATLAB Calibration Toolbox

(https://www.vision.caltech.edu/bouguetj/calib_doc/) to compute the intrinsic parameters (at least 20 images from different views). Verify the estimate with your solutions in (1) and (2).

1 Rotation Interpolation

(c) Interpolated View

Figure 2: Using two images of pure rotation (a and b), you will generate interpolated view by sampling rotation (c)

In HW 2, we generate a panoramic image from multiple images. In this problem, you will use two images of pure rotation (Figure 2(a) and 2(b)) to generate many views by interpolating between rotations as shown in Figure 2(c).

Write-up:

• Compute the pure rotation, R^R_L from the homography between two images, ^RH_L.
• Interpolate between two rotations, , from I₃ (Left image) to R (Right image) where N > 20 is the number of interpolated rotations using SLERP.
• Generate interpolated images similar to Figure 2(c).
  • Given ⁱR_L, compute homography from Left image to the interpolated image, ⁱH_L and warp it, im1 = ImageWarping(imageLeft, ⁱH_L).
  • Compute homography from Right image to the interpolated image, i.e., ⁱH_R =ⁱ H^R_LH_L¹. Warp image, im2 = ImageWarping(imageRight, ⁱH_R).
  • Composite two images based on distance, imInterpolated = w*im1+(1-w)*im2 where w is weight, inversely proportional to the distance.

2 Tour into Your Picture

Figure 3: (a) Given a single image taken by a cellphone, you will navigate into the image. (b) You rectify the image such that the Y axis of the camera aligns with the surface normal of the ground plane. (c) You define the five principal planes using vanishing points. (d) A 3D box can be texture mapped.

Take an photo with your camera of a scene where you can observe the Z vanishing point. You will virtually tour into your photo that creates 3D sensation.

Write-up:

• Rectification: Rectify your image such that the surface normal of the ground plane is aligned with the Y axis of your camera using the homography between ground plane and the camera. Derive and compute the homography for the rectification. Visualize rectified image similar to Figure 3(b).
• Z vanishing point: Given the rectified image, derive and compute the Z vanishing point, p (Figure 3(c)) using the four edges from the ground and ceiling planes using linear least squares (the intersection of four lines), i.e., Ax = b.
• 3D box generation:
  1. Define the four corners (u₁₁, u₁₂, u₂₁, u₂₂) of the purple plane (Figure 3(c)) in the image.
  2. Define the depth of the four corner, e.g., d=1 and compute the 3D locations of the corners, U = dK¹u.
  3. Express the 3D locations of frontal plane, V₁₁, V₁₂, V₂₁, and V₂₂ using d, p, K, u11, u12, u21, u22, v11, v12, v21, v22.
  4. Express a 3D point, X, in the 3D plane (U₁₁, U₁₂, U₂₁, and U₂₂), using two parameters, e.g., X = a₁₁ + a₂₂ + a₃. Apply this for the rest four planes.
• Homography mapping:
  1. Derive and compute the homography per plane to the rectified image, i.e., ^sH_plane1, sHplane2, sHplane3, sHplane4, sHplane5.

₁

Hint: u = KX a Hplane 2 .

1

2. Derive and compute the homography per plane to a target image of pure rotation about Y axis of the camera with 20 degree (tHplane1, tHplane2, tHplane3, tHplane4, ^tH_plane5). Derive and compute the homography from the rectified image to the target image. Hint: tHs =t Hplane sHplane1 .
3. Derive and compute the homography per plane to a target image of pure translation along Z axis of the camera with 0.2d. Derive and compute the homography from the rectified image to the target image.

(4) Generate camera trajectory by interpolating rotation and translation (at least 5 discrete rotation and translation) and generate a video of navigation.