[SOLVED] COMP9517 Lab2 SIFT: Scale Invariant Feature Transform

SIFT: Scale-Invariant Feature Transform

A well-known algorithm in computer vision to detect and describe local features in images is the scale-invariant feature transform (SIFT). Its applications include object recognition, mapping and navigation, image stitching, 3D modelling, object tracking, and others.

A SIFT feature of an image is a salient keypoint with an associated descriptor. SIFT computation is commonly divided into two steps: 1) detection and 2) description.

At the end of the detection step, for each keypoint the SIFT algorithm computes the:

• keypoint spatial coordinates (x, y),
• keypoint scale (in the scale space),
• keypoint dominant orientation.

The description step computes a distinctive 128-dimensional feature vector for each keypoint. SIFT is designed to be invariant to scale and rotation. Moreover, the algorithm offers decent robustness to noise, illumination gradients, and affine transformations.

SIFT in OpenCV

In OpenCV the SIFT algorithm is available only in the non-free module (OpenCV has both free and non-free modules). The algorithm has been patented by the creator but can be freely used for academic and research purposes. The non-free modules can be found in the opencv_contrib package. You will first need to install this package, as shown below, and then you can use the SIFT module.

Initialize and activate the virtual environment (optional):

$ python3 -m venv env

$ source venv/bin/activate

Install the correct version of OpenCV and the contrib module:

$ pip install opencv-python==3.4.2.17

$ pip install opencv-contrib-python==3.4.2.17

Task 1 (0.5 mark): Compute the SIFT features of the given image.

1. Extract SIFT features with default parameters and show the keypoints on the image.
2. To achieve better visualization of the keypoints, reduce the number of keypoints. Hint: Vary the parameter contrastThreshold or nfeatures so that the number of keypoints becomes about 10% of all default keypoints.

Submit the images obtained in a) and b) and mention the approach you used in b).

Task 2 (1 mark): Change the scale of the image and recompute the SIFT features. a) Enlarge the given image by a scale percentage of 115.

1. Extract the SIFT features and show the keypoints on the scaled image using the same parameter setting as for Task 1 (for the reduced number of keypoints).
2. Inspect the keypoints visually: Are the keypoints of the scaled image roughly the same as those of the original image? What does this observation imply?
3. Match the SIFT descriptors of the keypoints of the scaled image with those of the original image using the nearest-neighbour distance ratio method. Show the keypoints of the 5 best-matching descriptors on both the original and the scaled image.

Hint: Brute-force matching is available in OpenCV for feature matching.

Submit the images obtained in b) and d) and your answers to the questions in c).

Task 3 (1 mark): Rotate the image and recompute the SIFT features. a) Rotate the given image clockwise by 60 degrees.

1. Extract the SIFT features and show the keypoints on the rotated image using the same parameter setting as for Task 1 (for the reduced number of keypoints).
2. Inspect the keypoints visually: Are the keypoints of the rotated image roughly the same as those of the original image? What does this observation imply?
3. Match the SIFT descriptors of the keypoints of the rotated image with those of the original image using the nearest-neighbour distance ratio method. Show the keypoints of the 5 best-matching descriptors on both the original and the rotated image.

Submit the images obtained in b) and d) and your answers to the questions in c).