[SOLVED] COMP9517 Lab 2- SIFT: Scale-Invariant Feature Transform

COMP9517: Computer Vision
2022 T2 Lab 2 Specification
Maximum Marks Achievable: 2.5
This lab is worth 2.5% of the total course marks.
Objective: This lab revisits important concepts covered in the lectures of Week 3 and aims to make you familiar with implementing specific algorithms.
Materials: The sample image as well as template code to be used in the tasks of this lab is available in WebCMS3. You are required to use OpenCV 3+ with Python 3+.
Submission: The tasks are assessable after the lab. Submit your source code as a Jupyter notebook (.ipynb) with output images (.png) in a single zip file by the above deadline. The submission link will be announced in due time.
The sample image Cathedral.png is to be used for all questions.
SIFT: Scale-Invariant Feature Transform
A well-known algorithm in computer vision to detect and describe local features in images is the scale-invariant feature transform (SIFT). Its applications include object recognition, mapping and navigation, image stitching, 3D modelling, object tracking, and others.
A SIFT feature of an image is a salient keypoint with an associated descriptor. SIFT computation is commonly divided into two steps:
1) detection,
2) description.
At the end of the detection step, for each keypoint the SIFT algorithm computes the:
keypoint spatial coordinates (x, y),
keypoint scale (in the scale space),
keypoint dominant orientation.
The lab files should be submitted online.
Instructions for submission will be posted closer to the deadline.
Deadline for submission is Week 4, Tuesday 21 June 2022, 18:00:00.
The subsequent description step computes a distinctive 128-dimensional feature vector for each keypoint. SIFT is designed in such a way that this descriptive feature vector is invariant to scaling and rotation. Moreover, the algorithm offers decent robustness to noise, illumination gradients, and affine transformations.
Task 1 (0.5 mark): Compute the SIFT features of the given image.
a) Extract SIFT features with default parameters and show the keypoints on the image.
b) To achieve better visualization of the keypoints, reduce the number of keypoints. Hint: Vary the parameter contrastThreshold or nfeatures so that the number of keypoints becomes about 10% of all default keypoints.
Submit the images obtained in a) and b) and briefly describe in your Jupyter notebook the approach you used for b).
Task 2 (1 mark): Change the scale of the image and recompute the SIFT features.
a) Enlarge the given image with a scaling factor of 115%.
b) Extract the SIFT features and show the keypoints on the scaled image using the same parameter setting as for Task 1 (for the reduced number of keypoints).
c) Inspect the keypoints visually: Are the keypoints of the scaled image roughly the same as those of the original image? What does this imply?
d) Match the SIFT descriptors of the keypoints of the scaled image with those of the original image using the nearest-neighbour distance ratio method. Show the keypoints of the 5 best-matching descriptors on both the original and the scaled image. Hint: Brute-force matching is available in OpenCV for feature matching.
Submit the images obtained in b) and d) and include in your Jupyter notebook your answers to the questions in c).
Task 3 (1 mark): Rotate the image and recompute the SIFT features.
a) Rotate the given image clockwise by 60 degrees and by 120 degrees.
b) For each rotated image, extract the SIFT features and show the keypoints on the image using the same parameter setting as for Task 1 (for the reduced number of keypoints).
c) For each rotated image, inspect the keypoints visually: Are the keypoints of the rotated image roughly the same as those of the original image? What does this imply?
d) For each rotated image, match the SIFT descriptors of the keypoints with those of the original image using the nearest-neighbour distance ratio method. Show the keypoints of the 5 best-matching descriptors on both the original and the rotated image.
Submit the images obtained in b) and d) and include in your Jupyter notebook your answers to the questions in c).
Coding Requirements and Suggestions
Check the OpenCV documentation for various in-built functions to find SIFT features, draw keypoints, and match keypoints in images. You should understand how the algorithm works, what parameters you can set in these in-built functions, and how these parameters affect the output. For your reference, below are links to relevant OpenCV functions.
2D Features Framework https://docs.opencv.org/4.6.0/da/d9b/group__features2d.html
Drawing Functions of Keypoints and Matches
https://docs.opencv.org/4.6.0/d4/d5d/group__features2d__draw.html
Descriptor Matchers
https://docs.opencv.org/4.6.0/d8/d9b/group__features2d__match.html
OpenCV SIFT Class reference
https://docs.opencv.org/4.6.0/d7/d60/classcv_1_1SIFT.html
In your Jupyter notebook, the input image should be readable from the location specified as an argument, and all output images and other requested results should be displayed in the notebook environment. All cells in your notebook should have been executed so that the tutor/marker does not have to execute the notebook again to see the results.
Refer to the following page to understand image features and various feature detectors:
https://docs.opencv.org/4.6.0/db/d27/tutorial_py_table_of_contents_feature2d.html
Also, refer to the following example of computing SIFT features and showing the keypoints:
https://docs.opencv.org/4.6.0/da/df5/tutorial_py_sift_intro.html
And finally see this page for an example of feature matching:
https://docs.opencv.org/4.6.0/dc/dc3/tutorial_py_matcher.html
A template is provided in WebCMS for this lab.
Reference: D. G. Lowe. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, vol. 60, no. 2, pp. 91-110, November 2004. https://doi.org/10.1023/B:VISI.0000029664.99615.94
Copyright: UNSW CSE COMP9517 Team
Released: 14 June 2022